climpred.classes.HindcastEnsemble.bootstrap¶
- HindcastEnsemble.bootstrap(metric=None, comparison=None, dim=None, alignment=None, reference=None, iterations=None, sig=95, resample_dim='member', pers_sig=None, **metric_kwargs)[source]¶
Bootstrap with replacement according to Goddard et al. 2013.
- Parameters
metric (str,
Metric
) – Metric to apply for verification, see metrics.comparison (str,
Comparison
) – How to compare to the observations/verification data, see comparisons.dim (str, list of str) – dimension(s) to apply metric over.
dim
is passed on to xskillscore.{metric} and includes xskillscore’smember_dim
.dim
should containmember
whencomparison
is probabilistic but should not containmember
whencomparison='e2o'
. Defaults toNone
meaning that all dimensions other thanlead
are reduced.reference (str, list of str) – Type of reference forecasts with which to verify. One or more of [‘uninitialized’, ‘persistence’, ‘climatology’]. If None or empty, returns no p value.
alignment (str) –
which inits or verification times should be aligned?
’maximize’: maximize the degrees of freedom by slicing
init
andverif
to a common time frame at each lead.’same_inits’: slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations.
’same_verif’: slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates.
iterations (int) – Number of resampling iterations for bootstrapping with replacement. Recommended >= 500.
sig (int, default 95) – Significance level in percent for deciding whether uninitialized and persistence beat initialized skill.
dimension to resample from. default: ‘member’.
’member’: select a different set of members from hind
’init’: select a different set of initializations from hind
pers_sig (int, default None) – If not None, the separate significance level for persistence.
**metric_kwargs (optional) – arguments passed to
metric
.
- Returns
with dimensions
result
(holdingskill
,p
,low_ci
andhigh_ci
) andskill
(holdinginitialized
,persistence
and/oruninitialized
):- result=’verify skill’, skill=’initialized’:
mean initialized skill
- result=’high_ci’, skill=’initialized’:
high confidence interval boundary for initialized skill
- result=’p’, skill=’uninitialized’:
p value of the hypothesis that the difference of skill between the initialized and uninitialized simulations is smaller or equal to zero based on bootstrapping with replacement.
- result=’p’, skill=’persistence’:
p value of the hypothesis that the difference of skill between the initialized and persistence simulations is smaller or equal to zero based on bootstrapping with replacement.
- Return type
xr.Datasets
Example
Calculate the Pearson’s Anomaly Correlation (‘acc’) comparing the ensemble mean forecast to the verification (
e2o
) over the same verification times (same_verifs
) for all leads reducing dimensionsinit
50 times after resamplingmember
dimension with replacement. Also calculate reference skill for thepersistence
,climatology
anduninitialized
forecast and compare whether initialized skill is better than reference skill: Returns verify skill, probability that reference forecast performs better than initialized and the lower and upper bound of the resample.>>> HindcastEnsemble.bootstrap(metric='acc', comparison='e2o', ... dim='init', iterations=50, resample_dim='member', ... alignment='same_verifs', ... reference=['persistence', 'climatology' ,'uninitialized']) <xarray.Dataset> Dimensions: (lead: 10, results: 4, skill: 4) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * results (results) <U12 'verify skill' 'p' 'low_ci' 'high_ci' * skill (skill) <U13 'initialized' 'persistence' ... 'uninitialized' Data variables: SST (skill, results, lead) float64 0.9313 0.9119 ... 0.8078 0.8078 Attributes: prediction_skill: calculated by climpred https://climpred.read... number_of_initializations: 61 number_of_members: 10 alignment: same_verifs metric: pearson_r comparison: e2o dim: ['init'] units: None confidence_interval_levels: 0.975-0.025 bootstrap_iterations: 50 p: probability that reference performs better t...