climpred.classes.HindcastEnsemble.bootstrap

HindcastEnsemble.bootstrap(metric: Optional[Union[str, climpred.metrics.Metric]] = None, comparison: Optional[Union[str, climpred.comparisons.Comparison]] = None, dim: Optional[Union[str, List[str]]] = None, alignment: Optional[str] = None, reference: Optional[Union[List[str], str]] = None, groupby: Optional[Union[str, xarray.DataArray]] = None, iterations: Optional[int] = None, sig: int = 95, resample_dim: str = 'member', pers_sig: Optional[int] = None, **metric_kwargs: Optional[Any]) → xarray.Dataset[source]

Bootstrap with replacement according to Goddard et al. [2013].

Parameters

  • metric – Metric to apply for verification, see metrics

  • comparison – How to compare to the observations/verification data. See comparisons.

  • dim – Dimension(s) to apply metric over. dim is passed on to xskillscore.{metric} and includes xskillscore’s member_dim. dim should contain member when comparison is probabilistic but should not contain member when comparison="e2o". Defaults to None meaning that all dimensions other than lead are reduced.

  • reference – Type of reference forecasts with which to verify against. One or more of ["uninitialized", "persistence", "climatology"]. Defaults to None meaning no reference. If None or [], returns no p value.

  • alignment – which inits or verification times should be aligned?

    • “”maximize: maximize the degrees of freedom by slicing init and verif to a common time frame at each lead.

    • "same_inits": slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations.

    • "same_verif": slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates.

  • iterations – Number of resampling iterations for bootstrapping with replacement. Recommended >= 500.

  • sig – Significance level in percent for deciding whether uninitialized and persistence beat initialized skill.

  • resample_dim – dimension to resample from. Default: "member".

    • "member": select a different set of members from hind

    • "init": select a different set of initializations from hind

  • pers_sig – If not None, the separate significance level for persistence.

  • groupby – group init before passing initialized to bootstrap.

  • **metric_kwargs – arguments passed to metric.

Returns

xarray.Dataset with dimensions results (holding skill, p, low_ci and high_ci) and skill (holding initialized, persistence and/or uninitialized):

  • results=”verify skill”, skill=”initialized”:

    mean initialized skill

  • results=”high_ci”, skill=”initialized”:

    high confidence interval boundary for initialized skill

  • results=”p”, skill=”uninitialized”:

    p value of the hypothesis that the difference of skill between the initialized and uninitialized simulations is smaller or equal to zero based on bootstrapping with replacement.

  • results=”p”, skill=”persistence”:

    p value of the hypothesis that the difference of skill between the initialized and persistence simulations is smaller or equal to zero based on bootstrapping with replacement.

References

Goddard et al. [2013]

Example

Continuous Ranked Probability Score ("crps") comparing every member forecast to the verification ("m2o") over the same initializations ("same_inits") for all leads reducing dimension member 50 times after resampling member dimension with replacement. Note that dimension init remains. Also calculate reference skill for the "persistence", "climatology" and "uninitialized" forecast and compare whether initialized skill is better than reference skill: Returns verify skill, probability that reference forecast performs better than initialized and the lower and upper bound of the resample.

>>> HindcastEnsemble.bootstrap(
...     metric="crps",
...     comparison="m2o",
...     dim="member",
...     iterations=50,
...     resample_dim="member",
...     alignment="same_inits",
...     reference=["persistence", "climatology", "uninitialized"],
... )
<xarray.Dataset>
Dimensions:     (skill: 4, results: 4, lead: 10, init: 51)
Coordinates:
  * init        (init) object 1955-01-01 00:00:00 ... 2005-01-01 00:00:00
  * lead        (lead) int32 1 2 3 4 5 6 7 8 9 10
    valid_time  (lead, init) object 1956-01-01 00:00:00 ... 2015-01-01 00:00:00
  * results     (results) <U12 'verify skill' 'p' 'low_ci' 'high_ci'
  * skill       (skill) <U13 'initialized' 'persistence' ... 'uninitialized'
Data variables:
    SST         (skill, results, lead, init) float64 0.1202 0.01764 ... 0.1033
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  HindcastEnsemble.bootstrap()
    number_of_members:             10
    alignment:                     same_inits
    metric:                        crps
    comparison:                    m2o
    dim:                           member
    reference:                     ['persistence', 'climatology', 'uninitiali...
    resample_dim:                  member
    sig:                           95
    iterations:                    50
    confidence_interval_levels:    0.975-0.025

previous

climpred.classes.HindcastEnsemble.verify

next

climpred.classes.HindcastEnsemble.generate_uninitialized