climpred.classes.HindcastEnsemble.verify

climpred.classes.HindcastEnsemble.verify#

HindcastEnsemble.verify(*, metric: str | Metric, comparison: str | Comparison, dim: str | List[str] | None = None, alignment: str | None = None, reference: str | List[str] | None = None, groupby: DataArray | str | None = None, **metric_kwargs: Any | None) → Dataset[source]#

Verify the initialized ensemble against observations.

Note

This will automatically verify against all shared variables between the initialized ensemble and observations/verification data.

Parameters:

  • metric – Metric to apply for verification, see metrics

  • comparison – How to compare to the observations/verification data. See comparisons.

  • dim – Dimension(s) to apply metric over. dim is passed on to xskillscore.{metric} and includes xskillscore’s member_dim. dim should contain member when comparison is probabilistic but should not contain member when comparison=e2o. Defaults to None meaning that all dimensions other than lead are reduced.

  • alignment – which inits or verification times should be aligned?

    • "maximize": maximize the degrees of freedom by slicing initialized and verif to a common time frame at each lead.

    • "same_inits": slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations.

    • "same_verif": slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates.

  • reference – Type of reference forecasts with which to verify against. One or more of ["uninitialized", "persistence", "climatology"]. Defaults to None meaning no reference.

  • groupby – group init before passing initialized to verify.

  • **metric_kwargs – arguments passed to metric.

Returns:

initialized and reference forecast skill reduced by dimensions dim

Example

Continuous Ranked Probability Score (crps) comparing every member with the verification (m2o) over the same verification time (same_verifs) for all leads reducing dimensions init and member:

>>> HindcastEnsemble.verify(
...     metric="crps",
...     comparison="m2o",
...     alignment="same_verifs",
...     dim=["init", "member"],
... )
<xarray.Dataset>
Dimensions:  (lead: 10)
Coordinates:
  * lead     (lead) int32 1 2 3 4 5 6 7 8 9 10
    skill    <U11 'initialized'
Data variables:
    SST      (lead) float64 0.05208 0.05009 0.05489 ... 0.09261 0.1083 0.1176
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  HindcastEnsemble.verify()
    number_of_initializations:     64
    number_of_members:             10
    alignment:                     same_verifs
    metric:                        crps
    comparison:                    m2o
    dim:                           ['init', 'member']
    reference:                     []

Root mean square error ("rmse") comparing the ensemble mean with the verification ("e2o") over the same initializations ("same_inits") for all leads reducing dimension init while also calculating reference skill for the "persistence", "climatology" and "uninitialized" forecast.

>>> HindcastEnsemble.verify(
...     metric="rmse",
...     comparison="e2o",
...     alignment="same_inits",
...     dim="init",
...     reference=["persistence", "climatology", "uninitialized"],
... )
<xarray.Dataset>
Dimensions:  (skill: 4, lead: 10)
Coordinates:
  * lead     (lead) int32 1 2 3 4 5 6 7 8 9 10
  * skill    (skill) <U13 'initialized' 'persistence' ... 'uninitialized'
Data variables:
    SST      (skill, lead) float64 0.08135 0.08254 0.086 ... 0.1012 0.1017
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  HindcastEnsemble.verify()
    number_of_initializations:     64
    number_of_members:             10
    alignment:                     same_inits
    metric:                        rmse
    comparison:                    e2o
    dim:                           init
    reference:                     ['persistence', 'climatology', 'uninitiali...

The verification does not need to reduce a dimension. To obtain the skill for each initialization, set dim=[].

>>> HindcastEnsemble.verify(
...     metric="rmse",
...     comparison="e2o",
...     alignment="same_verifs",
...     dim=[],
... )
<xarray.Dataset>
Dimensions:     (init: 61, lead: 10)
Coordinates:
  * init        (init) object 1954-01-01 00:00:00 ... 2014-01-01 00:00:00
  * lead        (lead) int32 1 2 3 4 5 6 7 8 9 10
    valid_time  (lead, init) object 1955-01-01 00:00:00 ... 2024-01-01 00:00:00
    skill       <U11 'initialized'
Data variables:
    SST         (lead, init) float64 nan nan nan nan nan ... nan nan nan nan nan
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  HindcastEnsemble.verify()
    number_of_members:             10
    alignment:                     same_verifs
    metric:                        rmse
    comparison:                    e2o
    dim:                           []
    reference:                     []
Contents