climpred.classes.PerfectModelEnsemble.verify#

PerfectModelEnsemble.verify(metric: Optional[Union[str, Metric]] = None, comparison: Optional[Union[str, Comparison]] = None, dim: Optional[Union[str, List[str]]] = None, reference: Optional[Union[List[str], str]] = None, groupby: Optional[Union[DataArray, str]] = None, **metric_kwargs: Optional[Any]) → Dataset[source]#

Verify initialized predictions against a configuration of its members.

Note

The configuration of the other ensemble members is based off of the comparison keyword argument.

Parameters:

  • metric – Metric to apply for verification, see metrics

  • comparison – How to compare the initialized prediction ensemble with itself, see comparisons.

  • dim – Dimension(s) over which to apply metric. dim is passed on to xskillscore.{metric} and includes xskillscore’s member_dim. dim should contain member when comparison is probabilistic but should not contain member when comparison=e2c. Defaults to None meaning that all dimensions other than lead are reduced.

  • reference – Type of reference forecasts with which to verify against. One or more of ["uninitialized", "persistence", "climatology"]. Defaults to None meaning no reference. For persistence, choose between set_options(PerfectModel_persistence_from_initialized_lead_0)=False (default) using compute_persistence() or set_options(PerfectModel_persistence_from_initialized_lead_0)=True using compute_persistence_from_first_lead().

  • groupby – group init before passing initialized to verify.

  • **metric_kwargs – Arguments passed to metric.

Returns:

initialized and reference forecast skill reduced by dimensions dim

Example

Root mean square error (rmse) comparing every member with the ensemble mean forecast (m2e) for all leads reducing dimensions init and member:

>>> PerfectModelEnsemble.verify(
...     metric="rmse", comparison="m2e", dim=["init", "member"]
... )
<xarray.Dataset>
Dimensions:  (lead: 20)
Coordinates:
  * lead     (lead) int64 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Data variables:
    tos      (lead) float32 0.1028 0.1249 0.1443 0.1707 ... 0.2113 0.2452 0.2297
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  PerfectModelEnsemble.verify()
    number_of_initializations:     12
    number_of_members:             10
    metric:                        rmse
    comparison:                    m2e
    dim:                           ['init', 'member']
    reference:                     []

Continuous Ranked Probability Score ("crps") comparing every member to every other member ("m2m") reducing dimensions member and init while also calculating reference skill for the persistence, climatology and uninitialized forecast.

>>> PerfectModelEnsemble.verify(
...     metric="crps",
...     comparison="m2m",
...     dim=["init", "member"],
...     reference=["persistence", "climatology", "uninitialized"],
... )
<xarray.Dataset>
Dimensions:  (skill: 4, lead: 20)
Coordinates:
  * lead     (lead) int64 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
  * skill    (skill) <U13 'initialized' 'persistence' ... 'uninitialized'
Data variables:
    tos      (skill, lead) float64 0.0621 0.07352 0.08678 ... 0.122 0.1246
Attributes:
    prediction_skill_software:                         climpred https://climp...
    skill_calculated_by_function:                      PerfectModelEnsemble.v...
    number_of_initializations:                         12
    number_of_members:                                 10
    metric:                                            crps
    comparison:                                        m2m
    dim:                                               ['init', 'member']
    reference:                                         ['persistence', 'clima...
    PerfectModel_persistence_from_initialized_lead_0:  False