climpred.metrics._threshold_brier_score

climpred.metrics._threshold_brier_score#

climpred.metrics._threshold_brier_score(forecast: Dataset, verif: Dataset, dim: str | List[str] | None = None, **metric_kwargs: Any) → Dataset | DataArray[source]#

Brier score of an ensemble for exceeding given thresholds.

CRPS = \int_f BS(F(f), H(f - o)) df

where F(o) = \int_{f \leq o} p(f) df is the cumulative distribution function (CDF) of the forecast distribution F, o is a point estimate of the true observation (observational error is neglected), BS denotes the Brier score and H(x) denotes the Heaviside step function, which we define here as equal to 1 for x \geq 0 and 0 otherwise.

Parameters:

  • forecast – Forecast with member dim.

  • verif – Verification data without member dim.

  • dim – Dimension to apply metric over. Expects at least member. Other dimensions are passed to xskillscore and averaged.

  • threshold (int, float, xarray.Dataset, xr.DataArray) – Threshold to check exceedance, see xskillscore.threshold_brier_score().

  • metric_kwargs – optional, see xskillscore.threshold_brier_score()

Notes

minimum

0.0

maximum

1.0

perfect

0.0

orientation

negative

References

See also

Example

>>> # get threshold brier score for each init
>>> HindcastEnsemble.verify(
...     metric="threshold_brier_score",
...     comparison="m2o",
...     dim="member",
...     threshold=0.2,
...     alignment="same_inits",
... )
<xarray.Dataset> Size: 9kB
Dimensions:     (lead: 10, init: 52)
Coordinates:
  * lead        (lead) int32 40B 1 2 3 4 5 6 7 8 9 10
    valid_time  (lead, init) object 4kB 1955-01-01 00:00:00 ... 2015-01-01 00...
    threshold   float64 8B 0.2
    init        (init) object 416B 1954-01-01 00:00:00 ... 2005-01-01 00:00:00
    skill       <U11 44B 'initialized'
Data variables:
    SST         (lead, init) float64 4kB 0.0 0.0 0.0 0.0 ... 0.25 0.36 0.09 0.01
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  HindcastEnsemble.verify()
    number_of_members:             10
    alignment:                     same_inits
    metric:                        threshold_brier_score
    comparison:                    m2o
    dim:                           member
    reference:                     []
    threshold:                     0.2

>>> # multiple thresholds averaging over init dimension
>>> HindcastEnsemble.verify(
...     metric="threshold_brier_score",
...     comparison="m2o",
...     dim=["member", "init"],
...     threshold=[0.2, 0.3],
...     alignment="same_verifs",
... )
<xarray.Dataset> Size: 260B
Dimensions:    (lead: 10, threshold: 2)
Coordinates:
  * lead       (lead) int32 40B 1 2 3 4 5 6 7 8 9 10
  * threshold  (threshold) float64 16B 0.2 0.3
    skill      <U11 44B 'initialized'
Data variables:
    SST        (lead, threshold) float64 160B 0.08712 0.005769 ... 0.01923
Attributes:
    prediction_skill_software:     climpred https://climpred.readthedocs.io/
    skill_calculated_by_function:  HindcastEnsemble.verify()
    number_of_initializations:     64
    number_of_members:             10
    alignment:                     same_verifs
    metric:                        threshold_brier_score
    comparison:                    m2o
    dim:                           ['member', 'init']
    reference:                     []
    threshold:                     [0.2, 0.3]
Contents