climpred.metrics._reliability¶
- climpred.metrics._reliability(forecast, verif, dim=None, **metric_kwargs)[source]¶
Returns the data required to construct the reliability diagram for an event. The the relative frequencies of occurrence of an event for a range of forecast probability bins.
- Parameters
forecast (xr.object) – Raw forecasts with
member
dimension if logical provided in metric_kwargs. Probability forecasts in [0,1] if logical is not provided.verif (xr.object) – Verification data without
member
dim. Raw verification if logical provided, else binary verification.dim (list or str) – Dimensions to aggregate. Requires member if logical provided in metric_kwargs to create probability forecasts. If logical not provided in metric_kwargs, should not include member.
logical (callable, optional) – Function with bool result to be applied to verification data and forecasts and then
mean('member')
to get forecasts and verification data in interval [0,1]. Passed via metric_kwargs.probability_bin_edges (array_like, optional) – Probability bin edges used to compute the reliability. Bins include the left most edge, but not the right. Passed via metric_kwargs. Defaults to 6 equally spaced edges between 0 and 1+1e-8.
- Returns
- The relative frequency of occurrence for each
probability bin
- Return type
reliability (xr.object)
- Details:
perfect
flat distribution
See also
Example
Define a boolean/logical function for binary scoring:
>>> def pos(x): return x > 0 # checking binary outcomes
Option 1. Pass with keyword
logical
: (especially designed forPerfectModelEnsemble
, where binary verification can only be created after comparison))>>> HindcastEnsemble.verify(metric='reliability', comparison='m2o', ... dim=['member','init'], alignment='same_verifs', logical=pos) <xarray.Dataset> Dimensions: (lead: 10, forecast_probability: 5) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * forecast_probability (forecast_probability) float64 0.1 0.3 0.5 0.7 0.9 SST_samples (forecast_probability) float64 22.0 5.0 1.0 3.0 21.0 skill <U11 'initialized' Data variables: SST (lead, forecast_probability) float64 0.09091 ... 1.0
Option 2. Pre-process to generate a binary forecast and verification product:
>>> HindcastEnsemble.map(pos).verify(metric='reliability', ... comparison='m2o', dim=['init', 'member'], alignment='same_verifs') <xarray.Dataset> Dimensions: (lead: 10, forecast_probability: 5) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * forecast_probability (forecast_probability) float64 0.1 0.3 0.5 0.7 0.9 SST_samples (forecast_probability) float64 22.0 5.0 1.0 3.0 21.0 skill <U11 'initialized' Data variables: SST (lead, forecast_probability) float64 0.09091 ... 1.0
Option 3. Pre-process to generate a probability forecast and binary verification product. because
member
not present inhindcast
, usecomparison='e2o'
anddim='init'
:>>> HindcastEnsemble.map(pos).mean('member').verify(metric='reliability', ... comparison='e2o', dim='init', alignment='same_verifs') <xarray.Dataset> Dimensions: (lead: 10, forecast_probability: 5) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * forecast_probability (forecast_probability) float64 0.1 0.3 0.5 0.7 0.9 SST_samples (forecast_probability) float64 22.0 5.0 1.0 3.0 21.0 skill <U11 'initialized' Data variables: SST (lead, forecast_probability) float64 0.09091 ... 1.0