# What’s New

## Contents

# What’s New#

## climpred v2.3.0 (2021-11-25)#

Note

As both maintainers moved out of academia into industry, this will be probably the last release for a while. If you are interested in maintaining climpred, please ping us.

### Bug Fixes#

Fix

`reference="persistence"`

for resampled`init`

. (GH#730, GH#731) Aaron Spring.`HindcastEnsemble.verify()`

`(comparison="m2o", reference="uninitialized", dim="init")`

. (GH#735, GH#731) Aaron Spring.`HindcastEnsemble.remove_bias()`

does not drop single item`lead`

dimension. (GH#771, GH#773) Aaron Spring.

### New Features#

Refactored

`HindcastEnsemble.bootstrap()`

and`PerfectModelEnsemble.bootstrap()`

based on`HindcastEnsemble.verify()`

and`PerfectModelEnsemble.verify()`

, which makes them more comparable.`pers_sig`

is removed. Also`reference=["climatology", "persistence"]`

skill has variance if`resample_dim='init'`

.`bootstrap`

relies on either`set_option(resample_skill_func="...")`

:`"loop"`

: calls`climpred.bootstrap.resample_skill_loop()`

which loops over iterations and calls`verify`

every single time. Most understandable and stable, but slow.`"exclude_resample_dim_from_dim"`

: calls`climpred.bootstrap.resample_skill_exclude_resample_dim_from_dim()`

which calls`verify(dim=dim_without_resample_dim)`

, resamples over`resample_dim`

and then takes a mean over`resample_dim`

if in`dim`

. Enables`HindcastEnsemble.bootstrap(resample_dim="init", alignment="same_verifs")`

. Fast alternative for`resample_dim="init"`

.`"resample_before"`

: calls`climpred.bootstrap.resample_skill_resample_before()`

which resamples`iteration`

dimension and then calls`verify`

vectorized. Fast alternative for`resample_dim="member"`

.`"default"`

:`climpred`

decides which to use

(relates to GH#375, GH#731) Aaron Spring.

`climpred.set_option(resample_skill_func='exclude_resample_dim_from_dim')`

allows`HindcastEnsemble.bootstrap(alignment='same_verifs', resample_dim='init')`

. Does not work for`pearson_r`

-derived metrics. (GH#582, GH#731) Aaron Spring.`climpred.utils.convert_init_lead_to_valid_time_lead()`

converts`data(init, lead)`

to`data(valid_time, lead)`

to visualize predictability barrier and the reverse`climpred.utils.convert_valid_time_lead_to_init_lead()`

. (GH#774, GH#775, GH#783) Aaron Spring.

### Internals/Minor Fixes#

Refactor

`asv`

benchmarking. Add`run-benchmarks`

label to`PR`

to run`asv`

via Github Actions. (GH#664, GH#718) Aaron Spring.Remove

`ipython`

from`requirements.txt`

. (GH#720) Aaron Spring.Calculating

`np.isin`

on`asi8`

instead of`xr.CFTimeIndex`

speeds up`HindcastEnsemble.verify()`

and`HindcastEnsemble.bootstrap()`

with large number of inits. (GH#414, GH#724) Aaron Spring.Add option

`bootstrap_resample_skill_func`

for they what skill is resampled in`HindcastEnsemble.bootstrap()`

and`PerfectModelEnsemble.bootstrap()`

, see`set_options`

. (GH#731) Aaron Spring.Add option

`resample_iterations_func`

to decide whether`xskillscore.resampling.resample_iterations()`

or`xskillscore.resampling.resample_iterations()`

should be used, see`set_options`

. (GH#731) Aaron Spring. - Add option`bootstrap_uninitialized_from_iterations_mean`

to exchange`uninitialized`

skill with the iteration mean`uninitialized`

. Defaults to False., see`set_options`

. (GH#731) Aaron Spring.`alignment="same_verifs"`

will not result in`NaN``s in ``valid_time`

. (GH#777) Aaron Spring.`HindcastEnsemble.plot_alignment()`

`(return_xr=True)`

contains`valid_time`

coordinate. (GH#779) Aaron Spring.

### Bug Fixes#

Fix

`PerfectModel_persistence_from_initialized_lead_0=True`

with multiple references. (GH#732, GH#733) Aaron Spring.

### Documentation#

Add verify dim example showing how

`HindcastEnsemble.verify()`

and`PerfectModelEnsemble.verify()`

are sensitive to`dim`

and how`dim`

answers different research questions. (GH#740) Aaron Spring.

## climpred v2.2.0 (2021-12-20)#

### Bug Fixes#

Fix when creating

`valid_time`

from`lead.attrs["units"]`

in`["seasons", "years"]`

with multi-month stride in`init`

. (GH#698, GH#700) Aaron Spring.Fix

`seasonality="season"`

in`reference="climatology"`

. (GH#641, GH#703) Aaron Spring.

### New Features#

Upon instantiation,

`PredictionEnsemble`

generates new 2-dimensional coordinate`valid_time`

for`initialized`

from`init`

and`lead`

, which is matched with`time`

from`verification`

during alignment. (GH#575, GH#675, GH#678) Aaron Spring.

```
>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST")
>>> hind.lead.attrs["units"] = "years"
>>> climpred.HindcastEnsemble(hind).get_initialized()
<xarray.Dataset>
Dimensions: (lead: 10, member: 10, init: 64)
Coordinates:
* lead (lead) int32 1 2 3 4 5 6 7 8 9 10
* member (member) int32 1 2 3 4 5 6 7 8 9 10
* init (init) object 1954-01-01 00:00:00 ... 2017-01-01 00:00:00
valid_time (lead, init) object 1955-01-01 00:00:00 ... 2027-01-01 00:00:00
Data variables:
SST (init, lead, member) float64 ...
```

Allow

`lead`

as`float`

also if`calendar="360_day"`

or`lead.attrs["units"]`

not in`["years","seasons","months"]`

. (GH#564, GH#675) Aaron Spring.Implement

`HindcastEnsemble.generate_uninitialized()`

resampling years without replacement from`initialized`

. (GH#589, GH#591) Aaron Spring.Implement Logarithmic Ensemble Skill Score

`_less()`

. (GH#239, GH#687) Aaron Spring.`HindcastEnsemble.remove_seasonality()`

and`PerfectModelEnsemble.remove_seasonality()`

remove the seasonality of all`climpred`

datasets. (GH#530, GH#688) Aaron Spring.Add keyword

`groupby`

in`HindcastEnsemble.verify()`

,`PerfectModelEnsemble.verify()`

,`HindcastEnsemble.bootstrap()`

and`PerfectModelEnsemble.bootstrap()`

to group skill by initializations seasonality. (GH#635, GH#690) Aaron Spring.

```
>>> import climpred
>>> hind = climpred.tutorial.load_dataset("NMME_hindcast_Nino34_sst")
>>> obs = climpred.tutorial.load_dataset("NMME_OIv2_Nino34_sst")
>>> hindcast = climpred.HindcastEnsemble(hind).add_observations(obs)
>>> # skill for each init month separated
>>> skill = hindcast.verify(
... metric="rmse",
... dim="init",
... comparison="e2o",
... skipna=True,
... alignment="maximize",
... groupby="month",
... )
>>> skill
<xarray.Dataset>
Dimensions: (month: 12, lead: 12, model: 12)
Coordinates:
* lead (lead) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0
* model (model) object 'NCEP-CFSv2' 'NCEP-CFSv1' ... 'GEM-NEMO'
skill <U11 'initialized'
* month (month) int64 1 2 3 4 5 6 7 8 9 10 11 12
Data variables:
sst (month, lead, model) float64 0.4127 0.3837 0.3915 ... 1.255 3.98
>>> skill.sst.plot(hue="model", col="month", col_wrap=3)
```

`HindcastEnsemble.plot_alignment()`

shows how forecast and observations are aligned based on the alignment keyword. This may help understanding which dates are matched for the different`alignment`

approaches. (GH#701, GH#702) Aaron Spring.In [1]: from climpred.tutorial import load_dataset In [2]: hindcast = climpred.HindcastEnsemble( ...: load_dataset("CESM-DP-SST") ...: ).add_observations(load_dataset("ERSST")) ...: In [3]: hindcast.plot_alignment(edgecolor="w") Out[3]: <xarray.plot.facetgrid.FacetGrid at 0x7f97810fc5b0>

Add

`attrs`

to new`coordinates`

created by`climpred`

. (GH#695, GH#697) Aaron Spring.Add

`seasonality="weekofyear"`

in`reference="climatology"`

. (GH#703) Aaron Spring.Compute

`reference="persistence"`

in`PerfectModelEnsemble`

from`initialized`

first`lead`

if`set_options`

`(PerfectModel_persistence_from_initialized_lead_0=True)`

(`False`

by default) using`compute_persistence_from_first_lead()`

. (GH#637, GH#706) Aaron Spring.

### Internals/Minor Fixes#

Reduce dependencies. (GH#686) Aaron Spring.

Add typing. (GH#685, GH#692) Aaron Spring.

refactor

`add_attrs`

into`HindcastEnsemble.verify()`

and`HindcastEnsemble.bootstrap()`

. Now all keywords are captured in the skill dataset attributes`.attrs`

. (GH#475, GH#694) Aaron Spring.docstrings formatting with blackdocs. (GH#708) Aaron Spring.

### Documentation#

Refresh all docs with

`sphinx_book_theme`

and`myst_nb`

. (GH#707, GH#708, GH#709, GH#710) Aaron Spring.

## climpred v2.1.6 (2021-08-31)#

Adding on to `v2.1.5`

, more bias reduction methods wrapped from
xclim
are implemented.

### Bug Fixes#

Fix

`results="p"`

in`HindcastEnsemble.bootstrap()`

and`PerfectModelEnsemble.bootstrap()`

when`reference='climatology'`

. (GH#668, GH#670) Aaron Spring.`HindcastEnsemble.remove_bias()`

for`how`

in`["modified_quantile", "basic_quantile", "gamma_mapping", "normal_mapping"]`

from bias_correction takes all`member`

to create model distribution. (GH#667) Aaron Spring.

### New Features#

allow more bias reduction methods wrapped from xclim in

`HindcastEnsemble.remove_bias()`

:`how="EmpiricalQuantileMapping"`

:`xclim.sdba.adjustment.EmpiricalQuantileMapping`

`how="DetrendedQuantileMapping"`

:`xclim.sdba.adjustment.DetrendedQuantileMapping`

`how="PrincipalComponents"`

:`xclim.sdba.adjustment.PrincipalComponents`

`how="QuantileDeltaMapping"`

:`xclim.sdba.adjustment.QuantileDeltaMapping`

`how="Scaling"`

:`xclim.sdba.adjustment.Scaling`

`how="LOCI"`

:`xclim.sdba.adjustment.LOCI`

These methods do not respond to

`OPTIONS['seasonality']`

like the other methods. Provide`group="init.month"`

to group by month or`group='init'`

to skip grouping. Provide`group=None`

or skip`group`

to use`init.{OPTIONS['seasonality']}`

. (GH#525, GH#662, GH#666, GH#671) Aaron Spring.

## climpred v2.1.5 (2021-08-12)#

While `climpred`

has used in the
ASP summer colloquium 2021,
many new features in `HindcastEnsemble.remove_bias()`

were
implemented.

### Breaking changes#

renamed

`cross_validate`

to`cv=False`

in`HindcastEnsemble.remove_bias()`

. Only used when`train_test_split='unfair-cv'`

. (GH#648, GH#655). Aaron Spring.

### Bug Fixes#

Shift back

`init`

by`lead`

after`HindcastEnsemble.verify()`

. (GH#644, GH#645) Aaron Spring.

### New Features#

`HindcastEnsemble.remove_bias()`

accepts new keyword`train_test_split='fair/unfair/unfair-cv'`

(default`unfair`

) following Risbey et al. 2021. (GH#648, GH#655) Aaron Spring.allow more bias reduction methods in

`HindcastEnsemble.remove_bias()`

:`how="additive_mean"`

: correcting the mean forecast additively (already implemented)`how="multiplicative_mean"`

: correcting the mean forecast multiplicatively`how="multiplicative_std"`

: correcting the standard deviation multiplicatively

Wrapped from bias_correction:

`how="modified_quantile"`

: Bai et al. 2016`how="basic_quantile"`

: Themeßl et al. 2011`how="gamma_mapping"`

and`how="normal_mapping"`

: Switanek et al. 2017

`HindcastEnsemble.remove_bias()`

now does leave-one-out cross validation when passing`cv='LOO'`

and`train_test_split='unfair-cv'`

.`cv=True`

falls back to`cv='LOO'`

. (GH#643, GH#646) Aaron Spring.Add new metrics

`_spread()`

and`_mul_bias()`

(GH#638) Aaron Spring.Add new tutorial datasets: (GH#651) Aaron Spring.

`NMME_OIv2_Nino34_sst`

and`NMME_hindcast_Nino34_sst`

with monthly leads`Observations_Germany`

and`ECMWF_S2S_Germany`

with daily leads

Metadata from CF convenctions are automatically attached by cf_xarray. (GH#639, GH#656) Aaron Spring.

Raise warning when dimensions

`time`

,`init`

or`member`

are chunked to show user how to circumvent`xskillscore`

chunking`ValueError`

when passing these dimensions as`dim`

in`HindcastEnsemble.verify()`

or`HindcastEnsemble.bootstrap()`

. (GH#509, GH#658) Aaron Spring.Implement

`PredictionEnsemble.chunks`

. (GH#658) Aaron Spring.

### Documentation#

Speed up ENSO monthly example with IRIDL server-side preprocessing (see context) (GH#594, GH#633) Aaron Spring.

Add CITATION.cff. Please cite Brady and Spring, 2020. (GH) Aaron Spring.

Use

`NMME_OIv2_Nino34_sst`

and`NMME_hindcast_Nino34_sst`

with monthly leads for bias reduction demonstrating`HindcastEnsemble.remove_bias()`

. (GH#646) Aaron Spring.

## climpred v2.1.4 (2021-06-28)#

### New Features#

Allow

`hours`

,`minutes`

and`seconds`

as`lead.attrs['units']`

. (GH#404, GH#603) Aaron Spring.Allow to set

`seasonality`

via`set_options`

to specify how to group in`verify(reference='climatology'`

or in`HindcastEnsemble.remove_bias()`

. (GH#529, GH#593, GH#603) Aaron Spring.Allow

`weekofyear`

via`datetime`

in`HindcastEnsemble.remove_bias()`

, but not yet implemented in`verify(reference='climatology')`

. (GH#529, GH#603) Aaron Spring.Allow more dimensions in

`initialized`

than in`observations`

. This is particular useful if you have forecasts from multiple models (in a`model`

dimension) and want to verify against the same observations. (GH#129, GH#528, GH#619) Aaron Spring.Automatically rename dimensions to

`CLIMPRED_ENSEMBLE_DIMS`

[`"init"`

,`"member"`

,`"lead"`

] if CF standard_names in coordinate attributes match: (GH#613, GH#622) Aaron Spring.`"init"`

:`"forecast_reference_time"`

`"member"`

:`"realization"`

`"lead"`

:`"forecast_period"`

If

`lead`

coordinate is`pd.Timedelta`

,`PredictionEnsemble`

converts`lead`

coordinate upon instantiation to integer`lead`

and corresponding`lead.attrs["units"]`

. (GH#606, GH#627) Aaron Spring.Require

`xskillscore >= 0.0.20`

.`_rps()`

now works with different`category_edges`

for observations and forecasts, see daily ECMWF example. (GH#629, GH#630) Aaron Spring.Set options

`warn_for_failed_PredictionEnsemble_xr_call`

,`warn_for_rename_to_climpred_dims`

,`warn_for_init_coords_int_to_annual`

,`climpred_warnings`

via`set_options`

. (GH#628, GH#631) Aaron Spring.`PredictionEnsemble`

acts like`xarray.Dataset`

and understands`data_vars`

,`dims`

,`sizes`

,`coords`

,`nbytes`

,`equals`

,`identical`

,`__iter__`

,`__len__`

,`__contains__`

,`__delitem__`

. (GH#568, GH#632) Aaron Spring.

### Documentation#

Add documentation page about publicly available initialized datasets and corresponding `climpred examples <initialized-datasets.html>`_. (GH#510, GH#561, GH#600) Aaron Spring.

Add GEFS example for numerical weather prediction. (GH#602, GH#603) Aaron Spring.

Add subseasonal daily ECMWF example using climetlab to access hindcasts from ECMWF cloud. (GH#587, GH#603) Aaron Spring.

Add subseasonal daily S2S example accessing S2S output on IRIDL with a cookie and working with “on-the-fly” reforecasts with

`hdate`

dimension. (GH#588, GH#593) Aaron Spring.Added example climpred on GPU. Running

`PerfectModelEnsemble.verify()`

on GPU with cupy-xarray finishes 10x faster. (GH#592, GH#607) Aaron Spring.How to work with biweekly aggregates in

`climpred`

, see daily ECMWF example. (GH#625, GH#630) Aaron Spring.

### Internals/Minor Fixes#

Add weekly upstream CI, which raises issues for failures. Adapted from

`xarray`

. Manually trigger by`git commit -m '[test-upstream]'`

. Skip climpred_testing CI by`git commit -m '[skip-ci]'`

(GH#518, GH#596) Aaron Spring.

## climpred v2.1.3 (2021-03-23)#

### Breaking changes#

### New Features#

`HindcastEnsemble.verify()`

,`PerfectModelEnsemble.verify()`

,`HindcastEnsemble.bootstrap()`

and`PerfectModelEnsemble.bootstrap()`

accept reference`climatology`

. Furthermore, reference`persistence`

also allows probabilistic metrics (GH#202, GH#565, GH#566) Aaron Spring.Added new metric

`_roc`

Receiver Operating Characteristic as`metric='roc'`

. (GH#566) Aaron Spring.

### Bug fixes#

`HindcastEnsemble.verify()`

and`HindcastEnsemble.bootstrap()`

accept`dim`

as`list`

,`set`

,`tuple`

or`str`

(GH#519, GH#558) Aaron Spring.`PredictionEnsemble.map()`

now does not fail silently when applying a function to all`xr.Datasets`

of`PredictionEnsemble`

. Instead,`UserWarning``s are raised. Furthermore, ``PredictionEnsemble.map(func, *args, **kwargs)`

applies only function to Datasets with matching dims if`dim="dim0_or_dim1"`

is passed as`**kwargs`

. (GH#417, GH#437, GH#552) Aaron Spring.`_rpc`

was fixed in`xskillscore>=0.0.19`

and hence is not falsely limited to 1 anymore (GH#562, GH#566) Aaron Spring.

### Internals/Minor Fixes#

Docstrings are now tested in GitHub actions continuous integration. (GH#545, GH#560) Aaron Spring.

Github actions now cancels previous commits, instead of running the full testing suite on every single commit. (GH#560) Aaron Spring.

`PerfectModelEnsemble.verify()`

does not add climpred attributes to skill by default anymore. (GH#560) Aaron Spring.Drop

`python==3.6`

support. (GH#573) Aaron Spring.Notebooks are now linted with nb_black using

`%load_ext nb_black`

or`%load_ext lab_black`

for Jupyter notebooks and Jupyter lab. (GH#526, GH#572) Aaron Spring.Reduce dependencies to install climpred. (GH#454, GH#572) Aaron Spring.

Examples from documentation available via Binder. Find further examples in the

`examples`

folder. (GH#549, GH#578) Aaron Spring.Rename branch

`master`

to`main`

. (GH#579) Aaron Spring.

## climpred v2.1.2 (2021-01-22)#

This release is the fixed version for our Journal of Open Source Software (JOSS)
article about `climpred`

, see review.

### New Features#

Function to calculate predictability horizon

`predictability_horizon()`

based on condition. (GH#46, GH#521) Aaron Spring.

### Bug fixes#

`PredictionEnsemble.smooth()`

now carries`lead.attrs`

(GH#527, pr:521) Aaron Spring.`PerfectModelEnsemble.verify()`

now works with`references`

also for geospatial inputs, which returned`NaN`

before. (GH#522, pr:521) Aaron Spring.`PredictionEnsemble.plot()`

now shifts composite lead frequencies like`days`

,`pentads`

,`seasons`

correctly. (GH#532, GH#533) Aaron Spring.Adapt to

`xesmf>=0.5.2`

for spatial xesmf smoothing. (GH#543, GH#548) Aaron Spring.`HindcastEnsemble.remove_bias()`

now carries attributes. (GH#531, GH#551) Aaron Spring.

## climpred v2.1.1 (2020-10-13)#

### Breaking changes#

This version introduces a lot of breaking changes. We are trying to overhaul
`climpred`

to have an intuitive API that also forces users to think about methodology
choices when running functions. The main breaking changes we introduced are for
`HindcastEnsemble.verify()`

and
`PerfectModelEnsemble.verify()`

. Now, instead of assuming
defaults for most keywords, we require the user to define `metric`

, `comparison`

,
`dim`

, and `alignment`

(for hindcast systems). We also require users to designate
the number of `iterations`

for bootstrapping.

User now has to designate number of iterations with

`iterations=...`

in`HindcastEnsemble.bootstrap()`

(GH#384, GH#436) Aaron Spring and Riley X. Brady.Make

`metric`

,`comparison`

,`dim`

, and`alignment`

required (previous default`None`

) arguments for`HindcastEnsemble.verify()`

(GH#384, GH#436) Aaron Spring and Riley X. Brady.Metric

`_brier_score`

and`_threshold_brier_score()`

now requires callable keyword argument`logical`

instead of`func`

(GH#388) Aaron Spring.`HindcastEnsemble.verify()`

does not correct`dim`

automatically to`member`

for probabilistic metrics. (GH#282, GH#407) Aaron Spring.Users can no longer add multiple observations to

`HindcastEnsemble`

. This will make current and future development much easier on maintainers (GH#429, GH#453) Riley X. Brady.Standardize the names of the output coordinates for

`PredictionEnsemble.verify()`

and`PredictionEnsemble.bootstrap()`

to`initialized`

,`uninitialized`

, and`persistence`

.`initialized`

showcases the metric result after comparing the initialized ensemble to the verification data;`uninitialized`

when comparing the uninitialized (historical) ensemble to the verification data;`persistence`

is the evaluation of the persistence forecast (GH#460, GH#478, GH#476, GH#480) Aaron Spring.`reference`

keyword in`HindcastEnsemble.verify()`

should be choosen from [`uninitialized`

,`persistence`

].`historical`

no longer works. (GH#460, GH#478, GH#476, GH#480) Aaron Spring.`HindcastEnsemble.verify()`

returns no`skill`

dimension if`reference=None`

(GH#480) Aaron Spring.`comparison`

is not applied to uninitialized skill in`HindcastEnsemble.bootstrap()`

. (GH#352, GH#418) Aaron Spring.

### New Features#

This release is accompanied by a bunch of new features. Math operations can now be used
with our `PredictionEnsemble`

objects and their variables
can be sub-selected. Users can now quick plot time series forecasts with these objects.
Bootstrapping is available for `HindcastEnsemble`

. Spatial
dimensions can be passed to metrics to do things like pattern correlation. New metrics
have been implemented based on Contingency tables. We now include an early version
of bias removal for `HindcastEnsemble`

.

Use math operations like

`+-*/`

with`HindcastEnsemble`

and`PerfectModelEnsemble`

. See demo Arithmetic-Operations-with-PredictionEnsemble-Objects. (GH#377) Aaron Spring.Subselect data variables from

`PerfectModelEnsemble`

as from`xarray.Dataset`

:`PredictionEnsemble[["var1", "var3"]]`

(GH#409) Aaron Spring.Plot all datasets in

`HindcastEnsemble`

or`PerfectModelEnsemble`

by`PredictionEnsemble.plot()`

if no other spatial dimensions are present. (GH#383) Aaron Spring.Bootstrapping now available for

`HindcastEnsemble`

as`HindcastEnsemble.bootstrap()`

, which is analogous to the`PerfectModelEnsemble`

method. (GH#257, GH#418) Aaron Spring.`HindcastEnsemble.verify()`

allows all dimensions from`initialized`

ensemble as`dim`

. This allows e.g. spatial dimensions to be used for pattern correlation. Make sure to use`skipna=True`

when using spatial dimensions and output has NaNs (in the case of land, for instance). (GH#282, GH#407) Aaron Spring.Allow binary forecasts at when calling

`HindcastEnsemble.verify()`

, rather than needing to supply binary results beforehand. In other words,`hindcast.verify(metric='bs', comparison='m2o', dim='member', logical=logical)`

is now the same as`hindcast.map(logical).verify(metric='brier_score', comparison='m2o', dim='member'`

. (GH#431) Aaron Spring.Check

`calendar`

types when using`HindcastEnsemble.add_observations()`

,`HindcastEnsemble.add_uninitialized()`

,`PerfectModelEnsemble.add_control()`

to ensure that the verification data calendars match that of the initialized ensemble. (GH#300, GH#452, GH#422, GH#462) Riley X. Brady and Aaron Spring.Implement new metrics which have been ported over from https://github.com/csiro-dcfp/doppyo/ to

`xskillscore`

by Dougie Squire. (GH#439, GH#456) Aaron Springrank histogram

`_rank_histogram()`

discrimination

`_discrimination()`

reliability

`_reliability()`

ranked probability score

`_rps()`

contingency table and related scores

`_contingency()`

Perfect Model

`PerfectModelEnsemble.verify()`

no longer requires`control`

in`PerfectModelEnsemble`

. It is only required when`reference=['persistence']`

. (GH#461) Aaron Spring.Implemented bias removal

`remove_bias`

.`remove_bias(how='mean')`

removes the mean bias of initialized hindcasts with respect to observations. See example. (GH#389, GH#443, GH#459) Aaron Spring and Riley X. Brady.

### Deprecated#

`spatial_smoothing_xrcoarsen`

no longer used for spatial smoothing. (GH#391) Aaron Spring.`compute_metric`

,`compute_uninitialized`

and`compute_persistence`

no longer in use for`PerfectModelEnsemble`

in favor of`PerfectModelEnsemble.verify()`

with the`reference`

keyword instead. (GH#436, GH#468, GH#472) Aaron Spring and Riley X. Brady.`'historical'`

no longer a valid choice for`reference`

. Use`'uninitialized'`

instead. (GH#478) Aaron Spring.

### Bug Fixes#

`PredictionEnsemble.verify()`

and`PredictionEnsemble.bootstrap()`

now accept`metric_kwargs`

. (GH#387) Aaron Spring.`PerfectModelEnsemble.verify()`

now accepts`'uninitialized'`

as a reference. (GH#395) Riley X. Brady.Spatial and temporal smoothing

`PredictionEnsemble.smooth()`

now work as expected and rename time dimensions after`verify()`

. (GH#391) Aaron Spring.`PredictionEnsemble.verify(comparison='m2o', references=['uninitialized', 'persistence']`

does not fail anymore. (GH#385, GH#400) Aaron Spring.Remove bias using

`dayofyear`

in`HindcastEnsemble.reduce_bias()`

. (GH#443) Aaron Spring.`climpred`

works with`dask=>2.28`

. (GH#479, GH#482) Aaron Spring.

### Documentation#

Updates

`climpred`

tagline to “Verification of weather and climate forecasts.” (GH#420) Riley X. Brady.Adds section on how to use arithmetic with

`HindcastEnsemble`

. (GH#378) Riley X. Brady.Add docs section for similar open-source forecasting packages. (GH#432) Riley X. Brady.

Add all metrics to main API in addition to metrics page. (GH#438) Riley X. Brady.

Add page on bias removal Aaron Spring.

### Internals/Minor Fixes#

`PredictionEnsemble.verify()`

replaces deprecated`PerfectModelEnsemble.compute_metric()`

and accepts`reference`

as keyword. (GH#387) Aaron Spring.Cleared out unnecessary statistics functions from

`climpred`

and migrated them to`esmtools`

. Add`esmtools`

as a required package. (GH#395) Riley X. Brady.Remove fixed pandas dependency from

`pandas=0.25`

to stable`pandas`

. (GH#402, GH#403) Aaron Spring.`dim`

is expected to be a list of strings in`compute_perfect_model()`

and`compute_hindcast()`

. (GH#282, GH#407) Aaron Spring.Update

`cartopy`

requirement to 0.0.18 or greater to release lock on`matplotlib`

version. Update`xskillscore`

requirement to 0.0.18 to cooperate with new`xarray`

version. (GH#451, GH#449) Riley X. BradySwitch from Travis CI and Coveralls to Github Actions and CodeCov. (GH#471) Riley X. Brady

Assertion functions added for

`PerfectModelEnsemble`

:`assert_PredictionEnsemble()`

. (GH#391) Aaron Spring.Test all metrics against synthetic data. (GH#388) Aaron Spring.

## climpred v2.1.0 (2020-06-08)#

### Breaking Changes#

Keyword

`bootstrap`

has been replaced with`iterations`

. We feel that this more accurately describes the argument, since “bootstrap” is really the process as a whole. (GH#354) Aaron Spring.

### New Features#

`HindcastEnsemble`

and`PerfectModelEnsemble`

now use an HTML representation, following the more recent versions of`xarray`

. (GH#371) Aaron Spring.`HindcastEnsemble.verify()`

now takes`reference=...`

keyword. Current options are`'persistence'`

for a persistence forecast of the observations and`'uninitialized'`

for an uninitialized/historical reference, such as an uninitialized/forced run. (GH#341) Riley X. Brady.We now only enforce a union of the initialization dates with observations if

`reference='persistence'`

for`HindcastEnsemble`

. This is to ensure that the same set of initializations is used by the observations to construct a persistence forecast. (GH#341) Riley X. Brady.`compute_perfect_model()`

now accepts initialization (`init`

) as`cftime`

and`int`

.`cftime`

is now implemented into the bootstrap uninitialized functions for the perfect model configuration. (GH#332) Aaron Spring.New explicit keywords in bootstrap functions for

`resampling_dim`

and`reference_compute`

(GH#320) Aaron Spring.Logging now included for

`compute_hindcast`

which displays the`inits`

and verification dates used at each lead (GH#324) Aaron Spring, (GH#338) Riley X. Brady. See (logging).New explicit keywords added for

`alignment`

of verification dates and initializations. (GH#324) Aaron Spring. See (alignment)`'maximize'`

: Maximize the degrees of freedom by slicing`hind`

and`verif`

to a common time frame at each lead. (GH#338) Riley X. Brady.`'same_inits'`

: slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (GH#328) Riley X. Brady.`'same_verifs'`

: slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (GH#331) Riley X. Brady.

### Performance#

The major change for this release is a dramatic speedup in bootstrapping functions, led
by Aaron Spring. We focused on scalability with `dask`

and found many places we
could compute skill simultaneously over all bootstrapped ensemble members rather than
at each iteration.

Bootstrapping uninitialized skill in the perfect model framework is now sped up significantly for annual lead resolution. (GH#332) Aaron Spring.

General speedup in

`bootstrap_hindcast()`

and`bootstrap_perfect_model()`

: (GH#285) Aaron Spring.Properly implemented handling for lazy results when inputs are chunked.

User gets warned when chunking potentially unnecessarily and/or inefficiently.

### Bug Fixes#

Alignment options now account for differences in the historical time series if

`reference='historical'`

. (GH#341) Riley X. Brady.

### Internals/Minor Fixes#

Added a Code of Conduct (GH#285) Aaron Spring.

Gather

`pytest.fixture in ``conftest.py`

. (GH#313) Aaron Spring.Move

`x_METRICS`

and`COMPARISONS`

to`metrics.py`

and`comparisons.py`

in order to avoid circular import dependencies. (GH#315) Aaron Spring.`asv`

benchmarks added for`HindcastEnsemble`

(GH#285) Aaron Spring.Ignore irrelevant warnings in

`pytest`

and mark slow tests (GH#333) Aaron Spring.Default

`CONCAT_KWARGS`

now in all`xr.concat`

to speed up bootstrapping. (GH#330) Aaron Spring.Remove

`member`

coords for`m2c`

comparison for probabilistic metrics. (GH#330) Aaron Spring.Refactored

`compute_hindcast()`

and`compute_perfect_model()`

. (GH#330) Aaron Spring.Changed lead0 coordinate modifications to be compliant with

`xarray=0.15.1`

in`compute_persistence()`

. (GH#348) Aaron Spring.Exchanged

`my_quantile`

with`xr.quantile(skipna=False)`

. (GH#348) Aaron Spring.Remove

`sig`

from`plot_bootstrapped_skill_over_leadyear()`

. (GH#351) Aaron Spring.Require

`xskillscore v0.0.15`

and use their functions for effective sample size-based metrics. (:pr: 353) Riley X. Brady.Faster bootstrapping without replacement used in threshold functions of

`climpred.stats`

(GH#354) Aaron Spring.Require

`cftime v1.1.2`

, which modifies their object handling to create 200-400x speedups in some basic operations. (GH#356) Riley X. Brady.Resample first and then calculate skill in

`bootstrap_perfect_model()`

and`bootstrap_hindcast()`

(GH#355) Aaron Spring.

### Documentation#

Added demo to setup your own raw model output compliant to

`climpred`

(GH#296) Aaron Spring. See (here).Added demo using

`intake-esm`

with`climpred`

. See demo. (GH#296) Aaron Spring.Added Verification Alignment page explaining how initializations are selected and aligned with verification data. (GH#328) Riley X. Brady. See (here).

## climpred v2.0.0 (2020-01-22)#

### New Features#

Add support for

`days`

,`pentads`

,`weeks`

,`months`

,`seasons`

for lead time resolution.`climpred`

now requires a`lead`

attribute “units” to decipher what resolution the predictions are at. (GH#294) Kathy Pegion and Riley X. Brady.

`HindcastEnsemble`

now has`HindcastEnsemble.add_observations()`

and`HindcastEnsemble.get_observations()`

methods. These are the same as`.add_reference()`

and`.get_reference()`

, which will be deprecated eventually. The name change clears up confusion, since “reference” is the appropriate name for a reference forecast, e.g.`"persistence"`

. (GH#310) Riley X. Brady.`HindcastEnsemble`

now has`.verify()`

function, which duplicates the`.compute_metric()`

function. We feel that`.verify()`

is more clear and easy to write, and follows the terminology of the field. (GH#310) Riley X. Brady.`e2o`

and`m2o`

are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (GH#310) Riley X. Brady.

### Documentation#

New example pages for subseasonal-to-seasonal prediction using

`climpred`

. (GH#294) Kathy PegionComparisons page rewritten for more clarity. (GH#310) Riley X. Brady.

### Bug Fixes#

Fixed m2m broken comparison issue and removed correction. (GH#290) Aaron Spring.

### Internals/Minor Fixes#

Updates to

`xskillscore`

v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (GH#309) Riley X. Brady.Stacking dims is handled by

`comparisons`

, no need for internal keyword`stack_dims`

. Therefore`comparison`

now takes`metric`

as argument instead. (GH#290) Aaron Spring.`assign_attrs`

now carries dim (GH#290) Aaron Spring.`reference`

changed to`verif`

throughout hindcast compute functions. This is more clear, since`reference`

usually refers to a type of forecast, such as persistence. (GH#310) Riley X. Brady.`Comparison`

objects can now have aliases. (GH#310) Riley X. Brady.

## climpred v1.2.1 (2020-01-07)#

### Depreciated#

`mad`

no longer a keyword for the median absolute error metric. Users should now use`median_absolute_error`

, which is identical to changes in`xskillscore`

version 0.0.10. (GH#283) Riley X. Brady`pacc`

no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (GH#283) Riley X. Brady`msss`

no longer a keyword for the Murphy’s MSSS, since it is reserved for the standard MSSS. (GH#283) Riley X. Brady

### New Features#

Metrics

`pearson_r_eff_p_value`

and`spearman_r_eff_p_value`

account for autocorrelation in computing p values. (GH#283) Riley X. BradyMetric

`effective_sample_size`

computes number of independent samples between two time series being correlated. (GH#283) Riley X. BradyAdded keywords for metrics: (GH#283) Riley X. Brady

`'pval'`

for`pearson_r_p_value`

`['n_eff', 'eff_n']`

for`effective_sample_size`

`['p_pval_eff', 'pvalue_eff', 'pval_eff']`

for`pearson_r_eff_p_value`

`['spvalue', 'spval']`

for`spearman_r_p_value`

`['s_pval_eff', 'spvalue_eff', 'spval_eff']`

for`spearman_r_eff_p_value`

`'nev'`

for`nmse`

### Internals/Minor Fixes#

`climpred`

now requires`xarray`

version 0.14.1 so that the`drop_vars()`

keyword used in our package does not throw an error. (GH#276) Riley X. BradyUpdate to

`xskillscore`

version 0.0.10 to fix errors in weighted metrics with pairwise NaNs. (GH#283) Riley X. Brady`doc8`

added to`pre-commit`

to have consistent formatting on`.rst`

files. (GH#283) Riley X. BradyRemove

`proper`

attribute on`Metric`

class since it isn’t used anywhere. (GH#283) Riley X. BradyAdd testing for effective p values. (GH#283) Riley X. Brady

Add testing for whether metric aliases are repeated/overwrite each other. (GH#283) Riley X. Brady

`ppp`

changed to`msess`

, but keywords allow for`ppp`

and`msss`

still. (GH#283) Riley X. Brady

### Documentation#

Expansion of metrics documentation with much more detail on how metrics are computed, their keywords, references, min/max/perfect scores, etc. (GH#283) Riley X. Brady

Update terminology page with more information on metrics terminology. (GH#283) Riley X. Brady

## climpred v1.2.0 (2019-12-17)#

### Depreciated#

Abbreviation

`pval`

depreciated. Use`p_pval`

for`pearson_r_p_value`

instead. (GH#264) Aaron Spring.

### New Features#

Users can now pass a custom

`metric`

or`comparison`

to compute functions. (GH#268) Aaron Spring.New deterministic metrics (see metrics). (GH#264) Aaron Spring.

Spearman ranked correlation (spearman_r)

Spearman ranked correlation p-value (spearman_r_p_value)

Mean Absolute Deviation (mad)

Mean Absolute Percent Error (mape)

Symmetric Mean Absolute Percent Error (smape)

Users can now apply arbitrary

`xarray`

methods to`HindcastEnsemble`

and`PerfectModelEnsemble`

. (GH#243) Riley X. Brady.Add “getter” methods to

`HindcastEnsemble`

and`PerfectModelEnsemble`

to retrieve`xarray`

datasets from the objects. (GH#243) Riley X. Brady.

```
>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST")
>>> ref = climpred.tutorial.load_dataset("ERSST")
>>> hindcast = climpred.HindcastEnsemble(hind)
>>> hindcast = hindcast.add_reference(ref, "ERSST")
>>> print(hindcast)
<climpred.HindcastEnsemble>
Initialized Ensemble:
SST (init, lead, member) float64 ...
ERSST:
SST (time) float32 ...
Uninitialized:
None
>>> print(hindcast.get_initialized())
<xarray.Dataset>
Dimensions: (init: 64, lead: 10, member: 10)
Coordinates:
* lead (lead) int32 1 2 3 4 5 6 7 8 9 10
* member (member) int32 1 2 3 4 5 6 7 8 9 10
* init (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0
Data variables:
SST (init, lead, member) float64 ...
>>> print(hindcast.get_reference("ERSST"))
<xarray.Dataset>
Dimensions: (time: 61)
Coordinates:
* time (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015
Data variables:
SST (time) float32 ...
```

`metric_kwargs`

can be passed to`Metric`

. (GH#264) Aaron Spring.See

`metric_kwargs`

under metrics.

### Bug Fixes#

`HindcastEnsemble.compute_metric()`

doesn’t drop coordinates from the initialized hindcast ensemble anymore. (GH#258) Aaron Spring.Metric

`uacc`

does not crash when`ppp`

negative anymore. (GH#264) Aaron Spring.Update

`xskillscore`

to version 0.0.9 to fix all-NaN issue with`pearson_r`

and`pearson_r_p_value`

when there’s missing data. (GH#269) Riley X. Brady.

### Internals/Minor Fixes#

Rewrote

`varweighted_mean_period()`

based on`xrft`

. Changed`time_dim`

to`dim`

. Function no longer drops coordinates. (GH#258) Aaron SpringAdd

`dim='time'`

in`dpp()`

. (GH#258) Aaron SpringComparisons

`m2m`

,`m2e`

rewritten to not stack dims into supervector because this is now done in`xskillscore`

. (GH#264) Aaron SpringAdd

`tqdm`

progress bar to`bootstrap_compute()`

. (GH#244) Aaron SpringRemove inplace behavior for

`HindcastEnsemble`

and`PerfectModelEnsemble`

. (GH#243) Riley X. BradyAdded tests for chunking with

`dask`

. (GH#258) Aaron SpringFix test issues with esmpy 8.0 by forcing esmpy 7.1 (GH#269). Riley X. Brady

Rewrote

`metrics`

and`comparisons`

as classes to accomodate custom metrics and comparisons. (GH#268) Aaron Spring

### Documentation#

Add examples notebook for temporal and spatial smoothing. (GH#244) Aaron Spring

Add documentation for computing a metric over a specified dimension. (GH#244) Aaron Spring

Update API to be more organized with individual function/class pages. (GH#243) Riley X. Brady.

Add page describing the

`HindcastEnsemble`

and`PerfectModelEnsemble`

objects more clearly. (GH#243) Riley X. BradyAdd page for publications and helpful links. (GH#270) Riley X. Brady.

## climpred v1.1.0 (2019-09-23)#

### Features#

Write information about skill computation to netcdf attributes(GH#213) Aaron Spring

Temporal and spatial smoothing module (GH#224) Aaron Spring

Add metrics brier_score, threshold_brier_score and crpss_es (GH#232) Aaron Spring

Allow compute_hindcast and compute_perfect_model to specify which dimension dim to calculate metric over (GH#232) Aaron Spring

### Bug Fixes#

Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (GH#232) Aaron Spring

### Internals/Minor Fixes#

Rename .stats.DPP to dpp (GH#232) Aaron Spring

Add matplotlib as a main dependency so that a direct pip installation works (GH#211) Riley X. Brady.

`climpred`

is now installable from conda-forge (GH#212) Riley X. Brady.Fix erroneous descriptions of sample datasets (GH#226) Riley X. Brady.

Benchmarking time and peak memory of compute functions with asv (GH#231) Aaron Spring

### Documentation#

Add scope of package to docs for clarity for users and developers. (GH#235) Riley X. Brady.

## climpred v1.0.1 (2019-07-04)#

### Bug Fixes#

Accomodate for lead-zero within the

`lead`

dimension (GH#196) Riley X. Brady.Fix issue with adding uninitialized ensemble to

`HindcastEnsemble`

object (GH#199) Riley X. Brady.Allow

`max_dof`

keyword to be passed to`compute_metric`

and`compute_persistence`

for`HindcastEnsemble`

. (GH#199) Riley X. Brady.

### Internals/Minor Fixes#

Force

`xskillscore`

version 0.0.4 or higher to avoid`ImportError`

(GH#204) Riley X. Brady.Change

`max_dfs`

keyword to`max_dof`

(GH#199) Riley X. Brady.Add tests for

`HindcastEnsemble`

and`PerfectModelEnsemble`

. (GH#199) Riley X. Brady

## climpred v1.0.0 (2019-07-03)#

`climpred`

v1.0.0 represents the first stable release of the package. It includes
`HindcastEnsemble`

and `PerfectModelEnsemble`

objects to
perform analysis with.
It offers a suite of deterministic and probabilistic metrics that are optimized to be
run on single time series or grids of data (e.g., lat, lon, and depth). Currently,
`climpred`

only supports annual forecasts.

### Features#

Bootstrap prediction skill based on resampling with replacement consistently in

`ReferenceEnsemble`

and`PerfectModelEnsemble`

. (GH#128) Aaron SpringConsistent bootstrap function for

`climpred.stats`

functions via`bootstrap_func`

wrapper. (GH#167) Aaron Springmany more metrics:

`_msss_murphy`

,`_less`

and probabilistic`_crps`

,`_crpss`

(GH#128) Aaron Spring

### Bug Fixes#

`compute_uninitialized`

now trims input data to the same time window. (GH#193) Riley X. Brady`rm_poly`

now properly interpolates/fills NaNs. (GH#192) Riley X. Brady

### Internals/Minor Fixes#

The

`climpred`

version can be printed. (GH#195) Riley X. BradyConstants are made elegant and pushed to a separate module. (GH#184) Andrew Huang

Checks are consolidated to their own module. (GH#173) Andrew Huang

### Documentation#

Documentation built extensively in multiple PRs.

## climpred v0.3 (2019-04-27)#

`climpred`

v0.3 really represents the entire development phase leading up to the
version 1 release. This was done in collaboration between Riley X. Brady,
Aaron Spring, and Andrew Huang. Future releases will have less additions.

### Features#

Introduces object-oriented system to

`climpred`

, with classes`ReferenceEnsemble`

and`PerfectModelEnsemble`

. (GH#86) Riley X. BradyExpands bootstrapping module for perfect-module configurations. (GH#78, GH#87) Aaron Spring

Adds functions for computing Relative Entropy (GH#73) Aaron Spring

Sets more intelligible dimension expectations for

`climpred`

(GH#98, GH#105) Riley X. Brady and Aaron Spring:`init`

: initialization dates for the prediction ensemble`lead`

: retrospective forecasts from prediction ensemble; returned dimension for prediction calculations`time`

: time dimension for control runs, references, etc.`member`

: ensemble member dimension.

Updates

`open_dataset`

to display available dataset names when no argument is passed. (GH#123) Riley X. BradyChange

`ReferenceEnsemble`

to`HindcastEnsemble`

. (GH#124) Riley X. BradyAdd probabilistic metrics to

`climpred`

. (GH#128) Aaron SpringConsolidate separate perfect-model and hindcast functions into singular functions (GH#128) Aaron Spring

Add option to pass proxy through to

`open_dataset`

for firewalled networks. (GH#138) Riley X. Brady

### Bug Fixes#

`xr_rm_poly`

can now operate on Datasets and with multiple variables. It also interpolates across NaNs in time series. (GH#94) Andrew HuangTravis CI,

`treon`

, and`pytest`

all run for automated testing of new features. (GH#98, GH#105, GH#106) Riley X. Brady and Aaron SpringClean up

`check_xarray`

decorators and make sure that they work. (GH#142) Andrew HuangEnsures that

`help()`

returns proper docstring even with decorators. (GH#149) Andrew HuangFixes bootstrap so p values are correct. (GH#170) Aaron Spring

### Internals/Minor Fixes#

Adds unit testing for all perfect-model comparisons. (GH#107) Aaron Spring

Updates CESM-LE uninitialized ensemble sample data to have 34 members. (GH#113) Riley X. Brady

Adds MPI-ESM hindcast, historical, and assimilation sample data. (GH#119) Aaron Spring

Replaces

`check_xarray`

with a decorator for checking that input arguments are xarray objects. (GH#120) Andrew HuangAdd custom exceptions for clearer error reporting. (GH#139) Riley X. Brady

Remove “xr” prefix from stats module. (GH#144) Riley X. Brady

Add codecoverage for testing. (GH#152) Riley X. Brady

Update exception messages for more pretty error reporting. (GH#156) Andrew Huang

Add

`pre-commit`

and`flake8`

/`black`

check in CI. (GH#163) Riley X. BradyChange

`loadutils`

module to`tutorial`

and`open_dataset`

to`load_dataset`

. (GH#164) Riley X. BradyRemove predictability horizon function to revisit for v2. (GH#165) Riley X. Brady

Increase code coverage through more testing. (GH#167) Aaron Spring

Consolidates checks and constants into modules. (GH#173) Andrew Huang

## climpred v0.2 (2019-01-11)#

Name changed to `climpred`

, developed enough for basic decadal prediction tasks on a
perfect-model ensemble and reference-based ensemble.

## climpred v0.1 (2018-12-20)#

Collaboration between Riley Brady and Aaron Spring begins.