# What’s New¶

## climpred v2.1.0 (2020-06-08)¶

### Breaking Changes¶

- Keyword
`bootstrap`

has been replaced with`iterations`

. We feel that this more accurately describes the argument, since “bootstrap” is really the process as a whole. (GH#354) Aaron Spring.

### New Features¶

`HindcastEnsemble`

and`PerfectModelEnsemble`

now use an HTML representation, following the more recent versions of`xarray`

. (GH#371) Aaron Spring.`HindcastEnsemble.verify()`

now takes`reference=...`

keyword. Current options are`'persistence'`

for a persistence forecast of the observations and`'historical'`

for some historical reference, such as an uninitialized/forced run. (GH#341) Riley X. Brady.We now only enforce a union of the initialization dates with observations if

`reference='persistence'`

for`HindcastEnsemble`

. This is to ensure that the same set of initializations is used by the observations to construct a persistence forecast. (GH#341) Riley X. Brady.`compute_perfect_model()`

now accepts initialization (`init`

) as`cftime`

and`int`

.`cftime`

is now implemented into the bootstrap uninitialized functions for the perfect model configuration. (GH#332) Aaron Spring.New explicit keywords in bootstrap functions for

`resampling_dim`

and`reference_compute`

(GH#320) Aaron Spring.Logging now included for

`compute_hindcast`

which displays the`inits`

and verification dates used at each lead (GH#324) Aaron Spring, (GH#338) Riley X. Brady. See (logging).New explicit keywords added for

`alignment`

of verification dates and initializations. (GH#324) Aaron Spring. See (alignment)`'maximize'`

: Maximize the degrees of freedom by slicing`hind`

and`verif`

to a common time frame at each lead. (GH#338) Riley X. Brady.`'same_inits'`

: slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (GH#328) Riley X. Brady.`'same_verifs'`

: slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (GH#331) Riley X. Brady.

### Performance¶

The major change for this release is a dramatic speedup in bootstrapping functions, led by
Aaron Spring. We focused on scalability with `dask`

and found many places we could compute
skill simultaneously over all bootstrapped ensemble members rather than at each iteration.

Bootstrapping uninitialized skill in the perfect model framework is now sped up significantly for annual lead resolution. (GH#332) Aaron Spring.

General speedup in

`bootstrap_hindcast()`

and`bootstrap_perfect_model()`

: (GH#285) Aaron Spring.- Properly implemented handling for lazy results when inputs are chunked.
- User gets warned when chunking potentially unnecessarily and/or inefficiently.

### Bug Fixes¶

- Alignment options now account for differences in the historical time series if
`reference='historical'`

. (GH#341) Riley X. Brady.

### Internals/Minor Fixes¶

- Added a Code of Conduct (GH#285) Aaron Spring.
- Gather all
`pytest.fixture``s in ``conftest.py`

. (GH#313) Aaron Spring. - Move
`x_METRICS`

and`COMPARISONS`

to`metrics.py`

and`comparisons.py`

in order to avoid circular import dependencies. (GH#315) Aaron Spring. `asv`

benchmarks added for`HindcastEnsemble`

(GH#285) Aaron Spring.- Ignore irrelevant warnings in
`pytest`

and mark slow tests (GH#333) Aaron Spring. - Default
`CONCAT_KWARGS`

now in all`xr.concat`

to speed up bootstrapping. (GH#330) Aaron Spring. - Remove
`member`

coords for`m2c`

comparison for probabilistic metrics. (GH#330) Aaron Spring. - Refactored
`compute_hindcast()`

and`compute_perfect_model()`

. (GH#330) Aaron Spring. - Changed lead0 coordinate modifications to be compliant with
`xarray=0.15.1`

in`compute_persistence()`

. (GH#348) Aaron Spring. - Exchanged
`my_quantile`

with`xr.quantile(skipna=False)`

. (GH#348) Aaron Spring. - Remove
`sig`

from`plot_bootstrapped_skill_over_leadyear()`

. (GH#351) Aaron Spring. - Require
`xskillscore v0.0.15`

and use their functions for effective sample size-based metrics. (:pr: 353) Riley X. Brady. - Faster bootstrapping without replacement used in threshold functions of
`climpred.stats`

(GH#354) Aaron Spring. - Require
`cftime v1.1.2`

, which modifies their object handling to create 200-400x speedups in some basic operations. (GH#356) Riley X. Brady. - Resample first and then calculate skill in
`bootstrap_perfect_model()`

and`bootstrap_hindcast()`

(GH#355) Aaron Spring.

### Documentation¶

- Added demo to setup your own raw model output compliant to
`climpred`

(GH#296) Aaron Spring. See (here). - Added demo using
`intake-esm`

with`climpred`

(GH#296) Aaron Spring. See (here). - Added Verification Alignment page explaining how initializations are selected and aligned with verification data. (GH#328) Riley X. Brady. See (here).

## climpred v2.0.0 (2020-01-22)¶

### New Features¶

- Add support for
`days`

,`pentads`

,`weeks`

,`months`

,`seasons`

for lead time resolution.`climpred`

now requires a`lead`

attribute “units” to decipher what resolution the predictions are at. (GH#294) Kathy Pegion and Riley X. Brady.

```
>>> hind = climpred.tutorial.load_dataset('CESM-DP-SST')
>>> hind.lead.attrs['units'] = 'years'
```

`HindcastEnsemble`

now has`.add_observations()`

and`.get_observations()`

methods. These are the same as`.add_reference()`

and`.get_reference()`

, which will be deprecated eventually. The name change clears up confusion, since “reference” is the appropriate name for a reference forecast, e.g. persistence. (GH#310) Riley X. Brady.`HindcastEnsemble`

now has`.verify()`

function, which duplicates the`.compute_metric()`

function. We feel that`.verify()`

is more clear and easy to write, and follows the terminology of the field. (GH#310) Riley X. Brady.`e2o`

and`m2o`

are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (GH#310) Riley X. Brady.

### Documentation¶

New example pages for subseasonal-to-seasonal prediction using

`climpred`

. (GH#294) Kathy PegionComparisons page rewritten for more clarity. (GH#310) Riley X. Brady.

### Bug Fixes¶

- Fixed m2m broken comparison issue and removed correction (GH#290) Aaron Spring.

### Internals/Minor Fixes¶

- Updates to
`xskillscore`

v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (GH#309) Riley X. Brady. - Stacking dims is handled by
`comparisons`

, no need for internal keyword`stack_dims`

. Therefore`comparison`

now takes`metric`

as argument instead. (GH#290) Aaron Spring. `assign_attrs`

now carries dim (GH#290) Aaron Spring.- “reference” changed to “verif” throughout hindcast compute functions. This is more clear, since “reference” usually refers to a type of forecast, such as persistence. (GH#310) Riley X. Brady.
`Comparison`

objects can now have aliases. (GH#310) Riley X. Brady.

## climpred v1.2.1 (2020-01-07)¶

### Depreciated¶

`mad`

no longer a keyword for the median absolute error metric. Users should now use`median_absolute_error`

, which is identical to changes in`xskillscore`

version 0.0.10. (GH#283) Riley X. Brady`pacc`

no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (GH#283) Riley X. Brady`msss`

no longer a keyword for the Murphy’s MSSS, since it is reserved for the standard MSSS. (GH#283) Riley X. Brady

### New Features¶

Metrics

`pearson_r_eff_p_value`

and`spearman_r_eff_p_value`

account for autocorrelation in computing p values. (GH#283) Riley X. BradyMetric

`effective_sample_size`

computes number of independent samples between two time series being correlated. (GH#283) Riley X. BradyAdded keywords for metrics: (GH#283) Riley X. Brady

`'pval'`

for`pearson_r_p_value`

`['n_eff', 'eff_n']`

for`effective_sample_size`

`['p_pval_eff', 'pvalue_eff', 'pval_eff']`

for`pearson_r_eff_p_value`

`['spvalue', 'spval']`

for`spearman_r_p_value`

`['s_pval_eff', 'spvalue_eff', 'spval_eff']`

for`spearman_r_eff_p_value`

`'nev'`

for`nmse`

### Internals/Minor Fixes¶

`climpred`

now requires`xarray`

version 0.14.1 so that the`drop_vars()`

keyword used in our package does not throw an error. (GH#276) Riley X. Brady- Update to
`xskillscore`

version 0.0.10 to fix errors in weighted metrics with pairwise NaNs. (GH#283) Riley X. Brady `doc8`

added to`pre-commit`

to have consistent formatting on`.rst`

files. (GH#283) Riley X. Brady- Remove
`proper`

attribute on`Metric`

class since it isn’t used anywhere. (GH#283) Riley X. Brady - Add testing for effective p values. (GH#283) Riley X. Brady
- Add testing for whether metric aliases are repeated/overwrite each other. (GH#283) Riley X. Brady
`ppp`

changed to`msess`

, but keywords allow for`ppp`

and`msss`

still. (GH#283) Riley X. Brady

### Documentation¶

- Expansion of metrics documentation with much more detail on how metrics are computed, their keywords, references, min/max/perfect scores, etc. (GH#283) Riley X. Brady
- Update terminology page with more information on metrics terminology. (GH#283) Riley X. Brady

## climpred v1.2.0 (2019-12-17)¶

### Depreciated¶

- Abbreviation
`pval`

depreciated. Use`p_pval`

for`pearson_r_p_value`

instead. (GH#264) Aaron Spring.

### New Features¶

Users can now pass a custom

`metric`

or`comparison`

to compute functions. (GH#268) Aaron Spring.New deterministic metrics (see metrics). (GH#264) Aaron Spring.

- Spearman ranked correlation (spearman_r)
- Spearman ranked correlation p-value (spearman_r_p_value)
- Mean Absolute Deviation (mad)
- Mean Absolute Percent Error (mape)
- Symmetric Mean Absolute Percent Error (smape)

Users can now apply arbitrary

`xarray`

methods to`HindcastEnsemble`

and`PerfectModelEnsemble`

. (GH#243) Riley X. Brady.Add “getter” methods to

`HindcastEnsemble`

and`PerfectModelEnsemble`

to retrieve`xarray`

datasets from the objects. (GH#243) Riley X. Brady.>>> hind = climpred.tutorial.load_dataset('CESM-DP-SST') >>> ref = climpred.tutorial.load_dataset('ERSST') >>> hindcast = climpred.HindcastEnsemble(hind) >>> hindcast = hindcast.add_reference(ref, 'ERSST') >>> print(hindcast) <climpred.HindcastEnsemble> Initialized Ensemble: SST (init, lead, member) float64 ... ERSST: SST (time) float32 ... Uninitialized: None >>> print(hindcast.get_initialized()) <xarray.Dataset> Dimensions: (init: 64, lead: 10, member: 10) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * member (member) int32 1 2 3 4 5 6 7 8 9 10 * init (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0 Data variables: SST (init, lead, member) float64 ... >>> print(hindcast.get_reference('ERSST')) <xarray.Dataset> Dimensions: (time: 61) Coordinates: * time (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015 Data variables: SST (time) float32 ...

`metric_kwargs`

can be passed to`Metric`

. (GH#264) Aaron Spring.- See
`metric_kwargs`

under metrics.

- See

### Bug Fixes¶

`compute_metric()`

doesn’t drop coordinates from the initialized hindcast ensemble anymore. (GH#258) Aaron Spring.- Metric
`uacc`

does not crash when`ppp`

negative anymore. (GH#264) Aaron Spring. - Update
`xskillscore`

to version 0.0.9 to fix all-NaN issue with`pearson_r`

and`pearson_r_p_value`

when there’s missing data. (GH#269) Riley X. Brady.

### Internals/Minor Fixes¶

Rewrote

`varweighted_mean_period()`

based on`xrft`

. Changed`time_dim`

to`dim`

. Function no longer drops coordinates. (GH#258) Aaron SpringAdd

`dim='time'`

in`dpp()`

. (GH#258) Aaron SpringComparisons

`m2m`

,`m2e`

rewritten to not stack dims into supervector because this is now done in`xskillscore`

. (GH#264) Aaron SpringAdd

`tqdm`

progress bar to`bootstrap_compute()`

. (GH#244) Aaron SpringRemove inplace behavior for

`HindcastEnsemble`

and`PerfectModelEnsemble`

. (GH#243) Riley X. BradyAdded tests for chunking with

`dask`

. (GH#258) Aaron SpringFix test issues with esmpy 8.0 by forcing esmpy 7.1 (GH#269). Riley X. Brady

Rewrote

`metrics`

and`comparisons`

as classes to accomodate custom metrics and comparisons. (GH#268) Aaron Spring

### Documentation¶

- Add examples notebook for temporal and spatial smoothing. (GH#244) Aaron Spring
- Add documentation for computing a metric over a specified dimension. (GH#244) Aaron Spring
- Update API to be more organized with individual function/class pages. (GH#243) Riley X. Brady.
- Add page describing the
`HindcastEnsemble`

and`PerfectModelEnsemble`

objects more clearly. (GH#243) Riley X. Brady - Add page for publications and helpful links. (GH#270) Riley X. Brady.

## climpred v1.1.0 (2019-09-23)¶

### Features¶

- Write information about skill computation to netcdf attributes(GH#213) Aaron Spring
- Temporal and spatial smoothing module (GH#224) Aaron Spring
- Add metrics brier_score, threshold_brier_score and crpss_es (GH#232) Aaron Spring
- Allow compute_hindcast and compute_perfect_model to specify which dimension dim to calculate metric over (GH#232) Aaron Spring

### Bug Fixes¶

- Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (GH#232) Aaron Spring

### Internals/Minor Fixes¶

- Rename .stats.DPP to dpp (GH#232) Aaron Spring
- Add matplotlib as a main dependency so that a direct pip installation works (GH#211) Riley X. Brady.
`climpred`

is now installable from conda-forge (GH#212) Riley X. Brady.- Fix erroneous descriptions of sample datasets (GH#226) Riley X. Brady.
- Benchmarking time and peak memory of compute functions with asv (GH#231) Aaron Spring

### Documentation¶

- Add scope of package to docs for clarity for users and developers. (GH#235) Riley X. Brady.

## climpred v1.0.1 (2019-07-04)¶

### Bug Fixes¶

- Accomodate for lead-zero within the
`lead`

dimension (GH#196) Riley X. Brady. - Fix issue with adding uninitialized ensemble to
`HindcastEnsemble`

object (GH#199) Riley X. Brady. - Allow
`max_dof`

keyword to be passed to`compute_metric`

and`compute_persistence`

for`HindcastEnsemble`

(GH#199) Riley X. Brady.

### Internals/Minor Fixes¶

- Force
`xskillscore`

version 0.0.4 or higher to avoid`ImportError`

(GH#204) Riley X. Brady. - Change
`max_dfs`

keyword to`max_dof`

(GH#199) Riley X. Brady. - Add testing for
`HindcastEnsemble`

and`PerfectModelEnsemble`

(GH#199) Riley X. Brady

## climpred v1.0.0 (2019-07-03)¶

`climpred`

v1.0.0 represents the first stable release of the package. It includes
`HindcastEnsemble`

and `PerfectModelEnsemble`

objects to perform analysis with.
It offers a suite of deterministic and probabilistic metrics that are optimized to be
run on single time series or grids of data (e.g., lat, lon, and depth). Currently,
`climpred`

only supports annual forecasts.

### Features¶

- Bootstrap prediction skill based on resampling with replacement consistently in
`ReferenceEnsemble`

and`PerfectModelEnsemble`

. (GH#128) Aaron Spring - Consistent bootstrap function for
`climpred.stats`

functions via`bootstrap_func`

wrapper. (GH#167) Aaron Spring - many more metrics:
`_msss_murphy`

,`_less`

and probabilistic`_crps`

,`_crpss`

(GH#128) Aaron Spring

### Bug Fixes¶

`compute_uninitialized`

now trims input data to the same time window. (GH#193) Riley X. Brady`rm_poly`

now properly interpolates/fills NaNs. (GH#192) Riley X. Brady

### Internals/Minor Fixes¶

- The
`climpred`

version can be printed. (GH#195) Riley X. Brady - Constants are made elegant and pushed to a separate module. (GH#184) Andrew Huang
- Checks are consolidated to their own module. (GH#173) Andrew Huang

### Documentation¶

- Documentation built extensively in multiple PRs.

## climpred v0.3 (2019-04-27)¶

`climpred`

v0.3 really represents the entire development phase leading up to the
version 1 release. This was done in collaboration between Riley X. Brady,
Aaron Spring, and Andrew Huang. Future releases will have less additions.

### Features¶

Introduces object-oriented system to

`climpred`

, with classes`ReferenceEnsemble`

and`PerfectModelEnsemble`

. (GH#86) Riley X. BradyExpands bootstrapping module for perfect-module configurations. (GH#78, GH#87) Aaron Spring

Adds functions for computing Relative Entropy (GH#73) Aaron Spring

Sets more intelligible dimension expectations for

`climpred`

(GH#98, GH#105) Riley X. Brady and Aaron Spring:`init`

: initialization dates for the prediction ensemble`lead`

: retrospective forecasts from prediction ensemble; returned dimension for prediction calculations`time`

: time dimension for control runs, references, etc.`member`

: ensemble member dimension.

Updates

`open_dataset`

to display available dataset names when no argument is passed. (GH#123) Riley X. BradyChange

`ReferenceEnsemble`

to`HindcastEnsemble`

. (GH#124) Riley X. BradyAdd probabilistic metrics to

`climpred`

. (GH#128) Aaron SpringConsolidate separate perfect-model and hindcast functions into singular functions (GH#128) Aaron Spring

Add option to pass proxy through to

`open_dataset`

for firewalled networks. (GH#138) Riley X. Brady

### Bug Fixes¶

`xr_rm_poly`

can now operate on Datasets and with multiple variables. It also interpolates across NaNs in time series. (GH#94) Andrew Huang- Travis CI,
`treon`

, and`pytest`

all run for automated testing of new features. (GH#98, GH#105, GH#106) Riley X. Brady and Aaron Spring - Clean up
`check_xarray`

decorators and make sure that they work. (GH#142) Andrew Huang - Ensures that
`help()`

returns proper docstring even with decorators. (GH#149) Andrew Huang - Fixes bootstrap so p values are correct. (GH#170) Aaron Spring

### Internals/Minor Fixes¶

- Adds unit testing for all perfect-model comparisons. (GH#107) Aaron Spring
- Updates CESM-LE uninitialized ensemble sample data to have 34 members. (GH#113) Riley X. Brady
- Adds MPI-ESM hindcast, historical, and assimilation sample data. (GH#119) Aaron Spring
- Replaces
`check_xarray`

with a decorator for checking that input arguments are xarray objects. (GH#120) Andrew Huang - Add custom exceptions for clearer error reporting. (GH#139) Riley X. Brady
- Remove “xr” prefix from stats module. (GH#144) Riley X. Brady
- Add codecoverage for testing. (GH#152) Riley X. Brady
- Update exception messages for more pretty error reporting. (GH#156) Andrew Huang
- Add
`pre-commit`

and`flake8`

/`black`

check in CI. (GH#163) Riley X. Brady - Change
`loadutils`

module to`tutorial`

and`open_dataset`

to`load_dataset`

. (GH#164) Riley X. Brady - Remove predictability horizon function to revisit for v2. (GH#165) Riley X. Brady
- Increase code coverage through more testing. (GH#167) Aaron Spring
- Consolidates checks and constants into modules. (GH#173) Andrew Huang

## climpred v0.2 (2019-01-11)¶

Name changed to `climpred`

, developed enough for basic decadal prediction tasks on a
perfect-model ensemble and reference-based ensemble.

## climpred v0.1 (2018-12-20)¶

Collaboration between Riley Brady and Aaron Spring begins.