What’s New#

climpred v2.3.0 (2021-11-25)#

Note

As both maintainers moved out of academia into industry, this will be probably the last release for a while. If you are interested in maintaining climpred, please ping us.

Bug Fixes#

Fix reference="persistence" for resampled init. (GH#730, GH#731) Aaron Spring.
HindcastEnsemble.verify() (comparison="m2o", reference="uninitialized", dim="init"). (GH#735, GH#731) Aaron Spring.
HindcastEnsemble.remove_bias() does not drop single item lead dimension. (GH#771, GH#773) Aaron Spring.

New Features#

Refactored HindcastEnsemble.bootstrap() and PerfectModelEnsemble.bootstrap() based on HindcastEnsemble.verify() and PerfectModelEnsemble.verify(), which makes them more comparable. pers_sig is removed. Also reference=["climatology", "persistence"] skill has variance if resample_dim='init'. bootstrap relies on either set_option(resample_skill_func="..."):
- "loop": calls climpred.bootstrap.resample_skill_loop() which loops over iterations and calls verify every single time. Most understandable and stable, but slow.
- "exclude_resample_dim_from_dim": calls climpred.bootstrap.resample_skill_exclude_resample_dim_from_dim() which calls verify(dim=dim_without_resample_dim), resamples over resample_dim and then takes a mean over resample_dim if in dim. Enables HindcastEnsemble.bootstrap(resample_dim="init", alignment="same_verifs"). Fast alternative for resample_dim="init".
- "resample_before": calls climpred.bootstrap.resample_skill_resample_before() which resamples iteration dimension and then calls verify vectorized. Fast alternative for resample_dim="member".
- "default": climpred decides which to use
(relates to GH#375, GH#731) Aaron Spring.
climpred.set_option(resample_skill_func='exclude_resample_dim_from_dim') allows HindcastEnsemble.bootstrap(alignment='same_verifs', resample_dim='init'). Does not work for pearson_r-derived metrics. (GH#582, GH#731) Aaron Spring.
climpred.utils.convert_init_lead_to_valid_time_lead() converts data(init, lead) to data(valid_time, lead) to visualize predictability barrier and the reverse climpred.utils.convert_valid_time_lead_to_init_lead(). (GH#774, GH#775, GH#783) Aaron Spring.

Internals/Minor Fixes#

Refactor asv benchmarking. Add run-benchmarks label to PR to run asv via Github Actions. (GH#664, GH#718) Aaron Spring.
Remove ipython from requirements.txt. (GH#720) Aaron Spring.
Calculating np.isin on asi8 instead of xr.CFTimeIndex speeds up HindcastEnsemble.verify() and HindcastEnsemble.bootstrap() with large number of inits. (GH#414, GH#724) Aaron Spring.
Add option bootstrap_resample_skill_func for they what skill is resampled in HindcastEnsemble.bootstrap() and PerfectModelEnsemble.bootstrap(), see set_options. (GH#731) Aaron Spring.
Add option resample_iterations_func to decide whether xskillscore.resampling.resample_iterations() or xskillscore.resampling.resample_iterations() should be used, see set_options. (GH#731) Aaron Spring. - Add option bootstrap_uninitialized_from_iterations_mean to exchange uninitialized skill with the iteration mean uninitialized. Defaults to False., see set_options. (GH#731) Aaron Spring.
alignment="same_verifs" will not result in NaN``s in ``valid_time. (GH#777) Aaron Spring.
HindcastEnsemble.plot_alignment() (return_xr=True) contains valid_time coordinate. (GH#779) Aaron Spring.

Bug Fixes#

Fix PerfectModel_persistence_from_initialized_lead_0=True with multiple references. (GH#732, GH#733) Aaron Spring.

Documentation#

Add verify dim example showing how HindcastEnsemble.verify() and PerfectModelEnsemble.verify() are sensitive to dim and how dim answers different research questions. (GH#740) Aaron Spring.

climpred v2.2.0 (2021-12-20)#

Bug Fixes#

Fix when creating valid_time from lead.attrs["units"] in ["seasons", "years"] with multi-month stride in init. (GH#698, GH#700) Aaron Spring.
Fix seasonality="season" in reference="climatology". (GH#641, GH#703) Aaron Spring.

New Features#

Upon instantiation, PredictionEnsemble generates new 2-dimensional coordinate valid_time for initialized from init and lead, which is matched with time from verification during alignment. (GH#575, GH#675, GH#678) Aaron Spring.

>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST")
>>> hind.lead.attrs["units"] = "years"
>>> climpred.HindcastEnsemble(hind).get_initialized()
<xarray.Dataset>
Dimensions:     (lead: 10, member: 10, init: 64)
Coordinates:
  * lead        (lead) int32 1 2 3 4 5 6 7 8 9 10
  * member      (member) int32 1 2 3 4 5 6 7 8 9 10
  * init        (init) object 1954-01-01 00:00:00 ... 2017-01-01 00:00:00
    valid_time  (lead, init) object 1955-01-01 00:00:00 ... 2027-01-01 00:00:00
Data variables:
    SST         (init, lead, member) float64 ...

Allow lead as float also if calendar="360_day" or lead.attrs["units"] not in ["years","seasons","months"]. (GH#564, GH#675) Aaron Spring.
Implement HindcastEnsemble.generate_uninitialized() resampling years without replacement from initialized. (GH#589, GH#591) Aaron Spring.
Implement Logarithmic Ensemble Skill Score _less(). (GH#239, GH#687) Aaron Spring.
HindcastEnsemble.remove_seasonality() and PerfectModelEnsemble.remove_seasonality() remove the seasonality of all climpred datasets. (GH#530, GH#688) Aaron Spring.
Add keyword groupby in HindcastEnsemble.verify(), PerfectModelEnsemble.verify(), HindcastEnsemble.bootstrap() and PerfectModelEnsemble.bootstrap() to group skill by initializations seasonality. (GH#635, GH#690) Aaron Spring.

>>> import climpred
>>> hind = climpred.tutorial.load_dataset("NMME_hindcast_Nino34_sst")
>>> obs = climpred.tutorial.load_dataset("NMME_OIv2_Nino34_sst")
>>> hindcast = climpred.HindcastEnsemble(hind).add_observations(obs)
>>> # skill for each init month separated
>>> skill = hindcast.verify(
...     metric="rmse",
...     dim="init",
...     comparison="e2o",
...     skipna=True,
...     alignment="maximize",
...     groupby="month",
... )
>>> skill
<xarray.Dataset>
Dimensions:  (month: 12, lead: 12, model: 12)
Coordinates:
  * lead     (lead) float64 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 10.0 11.0
  * model    (model) object 'NCEP-CFSv2' 'NCEP-CFSv1' ... 'GEM-NEMO'
    skill    <U11 'initialized'
  * month    (month) int64 1 2 3 4 5 6 7 8 9 10 11 12
Data variables:
    sst      (month, lead, model) float64 0.4127 0.3837 0.3915 ... 1.255 3.98
>>> skill.sst.plot(hue="model", col="month", col_wrap=3)

HindcastEnsemble.plot_alignment() shows how forecast and observations are aligned based on the alignment keyword. This may help understanding which dates are matched for the different alignment approaches. (GH#701, GH#702) Aaron Spring.

In [1]: from climpred.tutorial import load_dataset

In [2]: hindcast = climpred.HindcastEnsemble(
   ...:     load_dataset("CESM-DP-SST")
   ...: ).add_observations(load_dataset("ERSST"))
   ...: 

In [3]: hindcast.plot_alignment(edgecolor="w")
Out[3]: <xarray.plot.facetgrid.FacetGrid at 0x7f97810fc5b0>

Add attrs to new coordinates created by climpred. (GH#695, GH#697) Aaron Spring.
Add seasonality="weekofyear" in reference="climatology". (GH#703) Aaron Spring.
Compute reference="persistence" in PerfectModelEnsemble from initialized first lead if set_options (PerfectModel_persistence_from_initialized_lead_0=True) (False by default) using compute_persistence_from_first_lead(). (GH#637, GH#706) Aaron Spring.

Internals/Minor Fixes#

Reduce dependencies. (GH#686) Aaron Spring.
Add typing. (GH#685, GH#692) Aaron Spring.
refactor add_attrs into HindcastEnsemble.verify() and HindcastEnsemble.bootstrap(). Now all keywords are captured in the skill dataset attributes .attrs. (GH#475, GH#694) Aaron Spring.
docstrings formatting with blackdocs. (GH#708) Aaron Spring.

Documentation#

Refresh all docs with sphinx_book_theme and myst_nb. (GH#707, GH#708, GH#709, GH#710) Aaron Spring.

climpred v2.1.6 (2021-08-31)#

Adding on to v2.1.5, more bias reduction methods wrapped from xclim are implemented.

Bug Fixes#

Fix results="p" in HindcastEnsemble.bootstrap() and PerfectModelEnsemble.bootstrap() when reference='climatology'. (GH#668, GH#670) Aaron Spring.
HindcastEnsemble.remove_bias() for how in ["modified_quantile", "basic_quantile", "gamma_mapping", "normal_mapping"] from bias_correction takes all member to create model distribution. (GH#667) Aaron Spring.

New Features#

allow more bias reduction methods wrapped from xclim in HindcastEnsemble.remove_bias():
- how="EmpiricalQuantileMapping": xclim.sdba.adjustment.EmpiricalQuantileMapping
- how="DetrendedQuantileMapping": xclim.sdba.adjustment.DetrendedQuantileMapping
- how="PrincipalComponents": xclim.sdba.adjustment.PrincipalComponents
- how="QuantileDeltaMapping": xclim.sdba.adjustment.QuantileDeltaMapping
- how="Scaling": xclim.sdba.adjustment.Scaling
- how="LOCI": xclim.sdba.adjustment.LOCI
These methods do not respond to OPTIONS['seasonality'] like the other methods. Provide group="init.month" to group by month or group='init' to skip grouping. Provide group=None or skip group to use init.{OPTIONS['seasonality']}. (GH#525, GH#662, GH#666, GH#671) Aaron Spring.

climpred v2.1.5 (2021-08-12)#

While climpred has used in the ASP summer colloquium 2021, many new features in HindcastEnsemble.remove_bias() were implemented.

Breaking changes#

renamed cross_validate to cv=False in HindcastEnsemble.remove_bias(). Only used when train_test_split='unfair-cv'. (GH#648, GH#655). Aaron Spring.

Bug Fixes#

Shift back init by lead after HindcastEnsemble.verify(). (GH#644, GH#645) Aaron Spring.

New Features#

HindcastEnsemble.remove_bias() accepts new keyword train_test_split='fair/unfair/unfair-cv' (default unfair) following Risbey et al. 2021. (GH#648, GH#655) Aaron Spring.
allow more bias reduction methods in HindcastEnsemble.remove_bias():
- how="additive_mean": correcting the mean forecast additively (already implemented)
- how="multiplicative_mean": correcting the mean forecast multiplicatively
- how="multiplicative_std": correcting the standard deviation multiplicatively
Wrapped from bias_correction:
- how="modified_quantile": Bai et al. 2016
- how="basic_quantile": Themeßl et al. 2011
- how="gamma_mapping" and how="normal_mapping": Switanek et al. 2017
HindcastEnsemble.remove_bias() now does leave-one-out cross validation when passing cv='LOO' and train_test_split='unfair-cv'. cv=True falls back to cv='LOO'. (GH#643, GH#646) Aaron Spring.
Add new metrics _spread() and _mul_bias() (GH#638) Aaron Spring.
Add new tutorial datasets: (GH#651) Aaron Spring.
- NMME_OIv2_Nino34_sst and NMME_hindcast_Nino34_sst with monthly leads
- Observations_Germany and ECMWF_S2S_Germany with daily leads
Metadata from CF convenctions are automatically attached by cf_xarray. (GH#639, GH#656) Aaron Spring.
Raise warning when dimensions time, init or member are chunked to show user how to circumvent xskillscore chunking ValueError when passing these dimensions as dim in HindcastEnsemble.verify() or HindcastEnsemble.bootstrap(). (GH#509, GH#658) Aaron Spring.
Implement PredictionEnsemble.chunks. (GH#658) Aaron Spring.

Documentation#

Speed up ENSO monthly example with IRIDL server-side preprocessing (see context) (GH#594, GH#633) Aaron Spring.
Add CITATION.cff. Please cite Brady and Spring, 2020. (GH) Aaron Spring.
Use NMME_OIv2_Nino34_sst and NMME_hindcast_Nino34_sst with monthly leads for bias reduction demonstrating HindcastEnsemble.remove_bias(). (GH#646) Aaron Spring.

climpred v2.1.4 (2021-06-28)#

New Features#

Allow hours, minutes and seconds as lead.attrs['units']. (GH#404, GH#603) Aaron Spring.
Allow to set seasonality via set_options to specify how to group in verify(reference='climatology' or in HindcastEnsemble.remove_bias(). (GH#529, GH#593, GH#603) Aaron Spring.
Allow weekofyear via datetime in HindcastEnsemble.remove_bias(), but not yet implemented in verify(reference='climatology'). (GH#529, GH#603) Aaron Spring.
Allow more dimensions in initialized than in observations. This is particular useful if you have forecasts from multiple models (in a model dimension) and want to verify against the same observations. (GH#129, GH#528, GH#619) Aaron Spring.
Automatically rename dimensions to CLIMPRED_ENSEMBLE_DIMS ["init", "member", "lead"] if CF standard_names in coordinate attributes match: (GH#613, GH#622) Aaron Spring.
- "init": "forecast_reference_time"
- "member": "realization"
- "lead": "forecast_period"
If lead coordinate is pd.Timedelta, PredictionEnsemble converts lead coordinate upon instantiation to integer lead and corresponding lead.attrs["units"]. (GH#606, GH#627) Aaron Spring.
Require xskillscore >= 0.0.20. _rps() now works with different category_edges for observations and forecasts, see daily ECMWF example. (GH#629, GH#630) Aaron Spring.
Set options warn_for_failed_PredictionEnsemble_xr_call, warn_for_rename_to_climpred_dims, warn_for_init_coords_int_to_annual, climpred_warnings via set_options. (GH#628, GH#631) Aaron Spring.
PredictionEnsemble acts like xarray.Dataset and understands data_vars, dims, sizes, coords, nbytes, equals, identical, __iter__, __len__, __contains__, __delitem__. (GH#568, GH#632) Aaron Spring.

Documentation#

Add documentation page about publicly available initialized datasets and corresponding `climpred examples <initialized-datasets.html>`_. (GH#510, GH#561, GH#600) Aaron Spring.
Add GEFS example for numerical weather prediction. (GH#602, GH#603) Aaron Spring.
Add subseasonal daily ECMWF example using climetlab to access hindcasts from ECMWF cloud. (GH#587, GH#603) Aaron Spring.
Add subseasonal daily S2S example accessing S2S output on IRIDL with a cookie and working with “on-the-fly” reforecasts with hdate dimension. (GH#588, GH#593) Aaron Spring.
Added example climpred on GPU. Running PerfectModelEnsemble.verify() on GPU with cupy-xarray finishes 10x faster. (GH#592, GH#607) Aaron Spring.
How to work with biweekly aggregates in climpred, see daily ECMWF example. (GH#625, GH#630) Aaron Spring.

Internals/Minor Fixes#

Add weekly upstream CI, which raises issues for failures. Adapted from xarray. Manually trigger by git commit -m '[test-upstream]'. Skip climpred_testing CI by git commit -m '[skip-ci]' (GH#518, GH#596) Aaron Spring.

climpred v2.1.3 (2021-03-23)#

Breaking changes#

New Features#

HindcastEnsemble.verify(), PerfectModelEnsemble.verify(), HindcastEnsemble.bootstrap() and PerfectModelEnsemble.bootstrap() accept reference climatology. Furthermore, reference persistence also allows probabilistic metrics (GH#202, GH#565, GH#566) Aaron Spring.
Added new metric _roc Receiver Operating Characteristic as metric='roc'. (GH#566) Aaron Spring.

Bug fixes#

HindcastEnsemble.verify() and HindcastEnsemble.bootstrap() accept dim as list, set, tuple or str (GH#519, GH#558) Aaron Spring.
PredictionEnsemble.map() now does not fail silently when applying a function to all xr.Datasets of PredictionEnsemble. Instead, UserWarning``s are raised. Furthermore, ``PredictionEnsemble.map(func, *args, **kwargs) applies only function to Datasets with matching dims if dim="dim0_or_dim1" is passed as **kwargs. (GH#417, GH#437, GH#552) Aaron Spring.
_rpc was fixed in xskillscore>=0.0.19 and hence is not falsely limited to 1 anymore (GH#562, GH#566) Aaron Spring.

Internals/Minor Fixes#

Docstrings are now tested in GitHub actions continuous integration. (GH#545, GH#560) Aaron Spring.
Github actions now cancels previous commits, instead of running the full testing suite on every single commit. (GH#560) Aaron Spring.
PerfectModelEnsemble.verify() does not add climpred attributes to skill by default anymore. (GH#560) Aaron Spring.
Drop python==3.6 support. (GH#573) Aaron Spring.
Notebooks are now linted with nb_black using %load_ext nb_black or %load_ext lab_black for Jupyter notebooks and Jupyter lab. (GH#526, GH#572) Aaron Spring.
Reduce dependencies to install climpred. (GH#454, GH#572) Aaron Spring.
Examples from documentation available via Binder. Find further examples in the examples folder. (GH#549, GH#578) Aaron Spring.
Rename branch master to main. (GH#579) Aaron Spring.

climpred v2.1.2 (2021-01-22)#

This release is the fixed version for our Journal of Open Source Software (JOSS) article about climpred, see review.

New Features#

Function to calculate predictability horizon predictability_horizon() based on condition. (GH#46, GH#521) Aaron Spring.

Bug fixes#

PredictionEnsemble.smooth() now carries lead.attrs (GH#527, pr:521) Aaron Spring.
PerfectModelEnsemble.verify() now works with references also for geospatial inputs, which returned NaN before. (GH#522, pr:521) Aaron Spring.
PredictionEnsemble.plot() now shifts composite lead frequencies like days, pentads, seasons correctly. (GH#532, GH#533) Aaron Spring.
Adapt to xesmf>=0.5.2 for spatial xesmf smoothing. (GH#543, GH#548) Aaron Spring.
HindcastEnsemble.remove_bias() now carries attributes. (GH#531, GH#551) Aaron Spring.

climpred v2.1.1 (2020-10-13)#

Breaking changes#

This version introduces a lot of breaking changes. We are trying to overhaul climpred to have an intuitive API that also forces users to think about methodology choices when running functions. The main breaking changes we introduced are for HindcastEnsemble.verify() and PerfectModelEnsemble.verify(). Now, instead of assuming defaults for most keywords, we require the user to define metric, comparison, dim, and alignment (for hindcast systems). We also require users to designate the number of iterations for bootstrapping.

User now has to designate number of iterations with iterations=... in HindcastEnsemble.bootstrap() (GH#384, GH#436) Aaron Spring and Riley X. Brady.
Make metric, comparison, dim, and alignment required (previous default None) arguments for HindcastEnsemble.verify() (GH#384, GH#436) Aaron Spring and Riley X. Brady.
Metric _brier_score and _threshold_brier_score() now requires callable keyword argument logical instead of func (GH#388) Aaron Spring.
HindcastEnsemble.verify() does not correct dim automatically to member for probabilistic metrics. (GH#282, GH#407) Aaron Spring.
Users can no longer add multiple observations to HindcastEnsemble. This will make current and future development much easier on maintainers (GH#429, GH#453) Riley X. Brady.
Standardize the names of the output coordinates for PredictionEnsemble.verify() and PredictionEnsemble.bootstrap() to initialized, uninitialized, and persistence. initialized showcases the metric result after comparing the initialized ensemble to the verification data; uninitialized when comparing the uninitialized (historical) ensemble to the verification data; persistence is the evaluation of the persistence forecast (GH#460, GH#478, GH#476, GH#480) Aaron Spring.
reference keyword in HindcastEnsemble.verify() should be choosen from [uninitialized, persistence]. historical no longer works. (GH#460, GH#478, GH#476, GH#480) Aaron Spring.
HindcastEnsemble.verify() returns no skill dimension if reference=None (GH#480) Aaron Spring.
comparison is not applied to uninitialized skill in HindcastEnsemble.bootstrap(). (GH#352, GH#418) Aaron Spring.

New Features#

This release is accompanied by a bunch of new features. Math operations can now be used with our PredictionEnsemble objects and their variables can be sub-selected. Users can now quick plot time series forecasts with these objects. Bootstrapping is available for HindcastEnsemble. Spatial dimensions can be passed to metrics to do things like pattern correlation. New metrics have been implemented based on Contingency tables. We now include an early version of bias removal for HindcastEnsemble.

Use math operations like +-*/ with HindcastEnsemble and PerfectModelEnsemble. See demo Arithmetic-Operations-with-PredictionEnsemble-Objects. (GH#377) Aaron Spring.
Subselect data variables from PerfectModelEnsemble as from xarray.Dataset: PredictionEnsemble[["var1", "var3"]] (GH#409) Aaron Spring.
Plot all datasets in HindcastEnsemble or PerfectModelEnsemble by PredictionEnsemble.plot() if no other spatial dimensions are present. (GH#383) Aaron Spring.
Bootstrapping now available for HindcastEnsemble as HindcastEnsemble.bootstrap(), which is analogous to the PerfectModelEnsemble method. (GH#257, GH#418) Aaron Spring.
HindcastEnsemble.verify() allows all dimensions from initialized ensemble as dim. This allows e.g. spatial dimensions to be used for pattern correlation. Make sure to use skipna=True when using spatial dimensions and output has NaNs (in the case of land, for instance). (GH#282, GH#407) Aaron Spring.
Allow binary forecasts at when calling HindcastEnsemble.verify(), rather than needing to supply binary results beforehand. In other words, hindcast.verify(metric='bs', comparison='m2o', dim='member', logical=logical) is now the same as hindcast.map(logical).verify(metric='brier_score', comparison='m2o', dim='member'. (GH#431) Aaron Spring.
Check calendar types when using HindcastEnsemble.add_observations(), HindcastEnsemble.add_uninitialized(), PerfectModelEnsemble.add_control() to ensure that the verification data calendars match that of the initialized ensemble. (GH#300, GH#452, GH#422, GH#462) Riley X. Brady and Aaron Spring.
Implement new metrics which have been ported over from https://github.com/csiro-dcfp/doppyo/ to xskillscore by Dougie Squire. (GH#439, GH#456) Aaron Spring
- rank histogram _rank_histogram()
- discrimination _discrimination()
- reliability _reliability()
- ranked probability score _rps()
- contingency table and related scores _contingency()
Perfect Model PerfectModelEnsemble.verify() no longer requires control in PerfectModelEnsemble. It is only required when reference=['persistence']. (GH#461) Aaron Spring.
Implemented bias removal remove_bias. remove_bias(how='mean') removes the mean bias of initialized hindcasts with respect to observations. See example. (GH#389, GH#443, GH#459) Aaron Spring and Riley X. Brady.

Deprecated#

spatial_smoothing_xrcoarsen no longer used for spatial smoothing. (GH#391) Aaron Spring.
compute_metric, compute_uninitialized and compute_persistence no longer in use for PerfectModelEnsemble in favor of PerfectModelEnsemble.verify() with the reference keyword instead. (GH#436, GH#468, GH#472) Aaron Spring and Riley X. Brady.
'historical' no longer a valid choice for reference. Use 'uninitialized' instead. (GH#478) Aaron Spring.

Bug Fixes#

PredictionEnsemble.verify() and PredictionEnsemble.bootstrap() now accept metric_kwargs. (GH#387) Aaron Spring.
PerfectModelEnsemble.verify() now accepts 'uninitialized' as a reference. (GH#395) Riley X. Brady.
Spatial and temporal smoothing PredictionEnsemble.smooth() now work as expected and rename time dimensions after verify(). (GH#391) Aaron Spring.
PredictionEnsemble.verify(comparison='m2o', references=['uninitialized', 'persistence'] does not fail anymore. (GH#385, GH#400) Aaron Spring.
Remove bias using dayofyear in HindcastEnsemble.reduce_bias(). (GH#443) Aaron Spring.
climpred works with dask=>2.28. (GH#479, GH#482) Aaron Spring.

Documentation#

Updates climpred tagline to “Verification of weather and climate forecasts.” (GH#420) Riley X. Brady.
Adds section on how to use arithmetic with HindcastEnsemble. (GH#378) Riley X. Brady.
Add docs section for similar open-source forecasting packages. (GH#432) Riley X. Brady.
Add all metrics to main API in addition to metrics page. (GH#438) Riley X. Brady.
Add page on bias removal Aaron Spring.

Internals/Minor Fixes#

PredictionEnsemble.verify() replaces deprecated PerfectModelEnsemble.compute_metric() and accepts reference as keyword. (GH#387) Aaron Spring.
Cleared out unnecessary statistics functions from climpred and migrated them to esmtools. Add esmtools as a required package. (GH#395) Riley X. Brady.
Remove fixed pandas dependency from pandas=0.25 to stable pandas. (GH#402, GH#403) Aaron Spring.
dim is expected to be a list of strings in compute_perfect_model() and compute_hindcast(). (GH#282, GH#407) Aaron Spring.
Update cartopy requirement to 0.0.18 or greater to release lock on matplotlib version. Update xskillscore requirement to 0.0.18 to cooperate with new xarray version. (GH#451, GH#449) Riley X. Brady
Switch from Travis CI and Coveralls to Github Actions and CodeCov. (GH#471) Riley X. Brady
Assertion functions added for PerfectModelEnsemble: assert_PredictionEnsemble(). (GH#391) Aaron Spring.
Test all metrics against synthetic data. (GH#388) Aaron Spring.

climpred v2.1.0 (2020-06-08)#

Breaking Changes#

Keyword bootstrap has been replaced with iterations. We feel that this more accurately describes the argument, since “bootstrap” is really the process as a whole. (GH#354) Aaron Spring.

New Features#

HindcastEnsemble and PerfectModelEnsemble now use an HTML representation, following the more recent versions of xarray. (GH#371) Aaron Spring.
HindcastEnsemble.verify() now takes reference=... keyword. Current options are 'persistence' for a persistence forecast of the observations and 'uninitialized' for an uninitialized/historical reference, such as an uninitialized/forced run. (GH#341) Riley X. Brady.
We now only enforce a union of the initialization dates with observations if reference='persistence' for HindcastEnsemble. This is to ensure that the same set of initializations is used by the observations to construct a persistence forecast. (GH#341) Riley X. Brady.
compute_perfect_model() now accepts initialization (init) as cftime and int. cftime is now implemented into the bootstrap uninitialized functions for the perfect model configuration. (GH#332) Aaron Spring.
New explicit keywords in bootstrap functions for resampling_dim and reference_compute (GH#320) Aaron Spring.
Logging now included for compute_hindcast which displays the inits and verification dates used at each lead (GH#324) Aaron Spring, (GH#338) Riley X. Brady. See (logging).
New explicit keywords added for alignment of verification dates and initializations. (GH#324) Aaron Spring. See (alignment)
- 'maximize': Maximize the degrees of freedom by slicing hind and verif to a common time frame at each lead. (GH#338) Riley X. Brady.
- 'same_inits': slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (GH#328) Riley X. Brady.
- 'same_verifs': slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (GH#331) Riley X. Brady.

Performance#

The major change for this release is a dramatic speedup in bootstrapping functions, led by Aaron Spring. We focused on scalability with dask and found many places we could compute skill simultaneously over all bootstrapped ensemble members rather than at each iteration.

Bootstrapping uninitialized skill in the perfect model framework is now sped up significantly for annual lead resolution. (GH#332) Aaron Spring.
General speedup in bootstrap_hindcast() and bootstrap_perfect_model(): (GH#285) Aaron Spring.
- Properly implemented handling for lazy results when inputs are chunked.
- User gets warned when chunking potentially unnecessarily and/or inefficiently.

Bug Fixes#

Alignment options now account for differences in the historical time series if reference='historical'. (GH#341) Riley X. Brady.

Internals/Minor Fixes#

Added a Code of Conduct (GH#285) Aaron Spring.
Gather pytest.fixture in ``conftest.py. (GH#313) Aaron Spring.
Move x_METRICS and COMPARISONS to metrics.py and comparisons.py in order to avoid circular import dependencies. (GH#315) Aaron Spring.
asv benchmarks added for HindcastEnsemble (GH#285) Aaron Spring.
Ignore irrelevant warnings in pytest and mark slow tests (GH#333) Aaron Spring.
Default CONCAT_KWARGS now in all xr.concat to speed up bootstrapping. (GH#330) Aaron Spring.
Remove member coords for m2c comparison for probabilistic metrics. (GH#330) Aaron Spring.
Refactored compute_hindcast() and compute_perfect_model(). (GH#330) Aaron Spring.
Changed lead0 coordinate modifications to be compliant with xarray=0.15.1 in compute_persistence(). (GH#348) Aaron Spring.
Exchanged my_quantile with xr.quantile(skipna=False). (GH#348) Aaron Spring.
Remove sig from plot_bootstrapped_skill_over_leadyear(). (GH#351) Aaron Spring.
Require xskillscore v0.0.15 and use their functions for effective sample size-based metrics. (:pr: 353) Riley X. Brady.
Faster bootstrapping without replacement used in threshold functions of climpred.stats (GH#354) Aaron Spring.
Require cftime v1.1.2, which modifies their object handling to create 200-400x speedups in some basic operations. (GH#356) Riley X. Brady.
Resample first and then calculate skill in bootstrap_perfect_model() and bootstrap_hindcast() (GH#355) Aaron Spring.

Documentation#

Added demo to setup your own raw model output compliant to climpred (GH#296) Aaron Spring. See (here).
Added demo using intake-esm with climpred. See demo. (GH#296) Aaron Spring.
Added Verification Alignment page explaining how initializations are selected and aligned with verification data. (GH#328) Riley X. Brady. See (here).

climpred v2.0.0 (2020-01-22)#

New Features#

Add support for days, pentads, weeks, months, seasons for lead time resolution. climpred now requires a lead attribute “units” to decipher what resolution the predictions are at. (GH#294) Kathy Pegion and Riley X. Brady.

HindcastEnsemble now has HindcastEnsemble.add_observations() and HindcastEnsemble.get_observations() methods. These are the same as .add_reference() and .get_reference(), which will be deprecated eventually. The name change clears up confusion, since “reference” is the appropriate name for a reference forecast, e.g. "persistence". (GH#310) Riley X. Brady.
HindcastEnsemble now has .verify() function, which duplicates the .compute_metric() function. We feel that .verify() is more clear and easy to write, and follows the terminology of the field. (GH#310) Riley X. Brady.
e2o and m2o are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (GH#310) Riley X. Brady.

Documentation#

New example pages for subseasonal-to-seasonal prediction using climpred. (GH#294) Kathy Pegion
- Calculate the skill of the MJO index as a function of lead time (link).
- Calculate the skill of the MJO index as a function of lead time for weekly data (link).
- Calculate ENSO skill as a function of initial month vs. lead time (link).
- Calculate Seasonal ENSO skill (link).
Comparisons page rewritten for more clarity. (GH#310) Riley X. Brady.

Bug Fixes#

Fixed m2m broken comparison issue and removed correction. (GH#290) Aaron Spring.

Internals/Minor Fixes#

Updates to xskillscore v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (GH#309) Riley X. Brady.
Stacking dims is handled by comparisons, no need for internal keyword stack_dims. Therefore comparison now takes metric as argument instead. (GH#290) Aaron Spring.
assign_attrs now carries dim (GH#290) Aaron Spring.
reference changed to verif throughout hindcast compute functions. This is more clear, since reference usually refers to a type of forecast, such as persistence. (GH#310) Riley X. Brady.
Comparison objects can now have aliases. (GH#310) Riley X. Brady.

climpred v1.2.1 (2020-01-07)#

Depreciated#

mad no longer a keyword for the median absolute error metric. Users should now use median_absolute_error, which is identical to changes in xskillscore version 0.0.10. (GH#283) Riley X. Brady
pacc no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (GH#283) Riley X. Brady
msss no longer a keyword for the Murphy’s MSSS, since it is reserved for the standard MSSS. (GH#283) Riley X. Brady

New Features#

Metrics pearson_r_eff_p_value and spearman_r_eff_p_value account for autocorrelation in computing p values. (GH#283) Riley X. Brady
Metric effective_sample_size computes number of independent samples between two time series being correlated. (GH#283) Riley X. Brady
Added keywords for metrics: (GH#283) Riley X. Brady
- 'pval' for pearson_r_p_value
- ['n_eff', 'eff_n'] for effective_sample_size
- ['p_pval_eff', 'pvalue_eff', 'pval_eff'] for pearson_r_eff_p_value
- ['spvalue', 'spval'] for spearman_r_p_value
- ['s_pval_eff', 'spvalue_eff', 'spval_eff'] for spearman_r_eff_p_value
- 'nev' for nmse

Internals/Minor Fixes#

climpred now requires xarray version 0.14.1 so that the drop_vars() keyword used in our package does not throw an error. (GH#276) Riley X. Brady
Update to xskillscore version 0.0.10 to fix errors in weighted metrics with pairwise NaNs. (GH#283) Riley X. Brady
doc8 added to pre-commit to have consistent formatting on .rst files. (GH#283) Riley X. Brady
Remove proper attribute on Metric class since it isn’t used anywhere. (GH#283) Riley X. Brady
Add testing for effective p values. (GH#283) Riley X. Brady
Add testing for whether metric aliases are repeated/overwrite each other. (GH#283) Riley X. Brady
ppp changed to msess, but keywords allow for ppp and msss still. (GH#283) Riley X. Brady

Documentation#

Expansion of metrics documentation with much more detail on how metrics are computed, their keywords, references, min/max/perfect scores, etc. (GH#283) Riley X. Brady
Update terminology page with more information on metrics terminology. (GH#283) Riley X. Brady

climpred v1.2.0 (2019-12-17)#

Depreciated#

Abbreviation pval depreciated. Use p_pval for pearson_r_p_value instead. (GH#264) Aaron Spring.

New Features#

Users can now pass a custom metric or comparison to compute functions. (GH#268) Aaron Spring.
- See user-defined-metrics and user-defined-comparisons.
New deterministic metrics (see metrics). (GH#264) Aaron Spring.
- Spearman ranked correlation (spearman_r)
- Spearman ranked correlation p-value (spearman_r_p_value)
- Mean Absolute Deviation (mad)
- Mean Absolute Percent Error (mape)
- Symmetric Mean Absolute Percent Error (smape)

Users can now apply arbitrary xarray methods to HindcastEnsemble and PerfectModelEnsemble. (GH#243) Riley X. Brady.
- See the Prediction Ensemble objects demo page.
Add “getter” methods to HindcastEnsemble and PerfectModelEnsemble to retrieve xarray datasets from the objects. (GH#243) Riley X. Brady.

>>> hind = climpred.tutorial.load_dataset("CESM-DP-SST")
>>> ref = climpred.tutorial.load_dataset("ERSST")
>>> hindcast = climpred.HindcastEnsemble(hind)
>>> hindcast = hindcast.add_reference(ref, "ERSST")
>>> print(hindcast)
<climpred.HindcastEnsemble>
Initialized Ensemble:
    SST      (init, lead, member) float64 ...
ERSST:
    SST      (time) float32 ...
Uninitialized:
    None
>>> print(hindcast.get_initialized())
<xarray.Dataset>
Dimensions:  (init: 64, lead: 10, member: 10)
Coordinates:
* lead     (lead) int32 1 2 3 4 5 6 7 8 9 10
* member   (member) int32 1 2 3 4 5 6 7 8 9 10
* init     (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0
Data variables:
    SST      (init, lead, member) float64 ...
>>> print(hindcast.get_reference("ERSST"))
<xarray.Dataset>
Dimensions:  (time: 61)
Coordinates:
* time     (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015
Data variables:
    SST      (time) float32 ...

metric_kwargs can be passed to Metric. (GH#264) Aaron Spring.
- See metric_kwargs under metrics.

Bug Fixes#

HindcastEnsemble.compute_metric() doesn’t drop coordinates from the initialized hindcast ensemble anymore. (GH#258) Aaron Spring.
Metric uacc does not crash when ppp negative anymore. (GH#264) Aaron Spring.
Update xskillscore to version 0.0.9 to fix all-NaN issue with pearson_r and pearson_r_p_value when there’s missing data. (GH#269) Riley X. Brady.

Internals/Minor Fixes#

Rewrote varweighted_mean_period() based on xrft. Changed time_dim to dim. Function no longer drops coordinates. (GH#258) Aaron Spring
Add dim='time' in dpp(). (GH#258) Aaron Spring
Comparisons m2m, m2e rewritten to not stack dims into supervector because this is now done in xskillscore. (GH#264) Aaron Spring
Add tqdm progress bar to bootstrap_compute(). (GH#244) Aaron Spring
Remove inplace behavior for HindcastEnsemble and PerfectModelEnsemble. (GH#243) Riley X. Brady
- See demo page on prediction ensemble objects
Added tests for chunking with dask. (GH#258) Aaron Spring
Fix test issues with esmpy 8.0 by forcing esmpy 7.1 (GH#269). Riley X. Brady
Rewrote metrics and comparisons as classes to accomodate custom metrics and comparisons. (GH#268) Aaron Spring
- See user-defined-metrics and user-defined-comparisons.

Documentation#

Add examples notebook for temporal and spatial smoothing. (GH#244) Aaron Spring
Add documentation for computing a metric over a specified dimension. (GH#244) Aaron Spring
Update API to be more organized with individual function/class pages. (GH#243) Riley X. Brady.
Add page describing the HindcastEnsemble and PerfectModelEnsemble objects more clearly. (GH#243) Riley X. Brady
Add page for publications and helpful links. (GH#270) Riley X. Brady.

climpred v1.1.0 (2019-09-23)#

Features#

Write information about skill computation to netcdf attributes(GH#213) Aaron Spring
Temporal and spatial smoothing module (GH#224) Aaron Spring
Add metrics brier_score, threshold_brier_score and crpss_es (GH#232) Aaron Spring
Allow compute_hindcast and compute_perfect_model to specify which dimension dim to calculate metric over (GH#232) Aaron Spring

Bug Fixes#

Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (GH#232) Aaron Spring

Internals/Minor Fixes#

Rename .stats.DPP to dpp (GH#232) Aaron Spring
Add matplotlib as a main dependency so that a direct pip installation works (GH#211) Riley X. Brady.
climpred is now installable from conda-forge (GH#212) Riley X. Brady.
Fix erroneous descriptions of sample datasets (GH#226) Riley X. Brady.
Benchmarking time and peak memory of compute functions with asv (GH#231) Aaron Spring

Documentation#

Add scope of package to docs for clarity for users and developers. (GH#235) Riley X. Brady.

climpred v1.0.1 (2019-07-04)#

Bug Fixes#

Accomodate for lead-zero within the lead dimension (GH#196) Riley X. Brady.
Fix issue with adding uninitialized ensemble to HindcastEnsemble object (GH#199) Riley X. Brady.
Allow max_dof keyword to be passed to compute_metric and compute_persistence for HindcastEnsemble. (GH#199) Riley X. Brady.

Internals/Minor Fixes#

Force xskillscore version 0.0.4 or higher to avoid ImportError (GH#204) Riley X. Brady.
Change max_dfs keyword to max_dof (GH#199) Riley X. Brady.
Add tests for HindcastEnsemble and PerfectModelEnsemble. (GH#199) Riley X. Brady

climpred v1.0.0 (2019-07-03)#

climpred v1.0.0 represents the first stable release of the package. It includes HindcastEnsemble and PerfectModelEnsemble objects to perform analysis with. It offers a suite of deterministic and probabilistic metrics that are optimized to be run on single time series or grids of data (e.g., lat, lon, and depth). Currently, climpred only supports annual forecasts.

Features#

Bootstrap prediction skill based on resampling with replacement consistently in ReferenceEnsemble and PerfectModelEnsemble. (GH#128) Aaron Spring
Consistent bootstrap function for climpred.stats functions via bootstrap_func wrapper. (GH#167) Aaron Spring
many more metrics: _msss_murphy, _less and probabilistic _crps, _crpss (GH#128) Aaron Spring

Bug Fixes#

compute_uninitialized now trims input data to the same time window. (GH#193) Riley X. Brady
rm_poly now properly interpolates/fills NaNs. (GH#192) Riley X. Brady

Internals/Minor Fixes#

The climpred version can be printed. (GH#195) Riley X. Brady
Constants are made elegant and pushed to a separate module. (GH#184) Andrew Huang
Checks are consolidated to their own module. (GH#173) Andrew Huang

Documentation#

Documentation built extensively in multiple PRs.

climpred v0.3 (2019-04-27)#

climpred v0.3 really represents the entire development phase leading up to the version 1 release. This was done in collaboration between Riley X. Brady, Aaron Spring, and Andrew Huang. Future releases will have less additions.

Features#

Introduces object-oriented system to climpred, with classes ReferenceEnsemble and PerfectModelEnsemble. (GH#86) Riley X. Brady
Expands bootstrapping module for perfect-module configurations. (GH#78, GH#87) Aaron Spring
Adds functions for computing Relative Entropy (GH#73) Aaron Spring
Sets more intelligible dimension expectations for climpred (GH#98, GH#105) Riley X. Brady and Aaron Spring:
- init: initialization dates for the prediction ensemble
- lead: retrospective forecasts from prediction ensemble; returned dimension for prediction calculations
- time: time dimension for control runs, references, etc.
- member: ensemble member dimension.
Updates open_dataset to display available dataset names when no argument is passed. (GH#123) Riley X. Brady
Change ReferenceEnsemble to HindcastEnsemble. (GH#124) Riley X. Brady
Add probabilistic metrics to climpred. (GH#128) Aaron Spring
Consolidate separate perfect-model and hindcast functions into singular functions (GH#128) Aaron Spring
Add option to pass proxy through to open_dataset for firewalled networks. (GH#138) Riley X. Brady

Bug Fixes#

xr_rm_poly can now operate on Datasets and with multiple variables. It also interpolates across NaNs in time series. (GH#94) Andrew Huang
Travis CI, treon, and pytest all run for automated testing of new features. (GH#98, GH#105, GH#106) Riley X. Brady and Aaron Spring
Clean up check_xarray decorators and make sure that they work. (GH#142) Andrew Huang
Ensures that help() returns proper docstring even with decorators. (GH#149) Andrew Huang
Fixes bootstrap so p values are correct. (GH#170) Aaron Spring

Internals/Minor Fixes#

Adds unit testing for all perfect-model comparisons. (GH#107) Aaron Spring
Updates CESM-LE uninitialized ensemble sample data to have 34 members. (GH#113) Riley X. Brady
Adds MPI-ESM hindcast, historical, and assimilation sample data. (GH#119) Aaron Spring
Replaces check_xarray with a decorator for checking that input arguments are xarray objects. (GH#120) Andrew Huang
Add custom exceptions for clearer error reporting. (GH#139) Riley X. Brady
Remove “xr” prefix from stats module. (GH#144) Riley X. Brady
Add codecoverage for testing. (GH#152) Riley X. Brady
Update exception messages for more pretty error reporting. (GH#156) Andrew Huang
Add pre-commit and flake8/black check in CI. (GH#163) Riley X. Brady
Change loadutils module to tutorial and open_dataset to load_dataset. (GH#164) Riley X. Brady
Remove predictability horizon function to revisit for v2. (GH#165) Riley X. Brady
Increase code coverage through more testing. (GH#167) Aaron Spring
Consolidates checks and constants into modules. (GH#173) Andrew Huang

climpred v0.2 (2019-01-11)#

Name changed to climpred, developed enough for basic decadal prediction tasks on a perfect-model ensemble and reference-based ensemble.

climpred v0.1 (2018-12-20)#

Collaboration between Riley Brady and Aaron Spring begins.

climpred documentation

What’s New

Contents

What’s New#

climpred v2.3.0 (2021-11-25)#

Bug Fixes#

New Features#

Internals/Minor Fixes#

Bug Fixes#

Documentation#

climpred v2.2.0 (2021-12-20)#

Bug Fixes#

New Features#

Internals/Minor Fixes#

Documentation#

climpred v2.1.6 (2021-08-31)#

Bug Fixes#

New Features#

climpred v2.1.5 (2021-08-12)#

Breaking changes#

Bug Fixes#

New Features#

Documentation#

climpred v2.1.4 (2021-06-28)#

New Features#

Documentation#

Internals/Minor Fixes#

climpred v2.1.3 (2021-03-23)#

Breaking changes#

New Features#

Bug fixes#

Internals/Minor Fixes#

climpred v2.1.2 (2021-01-22)#

New Features#

Bug fixes#

climpred v2.1.1 (2020-10-13)#

Breaking changes#

New Features#

Deprecated#

Bug Fixes#

Documentation#

Internals/Minor Fixes#

climpred v2.1.0 (2020-06-08)#

Breaking Changes#

New Features#

Performance#

Bug Fixes#

Internals/Minor Fixes#

Documentation#

climpred v2.0.0 (2020-01-22)#

New Features#

Documentation#

Bug Fixes#

Internals/Minor Fixes#

climpred v1.2.1 (2020-01-07)#

Depreciated#

New Features#

Internals/Minor Fixes#

Documentation#

climpred v1.2.0 (2019-12-17)#

Depreciated#

New Features#

Bug Fixes#

Internals/Minor Fixes#

Documentation#

climpred v1.1.0 (2019-09-23)#

Features#

Bug Fixes#

Internals/Minor Fixes#

Documentation#

climpred v1.0.1 (2019-07-04)#

Bug Fixes#

Internals/Minor Fixes#

climpred v1.0.0 (2019-07-03)#

Features#

Bug Fixes#

Internals/Minor Fixes#

Documentation#

climpred v0.3 (2019-04-27)#

Features#