What’s New

climpred v2.1.5 (2021-08-12)

While climpred has used in the ASP summer colloquium 2021, many new features in remove_bias() were implemented.

Breaking changes

• renamed cross_validate to cv=False in remove_bias(). Only used when train_test_split='unfair-cv'. (GH#648, GH#655). Aaron Spring.

Bug Fixes

• Shift back init by lead after verify(). (GH#644, GH#645) Aaron Spring.

New Features

• remove_bias() accepts new keyword train_test_split='fair/unfair/unfair-cv' (default unfair) following Risbey et al. 2021. (GH#648, GH#655) Aaron Spring.

• allow more bias reduction methods in remove_bias():

  • how="additive_mean": correcting the mean forecast additively (already implemented)

  • how="multiplicative_mean": correcting the mean forecast multiplicatively

  • how="multiplicative_std": correcting the standard deviation multiplicatively

  Wrapped from bias_correction:

  • how="modified_quantile": Reference

  • how="basic_quantile": Reference

  • how="gamma_mapping": Reference

  • how="normal_mapping": Reference

• remove_bias() now does leave-one-out cross validation when passing cv='LOO' and train_test_split='unfair-cv'. cv=True falls back to cv='LOO'. (GH#643, GH#646) Aaron Spring.

• Add new metrics _spread() and _mul_bias() (GH#638) Aaron Spring.

• Add new tutorial datasets: (GH#651) Aaron Spring.

  • NMME_OIv2_Nino34_sst and NMME_hindcast_Nino34_sst with monthly leads

  • Observations_Germany and ECMWF_S2S_Germany with daily leads

• Metadata from CF convenctions are automatically attached by cf_xarray. (GH#639, GH#656) Aaron Spring.

• Raise warning when dimensions time, init or member are chunked to show user how to circumvent xskillscore chunking ValueError when passing these dimensions as dim in verify() or bootstrap(). (GH#509, GH#658) Aaron Spring.

• Implement PredictionEnsemble.chunks. (GH#658) Aaron Spring.

Documentation

• Speed up ENSO monthly example with IRIDL server-side preprocessing (see context) (GH#594, GH#633) Aaron Spring.

• Add CITATION.cff. Please cite Brady and Spring, 2020. (GH) Aaron Spring.

• Use NMME_OIv2_Nino34_sst and NMME_hindcast_Nino34_sst with monthly leads for bias reduction demonstrating remove_bias(). (GH#646) Aaron Spring.

climpred v2.1.4 (2021-06-28)

New Features

• Allow hours, minutes and seconds as lead.attrs['units']. (GH#404, GH#603) Aaron Spring.

• Allow to set seasonality via set_options to specify how to group in verify(reference='climatology' or in remove_bias(). (GH#529, GH#593, GH#603) Aaron Spring.

• Allow weekofyear via datetime in remove_bias(), but not yet implemented in verify(reference='climatology'). (GH#529, GH#603) Aaron Spring.

• Allow more dimensions in initialized than in observations. This is particular useful if you have forecasts from multiple models (in a model dimension) and want to verify against the same observations. (GH#129, GH#528, GH#619) Aaron Spring.

• Automatically rename dimensions to CLIMPRED_ENSEMBLE_DIMS ["init", "member", "lead"] if CF standard_names in coordinate attributes match: (GH#613, GH#622) Aaron Spring.

  • "init": "forecast_reference_time"

  • "member": "realization"

  • "lead": "forecast_period"

• If lead coordinate is pd.Timedelta, PredictionEnsemble converts lead coordinate upon instantiation to integer lead and corresponding lead.attrs["units"]. (GH#606, GH#627) Aaron Spring.

• Require xskillscore >= 0.0.20. _rps() now works with different category_edges for observations and forecasts, see daily ECMWF example. (GH#629, GH#630) Aaron Spring.

• Set options warn_for_failed_PredictionEnsemble_xr_call, warn_for_rename_to_climpred_dims, warn_for_init_coords_int_to_annual, climpred_warnings via set_options. (GH#628, GH#631) Aaron Spring.

• PredictionEnsemble acts like xr.Dataset and understands data_vars, dims, sizes, coords, nbytes, equals, identical, __iter__, __len__, __contains__, __delitem__. (GH#568, GH#632) Aaron Spring.

Documentation

• Add documentation page about publicly available initialized datasets and corresponding `climpred examples <initialized-datasets.html>`_. (GH#510, GH#561, GH#600) Aaron Spring.

• Add GEFS example for numerical weather prediction. (GH#602, GH#603) Aaron Spring.

• Add subseasonal daily ECMWF example using climetlab to access hindcasts from ECMWF cloud. (GH#587, GH#603) Aaron Spring.

• Add subseasonal daily S2S example accessing S2S output on IRIDL with a cookie and working with “on-the-fly” reforecasts with hdate dimension. (GH#588, GH#593) Aaron Spring.

• Added example climpred on GPU. Running verify() on GPU with cupy-xarray finishes 10x faster. (GH#592, GH#607) Aaron Spring.

• How to work with biweekly aggregates in climpred, see daily ECMWF example. (GH#625, GH#630) Aaron Spring.

Internals/Minor Fixes

• Add weekly upstream CI, which raises issues for failures. Adapted from xarray. Manually trigger by git commit -m '[test-upstream]'. Skip climpred_testing CI by git commit -m '[skip-ci]' (GH#518, GH#596) Aaron Spring.

climpred v2.1.3 (2021-03-23)

Breaking changes

New Features

• verify(), verify(), bootstrap() and bootstrap() accept reference climatology. Furthermore, reference persistence also allows probabilistic metrics (GH#202, GH#565, GH#566) Aaron Spring.

• Added new metric _roc Receiver Operating Characteristic as metric='roc'. (GH#566) Aaron Spring.

Bug fixes

• verify() and bootstrap() accept dim as list, set, tuple or str (GH#519, GH#558) Aaron Spring.

• map() now does not fail silently when applying a function to all xr.Datasets of PredictionEnsemble. Instead, UserWarning``s are raised. Furthermore, ``PredictionEnsemble.map(func, *args, **kwargs) applies only function to Datasets with matching dims if dim="dim0_or_dim1" is passed as **kwargs. (GH#417, GH#437, GH#552) Aaron Spring.

• _rpc was fixed in xskillscore>=0.0.19 and hence is not falsely limited to 1 anymore (GH#562, GH#566) Aaron Spring.

Internals/Minor Fixes

• Docstrings are now tested in GitHub actions continuous integration. (GH#545, GH#560) Aaron Spring.

• Github actions now cancels previous commits, instead of running the full testing suite on every single commit. (GH#560) Aaron Spring.

• verify() does not add climpred attributes to skill by default anymore. (GH#560) Aaron Spring.

• Drop python==3.6 support. (GH#573) Aaron Spring.

• Notebooks are now linted with nb_black using %load_ext nb_black or %load_ext lab_black for Jupyter notebooks and Jupyter lab. (GH#526, GH#572) Aaron Spring.

• Reduce dependencies to install climpred. (GH#454, GH#572) Aaron Spring.

• Examples from documentation available via Binder. Find further examples in the examples folder. (GH#549, GH#578) Aaron Spring.

• Rename branch master to main. (GH#579) Aaron Spring.

climpred v2.1.2 (2021-01-22)

This release is the fixed version for our Journal of Open Source Software (JOSS) article about climpred, see review.

New Features

• Function to calculate predictability horizon predictability_horizon() based on condition. (GH#46, GH#521) Aaron Spring.

Bug fixes

• smooth() now carries lead.attrs (GH#527, pr:521) Aaron Spring.

• verify() now works with references also for geospatial inputs, which returned NaN before. (GH#522, pr:521) Aaron Spring.

• plot() now shifts composite lead frequencies like days, pentads, seasons correctly. (GH#532, GH#533) Aaron Spring.

• Adapt to xesmf>=0.5.2 for spatial xesmf smoothing. (GH#543, GH#548) Aaron Spring.

• remove_bias() now carries attributes. (GH#531, GH#551) Aaron Spring.

climpred v2.1.1 (2020-10-13)

Breaking changes

This version introduces a lot of breaking changes. We are trying to overhaul climpred to have an intuitive API that also forces users to think about methodology choices when running functions. The main breaking changes we introduced are for verify() and verify(). Now, instead of assuming defaults for most keywords, we require the user to define metric, comparison, dim, and alignment (for hindcast systems). We also require users to designate the number of iterations for bootstrapping.

• User now has to designate number of iterations with iterations=... in bootstrap() (GH#384, GH#436) Aaron Spring and Riley X. Brady.

• Make metric, comparison, dim, and alignment required (previous default None) arguments for verify() (GH#384, GH#436) Aaron Spring and Riley X. Brady.

• Metric _brier_score and _threshold_brier_score() now requires callable keyword argument logical instead of func (GH#388) Aaron Spring.

• verify() does not correct dim automatically to member for probabilistic metrics. (GH#282, GH#407) Aaron Spring.

• Users can no longer add multiple observations to HindcastEnsemble. This will make current and future development much easier on maintainers (GH#429, GH#453) Riley X. Brady.

• Standardize the names of the output coordinates for verify() and bootstrap() to initialized, uninitialized, and persistence. initialized showcases the metric result after comparing the initialized ensemble to the verification data; uninitialized when comparing the uninitialized (historical) ensemble to the verification data; persistence is the evaluation of the persistence forecast (GH#460, GH#478, GH#476, GH#480) Aaron Spring.

• reference keyword in verify() should be choosen from [uninitialized, persistence]. historical no longer works (GH#460, GH#478, GH#476, GH#480) Aaron Spring.

• verify() returns no skill dimension if reference=None (GH#480) Aaron Spring.

• comparison is not applied to uninitialized skill in bootstrap(). (GH#352, GH#418) Aaron Spring.

New Features

This release is accompanied by a bunch of new features. Math operations can now be used with our PredictionEnsemble objects and their variables can be sub-selected. Users can now quick plot time series forecasts with these objects. Bootstrapping is available for HindcastEnsemble. Spatial dimensions can be passed to metrics to do things like pattern correlation. New metrics have been implemented based on Contingency tables. We now include an early version of bias removal for HindcastEnsemble.

• Use math operations like +-*/ with HindcastEnsemble and PerfectModelEnsemble. See a demo of this here (GH#377) Aaron Spring.

• Subselect data variables from PredictionEnsemble as from xr.Dataset: PredictionEnsemble[['var1', 'var3']] (GH#409) Aaron Spring.

• Plot all datasets in HindcastEnsemble or PerfectModelEnsemble by plot() if no other spatial dimensions are present. (GH#383) Aaron Spring.

• Bootstrapping now available for HindcastEnsemble as bootstrap(), which is analogous to the PerfectModelEnsemble method (GH#257, GH#418) Aaron Spring.

• verify() allows all dimensions from initialized ensemble as dim. This allows e.g. spatial dimensions to be used for pattern correlation. Make sure to use skipna=True when using spatial dimensions and output has nans (in the case of land, for instance) (GH#282, GH#407) Aaron Spring.

• Allow binary forecasts at when calling verify(), rather than needing to supply binary results beforehand. In other words, hindcast.verify(metric='brier_score', comparison='m2o', dim='member', logical=logical) is now the same as hindcast.map(logical).verify(metric='brier_score', comparison='m2o', dim='member'. (GH#431) Aaron Spring.

• Check calendar types when using add_observations(), add_uninitialized(), add_control() to ensure that the verification data calendars match that of the initialized ensemble. (GH#300, GH#452, GH#422, GH#462) Riley X. Brady and Aaron Spring.

• Implement new metrics which have been ported over from https://github.com/csiro-dcfp/doppyo/ to xskillscore by Dougie Squire. (GH#439, GH#456) Aaron Spring

  • rank histogram _rank_histogram()

  • discrimination _discrimination()

  • reliability _reliability()

  • ranked probability score _rps()

  • contingency table and related scores _contingency()

• Perfect Model verify() no longer requires control in PerfectModelEnsemble. It is only required when reference=['persistence']. (GH#461) Aaron Spring.

• Implemented bias removal remove_bias. remove_bias(how='mean') removes the mean bias of initialized hindcasts with respect to observations. See example. (GH#389, GH#443, GH#459) Aaron Spring and Riley X. Brady.

Deprecated

• spatial_smoothing_xrcoarsen no longer used for spatial smoothing. (GH#391) Aaron Spring.

• compute_metric, compute_uninitialized and compute_persistence no longer in use for PerfectModelEnsemble in favor of verify() with the reference keyword instead. (GH#436, GH#468, GH#472) Aaron Spring and Riley X. Brady.

• 'historical' no longer a valid choice for reference. Use 'uninitialized' instead. (GH#478) Aaron Spring.

Bug Fixes

• verify() and bootstrap() now accept metric_kwargs. (GH#387) Aaron Spring.

• verify() now accepts 'uninitialized' as a reference. (GH#395) Riley X. Brady.

• Spatial and temporal smoothing smooth() now work as expected and rename time dimensions after verify(). (GH#391) Aaron Spring.

• PredictionEnsemble.verify(comparison='m2o', references=['uninitialized', 'persistence'] does not fail anymore. (GH#385, GH#400) Aaron Spring.

• Remove bias using dayofyear in reduce_bias(). (GH#443) Aaron Spring.

• climpred works with dask=>2.28. (GH#479, GH#482) Aaron Spring.

Documentation

• Updates climpred tagline to “Verification of weather and climate forecasts.” (GH#420) Riley X. Brady.

• Adds section on how to use arithmetic with HindcastEnsemble. (GH#378) Riley X. Brady.

• Add docs section for similar open-source forecasting packages. (GH#432) Riley X. Brady.

• Add all metrics to main API in addition to metrics page. (GH#438) Riley X. Brady.

• Add page on bias removal Aaron Spring.

Internals/Minor Fixes

• verify() replaces deprecated PerfectModelEnsemble.compute_metric() and accepts reference as keyword. (GH#387) Aaron Spring.

• Cleared out unnecessary statistics functions from climpred and migrated them to esmtools. Add esmtools as a required package. (GH#395) Riley X. Brady.

• Remove fixed pandas dependency from pandas=0.25 to stable pandas. (GH#402, GH#403) Aaron Spring.

• dim is expected to be a list of strings in compute_perfect_model() and compute_hindcast(). (GH#282, GH#407) Aaron Spring.

• Update cartopy requirement to 0.0.18 or greater to release lock on matplotlib version. Update xskillscore requirement to 0.0.18 to cooperate with new xarray version. (GH#451, GH#449) Riley X. Brady

• Switch from Travis CI and Coveralls to Github Actions and CodeCov. (GH#471) Riley X. Brady

• Assertion functions added for PerfectModelEnsemble: assert_PredictionEnsemble(). (GH#391) Aaron Spring.

• Test all metrics against synthetic data. (GH#388) Aaron Spring.

climpred v2.1.0 (2020-06-08)

Breaking Changes

• Keyword bootstrap has been replaced with iterations. We feel that this more accurately describes the argument, since “bootstrap” is really the process as a whole. (GH#354) Aaron Spring.

New Features

• HindcastEnsemble and PerfectModelEnsemble now use an HTML representation, following the more recent versions of xarray. (GH#371) Aaron Spring.

• HindcastEnsemble.verify() now takes reference=... keyword. Current options are 'persistence' for a persistence forecast of the observations and 'uninitialized' for an uninitialized/historical reference, such as an uninitialized/forced run. (GH#341) Riley X. Brady.

• We now only enforce a union of the initialization dates with observations if reference='persistence' for HindcastEnsemble. This is to ensure that the same set of initializations is used by the observations to construct a persistence forecast. (GH#341) Riley X. Brady.

• compute_perfect_model() now accepts initialization (init) as cftime and int. cftime is now implemented into the bootstrap uninitialized functions for the perfect model configuration. (GH#332) Aaron Spring.

• New explicit keywords in bootstrap functions for resampling_dim and reference_compute (GH#320) Aaron Spring.

• Logging now included for compute_hindcast which displays the inits and verification dates used at each lead (GH#324) Aaron Spring, (GH#338) Riley X. Brady. See (logging).

• New explicit keywords added for alignment of verification dates and initializations. (GH#324) Aaron Spring. See (alignment)

  • 'maximize': Maximize the degrees of freedom by slicing hind and verif to a common time frame at each lead. (GH#338) Riley X. Brady.

  • 'same_inits': slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (GH#328) Riley X. Brady.

  • 'same_verifs': slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (GH#331) Riley X. Brady.

Performance

The major change for this release is a dramatic speedup in bootstrapping functions, led by Aaron Spring. We focused on scalability with dask and found many places we could compute skill simultaneously over all bootstrapped ensemble members rather than at each iteration.

• Bootstrapping uninitialized skill in the perfect model framework is now sped up significantly for annual lead resolution. (GH#332) Aaron Spring.

• General speedup in bootstrap_hindcast() and bootstrap_perfect_model(): (GH#285) Aaron Spring.

  • Properly implemented handling for lazy results when inputs are chunked.

  • User gets warned when chunking potentially unnecessarily and/or inefficiently.

Bug Fixes

• Alignment options now account for differences in the historical time series if reference='historical'. (GH#341) Riley X. Brady.

Internals/Minor Fixes

• Added a Code of Conduct (GH#285) Aaron Spring.

• Gather pytest.fixture in ``conftest.py. (GH#313) Aaron Spring.

• Move x_METRICS and COMPARISONS to metrics.py and comparisons.py in order to avoid circular import dependencies. (GH#315) Aaron Spring.

• asv benchmarks added for HindcastEnsemble (GH#285) Aaron Spring.

• Ignore irrelevant warnings in pytest and mark slow tests (GH#333) Aaron Spring.

• Default CONCAT_KWARGS now in all xr.concat to speed up bootstrapping. (GH#330) Aaron Spring.

• Remove member coords for m2c comparison for probabilistic metrics. (GH#330) Aaron Spring.

• Refactored compute_hindcast() and compute_perfect_model(). (GH#330) Aaron Spring.

• Changed lead0 coordinate modifications to be compliant with xarray=0.15.1 in compute_persistence(). (GH#348) Aaron Spring.

• Exchanged my_quantile with xr.quantile(skipna=False). (GH#348) Aaron Spring.

• Remove sig from plot_bootstrapped_skill_over_leadyear(). (GH#351) Aaron Spring.

• Require xskillscore v0.0.15 and use their functions for effective sample size-based metrics. (:pr: 353) Riley X. Brady.

• Faster bootstrapping without replacement used in threshold functions of climpred.stats (GH#354) Aaron Spring.

• Require cftime v1.1.2, which modifies their object handling to create 200-400x speedups in some basic operations. (GH#356) Riley X. Brady.

• Resample first and then calculate skill in bootstrap_perfect_model() and bootstrap_hindcast() (GH#355) Aaron Spring.

Documentation

• Added demo to setup your own raw model output compliant to climpred (GH#296) Aaron Spring. See (here).

• Added demo using intake-esm with climpred (GH#296) Aaron Spring. See (here).

• Added Verification Alignment page explaining how initializations are selected and aligned with verification data. (GH#328) Riley X. Brady. See (here).

climpred v2.0.0 (2020-01-22)

New Features

• Add support for days, pentads, weeks, months, seasons for lead time resolution. climpred now requires a lead attribute “units” to decipher what resolution the predictions are at. (GH#294) Kathy Pegion and Riley X. Brady.

>>> hind = climpred.tutorial.load_dataset('CESM-DP-SST')
>>> hind.lead.attrs['units'] = 'years'

• HindcastEnsemble now has .add_observations() and .get_observations() methods. These are the same as .add_reference() and .get_reference(), which will be deprecated eventually. The name change clears up confusion, since “reference” is the appropriate name for a reference forecast, e.g. persistence. (GH#310) Riley X. Brady.

• HindcastEnsemble now has .verify() function, which duplicates the .compute_metric() function. We feel that .verify() is more clear and easy to write, and follows the terminology of the field. (GH#310) Riley X. Brady.

• e2o and m2o are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (GH#310) Riley X. Brady.

Documentation

• New example pages for subseasonal-to-seasonal prediction using climpred. (GH#294) Kathy Pegion

  • Calculate the skill of the MJO index as a function of lead time (link).

  • Calculate the skill of the MJO index as a function of lead time for weekly data (link).

  • Calculate ENSO skill as a function of initial month vs. lead time (link).

  • Calculate Seasonal ENSO skill (link).

• Comparisons page rewritten for more clarity. (GH#310) Riley X. Brady.

Bug Fixes

• Fixed m2m broken comparison issue and removed correction (GH#290) Aaron Spring.

Internals/Minor Fixes

• Updates to xskillscore v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (GH#309) Riley X. Brady.

• Stacking dims is handled by comparisons, no need for internal keyword stack_dims. Therefore comparison now takes metric as argument instead. (GH#290) Aaron Spring.

• assign_attrs now carries dim (GH#290) Aaron Spring.

• reference changed to verif throughout hindcast compute functions. This is more clear, since reference usually refers to a type of forecast, such as persistence. (GH#310) Riley X. Brady.

• Comparison objects can now have aliases. (GH#310) Riley X. Brady.

climpred v1.2.1 (2020-01-07)

Depreciated

• mad no longer a keyword for the median absolute error metric. Users should now use median_absolute_error, which is identical to changes in xskillscore version 0.0.10. (GH#283) Riley X. Brady

• pacc no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (GH#283) Riley X. Brady

• msss no longer a keyword for the Murphy’s MSSS, since it is reserved for the standard MSSS. (GH#283) Riley X. Brady

New Features

• Metrics pearson_r_eff_p_value and spearman_r_eff_p_value account for autocorrelation in computing p values. (GH#283) Riley X. Brady

• Metric effective_sample_size computes number of independent samples between two time series being correlated. (GH#283) Riley X. Brady

• Added keywords for metrics: (GH#283) Riley X. Brady

  • 'pval' for pearson_r_p_value

  • ['n_eff', 'eff_n'] for effective_sample_size

  • ['p_pval_eff', 'pvalue_eff', 'pval_eff'] for pearson_r_eff_p_value

  • ['spvalue', 'spval'] for spearman_r_p_value

  • ['s_pval_eff', 'spvalue_eff', 'spval_eff'] for spearman_r_eff_p_value

  • 'nev' for nmse

Internals/Minor Fixes

• climpred now requires xarray version 0.14.1 so that the drop_vars() keyword used in our package does not throw an error. (GH#276) Riley X. Brady

• Update to xskillscore version 0.0.10 to fix errors in weighted metrics with pairwise NaNs. (GH#283) Riley X. Brady

• doc8 added to pre-commit to have consistent formatting on .rst files. (GH#283) Riley X. Brady

• Remove proper attribute on Metric class since it isn’t used anywhere. (GH#283) Riley X. Brady

• Add testing for effective p values. (GH#283) Riley X. Brady

• Add testing for whether metric aliases are repeated/overwrite each other. (GH#283) Riley X. Brady

• ppp changed to msess, but keywords allow for ppp and msss still. (GH#283) Riley X. Brady

Documentation

• Expansion of metrics documentation with much more detail on how metrics are computed, their keywords, references, min/max/perfect scores, etc. (GH#283) Riley X. Brady

• Update terminology page with more information on metrics terminology. (GH#283) Riley X. Brady

climpred v1.2.0 (2019-12-17)

Depreciated

• Abbreviation pval depreciated. Use p_pval for pearson_r_p_value instead. (GH#264) Aaron Spring.

New Features

• Users can now pass a custom metric or comparison to compute functions. (GH#268) Aaron Spring.

  • See user-defined-metrics and user-defined-comparisons.

• New deterministic metrics (see metrics). (GH#264) Aaron Spring.

  • Spearman ranked correlation (spearman_r)

  • Spearman ranked correlation p-value (spearman_r_p_value)

  • Mean Absolute Deviation (mad)

  • Mean Absolute Percent Error (mape)

  • Symmetric Mean Absolute Percent Error (smape)

• Users can now apply arbitrary xarray methods to HindcastEnsemble and PerfectModelEnsemble. (GH#243) Riley X. Brady.

  • See the Prediction Ensemble objects demo page.

• Add “getter” methods to HindcastEnsemble and PerfectModelEnsemble to retrieve xarray datasets from the objects. (GH#243) Riley X. Brady.

  >>> hind = climpred.tutorial.load_dataset('CESM-DP-SST')
  >>> ref = climpred.tutorial.load_dataset('ERSST')
  >>> hindcast = climpred.HindcastEnsemble(hind)
  >>> hindcast = hindcast.add_reference(ref, 'ERSST')
  >>> print(hindcast)
  <climpred.HindcastEnsemble>
  Initialized Ensemble:
      SST      (init, lead, member) float64 ...
  ERSST:
      SST      (time) float32 ...
  Uninitialized:
      None
  >>> print(hindcast.get_initialized())
  <xarray.Dataset>
  Dimensions:  (init: 64, lead: 10, member: 10)
  Coordinates:
  * lead     (lead) int32 1 2 3 4 5 6 7 8 9 10
  * member   (member) int32 1 2 3 4 5 6 7 8 9 10
  * init     (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0
  Data variables:
      SST      (init, lead, member) float64 ...
  >>> print(hindcast.get_reference('ERSST'))
  <xarray.Dataset>
  Dimensions:  (time: 61)
  Coordinates:
  * time     (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015
  Data variables:
      SST      (time) float32 ...
  

• metric_kwargs can be passed to Metric. (GH#264) Aaron Spring.

  • See metric_kwargs under metrics.

Bug Fixes

• compute_metric() doesn’t drop coordinates from the initialized hindcast ensemble anymore. (GH#258) Aaron Spring.

• Metric uacc does not crash when ppp negative anymore. (GH#264) Aaron Spring.

• Update xskillscore to version 0.0.9 to fix all-NaN issue with pearson_r and pearson_r_p_value when there’s missing data. (GH#269) Riley X. Brady.

Internals/Minor Fixes

• Rewrote varweighted_mean_period() based on xrft. Changed time_dim to dim. Function no longer drops coordinates. (GH#258) Aaron Spring

• Add dim='time' in dpp(). (GH#258) Aaron Spring

• Comparisons m2m, m2e rewritten to not stack dims into supervector because this is now done in xskillscore. (GH#264) Aaron Spring

• Add tqdm progress bar to bootstrap_compute(). (GH#244) Aaron Spring

• Remove inplace behavior for HindcastEnsemble and PerfectModelEnsemble. (GH#243) Riley X. Brady

  • See demo page on prediction ensemble objects

• Added tests for chunking with dask. (GH#258) Aaron Spring

• Fix test issues with esmpy 8.0 by forcing esmpy 7.1 (GH#269). Riley X. Brady

• Rewrote metrics and comparisons as classes to accomodate custom metrics and comparisons. (GH#268) Aaron Spring

  • See user-defined-metrics and user-defined-comparisons.

Documentation

• Add examples notebook for temporal and spatial smoothing. (GH#244) Aaron Spring

• Add documentation for computing a metric over a specified dimension. (GH#244) Aaron Spring

• Update API to be more organized with individual function/class pages. (GH#243) Riley X. Brady.

• Add page describing the HindcastEnsemble and PerfectModelEnsemble objects more clearly. (GH#243) Riley X. Brady

• Add page for publications and helpful links. (GH#270) Riley X. Brady.

climpred v1.1.0 (2019-09-23)

Features

• Write information about skill computation to netcdf attributes(GH#213) Aaron Spring

• Temporal and spatial smoothing module (GH#224) Aaron Spring

• Add metrics brier_score, threshold_brier_score and crpss_es (GH#232) Aaron Spring

• Allow compute_hindcast and compute_perfect_model to specify which dimension dim to calculate metric over (GH#232) Aaron Spring

Bug Fixes

• Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (GH#232) Aaron Spring

Internals/Minor Fixes

• Rename .stats.DPP to dpp (GH#232) Aaron Spring

• Add matplotlib as a main dependency so that a direct pip installation works (GH#211) Riley X. Brady.

• climpred is now installable from conda-forge (GH#212) Riley X. Brady.

• Fix erroneous descriptions of sample datasets (GH#226) Riley X. Brady.

• Benchmarking time and peak memory of compute functions with asv (GH#231) Aaron Spring

Documentation

• Add scope of package to docs for clarity for users and developers. (GH#235) Riley X. Brady.

climpred v1.0.1 (2019-07-04)

Bug Fixes

• Accomodate for lead-zero within the lead dimension (GH#196) Riley X. Brady.

• Fix issue with adding uninitialized ensemble to HindcastEnsemble object (GH#199) Riley X. Brady.

• Allow max_dof keyword to be passed to compute_metric and compute_persistence for HindcastEnsemble (GH#199) Riley X. Brady.

Internals/Minor Fixes

• Force xskillscore version 0.0.4 or higher to avoid ImportError (GH#204) Riley X. Brady.

• Change max_dfs keyword to max_dof (GH#199) Riley X. Brady.

• Add testing for HindcastEnsemble and PerfectModelEnsemble (GH#199) Riley X. Brady

climpred v1.0.0 (2019-07-03)

climpred v1.0.0 represents the first stable release of the package. It includes HindcastEnsemble and PerfectModelEnsemble objects to perform analysis with. It offers a suite of deterministic and probabilistic metrics that are optimized to be run on single time series or grids of data (e.g., lat, lon, and depth). Currently, climpred only supports annual forecasts.

Features

• Bootstrap prediction skill based on resampling with replacement consistently in ReferenceEnsemble and PerfectModelEnsemble. (GH#128) Aaron Spring

• Consistent bootstrap function for climpred.stats functions via bootstrap_func wrapper. (GH#167) Aaron Spring

• many more metrics: _msss_murphy, _less and probabilistic _crps, _crpss (GH#128) Aaron Spring

Bug Fixes

• compute_uninitialized now trims input data to the same time window. (GH#193) Riley X. Brady

• rm_poly now properly interpolates/fills NaNs. (GH#192) Riley X. Brady

Internals/Minor Fixes

• The climpred version can be printed. (GH#195) Riley X. Brady

• Constants are made elegant and pushed to a separate module. (GH#184) Andrew Huang

• Checks are consolidated to their own module. (GH#173) Andrew Huang

Documentation

• Documentation built extensively in multiple PRs.

climpred v0.3 (2019-04-27)

climpred v0.3 really represents the entire development phase leading up to the version 1 release. This was done in collaboration between Riley X. Brady, Aaron Spring, and Andrew Huang. Future releases will have less additions.

Features

• Introduces object-oriented system to climpred, with classes ReferenceEnsemble and PerfectModelEnsemble. (GH#86) Riley X. Brady

• Expands bootstrapping module for perfect-module configurations. (GH#78, GH#87) Aaron Spring

• Adds functions for computing Relative Entropy (GH#73) Aaron Spring

• Sets more intelligible dimension expectations for climpred (GH#98, GH#105) Riley X. Brady and Aaron Spring:

  • init: initialization dates for the prediction ensemble

  • lead: retrospective forecasts from prediction ensemble; returned dimension for prediction calculations

  • time: time dimension for control runs, references, etc.

  • member: ensemble member dimension.

• Updates open_dataset to display available dataset names when no argument is passed. (GH#123) Riley X. Brady

• Change ReferenceEnsemble to HindcastEnsemble. (GH#124) Riley X. Brady

• Add probabilistic metrics to climpred. (GH#128) Aaron Spring

• Consolidate separate perfect-model and hindcast functions into singular functions (GH#128) Aaron Spring

• Add option to pass proxy through to open_dataset for firewalled networks. (GH#138) Riley X. Brady

Bug Fixes

• xr_rm_poly can now operate on Datasets and with multiple variables. It also interpolates across NaNs in time series. (GH#94) Andrew Huang

• Travis CI, treon, and pytest all run for automated testing of new features. (GH#98, GH#105, GH#106) Riley X. Brady and Aaron Spring

• Clean up check_xarray decorators and make sure that they work. (GH#142) Andrew Huang

• Ensures that help() returns proper docstring even with decorators. (GH#149) Andrew Huang

• Fixes bootstrap so p values are correct. (GH#170) Aaron Spring

Internals/Minor Fixes

• Adds unit testing for all perfect-model comparisons. (GH#107) Aaron Spring

• Updates CESM-LE uninitialized ensemble sample data to have 34 members. (GH#113) Riley X. Brady

• Adds MPI-ESM hindcast, historical, and assimilation sample data. (GH#119) Aaron Spring

• Replaces check_xarray with a decorator for checking that input arguments are xarray objects. (GH#120) Andrew Huang

• Add custom exceptions for clearer error reporting. (GH#139) Riley X. Brady

• Remove “xr” prefix from stats module. (GH#144) Riley X. Brady

• Add codecoverage for testing. (GH#152) Riley X. Brady

• Update exception messages for more pretty error reporting. (GH#156) Andrew Huang

• Add pre-commit and flake8/black check in CI. (GH#163) Riley X. Brady

• Change loadutils module to tutorial and open_dataset to load_dataset. (GH#164) Riley X. Brady

• Remove predictability horizon function to revisit for v2. (GH#165) Riley X. Brady

• Increase code coverage through more testing. (GH#167) Aaron Spring

• Consolidates checks and constants into modules. (GH#173) Andrew Huang

climpred v0.2 (2019-01-11)

Name changed to climpred, developed enough for basic decadal prediction tasks on a perfect-model ensemble and reference-based ensemble.

climpred v0.1 (2018-12-20)

Collaboration between Riley Brady and Aaron Spring begins.