What’s New

climpred v2.1.1 (2020-10-13)

Breaking changes

This version introduces a lot of breaking changes. We are trying to overhaul climpred to have an intuitive API that also forces users to think about methodology choices when running functions. The main breaking changes we introduced are for verify() and verify(). Now, instead of assuming defaults for most keywords, we require the user to define metric, comparison, dim, and alignment (for hindcast systems). We also require users to designate the number of iterations for bootstrapping.

New Features

This release is accompanied by a bunch of new features. Math operations can now be used with our PredictionEnsemble objects and their variables can be sub-selected. Users can now quick plot time series forecasts with these objects. Bootstrapping is available for HindcastEnsemble. Spatial dimensions can be passed to metrics to do things like pattern correlation. New metrics have been implemented based on Contingency tables. We now include an early version of bias removal for HindcastEnsemble.

Deprecated

Bug Fixes

Documentation

Internals/Minor Fixes

climpred v2.1.0 (2020-06-08)

Breaking Changes

  • Keyword bootstrap has been replaced with iterations. We feel that this more accurately describes the argument, since “bootstrap” is really the process as a whole. (GH#354) Aaron Spring.

New Features

  • HindcastEnsemble and PerfectModelEnsemble now use an HTML representation, following the more recent versions of xarray. (GH#371) Aaron Spring.

  • HindcastEnsemble.verify() now takes reference=... keyword. Current options are 'persistence' for a persistence forecast of the observations and 'uninitialized' for an uninitialized/historical reference, such as an uninitialized/forced run. (GH#341) Riley X. Brady.

  • We now only enforce a union of the initialization dates with observations if reference='persistence' for HindcastEnsemble. This is to ensure that the same set of initializations is used by the observations to construct a persistence forecast. (GH#341) Riley X. Brady.

  • compute_perfect_model() now accepts initialization (init) as cftime and int. cftime is now implemented into the bootstrap uninitialized functions for the perfect model configuration. (GH#332) Aaron Spring.

  • New explicit keywords in bootstrap functions for resampling_dim and reference_compute (GH#320) Aaron Spring.

  • Logging now included for compute_hindcast which displays the inits and verification dates used at each lead (GH#324) Aaron Spring, (GH#338) Riley X. Brady. See (logging).

  • New explicit keywords added for alignment of verification dates and initializations. (GH#324) Aaron Spring. See (alignment)

    • 'maximize': Maximize the degrees of freedom by slicing hind and verif to a common time frame at each lead. (GH#338) Riley X. Brady.

    • 'same_inits': slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (GH#328) Riley X. Brady.

    • 'same_verifs': slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (GH#331) Riley X. Brady.

Performance

The major change for this release is a dramatic speedup in bootstrapping functions, led by Aaron Spring. We focused on scalability with dask and found many places we could compute skill simultaneously over all bootstrapped ensemble members rather than at each iteration.

Bug Fixes

  • Alignment options now account for differences in the historical time series if reference='historical'. (GH#341) Riley X. Brady.

Internals/Minor Fixes

Documentation

climpred v2.0.0 (2020-01-22)

New Features

  • Add support for days, pentads, weeks, months, seasons for lead time resolution. climpred now requires a lead attribute “units” to decipher what resolution the predictions are at. (GH#294) Kathy Pegion and Riley X. Brady.

>>> hind = climpred.tutorial.load_dataset('CESM-DP-SST')
>>> hind.lead.attrs['units'] = 'years'
  • HindcastEnsemble now has .add_observations() and .get_observations() methods. These are the same as .add_reference() and .get_reference(), which will be deprecated eventually. The name change clears up confusion, since “reference” is the appropriate name for a reference forecast, e.g. persistence. (GH#310) Riley X. Brady.

  • HindcastEnsemble now has .verify() function, which duplicates the .compute_metric() function. We feel that .verify() is more clear and easy to write, and follows the terminology of the field. (GH#310) Riley X. Brady.

  • e2o and m2o are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (GH#310) Riley X. Brady.

Documentation

  • New example pages for subseasonal-to-seasonal prediction using climpred. (GH#294) Kathy Pegion

    • Calculate the skill of the MJO index as a function of lead time (link).

    • Calculate the skill of the MJO index as a function of lead time for weekly data (link).

    • Calculate ENSO skill as a function of initial month vs. lead time (link).

    • Calculate Seasonal ENSO skill (link).

  • Comparisons page rewritten for more clarity. (GH#310) Riley X. Brady.

Bug Fixes

Internals/Minor Fixes

  • Updates to xskillscore v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (GH#309) Riley X. Brady.

  • Stacking dims is handled by comparisons, no need for internal keyword stack_dims. Therefore comparison now takes metric as argument instead. (GH#290) Aaron Spring.

  • assign_attrs now carries dim (GH#290) Aaron Spring.

  • reference changed to verif throughout hindcast compute functions. This is more clear, since reference usually refers to a type of forecast, such as persistence. (GH#310) Riley X. Brady.

  • Comparison objects can now have aliases. (GH#310) Riley X. Brady.

climpred v1.2.1 (2020-01-07)

Depreciated

  • mad no longer a keyword for the median absolute error metric. Users should now use median_absolute_error, which is identical to changes in xskillscore version 0.0.10. (GH#283) Riley X. Brady

  • pacc no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (GH#283) Riley X. Brady

  • msss no longer a keyword for the Murphy’s MSSS, since it is reserved for the standard MSSS. (GH#283) Riley X. Brady

New Features

  • Metrics pearson_r_eff_p_value and spearman_r_eff_p_value account for autocorrelation in computing p values. (GH#283) Riley X. Brady

  • Metric effective_sample_size computes number of independent samples between two time series being correlated. (GH#283) Riley X. Brady

  • Added keywords for metrics: (GH#283) Riley X. Brady

    • 'pval' for pearson_r_p_value

    • ['n_eff', 'eff_n'] for effective_sample_size

    • ['p_pval_eff', 'pvalue_eff', 'pval_eff'] for pearson_r_eff_p_value

    • ['spvalue', 'spval'] for spearman_r_p_value

    • ['s_pval_eff', 'spvalue_eff', 'spval_eff'] for spearman_r_eff_p_value

    • 'nev' for nmse

Internals/Minor Fixes

Documentation

climpred v1.2.0 (2019-12-17)

Depreciated

  • Abbreviation pval depreciated. Use p_pval for pearson_r_p_value instead. (GH#264) Aaron Spring.

New Features

  • Users can now apply arbitrary xarray methods to HindcastEnsemble and PerfectModelEnsemble. (GH#243) Riley X. Brady.

  • Add “getter” methods to HindcastEnsemble and PerfectModelEnsemble to retrieve xarray datasets from the objects. (GH#243) Riley X. Brady.

    >>> hind = climpred.tutorial.load_dataset('CESM-DP-SST')
    >>> ref = climpred.tutorial.load_dataset('ERSST')
    >>> hindcast = climpred.HindcastEnsemble(hind)
    >>> hindcast = hindcast.add_reference(ref, 'ERSST')
    >>> print(hindcast)
    <climpred.HindcastEnsemble>
    Initialized Ensemble:
        SST      (init, lead, member) float64 ...
    ERSST:
        SST      (time) float32 ...
    Uninitialized:
        None
    >>> print(hindcast.get_initialized())
    <xarray.Dataset>
    Dimensions:  (init: 64, lead: 10, member: 10)
    Coordinates:
    * lead     (lead) int32 1 2 3 4 5 6 7 8 9 10
    * member   (member) int32 1 2 3 4 5 6 7 8 9 10
    * init     (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0
    Data variables:
        SST      (init, lead, member) float64 ...
    >>> print(hindcast.get_reference('ERSST'))
    <xarray.Dataset>
    Dimensions:  (time: 61)
    Coordinates:
    * time     (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015
    Data variables:
        SST      (time) float32 ...
    
  • metric_kwargs can be passed to Metric. (GH#264) Aaron Spring.

Bug Fixes

  • compute_metric() doesn’t drop coordinates from the initialized hindcast ensemble anymore. (GH#258) Aaron Spring.

  • Metric uacc does not crash when ppp negative anymore. (GH#264) Aaron Spring.

  • Update xskillscore to version 0.0.9 to fix all-NaN issue with pearson_r and pearson_r_p_value when there’s missing data. (GH#269) Riley X. Brady.

Internals/Minor Fixes

Documentation

climpred v1.1.0 (2019-09-23)

Features

Bug Fixes

  • Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (GH#232) Aaron Spring

Internals/Minor Fixes

Documentation

climpred v1.0.1 (2019-07-04)

Bug Fixes

Internals/Minor Fixes

climpred v1.0.0 (2019-07-03)

climpred v1.0.0 represents the first stable release of the package. It includes HindcastEnsemble and PerfectModelEnsemble objects to perform analysis with. It offers a suite of deterministic and probabilistic metrics that are optimized to be run on single time series or grids of data (e.g., lat, lon, and depth). Currently, climpred only supports annual forecasts.

Features

  • Bootstrap prediction skill based on resampling with replacement consistently in ReferenceEnsemble and PerfectModelEnsemble. (GH#128) Aaron Spring

  • Consistent bootstrap function for climpred.stats functions via bootstrap_func wrapper. (GH#167) Aaron Spring

  • many more metrics: _msss_murphy, _less and probabilistic _crps, _crpss (GH#128) Aaron Spring

Bug Fixes

Internals/Minor Fixes

Documentation

  • Documentation built extensively in multiple PRs.

climpred v0.3 (2019-04-27)

climpred v0.3 really represents the entire development phase leading up to the version 1 release. This was done in collaboration between Riley X. Brady, Aaron Spring, and Andrew Huang. Future releases will have less additions.

Features

  • Introduces object-oriented system to climpred, with classes ReferenceEnsemble and PerfectModelEnsemble. (GH#86) Riley X. Brady

  • Expands bootstrapping module for perfect-module configurations. (GH#78, GH#87) Aaron Spring

  • Adds functions for computing Relative Entropy (GH#73) Aaron Spring

  • Sets more intelligible dimension expectations for climpred (GH#98, GH#105) Riley X. Brady and Aaron Spring:

    • init: initialization dates for the prediction ensemble

    • lead: retrospective forecasts from prediction ensemble; returned dimension for prediction calculations

    • time: time dimension for control runs, references, etc.

    • member: ensemble member dimension.

  • Updates open_dataset to display available dataset names when no argument is passed. (GH#123) Riley X. Brady

  • Change ReferenceEnsemble to HindcastEnsemble. (GH#124) Riley X. Brady

  • Add probabilistic metrics to climpred. (GH#128) Aaron Spring

  • Consolidate separate perfect-model and hindcast functions into singular functions (GH#128) Aaron Spring

  • Add option to pass proxy through to open_dataset for firewalled networks. (GH#138) Riley X. Brady

Bug Fixes

Internals/Minor Fixes

climpred v0.2 (2019-01-11)

Name changed to climpred, developed enough for basic decadal prediction tasks on a perfect-model ensemble and reference-based ensemble.

climpred v0.1 (2018-12-20)

Collaboration between Riley Brady and Aaron Spring begins.