What’s New¶
climpred v2.1.0 (2020-04-##)¶
Breaking change¶
- replace keyword bootstrap with iterations (GH#354) Aaron Spring.
- HTML
__repr__
forHindcastEnsemble
andPerfectModelEnsemble
following xarray.core.options.OPTIONS (GH#371) Aaron Spring.
New Features¶
Union with observations only enforced if
reference='persistence'
. (GH#341) Riley X. Brady.HindcastEnsemble.verify()
now takesreference=...
keyword. (GH#341) Riley X. Brady.compute_perfect_model now accepts inits as cftime and integer. Implemented cftime into bootstrap_uninit function, which requires a Leap or NoLeap calendar. (GH#332) Aaron Spring.
speed-up in bootstrapping uninitialized skill in perfect-model for annual leads by reshaping. (GH#332) Aaron Spring.
- speed-up in bootstrap functions: (GH#285) Aaron Spring.
xr.quantile
exchanged fordask.map_blocks(np.percentile)
- properly implemented handling for lazy results when chunked inputs
- user gets warned when chunking potentially (un)-necessary
new explicit keywords in bootstrap functions for
resampling_dim
andreference_compute
(GH#320) Aaron Spring.Logging now included for
compute_hindcast
which displays the inits and verification dates used at each lead (GH#324) Aaron Spring, (GH#338) Riley X. Brady..new explicit keywords added for
alignment
of verification dates and initializations. (GH#324) Aaron Spring.'maximize'
: Maximize the degrees of freedom by slicinghind
andverif
to a common time frame at each lead. (GH#338) Riley X. Brady.'same_inits'
: slice to a common init frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of initializations. (GH#328) Riley X. Brady.'same_verifs'
: slice to a common/consistent verification time frame prior to computing metric. This philosophy follows the thought that each lead should be based on the same set of verification dates. (GH#331) Riley X. Brady.
Bug Fixes¶
- Alignment options now account for differences in the historical time series if
reference='historical'
. (GH#341) Riley X. Brady.
Internals/Minor Fixes¶
- Added our Code of Conduct (GH#285) Aaron Spring.
- Gather all
pytest.fixture``s in ``conftest.py
. (GH#313) Aaron Spring. - Move
x_METRICS
andCOMPARISONS
tometrics.py
andcomparisons.py
in order to avoid circular import dependencies. (GH#315) Aaron Spring. asv
benchmarks forHindcastEnsemble
(GH#285) Aaron Spring.- ignore irrelevant warnings in
pytest
and mark slow tests (GH#333) Aaron Spring. - default
CONCAT_KWARGS
now in allxr.concat
to speed up bootstrapping. (GH#330) Aaron Spring. - Remove
member
coords form2c
comparison for probabilistic metrics. (GH#330) Aaron Spring. - Refactoring
compute_hindcast()
andcompute_perfect_model()
. (GH#330) Aaron Spring. - Changed lead0 coordinate modifications to be compliant with xarray=0.15.1 in
compute_persistence()
. (GH#348) Aaron Spring. - Exchanged my_quantile with xr.quantile(skipna=False). (GH#348) Aaron Spring.
- Remove
sig
fromplot_bootstrapped_skill_over_leadyear()
. (GH#351) Aaron Spring. - Require
xskillscore v0.0.15
and use their functions for effective sample size-based metrics. (:pr: 353) Riley X. Brady. - Faster bootstrapping without replacement used in threshold functions of climpred.stats (GH#354) Aaron Spring.
- Require
cftime v1.1.2
, which modifies their object handling to create 200-400x speedups in some basic operations. (GH#356) Riley X. Brady. - Resample first and then calculate skill in
bootstrap_perfect_model()
andbootstrap_hindcast()
(GH#355) Aaron Spring.
Documentation¶
- Demo and wrapper to setup your own raw model output compliant to
climpred
(GH#296) Aaron Spring. - Demo using
intake-esm
withclimpred
(GH#296) Aaron Spring. - Add Verification Alignment page explaining how initializations are selected and aligned with verification data. (GH#328) Riley X. Brady.
climpred v2.0.0 (2020-01-22)¶
New Features¶
- Add support for
days
,pentads
,weeks
,months
,seasons
for lead time resolution.climpred
now requires alead
attribute “units” to decipher what resolution the predictions are at. (GH#294) Kathy Pegion and Riley X. Brady.
>>> hind = climpred.tutorial.load_dataset('CESM-DP-SST')
>>> hind.lead.attrs['units'] = 'years'
HindcastEnsemble
now has.add_observations()
and.get_observations()
methods. These are the same as.add_reference()
and.get_reference()
, which will be deprecated eventually. The name change clears up confusion, since “reference” is the appropriate name for a reference forecast, e.g. persistence. (GH#310) Riley X. Brady.HindcastEnsemble
now has.verify()
function, which duplicates the.compute_metric()
function. We feel that.verify()
is more clear and easy to write, and follows the terminology of the field. (GH#310) Riley X. Brady.e2o
andm2o
are now the preferred keywords for comparing hindcast ensemble means and ensemble members to verification data, respectively. (GH#310) Riley X. Brady.
Documentation¶
New example pages for subseasonal-to-seasonal prediction using
climpred
. (GH#294) Kathy PegionComparisons page rewritten for more clarity. (GH#310) Riley X. Brady.
Bug Fixes¶
- Fixed m2m broken comparison issue and removed correction (GH#290) Aaron Spring.
Internals/Minor Fixes¶
- Updates to
xskillscore
v0.0.12 to get a 30-50% speedup in compute functions that rely on metrics from there. (GH#309) Riley X. Brady. - Stacking dims is handled by
comparisons
, no need for internal keywordstack_dims
. Thereforecomparison
now takesmetric
as argument instead. (GH#290) Aaron Spring. assign_attrs
now carries dim (GH#290) Aaron Spring.- “reference” changed to “verif” throughout hindcast compute functions. This is more clear, since “reference” usually refers to a type of forecast, such as persistence. (GH#310) Riley X. Brady.
Comparison
objects can now have aliases. (GH#310) Riley X. Brady.
climpred v1.2.1 (2020-01-07)¶
Depreciated¶
mad
no longer a keyword for the median absolute error metric. Users should now usemedian_absolute_error
, which is identical to changes inxskillscore
version 0.0.10. (GH#283) Riley X. Bradypacc
no longer a keyword for the p value associated with the Pearson product-moment correlation, since it is used by the correlation coefficient. (GH#283) Riley X. Bradymsss
no longer a keyword for the Murphy’s MSSS, since it is reserved for the standard MSSS. (GH#283) Riley X. Brady
New Features¶
Metrics
pearson_r_eff_p_value
andspearman_r_eff_p_value
account for autocorrelation in computing p values. (GH#283) Riley X. BradyMetric
effective_sample_size
computes number of independent samples between two time series being correlated. (GH#283) Riley X. BradyAdded keywords for metrics: (GH#283) Riley X. Brady
'pval'
forpearson_r_p_value
['n_eff', 'eff_n']
foreffective_sample_size
['p_pval_eff', 'pvalue_eff', 'pval_eff']
forpearson_r_eff_p_value
['spvalue', 'spval']
forspearman_r_p_value
['s_pval_eff', 'spvalue_eff', 'spval_eff']
forspearman_r_eff_p_value
'nev'
fornmse
Internals/Minor Fixes¶
climpred
now requiresxarray
version 0.14.1 so that thedrop_vars()
keyword used in our package does not throw an error. (GH#276) Riley X. Brady- Update to
xskillscore
version 0.0.10 to fix errors in weighted metrics with pairwise NaNs. (GH#283) Riley X. Brady doc8
added topre-commit
to have consistent formatting on.rst
files. (GH#283) Riley X. Brady- Remove
proper
attribute onMetric
class since it isn’t used anywhere. (GH#283) Riley X. Brady - Add testing for effective p values. (GH#283) Riley X. Brady
- Add testing for whether metric aliases are repeated/overwrite each other. (GH#283) Riley X. Brady
ppp
changed tomsess
, but keywords allow forppp
andmsss
still. (GH#283) Riley X. Brady
Documentation¶
- Expansion of metrics documentation with much more detail on how metrics are computed, their keywords, references, min/max/perfect scores, etc. (GH#283) Riley X. Brady
- Update terminology page with more information on metrics terminology. (GH#283) Riley X. Brady
climpred v1.2.0 (2019-12-17)¶
Depreciated¶
- Abbreviation
pval
depreciated. Usep_pval
forpearson_r_p_value
instead. (GH#264) Aaron Spring.
New Features¶
Users can now pass a custom
metric
orcomparison
to compute functions. (GH#268) Aaron Spring.New deterministic metrics (see metrics). (GH#264) Aaron Spring.
- Spearman ranked correlation (spearman_r)
- Spearman ranked correlation p-value (spearman_r_p_value)
- Mean Absolute Deviation (mad)
- Mean Absolute Percent Error (mape)
- Symmetric Mean Absolute Percent Error (smape)
Users can now apply arbitrary
xarray
methods toHindcastEnsemble
andPerfectModelEnsemble
. (GH#243) Riley X. Brady.Add “getter” methods to
HindcastEnsemble
andPerfectModelEnsemble
to retrievexarray
datasets from the objects. (GH#243) Riley X. Brady.>>> hind = climpred.tutorial.load_dataset('CESM-DP-SST') >>> ref = climpred.tutorial.load_dataset('ERSST') >>> hindcast = climpred.HindcastEnsemble(hind) >>> hindcast = hindcast.add_reference(ref, 'ERSST') >>> print(hindcast) <climpred.HindcastEnsemble> Initialized Ensemble: SST (init, lead, member) float64 ... ERSST: SST (time) float32 ... Uninitialized: None >>> print(hindcast.get_initialized()) <xarray.Dataset> Dimensions: (init: 64, lead: 10, member: 10) Coordinates: * lead (lead) int32 1 2 3 4 5 6 7 8 9 10 * member (member) int32 1 2 3 4 5 6 7 8 9 10 * init (init) float32 1954.0 1955.0 1956.0 1957.0 ... 2015.0 2016.0 2017.0 Data variables: SST (init, lead, member) float64 ... >>> print(hindcast.get_reference('ERSST')) <xarray.Dataset> Dimensions: (time: 61) Coordinates: * time (time) int64 1955 1956 1957 1958 1959 ... 2011 2012 2013 2014 2015 Data variables: SST (time) float32 ...
metric_kwargs
can be passed toMetric
. (GH#264) Aaron Spring.- See
metric_kwargs
under metrics.
- See
Bug Fixes¶
compute_metric()
doesn’t drop coordinates from the initialized hindcast ensemble anymore. (GH#258) Aaron Spring.- Metric
uacc
does not crash whenppp
negative anymore. (GH#264) Aaron Spring. - Update
xskillscore
to version 0.0.9 to fix all-NaN issue withpearson_r
andpearson_r_p_value
when there’s missing data. (GH#269) Riley X. Brady.
Internals/Minor Fixes¶
Rewrote
varweighted_mean_period()
based onxrft
. Changedtime_dim
todim
. Function no longer drops coordinates. (GH#258) Aaron SpringAdd
dim='time'
indpp()
. (GH#258) Aaron SpringComparisons
m2m
,m2e
rewritten to not stack dims into supervector because this is now done inxskillscore
. (GH#264) Aaron SpringAdd
tqdm
progress bar tobootstrap_compute()
. (GH#244) Aaron SpringRemove inplace behavior for
HindcastEnsemble
andPerfectModelEnsemble
. (GH#243) Riley X. BradyAdded tests for chunking with
dask
. (GH#258) Aaron SpringFix test issues with esmpy 8.0 by forcing esmpy 7.1 (GH#269). Riley X. Brady
Rewrote
metrics
andcomparisons
as classes to accomodate custom metrics and comparisons. (GH#268) Aaron Spring
Documentation¶
- Add examples notebook for temporal and spatial smoothing. (GH#244) Aaron Spring
- Add documentation for computing a metric over a specified dimension. (GH#244) Aaron Spring
- Update API to be more organized with individual function/class pages. (GH#243) Riley X. Brady.
- Add page describing the
HindcastEnsemble
andPerfectModelEnsemble
objects more clearly. (GH#243) Riley X. Brady - Add page for publications and helpful links. (GH#270) Riley X. Brady.
climpred v1.1.0 (2019-09-23)¶
Features¶
- Write information about skill computation to netcdf attributes(GH#213) Aaron Spring
- Temporal and spatial smoothing module (GH#224) Aaron Spring
- Add metrics brier_score, threshold_brier_score and crpss_es (GH#232) Aaron Spring
- Allow compute_hindcast and compute_perfect_model to specify which dimension dim to calculate metric over (GH#232) Aaron Spring
Bug Fixes¶
- Correct implementation of probabilistic metrics from xskillscore in compute_perfect_model, bootstrap_perfect_model, compute_hindcast and bootstrap_hindcast, now requires xskillscore>=0.05 (GH#232) Aaron Spring
Internals/Minor Fixes¶
- Rename .stats.DPP to dpp (GH#232) Aaron Spring
- Add matplotlib as a main dependency so that a direct pip installation works (GH#211) Riley X. Brady.
climpred
is now installable from conda-forge (GH#212) Riley X. Brady.- Fix erroneous descriptions of sample datasets (GH#226) Riley X. Brady.
- Benchmarking time and peak memory of compute functions with asv (GH#231) Aaron Spring
Documentation¶
- Add scope of package to docs for clarity for users and developers. (GH#235) Riley X. Brady.
climpred v1.0.1 (2019-07-04)¶
Bug Fixes¶
- Accomodate for lead-zero within the
lead
dimension (GH#196) Riley X. Brady. - Fix issue with adding uninitialized ensemble to
HindcastEnsemble
object (GH#199) Riley X. Brady. - Allow
max_dof
keyword to be passed tocompute_metric
andcompute_persistence
forHindcastEnsemble
(GH#199) Riley X. Brady.
Internals/Minor Fixes¶
- Force
xskillscore
version 0.0.4 or higher to avoidImportError
(GH#204) Riley X. Brady. - Change
max_dfs
keyword tomax_dof
(GH#199) Riley X. Brady. - Add testing for
HindcastEnsemble
andPerfectModelEnsemble
(GH#199) Riley X. Brady
climpred v1.0.0 (2019-07-03)¶
climpred
v1.0.0 represents the first stable release of the package. It includes
HindcastEnsemble
and PerfectModelEnsemble
objects to perform analysis with.
It offers a suite of deterministic and probabilistic metrics that are optimized to be
run on single time series or grids of data (e.g., lat, lon, and depth). Currently,
climpred
only supports annual forecasts.
Features¶
- Bootstrap prediction skill based on resampling with replacement consistently in
ReferenceEnsemble
andPerfectModelEnsemble
. (GH#128) Aaron Spring - Consistent bootstrap function for
climpred.stats
functions viabootstrap_func
wrapper. (GH#167) Aaron Spring - many more metrics:
_msss_murphy
,_less
and probabilistic_crps
,_crpss
(GH#128) Aaron Spring
Bug Fixes¶
compute_uninitialized
now trims input data to the same time window. (GH#193) Riley X. Bradyrm_poly
now properly interpolates/fills NaNs. (GH#192) Riley X. Brady
Internals/Minor Fixes¶
- The
climpred
version can be printed. (GH#195) Riley X. Brady - Constants are made elegant and pushed to a separate module. (GH#184) Andrew Huang
- Checks are consolidated to their own module. (GH#173) Andrew Huang
Documentation¶
- Documentation built extensively in multiple PRs.
climpred v0.3 (2019-04-27)¶
climpred
v0.3 really represents the entire development phase leading up to the
version 1 release. This was done in collaboration between Riley X. Brady,
Aaron Spring, and Andrew Huang. Future releases will have less additions.
Features¶
Introduces object-oriented system to
climpred
, with classesReferenceEnsemble
andPerfectModelEnsemble
. (GH#86) Riley X. BradyExpands bootstrapping module for perfect-module configurations. (GH#78, GH#87) Aaron Spring
Adds functions for computing Relative Entropy (GH#73) Aaron Spring
Sets more intelligible dimension expectations for
climpred
(GH#98, GH#105) Riley X. Brady and Aaron Spring:init
: initialization dates for the prediction ensemblelead
: retrospective forecasts from prediction ensemble; returned dimension for prediction calculationstime
: time dimension for control runs, references, etc.member
: ensemble member dimension.
Updates
open_dataset
to display available dataset names when no argument is passed. (GH#123) Riley X. BradyChange
ReferenceEnsemble
toHindcastEnsemble
. (GH#124) Riley X. BradyAdd probabilistic metrics to
climpred
. (GH#128) Aaron SpringConsolidate separate perfect-model and hindcast functions into singular functions (GH#128) Aaron Spring
Add option to pass proxy through to
open_dataset
for firewalled networks. (GH#138) Riley X. Brady
Bug Fixes¶
xr_rm_poly
can now operate on Datasets and with multiple variables. It also interpolates across NaNs in time series. (GH#94) Andrew Huang- Travis CI,
treon
, andpytest
all run for automated testing of new features. (GH#98, GH#105, GH#106) Riley X. Brady and Aaron Spring - Clean up
check_xarray
decorators and make sure that they work. (GH#142) Andrew Huang - Ensures that
help()
returns proper docstring even with decorators. (GH#149) Andrew Huang - Fixes bootstrap so p values are correct. (GH#170) Aaron Spring
Internals/Minor Fixes¶
- Adds unit testing for all perfect-model comparisons. (GH#107) Aaron Spring
- Updates CESM-LE uninitialized ensemble sample data to have 34 members. (GH#113) Riley X. Brady
- Adds MPI-ESM hindcast, historical, and assimilation sample data. (GH#119) Aaron Spring
- Replaces
check_xarray
with a decorator for checking that input arguments are xarray objects. (GH#120) Andrew Huang - Add custom exceptions for clearer error reporting. (GH#139) Riley X. Brady
- Remove “xr” prefix from stats module. (GH#144) Riley X. Brady
- Add codecoverage for testing. (GH#152) Riley X. Brady
- Update exception messages for more pretty error reporting. (GH#156) Andrew Huang
- Add
pre-commit
andflake8
/black
check in CI. (GH#163) Riley X. Brady - Change
loadutils
module totutorial
andopen_dataset
toload_dataset
. (GH#164) Riley X. Brady - Remove predictability horizon function to revisit for v2. (GH#165) Riley X. Brady
- Increase code coverage through more testing. (GH#167) Aaron Spring
- Consolidates checks and constants into modules. (GH#173) Andrew Huang
climpred v0.2 (2019-01-11)¶
Name changed to climpred
, developed enough for basic decadal prediction tasks on a
perfect-model ensemble and reference-based ensemble.
climpred v0.1 (2018-12-20)¶
Collaboration between Riley Brady and Aaron Spring begins.