**********************
Prediction Terminology
**********************

Terminology is often confusing and highly variable amongst those that make predictions
in the geoscience community. Here we define some common terms in climate prediction and
how we use them in ``climpred``.

Simulation Design
#################

*Hindcast Ensemble* (:py:class:`.HindcastEnsemble`):
Ensemble members are initialized from a simulation (generally a reconstruction from
reanalysis) or an analysis (representing the current state of the atmosphere, land, and
ocean by assimilation of observations) at initialization dates and integrated for some
lead years :cite:p:`Boer2016`.

*Perfect Model Experiment* (:py:class:`.PerfectModelEnsemble`):
Ensemble members are initialized from a control simulation
(:py:meth:`.PerfectModelEnsemble.add_control`) at randomly chosen
initialization dates and integrated for some lead years :cite:p:`Griffies1997`.

*Reconstruction/Assimilation*: (:py:meth:`.HindcastEnsemble.add_observations`)
A "reconstruction" is a model solution that uses
observations in some capacity to approximate historical or current conditions of the
atmosphere, ocean, sea ice, and/or land. This could be done via a forced simulation,
such as an OMIP run that uses a dynamical ocean/sea ice core with reanalysis forcing
from atmospheric winds. This could also be a fully data assimilative model, which
assimilates observations into the model solution.  For weather, subseasonal, and
seasonal predictions, the terms re-analysis and analysis are the terms typically used,
while reconstruction is more commonly used for decadal predictions.

*Uninitialized Ensemble*: (:py:meth:`.HindcastEnsemble.add_uninitialized`)
In this framework, an *uninitialized ensemble* is one that
is generated by perturbing initial conditions only at one point in the historical run.
These are generated via micro (round-off error perturbations) or macro (starting from
completely different restart files) methods. Uninitialized ensembles are used to
approximate the magnitude of internal climate variability and to confidently extract
the forced response (ensemble mean) in the climate system. In ``climpred``, we use
uninitialized ensembles as a baseline for how important (reoccurring) initializations
are for lending predictability to the system. Some modeling centers (such as NCAR)
provide a dynamical uninitialized ensemble (the CESM Large Ensemble) along with their
initialized prediction system (the CESM Decadal Prediction Large Ensemble). If this
isn't available, one can approximate the unintiailized response by bootstrapping a
control simulation.

Forecast Assessment
###################

*Accuracy*: The average degree of correspondence between individual pairs of forecasts
and observations :cite:p:`Murphy1988,Jolliffe2011`. Examples include Mean Absolute Error
(MAE) :py:func:`~.climpred.metrics._mae` and Mean Square Error (MSE)
:py:func:`~.climpred.metrics._mse`. See `metrics <metrics.html>`_.

*Association*: The overall strength of the relationship between individual pairs of
forecasts and observations :cite:p:`Jolliffe2011`. The primary measure of association
is the Anomaly Correlation Coefficient (ACC), which can be measured using the Pearson
product-moment correlation :py:func:`~.climpred.metrics._pearson_r` or
Spearman's Rank correlation :py:func:`~.climpred.metrics._spearman_r`. See
`metrics <metrics.html>`_.

*(Potential) Predictability*: This characterizes the "ability to be predicted"
rather than the current "capability to predict." One estimates this by computing a
metric (like the anomaly correlation coefficient (ACC)) between the prediction
ensemble and a member (or collection of members) selected as the verification member(s)
(in a perfect-model setup) or the reconstruction that initialized it
(in a hindcast setup) :cite:p:`Meehl2013,Pegion2019`.

*(Prediction) Skill*: (:py:meth:`.HindcastEnsemble.verify`)
This characterizes the current ability of the ensemble
forecasting system to predict the real world. This is derived by computing a metric
between the prediction ensemble and observations, reanalysis, or analysis of the real
world :cite:p:`Meehl2013,Pegion2019`.

*Skill Score*: The most generic skill score can be defined as the following
:cite:t:`Murphy1988`:

.. math::
    S = \frac{A_{f} - A_{r}}{A_{p} - A_{r}},

where :math:`A_{f}`, :math:`A_{p}`, and :math:`A_{r}` represent the accuracy of the
forecast being assessed, the accuracy of a perfect forecast, and the accuracy of the
reference forecast (e.g. persistence), respectively :cite:p:`Murphy1985`. Here,
:math:`S` represents the improvement in accuracy of the forecasts over the reference
forecasts relative to the total possible improvement in accuracy. They are typically
designed to take a value of 1 for a perfect forecast and 0 for equivalent to the
reference forecast :cite:p:`Jolliffe2011`.

Forecasting
###########

*Hindcast*: Retrospective forecasts of the past initialized from a reconstruction
integrated forward in time, also called re-forcasts.  Depending on the length of time
of the integration, external forcings may or may not be included.  The longer the
integration (e.g. decadal vs. daily), the more important it is to include external
forcing :cite:p:`Boer2016`.  Because they represent so-called forecasts over periods
that already occurred, their prediction skill can be evaluated.

*Prediction*: Forecasts initialized from a reconstruction integrated into the future.
Depending on the length of time of the integration, external forcings may or may not
be included.  The longer the integration (e.g. decadal vs. daily), the more important
it is to include external forcing :cite:p:`Boer2016`. Because predictions are made into
the future, it is necessary to wait until the forecast occurs before one can quantify
the skill of the forecast.

*Projection* An estimate of the future climate that is dependent on the externally
forced climate response, such as anthropogenic greenhouse gases, aerosols, and
volcanic eruptions :cite:p:`Meehl2013`.

References
##########

.. bibliography::
  :filter: docname in docnames