Development Page--Not for Official Use

Performance Analysis

The performance of each model is assessed based on criteria used by NOAA for the development and implementation of operational nowcast and forecast systems (NOAA, 1999). A single forecasting error or ei is defined as the difference between the predicted value pi and the observed value ri or ei = pi-ri. The models are assessed by averaging the individual errors over the full data sets, typically one year of water level measurements. The skill assessment variables used are the following:

Average error: Eavg = (1/N) S ei

Absolute Average Error: ½Eavg½ = (1/N) S ½ei½

Root Mean Square Error: Erms = ((1/N) S ei2)1/2

POF(X) – Positive Outlier Frequency or percentage of the forecasts X cm or more above the actual measurement.

NOF(X) – Negative Outlier Frequency or percentage of the forecasts X cm or more below the actual measurement.

MDPO(X) – Maximum Duration of Positive Outlier.

MDNO(X) – Maximum Duration of Negative Outlier.

The value defining an outlier is set at 15 cm for this study. An X=15 cm requirement limits water level errors to within +/- ½ foot and is based on NOAA’s estimates of pilots’ needs for under keel clearance. Additionally a skill assessment variable, the Normalized RMS Error is defined to compare model performance at different locations (Cox et al., 2002a). The root mean square of the error is divided by the root mean square of the signal to normalize the error with the variability of the signal. Root Mean Square Signal: Rrms = ((1/N) S ri2)1/2 Normalized RMS Error: NE = Erms/Rrms

<< PersistenceModel | ForecastsSiteMap | SkillAssessment >>