WL Model Comparisons
You will find listed in the tables below performance comparisons for several water level forecasting models. The models are Harmonic Forecasts (HR) (also known as tide charts), Persistent Model (Pers), Linear Regression Model (LR), and Neural Networks Model (NN). The comparisons are for 24 hour forecasts and 48 hour forecasts. 2 benchmark tests are presented in the tables. The first test is labeled Operational Model Performance Comparisons and the second test is labeled Average Model Performance Comparisons. Read below for more details on the tests. The models are compared using the root mean square error (RMSE) and the following NOS criteria: Central Frequency (15 cm), Positive Outlier Frequency (30 cm), Negative Outlier Frequency (30 cm), Maximum Duration of Positive Outliers (30 cm) and Maximum Duration of Negative Outliers (30 cm).
TODO: add units for skill variables
NOS Procedures for Developing and Implementing Operational Nowcast and Forecast Systems for PORTS
OPERATIONAL MODEL PERFORMANCE COMPARISONS:
This test compares the performance of the models when applied to the three year span starting January 1st 2000 and ending December 31st 2002. The models are all trained and optimized over data originating before January 1st 2000. Predictions for the Harmonic, Persistence, and Neural Network models are updated at least hourly for all the stations below.
Bob Hall Pier (014)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.114 | 84.22 | 0.351 | 1.648 | 17 | 54 |
| Pers 24hr | 0.086 | 91.96 | 0.454 | 0.166 | 8 | 8 |
| LR 24hr | 0.224 | 93.23 | 0.330 | 0.166 | 17 | 16 |
| NN 24hr | 0.075 | 94.64 | 0.301 | 0.104 | 8 | 6 |
| Pers 48hr | 0.114 | 84.30 | 1.522 | 0.879 | 30 | 14 |
| LR 48hr | 0.188 | 76.63 | 0.726 | | 23 | 46 |
| NN 48hr | 0.102 | 88.22 | 0.902 | 0.570 | 22 | 16 |
Rockport (015)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.116 | 81.55 | 0.08 | 1.73 | 6 | 60 |
| Pers 24hr | 0.064 | 96.53 | 0.08 | 0.00 | 5 | 0 |
| LR 24hr | 0.074 | 77.86 | 0.163 | 0.00 | 6 | 0 |
| NN 24hr | 0.050 | 98.72 | 0.02 | 0.00 | 2 | 0 |
| Pers 48hr | 0.095 | 88.57 | 0.40 | 0.28 | 14 | 16 |
| LR 48hr | 0.103 | 82.54 | 0.367 | 0.230 | 14 | 17 |
| NN 48hr | 0.082 | 92.85 | 0.21 | 0.06 | 9 | 2 |
Port Aransas (009)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.112 | 83.73 | 0.31 | 1.53 | 19 | 43 |
| Pers 24hr | 0.075 | 94.18 | 0.25 | 0.03 | 9 | 0 |
| LR 24hr | 0.172 | 94.85 | 1.053 | 0.016 | 24 | 2 |
| NN 24hr | 0.070 | 95.68 | 0.16 | 0.02 | 7 | 0 |
| Pers 48hr | 0.103 | 86.98 | 1.14 | 0.41 | 20 | 10 |
| LR 48hr | 0.208 | 89.76 | 1.757 | 0.256 | 48 | 36 |
| NN 48hr | 0.093 | 89.89 | 0.51 | 0.24 | 23 | 9 |
Sabine Pass ()
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | | | | | | |
| Pers 24hr | | | | | | |
| LR 24hr | | | | | | |
| NN 24hr | | | | | | |
| Pers 48hr | | | | | | |
| LR 48hr | | | | | | |
| NN 48hr | | | | | | |
Packery Channel / JFK Causeway (005)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.108 | 85.14 | 0.00 | 2.31 | 0 | 71 |
| Pers 24hr | 0.055 | 97.48 | 0.06 | 0.01 | 5 | 0 |
| LR 24hr | 0.101 | 97.31 | 0.713 | 0.002 | 24 | 0 |
| NN 24hr | 0.044 | 99.20 | 0.00 | 0.00 | 0 | 0 |
| Pers 48hr | 0.078 | 93.02 | 0.25 | 0.12 | 12 | 7 |
| LR 48hr | 0.124 | 93.27 | 1.115 | 0.161 | 48 | 22 |
| NN 48hr | 0.068 | 96.00 | 0.12 | 0.06 | 5 | 5 |
Pleasure Pier (021)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.149 | 72.81 | 1.41 | 3.39 | 28 | 72 |
| Pers 24hr | 0.146 | 79.65 | 3.07 | 2.29 | 25 | 29 |
| LR 24hr | 0.149 | 83.69 | 2.428 | 1.314 | 26 | 34 |
| NN 24hr | 0.123 | 84.60 | 2.34 | 0.80 | 22 | 20 |
| Pers 48hr | 0.172 | 71.20 | 4.49 | 3.80 | 39 | 33 |
| LR 48hr | 0.137 | 71.47 | 2.501 | 1.596 | 35 | 47 |
| NN 48hr | 0.140 | 79.05 | 2.38 | 2.38 | 38 | 24 |
Port O'Connor (057)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.119 | 80.56 | 0.83 | 1.23 | 19 | 46 |
| Pers 24hr | 0.087 | 91.11 | 0.60 | 0.09 | 14 | 5 |
| LR 24hr | 0.338 | 91.92 | 1.416 | 0.106 | 33 | 6 |
| NN 24hr | 0.070 | 95.58 | 0.13 | 0.10 | 7 | 4 |
| Pers 48hr | 0.117 | 82.58 | 1.57 | 0.90 | 31 | 24 |
| LR 48hr | 0.346 | 68.55 | 2.016 | 0.311 | 38 | 28 |
| NN 48hr | 0.099 | 88.15 | 0.85 | 0.52 | 18 | 11 |
Port Isabel (018)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.093 | 90.26 | 0.16 | 0.93 | 8 | 31 |
| Pers 24hr | 0.069 | 95.62 | 0.14 | 0.05 | 8 | 3 |
| LR 24hr | 0.078 | 96.42 | 0.175 | 0.039 | 17 | 5 |
| NN 24hr | 0.058 | 98.16 | 0.02 | 0.03 | 1 | 2 |
| Pers 48hr | 0.090 | 90.77 | 0.65 | 0.14 | 17 | 6 |
| LR 48hr | 0.114 | 93.14 | 0.598 | 0.229 | 27 | 44 |
| NN 48hr | 0.077 | 94.35 | 0.27 | 0.15 | 11 | 12 |
Morgans Point (503)
| Model | RMSE | CF | POF | NOF | MDPO | MDNO |
| Harmonic | 0.174 | 67.31 | 3.88 | 4.56 | 47 | 74 |
| Pers 24hr | 0.178 | 71.24 | 5.29 | 4.18 | 31 | 34 |
| LR 24hr | 0.110 | 55.61 | 2.857 | 1.328 | 18 | 22 |
| NN 24hr | 0.142 | 80.36 | 4.13 | 0.68 | 26 | 12 |
| Pers 48hr | 0.219 | 61.64 | 7.83 | 7.22 | 47 | 44 |
| LR 48hr | 0.173 | 64.54 | 4.742 | 3.024 | 51 | 65 |
| NN 48hr | 0.175 | 71.35 | 6.26 | 1.69 | 59 | 21 |