The numbers you will see on the next page will seem very useless until
you understand how to read them and what they mean. Only one of them
is actually used to come up with the rankings. The others are for your
personal knowledge to let you know where you may need to improve.
There are three things being calculated for each forecaster. They are
the average absolute error, average bias, and normalized error. Each
is done for Day 1, Day 2, and Day 3 forecasts as well as for the overall
total.
Average absolute error is calculated by summing up the absolute value of
the differences between your forecast and the observed values for that day
and then dividing by the number of forecasts. Obviously, the higher this
this number is the further away you are. For any of the numerical paramters
(i.e. temperature, humidity, wind speed, mslp, and heat idex or wind chill)
the absolute value of the difference is used. For all of the other parameters
except for precipitation it is simply a right or wrong calculation. For example
if you forecast the cloud cover as scattered and it turns out to be scattered
then nothing is added on to your accumulated score. However, if it is actually
overcast then one point will be added to the score. For precip. the calculation
is a little more involved. This calculation punishes you based on the percent
chance that you put into each "bucket" of total precip. The buckets are numbered
from 1 to 5. The points added are found as follows. The total precipitation is
put into its correct bucket. The difference between the actual bucket and your
forecast bucket is multiplied by the percentage that you forecast in the wrong
bucket. This result is then divided by ten and gets added to the accumulated
error. For example if there is no rain observed and you forecast a 50% chance
of no rain, a 10% chance of a trace to .1", a 20% chance of .11" to .25", and a
20% chance of .26" to .5" then you would have eleven points added on. This
number is not useful on its own but provides the basis for the normalized error.
The average bias is figured using only the numerical parameters and doesn't deal
with the absolute value. From this number you can see if you tend to be too high
or too low with these parameters and by about how much.
The actual rankings are found by using the normalized error. The average absolute
error value is normalized based on the persistence forecast so that everyone is on
a level playing field regardless of the number of forecasts made. The persistence
forecast score is always zero. If your score is negative then you are doing
better than persistence and vice versa. The more negative your score is then the
higher ranked you are. This cannot effectively be used as an actual accuracy
measurement because your score will fluctuate based not only on your forecasting
accuracy but also on the persistence accuracy.
If you have any questions or concerns about this process let me know and I'll try
to make it more clear.