event
Thursday 06 Dec 2018: Statistical Science -Hailang Du "Beyond Strictly Proper Scores: The Importance of Being Local"
Dr Hailiang Du - Durham University
H101 14:30-15:30
The evaluation of probabilistic forecasts plays a central role both in the interpretation and in the use of forecast systems and their development. Probabilistic scores provide statistical measures to assess the quality of probabilistic forecasts. Often, many probabilistic forecast systems are available while evaluations of their performance are not standardized, with different scores being used to measure different aspects of forecast performance. Even when the discussion is restricted to strictly proper scores, there remains considerable variability between scores; indeed strictly proper scores need not rank competing forecast systems in the same order when none of these systems are perfect. The locality property is explored to further distinguish skill scores. The only local strictly proper score, the logarithmic score, has an immediate interpretation in terms of bits of information. The interpretation of nonlocal strictly proper scores, on the other hand, relies on information regarding the unknown (if it even exists) True underlying distribution. The nonlocal strictly proper scores considered are shown to have properties that can produce "unfortunate" evaluations. It is therefore suggested that the logarithmic score always be included in the evaluation of probabilistic forecasts.
