Predicting future outcomes is one of the fundamental goals of statistical analysis. Predictions for events with significant inherent uncertainty should be probabilistic in nature to convey information on the uncertainty associated with the outcome. This is particularly of importance when the prediction problems involves the prediction of a risk measure or a functional of the outcome distribution. For extreme events or risks, the evaluation of the prediction falls in three distinct categories, depending on the question being asked:
A probabilistic forecast is issued for the extremes only and we want to know how good it is;
A probabilistic forecast is issued for every type of outcome, and we want to know how good it is at predicting extreme outcomes;
A probabilistic forecast is issued for every type of outcome, and we want to know how well certain tail properties or functionals of the predictive distribution match those of the true data distribution.
When predicting extreme events and assessing risk, the evaluation of the forecasts is additionally complicated by a lack of substantial observation set due to the rarity of the outcome of interest. We discuss how to perform the evaluation for all three categories above under these constraints within the frameworks of proper scoring rules and consistent scoring functions. For functional predictions, a well-matched scoring function may not exist as is the case, for example, for the variance functional, making the functional non-elicitable. For this case, we present some new results that show that non-elicitibility of the predicted functional may be solved by instead evaluating against the observed, or the estimated, functional rather than observed outcomes if sufficient data is available.