Skip to content
Chapter 3 · Part 1

Forecasting Metrics

The deepest treatment of forecasting metrics in any modern Python book. Point accuracy, probabilistic scoring (CRPS, quantile loss, Winkler), shape metrics, hierarchical metrics, Forecast Value Added, and a practical selection framework for picking the right metric for the right business question.

Learning objectives

By the end of this chapter you will be able to:

  • ▸Choose between point, probabilistic, and shape metrics for a given task
  • ▸Compute and interpret CRPS, quantile loss, and the Winkler interval score
  • ▸Apply Forecast Value Added (FVA) to justify forecasting investments
  • ▸Avoid the most common metric pitfalls (asymmetry of MAPE, division-by-zero, scale mixing)
  • ▸Aggregate metrics across horizons and series in defensible ways
  • 1.1Point Forecast Accuracy Metrics (Deterministic Forecasts)
  • 2.1Forecast Value Added (FVA)
  • 3.1Probabilistic Forecast Metrics (Uncertainty & Distributional Accuracy)
  • 4.1Temporal Pattern and Shape Metrics
  • 5.1Hierarchical and Group Forecast Metrics
  • 6.1Using Forecastability to Inform Metrics and Models
  • 7.1Aggregating Metrics Across Horizons and Time Series
  • 8.1Practical Metric Selection Framework
  • 9.1Advanced and Emerging Metrics
  • 10.1Metric Trait Reference: Selecting Metrics by Properties
  • 11.1Quick Metric Selection Guide
  • 12.1Chapter Summary
  • 13.1Appendix - Forecast Metric Crosswalk
Get the full chapter
All eleven chapters · 478 references · private Discord community.
← Previous · Chapter 2
The Forecastability of Time Series
Next · Chapter 4 →
ARIMA Modeling