Chapter 3 · Part 1
Forecasting Metrics
The deepest treatment of forecasting metrics in any modern Python book. Point accuracy, probabilistic scoring (CRPS, quantile loss, Winkler), shape metrics, hierarchical metrics, Forecast Value Added, and a practical selection framework for picking the right metric for the right business question.
Learning objectives
By the end of this chapter you will be able to:
- ▸Choose between point, probabilistic, and shape metrics for a given task
- ▸Compute and interpret CRPS, quantile loss, and the Winkler interval score
- ▸Apply Forecast Value Added (FVA) to justify forecasting investments
- ▸Avoid the most common metric pitfalls (asymmetry of MAPE, division-by-zero, scale mixing)
- ▸Aggregate metrics across horizons and series in defensible ways
- 1.1Point Forecast Accuracy Metrics (Deterministic Forecasts)
- 2.1Forecast Value Added (FVA)
- 3.1Probabilistic Forecast Metrics (Uncertainty & Distributional Accuracy)
- 4.1Temporal Pattern and Shape Metrics
- 5.1Hierarchical and Group Forecast Metrics
- 6.1Using Forecastability to Inform Metrics and Models
- 7.1Aggregating Metrics Across Horizons and Time Series
- 8.1Practical Metric Selection Framework
- 9.1Advanced and Emerging Metrics
- 10.1Metric Trait Reference: Selecting Metrics by Properties
- 11.1Quick Metric Selection Guide
- 12.1Chapter Summary
- 13.1Appendix - Forecast Metric Crosswalk
Get the full chapter
All eleven chapters · 478 references · private Discord community.
← Previous · Chapter 2
The Forecastability of Time Series
Next · Chapter 4 →
ARIMA Modeling