Chapter 6 · Part 3
State-of-the-Art Performance Evaluation
Forecast stability, drift detection, rolling-origin (walk-forward) cross-validation, blocked CV, and the techniques that make CV honest on dependent data — purging and embargoing.
Learning objectives
By the end of this chapter you will be able to:
- ▸Implement rolling-origin cross-validation correctly
- ▸Apply purging and embargoing to prevent leakage in dependent data
- ▸Detect concept drift in production forecasts
- ▸Run statistically valid model-comparison tests (Diebold-Mariano, MCS)
- 1.1Introduction: The Imperative of Robust Forecasting Evaluation
- 2.1Foundational Concepts of Robust Forecasting Performance
- 3.1Forecast Stability in Time Series Forecasting: Models, Metrics, and Industry Perspectives
- 4.1Drift Detection in Forecasting
- 5.1Rolling-Origin Cross-Validation (Walk-Forward Validation)
- 6.1Blocked Cross-Validation
- 7.1Techniques for Dependent Data: Purging and Embargoing
- 8.1Advanced Model Comparison and Selection Techniques
- 9.1Chapter Summary and Conclusions
Get the full chapter
All eleven chapters · 478 references · private Discord community.
← Previous · Chapter 5
Forecasting with Exponential Smoothing
Next · Chapter 7 →
Feature Engineering for Time Series