Skip to content
Chapter 6 · Part 3

State-of-the-Art Performance Evaluation

Forecast stability, drift detection, rolling-origin (walk-forward) cross-validation, blocked CV, and the techniques that make CV honest on dependent data — purging and embargoing.

Learning objectives

By the end of this chapter you will be able to:

  • ▸Implement rolling-origin cross-validation correctly
  • ▸Apply purging and embargoing to prevent leakage in dependent data
  • ▸Detect concept drift in production forecasts
  • ▸Run statistically valid model-comparison tests (Diebold-Mariano, MCS)
  • 1.1Introduction: The Imperative of Robust Forecasting Evaluation
  • 2.1Foundational Concepts of Robust Forecasting Performance
  • 3.1Forecast Stability in Time Series Forecasting: Models, Metrics, and Industry Perspectives
  • 4.1Drift Detection in Forecasting
  • 5.1Rolling-Origin Cross-Validation (Walk-Forward Validation)
  • 6.1Blocked Cross-Validation
  • 7.1Techniques for Dependent Data: Purging and Embargoing
  • 8.1Advanced Model Comparison and Selection Techniques
  • 9.1Chapter Summary and Conclusions
Get the full chapter
All eleven chapters · 478 references · private Discord community.
← Previous · Chapter 5
Forecasting with Exponential Smoothing
Next · Chapter 7 →
Feature Engineering for Time Series