Time-series forecasting: baselines, Holt-Winters and an honest backtest
Forecasting demand, traffic or sales starts with two habits: always compare with a simple baseline, and always test on the most recent period you held back. This simulation forecasts a daily series with a weekly pattern using five classic methods and scores each one honestly.
Level IntermediateAbout 12 minRead first: Linear regression1 challenge in the Lab
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
The grey line is the history; the last 7, 14 or 28 days are held back, and the white line shows what really happened. The blue line is the selected method's forecast with an approximate 80% interval. The panel on the right scores all five methods on the held-back days with MASE: the mean absolute error divided by the in-sample error of the seasonal naive forecast, so below 1 means better than 'same day last week'.
Things to try
- Compare Holt-Winters with seasonal naive on the default series: which has the lower MASE?
- Turn on the level shift: demand drops near the end. Raise the level smoothing α and see Holt-Winters adapt faster.
- Set the horizon to 28 days: the interval widens and the ranking of methods changes.
- Raise the noise: the gap between clever methods and simple baselines shrinks.
- Pick the trend-plus-weekday regression: it fits the calendar pattern well but cannot react to the level shift.
How it works in real systems
Real forecasting systems handle thousands of series at once, add holidays, promotions and weather as features, and use methods from ETS and ARIMA to gradient-boosted trees and neural models. Recent time-series foundation models such as TimesFM and Chronos forecast new series without training. Whatever the method, the evaluation is the same: rolling-origin backtests on the most recent data, compared against seasonal naive.
Key formulas
$$\ell_t=\alpha(y_t-s_{t-m})+(1-\alpha)(\ell_{t-1}+b_{t-1}),\quad b_t=\beta(\ell_t-\ell_{t-1})+(1-\beta)b_{t-1},\quad s_t=\gamma(y_t-\ell_t)+(1-\gamma)s_{t-m}$$
Additive Holt-Winters: level, trend and seasonal components.
$$\mathrm{MASE}=\frac{\operatorname{mean}|y_t-\hat y_t|}{\operatorname{mean}_{\text{train}}|y_t-y_{t-m}|}$$
Error scaled by the in-sample seasonal naive forecast; below 1 beats it.
Common mistakes
- Skipping simple baselines: a model that cannot beat the seasonal naive forecast is not ready.
- Splitting a time series at random: the test period must come after the training period.
- Reporting only point forecasts: planning needs the interval as well.
Real-world numbers
- The M4 competition (2018) compared 61 forecasting methods on 100,000 time series.
- MASE was proposed by Hyndman and Koehler (2006) so that errors can be compared across series of different scales.
- In M4, a hybrid of exponential smoothing and a neural network won, and combinations of statistical methods ranked near the top.
Further reading
- Forecasting: Principles and Practice (Hyndman and Athanasopoulos)
- Another look at measures of forecast accuracy (MASE)
- The M4 Competition: 100,000 time series and 61 forecasting methods
Code lab
The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.
Step 1 · Holt-Winters with statsmodels
Fit additive trend and seasonality, forecast 14 days and compare with seasonal naive.
pip install statsmodels numpy pandas
import numpy as np, pandas as pd
from statsmodels.tsa.holtwinters import ExponentialSmoothing
rng = np.random.default_rng(3)
t = np.arange(154)
weekly = np.array([0, 1.5, 2, 2.2, 3, -2, -4])
y = pd.Series(20 + 0.04 * t + weekly[t % 7] + rng.normal(0, 0.75, len(t)),
index=pd.date_range('2026-01-01', periods=len(t), freq='D'))
train, test = y[:-14], y[-14:]
hw = ExponentialSmoothing(train, trend='add', seasonal='add', seasonal_periods=7).fit()
forecast = hw.forecast(14)
snaive = np.tile(train.values[-7:], 2) # same day last week
Step 2 · Score with MASE
Scale the error by the in-sample error of the seasonal naive forecast.
scale = np.mean(np.abs(train.values[7:] - train.values[:-7]))
mase = lambda f: np.mean(np.abs(test.values - np.asarray(f))) / scale
print('Holt-Winters MASE :', round(mase(forecast), 3))
print('seasonal naive MASE:', round(mase(snaive), 3))
print(hw.params['smoothing_level'], hw.params['smoothing_trend'], hw.params['smoothing_seasonal'])
statsmodels fits α, β and γ by maximum likelihood; the simulation lets you set them by hand to see what each one does.
More simulations
← Recommender (matrix factorisation) · All AI Simulations · Conformal prediction →