ARIMA

Overview

ARIMA is a classical time-series model. The name stands for AutoRegressive Integrated Moving Average. It forecasts a series by combining three ideas: the next value depends on recent past values (autoregression), the series may need differencing to remove trends (integration), and the next value also depends on recent forecast errors (moving average). It is a standard baseline for short-horizon forecasts of a single series.

ARIMA is valued because it is compact, well understood and often hard to beat on small, regular series. It is also limited. It models one series at a time, assumes the pattern is stable after differencing and does not use extra information unless you add it. This article explains ARIMA in plain terms, walks through a full example, compares it with Prophet, N-BEATS, the Temporal Fusion Transformer, PatchTST and Chronos, and shows how it relates to modern models that learn from raw tables, as discussed on the model research page.

What it is

ARIMA is a statistical model for a sequence of numbers observed over time, such as daily sales, hourly temperature or monthly passenger counts. It has three integer settings, written as (p, d, q).

• p is the number of past values used by the autoregressive part.
• d is the number of times the series is differenced.
• q is the number of past forecast errors used by the moving average part.

So ARIMA(2, 1, 1) uses two lagged values, differences the series once and uses one lagged error.

The three parts in everyday words

Autoregression (AR) says that today looks like a blend of the last few days. If it was warm yesterday and the day before, it is probably warm today. The model learns how much weight to give each recent day.

Integration (I) means differencing. Instead of modelling the level, model the change from one step to the next. If sales climb steadily, the level keeps rising and never settles. But the day-to-day change may hover around a stable value. Differencing turns a climbing series into a steady one. Think of walking up a gentle ramp: your height keeps changing, but your step length stays about the same.

Moving average (MA) says that today also depends on the recent surprises. If yesterday's forecast was too low by a lot, today's value may also be a little higher than the simple pattern suggests. The model learns how much of recent surprises carries forward. The name is confusing, because it is not a plain moving average of values. It is a weighted sum of past errors.

Stationarity

A series is stationary when its basic statistics, such as the mean and variance, do not change over time, and its relationship between nearby points stays the same. ARIMA's AR and MA parts work on stationary data. That is why differencing exists: it removes trends so the rest can be modelled. If the variance grows with the level, a log or similar transform before modelling often helps.

Variants

Where it came from

The roots of ARIMA lie in the early twentieth century work on autoregressive models for time series. The approach was brought together and popularised in the 1970s by George Box and Gwilym Jenkins in their book on time series analysis. Their step-by-step procedure, often called the Box-Jenkins method, covers identification, estimation and diagnostic checking. The method remains the template for classical forecasting.

ARIMA became a standard tool in economics, supply planning, finance and engineering. Its strength was clear: with a small number of parameters, it produced forecasts and prediction intervals with solid statistical theory behind them. Seasonal extensions followed to handle patterns that repeat on a calendar cycle.

Later, other families arrived. Exponential smoothing methods offered a related, sometimes simpler, route. State-space formulations unified many of these models. In the 2010s, practical tools such as Prophet focused on easy handling of holidays and trend changes. In the late 2010s and after, neural methods such as N-BEATS, the Temporal Fusion Transformer and PatchTST were proposed, and more recently pre-trained forecasting models such as Chronos appeared. ARIMA is still the baseline they are compared with.

How it works

Step 1: plot and inspect the series

Look at the plot. Is there a trend? A repeating season? Sudden jumps? Missing points? Changing variance? Your eyes catch a lot that tests miss.

Step 2: stabilise the variance

If swings get larger as the level grows, apply a transform such as the logarithm. Then differences represent percentage-like changes.

Step 3: difference until stationary

Take the first difference: new value minus previous value. Check again. If a trend remains, difference again, though d = 2 is rare and d above 2 almost never used. For seasonal data, take a seasonal difference, such as the value minus the value one year earlier. Over-differencing is harmful, because it adds noise.

Formal tests exist. The augmented Dickey-Fuller test checks for a unit root, which is a form of non-stationarity. The KPSS test has the opposite null. Using both gives a fuller picture.

Step 4: inspect the ACF and PACF

Two plots guide the choice of p and q.

• The autocorrelation function (ACF) shows the correlation between the series and its own lagged copy at each lag.
• The partial autocorrelation function (PACF) shows the correlation at a lag after removing the effect of shorter lags.

Rules of thumb for a stationary series:

These are guides, not laws. Real data are messy, and several candidate models usually deserve a try.

Step 5: estimate the parameters

Given (p, d, q), software estimates the coefficients, usually by maximum likelihood. The result is a set of AR coefficients, MA coefficients, a variance of the errors and possibly a constant.

Step 6: compare candidates

Use information criteria such as AIC and BIC. Both reward fit and penalise complexity. Lower is better. Compare models only on the same differenced series. Also compare out-of-sample accuracy, which is the real test.

Step 7: check the residuals

If the model captured the structure, the residuals look like random noise. They should have no leftover autocorrelation, roughly constant variance and no strong pattern. The Ljung-Box test checks for leftover autocorrelation. If residuals still carry structure, revise the model.

Step 8: forecast

The model produces point forecasts and prediction intervals. For differenced models, the forecasts are transformed back to the original level. Intervals widen as the horizon grows, reflecting increasing uncertainty.

An analogy for the whole process

Imagine predicting tomorrow's water level in a pond. AR says the level tomorrow resembles the last few days. I says that if the pond slowly rises with the season, track how much it rises each day rather than the level itself. MA says that if a recent storm gave a surprise bump, some of that bump lingers. ARIMA mixes all three, and the Box-Jenkins steps help you decide how much of each.

The equation in words

An ARMA model says: the value now equals a constant, plus a weighted sum of the last p values, plus a weighted sum of the last q errors, plus a fresh error. With differencing, the same statement is applied to the changes instead of the levels. The coefficients are the weights found in estimation.

Key concepts and terms

A worked example

The following code fits a seasonal ARIMA to a monthly series. File names are placeholders.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.statespace.sarimax import SARIMAX
from statsmodels.tsa.stattools import adfuller
from statsmodels.stats.diagnostic import acorr_ljungbox

df = pd.read_csv("monthly_demand.csv", parse_dates=["month"], index_col="month")
y = np.log(df["units"])            # stabilise variance
y = y.asfreq("MS")                 # monthly start frequency

# 1. Check stationarity after differencing
d1 = y.diff().dropna()
print("ADF p-value (first difference):", adfuller(d1)[1])

# 2. Hold out the last 12 months
train, test = y[:-12], y[-12:]

# 3. Fit a seasonal model: ARIMA(1,1,1) with seasonal (0,1,1,12)
model = SARIMAX(
    train,
    order=(1, 1, 1),
    seasonal_order=(0, 1, 1, 12),
    enforce_stationarity=False,
    enforce_invertibility=False,
)
fit = model.fit(disp=False)
print(fit.summary().tables[1])
print("AIC:", fit.aic)

# 4. Check residuals
print(acorr_ljungbox(fit.resid.dropna(), lags=[12, 24]))

# 5. Forecast 12 months ahead with intervals
fc = fit.get_forecast(steps=12)
mean = np.exp(fc.predicted_mean)
ci = np.exp(fc.conf_int(alpha=0.05))

# 6. Measure error on the held-out year
mape = (np.abs(mean - np.exp(test)) / np.exp(test)).mean()
print("MAPE:", round(float(mape), 4))

What the choices mean

• The log transform makes seasonal swings look similar in size over time.
• order=(1, 1, 1) says one AR term, one difference and one MA term.
• seasonal_order=(0, 1, 1, 12) says a seasonal difference with period twelve and one seasonal MA term. This pattern is often called the airline model, after a classic example in the literature.
• The Ljung-Box test checks that residuals have no leftover autocorrelation. A large p-value is good here.
• The forecast is transformed back with the exponential function.

A small numeric walk-through of differencing and AR

Take the series 10, 12, 15, 19, 24. It climbs, with growing steps.

• First difference: 2, 3, 4, 5. The steps themselves grow by one each time.
• Second difference: 1, 1, 1. Now it is steady.

So d = 2 turns this series into a constant, a case where a simple model is enough.

For an AR(1) model with coefficient 0.6 and a mean of zero, if the last value is 10, the forecast for the next step is 0.6 x 10 = 6, and two steps ahead is 0.6 x 6 = 3.6. The forecast decays towards the mean. That is typical behaviour of a stationary AR model: far-ahead forecasts settle at the mean, and the prediction interval widens to the series' natural spread.

Automatic search

Libraries offer automatic selection that tries many (p, d, q) combinations and picks by an information criterion. This is convenient, but you still need to look at residuals and test on held-out data. An automatic model is a suggestion, not a verdict.

Rolling evaluation

A single train-test split is a noisy test. Use rolling-origin evaluation: train on the first part, forecast the next h steps, record errors, then move the origin forward and repeat. The average error over origins is a more reliable estimate of future accuracy.

errors = []
for cut in range(60, len(y) - 12, 6):
    fit = SARIMAX(y[:cut], order=(1,1,1),
                  seasonal_order=(0,1,1,12)).fit(disp=False)
    f = fit.forecast(12)
    errors.append(np.mean(np.abs(f.values - y[cut:cut+12].values)))
print(np.mean(errors))

Strengths and limitations

Strengths

• Compact: a handful of parameters describe the model.
• Strong on short, regular series, often competitive with complex models.
• Gives prediction intervals grounded in theory.
• Easy to diagnose with residual checks.
• Cheap to fit, so it can run on thousands of series.
• Interpretable parameters.
• Seasonal and exogenous extensions cover many practical cases.
• A natural baseline: any new forecasting method should beat it to justify its cost.

Limitations

• Handles one series at a time. It does not share information across related series.
• Assumes linear relationships.
• Needs a roughly stable structure after differencing. Sudden regime shifts break it.
• Struggles with multiple seasonalities, such as daily and weekly patterns in hourly data.
• Does not handle missing values natively without help in many implementations.
• Needs enough history, especially for seasonal models with long periods.
• Choosing orders takes judgement.
• Long-horizon forecasts of stationary models revert to the mean, and those of integrated models can drift.
• Poor at modelling intermittent demand with many zeros.
• Does not use rich covariates well without careful design.

Where it tends to win and lose

ARIMA often does well when you have one clean series, a modest history and a horizon of a few steps. It tends to lose when you have thousands of related series, strong effects from holidays and promotions, many covariates or complex shapes. In those settings, global models trained across many series often do better.

How it compares with alternatives

ARIMA versus gradient boosting on lag features

You can turn forecasting into a table problem. Each row is a time step, columns are lagged values, rolling averages, calendar fields and external features, and the target is the next value. Then XGBoost, LightGBM or CatBoost can learn the pattern across many series at once. This captures non-linear effects and shared structure that ARIMA misses, but loses ARIMA's compactness and its built-in intervals.

ARIMA versus pre-trained forecasting models

Pre-trained forecasters are trained on many series from many domains and can forecast a new series with no fitting. They shine when you have little history per series or too many series to model by hand. ARIMA remains useful as a transparent baseline and in settings where parameters must be explained.

Role in tabular foundation models

Tabular foundation models are pre-trained on many tables so they can predict, impute and flag oddities on a new table without per-table training. They read raw tables with rows, columns of mixed types, missing cells and often no helpful column names. ARIMA lives in a different corner, as a model for a single ordered series. Still, the two meet in several concrete ways.

Time series as tables

A time series can be written as a table: one row per time step, with columns for lagged values, calendar fields and covariates. This is the bridge. A tabular foundation model such as TabPFN or TabICL can be applied to such lagged tables to predict the next value in context, with no training on that dataset. ARIMA is the natural baseline in this comparison. Results depend heavily on how the lag table is built, and care is needed to respect time order. For how such models are evaluated, see the benchmarks page.

Respecting time order in table models

Standard table models assume rows are independent. Time series rows are not. When you feed lag tables to any tabular model:

• Split by time, never at random.
• Build lag features only from the past.
• Keep the context rows older than the query rows when using in-context models.
• Beware of leakage from rolling statistics that include the current or future value.

Features that ARIMA teaches

ARIMA's ideas turn into useful columns for any model: lagged values (AR), differences (I), and rolling means or lagged residuals (a rough MA analogue). A column of the first difference or the percentage change often helps tree models and table foundation models alike, because it removes trend.

Missing values and imputation

Real series have gaps. Classical ARIMA implementations in state-space form can handle missing observations through the Kalman filter, while simple ones need filled values. A tabular model trained with masked-cell objectives, related to imputation networks, can fill gaps using neighbouring columns and lags. Comparing a fill by ARIMA, by interpolation and by a table model on the same hidden points is a clean experiment. Use identical masks, and report error on the hidden cells only.

Anomaly detection

ARIMA residuals are a classic anomaly signal. Fit the model, compute one-step-ahead errors and flag points whose error exceeds several standard deviations. This is a transparent detector for single series. A tabular model can score rows in a lag table in a different way, using many columns at once. The anomaly detection page covers wider approaches, and Isolation Forest is a common tabular detector that can run on lag features. Each has strengths: residual methods give clear thresholds, while table methods catch combinations of columns.

Continual learning and drift

ARIMA is refitted when data changes, often with a rolling or expanding window. Its parameters can drift, which itself is a signal of changing behaviour. In a continual learning setting, a table model must adapt to new regimes without forgetting old ones. Monitoring ARIMA-style residual statistics offers a cheap drift alarm for the inputs and outputs of any deployed model.

Sector and semantic classification

Deciding whether a column or table represents prices, counts, temperatures or timestamps is a semantic task. Time-like columns have distinctive signatures: regular spacing, monotonic order and seasonal structure in other columns. A model that reads the raw table can use such hints, and a classical analysis such as checking for stationarity or a seasonal cycle can be a helpful engineered feature for a semantic head.

Where ARIMA still earns a place

• As a sanity check: if a large model cannot beat ARIMA on a series, question the setup.
• As a feature generator: ARIMA forecasts and residuals can feed table models as columns.
• As an explainable fallback in regulated settings that need simple parameters.
• As a teaching tool for ideas such as stationarity, autocorrelation and residual diagnostics.

See the architecture page for how modern table models represent rows and columns.

Practical guidance and best practices

Plot first, model second

Many mistakes are visible in a plot: outliers, level shifts, changes in frequency and missing stretches.

Handle outliers and breaks deliberately

A single wild value can distort estimates. Decide whether it is an error to fix, a real event to model with a dummy variable or a one-off to leave out. Document the choice.

Choose the horizon first

ARIMA is strongest at short horizons. If you need forecasts a long way ahead, set expectations, and compare with models designed for long horizons.

Use out-of-sample tests

Information criteria are helpful, but accuracy on later data is the final judge. Use rolling-origin evaluation.

Pick metrics that suit the data

Mean absolute error is easy to read. Mean absolute percentage error fails when values are near zero. Scaled errors compare across series. Choose and state the metric before looking at results.

Check residuals every time

No leftover autocorrelation, roughly constant spread, no outlier clusters. If residuals fail, the intervals are unreliable even if the point forecast looks fine.

Use seasonal terms only when there is seasonality

A seasonal period must be known and stable. Fitting seasonal terms to data without them adds noise.

Consider transforms

Logs, power transforms and differencing choices change the model. Keep track of them so forecasts can be mapped back correctly.

Scale to many series carefully

When forecasting thousands of series, automate order selection, but add guards: minimum history length, fallback to a simple method when a fit fails and flags for series with poor residuals.

Combine models

Averaging ARIMA with another method often beats either alone. Simple combinations are a strong and cheap improvement.

Document assumptions

State the frequency, the transform, the orders, the training window and the date of last refit.

A workflow checklist

1. Plot, clean and set the frequency.
2. Transform if variance changes with level.
3. Difference as needed and test.
4. Read ACF and PACF for candidate orders.
5. Fit several candidates and compare AIC.
6. Check residuals.
7. Evaluate with rolling origins against simple baselines.
8. Forecast with intervals.
9. Monitor and refit on a schedule.

Common mistakes

• Skipping the plot and trusting an automatic search.
• Over-differencing, which creates artificial patterns.
• Using random splits instead of time-ordered ones.
• Judging a model by in-sample fit only.
• Ignoring residual diagnostics.
• Comparing AIC across models with different differencing.
• Forgetting to invert transforms when reporting forecasts.
• Using MAPE on series that contain zeros or tiny values.
• Fitting a seasonal model with too little history.
• Treating prediction intervals as guarantees when residuals are not normal or have changing variance.
• Failing to beat a naive forecast and not noticing.
• Letting leakage into lag features when moving to table models.
• Ignoring structural breaks such as policy changes or sensor replacements.

The naive forecast check

Always compare against two simple baselines: the last observed value, and the value from the same season last year. If a sophisticated model does not beat these, it is not adding value. This check takes minutes and often reveals that a complicated model was no better than copying the past.

Reading the ACF and PACF with examples

A pure AR(1) series

Suppose each value is 0.8 times the previous value plus noise. The ACF decays slowly: about 0.8 at lag one, 0.64 at lag two, 0.51 at lag three. The PACF shows one large spike at lag one and then nothing. That cut-off at lag one points to p = 1.

A pure MA(1) series

Suppose each value is noise plus a share of the previous noise. The ACF shows one spike at lag one and then drops to near zero. The PACF decays gradually. The cut-off in the ACF points to q = 1.

A seasonal series

With monthly data and a yearly cycle, spikes appear at lags twelve, twenty-four and thirty-six. This suggests a seasonal difference with period twelve, or a seasonal AR or MA term.

A non-stationary series

When the ACF decays very slowly from a value near one, the series likely has a trend or unit root. Difference it and look again.

Handling special situations

Multiple seasonalities

Hourly data often has a daily and a weekly cycle. Plain SARIMA handles one season. Options include adding Fourier terms as exogenous regressors, using models designed for several seasons, or moving to a table approach with calendar columns.

Intermittent series

Series with many zeros, such as spare parts demand, break the usual assumptions. Specialised methods for intermittent demand work better than ARIMA.

Outliers and level shifts

Add dummy variables for known events, such as a store closure. For unknown ones, use robust methods or clean the data with documented rules.

Short histories

Seasonal models need several full cycles. With fewer than two cycles, prefer simple methods or borrow strength from related series with a global model.

Many series at once

Fit a simple automated ARIMA per series for a baseline, then test a global model trained across all series. Compare per-series errors, and look at the share of series where each approach wins.

From ARIMA to a lag table

The bridge to table models is a small piece of code.

import pandas as pd

s = pd.read_csv("monthly_demand.csv", parse_dates=["month"],
                index_col="month")["units"]

table = pd.DataFrame({"y": s})
for k in (1, 2, 3, 12):
    table[f"lag_{k}"] = s.shift(k)
table["diff_1"] = s.diff().shift(1)
table["roll_mean_3"] = s.shift(1).rolling(3).mean()
table["month_of_year"] = s.index.month
table = table.dropna()

split = int(len(table) * 0.8)
train, test = table.iloc[:split], table.iloc[split:]

Note the shift before every rolling or difference feature. Without it, the current value leaks into its own features. The split is by position, which keeps time order. Any tabular model can now be fitted on train and scored on test, and ARIMA can be run on the same split for a fair comparison.

Reporting a forecast well

A good forecast report answers five questions.

1. What was forecast, at which frequency and for which horizon?
2. How was the model chosen and tested?
3. How accurate was it on held-out data compared with naive baselines?
4. What range of outcomes is plausible?
5. What would make the forecast wrong?

Always show intervals, not only a single line. People plan better when they see the range.

Frequently asked questions

What does ARIMA stand for?

ARIMA stands for AutoRegressive Integrated Moving Average. The autoregressive part uses past values, the integrated part applies differencing to remove trends, and the moving average part uses past forecast errors.

What do p, d and q mean in ARIMA?

The value p is the number of lagged values used, d is how many times the series is differenced, and q is the number of lagged errors used. Together they define the model, so ARIMA(1, 1, 0) means one lag, one difference and no error terms.

How do I choose p, d and q?

Choose d first by differencing until the series looks stationary, usually zero, one or at most two. Then read the autocorrelation and partial autocorrelation plots to suggest p and q, fit several candidates, compare them with AIC or BIC and confirm with out-of-sample tests and clean residuals.

What is the difference between ARIMA and SARIMA?

SARIMA adds seasonal terms that model repeating patterns at a fixed period, such as twelve months. It has both the usual orders and seasonal orders with a season length, which makes it suitable for data with regular yearly, weekly or daily cycles.

When should I use ARIMA instead of a neural network?

Use ARIMA when you have one or a few series, a modest amount of history, a short horizon and a need for transparency. Neural and pre-trained models tend to help when you have many related series, rich covariates or complex shapes that a linear model cannot capture.

Why does ARIMA need a stationary series?

The autoregressive and moving average parts assume that relationships between points do not change over time. If the mean drifts or the variance grows, the estimated coefficients are unstable, so differencing and transforms are used to make the series steady before modelling.

Can ARIMA handle missing data?

Some implementations can, through a state-space form that treats gaps as unobserved points, while simpler ones need values to be filled first. Filling by interpolation is common, but it can hide real uncertainty, so test how sensitive the forecasts are to the fill method.

How does ARIMA relate to tabular foundation models?

A time series can be turned into a table of lagged values and covariates, and a tabular foundation model can predict the next value from it in context. ARIMA serves as the classical baseline for that comparison, and its ideas of lags and differences help create good columns.

Is ARIMA still useful today?

Yes. It is a fast, transparent baseline and often competitive on short, regular series. Even when you deploy a more complex model, an ARIMA forecast is a useful sanity check and can be a feature or a fallback.

How do I detect anomalies with ARIMA?

Fit the model, compute one-step-ahead forecast errors and flag points whose error is unusually large compared with the typical spread of residuals. Choose the threshold from the residual distribution, and be careful because an anomaly in the training window can distort the model itself.

Glossary

• ACF: the correlation of a series with its own lagged values.
• AIC: a score that balances fit and the number of parameters.
• Autoregression: predicting from past values of the same series.
• BIC: a stricter variant of AIC that penalises complexity more.
• Differencing: subtracting the previous value to remove trend.
• Exogenous variable: an outside input to the model.
• Forecast horizon: the number of steps predicted ahead.
• Lag: a previous time step.
• Ljung-Box test: a test for leftover autocorrelation in residuals.
• Moving average (model): a model using past errors.
• PACF: lag correlation with shorter lags removed.
• Prediction interval: a range for a future value.
• Residual: the difference between actual and fitted values.
• Seasonality: a repeating pattern with a fixed period.
• Stationarity: stable statistical behaviour over time.
• Unit root: a feature of a non-stationary series that behaves like a random walk.
• White noise: random values with no structure.

Summary

ARIMA combines autoregression, differencing and moving average terms into a compact model for forecasting a single series. The Box-Jenkins workflow of plotting, differencing, reading ACF and PACF, fitting, checking residuals and forecasting remains a clear way to build a reliable baseline. Seasonal and exogenous extensions widen its reach.

Its limits are linearity, a single-series focus and sensitivity to structural change. Modern alternatives such as Prophet, N-BEATS, TFT, PatchTST and Chronos fill gaps, and tabular foundation models can handle lag tables in context. ARIMA stays valuable as a baseline, a feature generator and an explainable fallback. Always test against naive forecasts, use time-ordered validation and check residuals. For related reading, see anomaly detection, XGBoost and the model research page.

Related

It belongs to Forecasting and tasks in my stack, under Foundation models and applied ML. Others in the same group: