Model monitoring: detecting drift and decay in production ML

Overview

Model monitoring is the practice of watching machine learning models after they are deployed. It tracks the data going into a model, the predictions coming out, the outcomes that eventually follow, and the health of the service around it. The aim is to notice early when a model is becoming less accurate, less fair or less reliable, and to act before the problem harms customers or the business.

An everyday picture: a car's dashboard. The engine might run perfectly when it leaves the factory, but tyres wear, fuel runs low and parts age. Warning lights and gauges tell the driver something is changing long before the car breaks down. Model monitoring is that dashboard for a deployed model. The model itself does not change, but the world around it does, so its performance slowly drifts.

Ordinary software usually fails loudly, with errors and crashes. Models usually fail quietly: they keep returning answers that look plausible but are increasingly wrong. That is why monitoring for models needs more than uptime checks. It needs statistical checks on data and predictions, and a way to compare predictions with what really happened.

What it is

Model monitoring covers several layers, from infrastructure to business outcomes.

Layer	What is watched	Example signals
Service health	Whether the model endpoint works	Latency, error rate, throughput, resource use
Data quality	Whether inputs are valid	Missing values, out-of-range values, schema changes
Data drift	Whether input distributions change	Population stability index, distribution distance tests
Prediction drift	Whether output distributions change	Share of positive predictions, score histograms
Performance	Whether predictions are correct	Accuracy, AUC, error, calibration, once labels arrive
Fairness	Whether performance differs across groups	Error or approval rate gaps by segment
Business impact	Whether the model still delivers value	Conversion, losses, manual review rates
LLM quality	Whether generated outputs are good and safe	Judge scores, groundedness, refusals, user feedback

Each layer answers a different question, and a mature setup watches all of them with alerts tuned to the cost of being wrong.

Where it came from

Statistics and credit scoring have long tracked whether the population being scored still resembles the population a model was built on; the population stability index comes from that tradition. Research on concept drift, where the relationship between inputs and outcomes changes over time, developed in the machine learning literature from the 1990s onwards.

As companies put many models into production in the 2010s, the operational side became pressing. A widely cited 2015 paper from Google researchers, "Hidden Technical Debt in Machine Learning Systems", described how models in production accumulate hidden dependencies on data and on each other. Cloud ML platforms then added built-in monitoring services, and a group of specialised open-source and commercial tools for ML observability appeared. Since 2023, monitoring has extended to large language model applications, where quality is often judged by evaluators and user feedback rather than simple labels.

How it works

Step 1: log everything the model sees and says

Every prediction is logged with a timestamp, a request ID, the input features (or a sample), the model version and the output. Without this record, nothing else is possible.

Step 2: choose a reference

Drift is a comparison, so a baseline is needed. Common choices are the training data, a validation set, or a recent stable window of production traffic.

Step 3: compute statistics per window

For each time window, such as an hour or a day, the system summarises each feature and the predictions, then compares them with the reference. Popular measures include the population stability index, the Kolmogorov-Smirnov test for numeric features, chi-squared tests for categories, and Jensen-Shannon or Wasserstein distances.

import numpy as np

def psi(ref, cur, bins=10):
    edges = np.quantile(ref, np.linspace(0, 1, bins + 1))
    r = np.histogram(ref, edges)[0] / len(ref) + 1e-6
    c = np.histogram(np.clip(cur, edges[0], edges[-1]), edges)[0] / len(cur) + 1e-6
    return float(np.sum((c - r) * np.log(c / r)))

A common rule of thumb treats a PSI below 0.1 as stable, 0.1 to 0.25 as a moderate shift and above 0.25 as a large shift, but thresholds should be set per feature.

Step 4: join ground truth when it arrives

Labels often arrive late: a loan default may take months, a fraud chargeback weeks. When outcomes arrive, they are joined back to logged predictions by request ID, and true performance metrics are computed per window and per segment. Until then, drift and prediction distributions act as early warnings, and some methods estimate performance from the model's own confidence.

Step 5: alert and act

Alerts go to the owning team with context: which feature moved, by how much and since when. Responses range from fixing a broken data pipeline, to retraining on recent data, to rolling back to an earlier version, to adding human review while the issue is investigated.

Monitoring LLM applications

For generative applications, monitoring adds traces of each request, automated scoring of outputs with rules or an LLM judge, user feedback, cost and token usage, and safety checks. Tools such as Langfuse and Phoenix focus on this.

Key concepts and terms

Term	Meaning
Data drift	A change in the distribution of model inputs
Concept drift	A change in the relationship between inputs and the correct output
Prediction drift	A change in the distribution of model outputs
Training-serving skew	A difference between features at training time and at serving time
Ground truth	The real outcome a prediction is later compared with
Label delay	The time between a prediction and its known outcome
Reference window	The baseline data used for drift comparison
Population stability index	A score of how much a distribution has shifted between two samples
Segment	A subgroup, such as region or customer type, monitored separately
Calibration	Whether predicted probabilities match observed frequencies
Retraining trigger	A rule that starts retraining when monitoring crosses a threshold

A worked example

This scenario is illustrative. An online lender uses a model to approve small loans.

Every application is logged with 40 features, the score, the decision and model version 12. The reference is the validation set from training.
Daily jobs compute PSI per feature. For three months all values stay below 0.08.
In week 14 the PSI for "declared monthly income" jumps to 0.31. The approval rate rises from 41 to 47 per cent the same week. An alert reaches the model owner.
Investigation shows that a form change now asks for gross rather than net income. The model was trained on net income, so applicants look richer than they are.
The team adds a temporary rule to convert values, adds human review for borderline scores and schedules a retrain with relabelled data.
Three months later, matured repayment outcomes confirm that loans approved in week 14 had a default rate 1.8 points higher, a cost the early alert helped contain.

Strengths and limitations

Strengths:

Catches silent failures that ordinary service monitoring misses.
Gives early warning before ground truth arrives.
Provides evidence for retraining decisions instead of fixed schedules.
Supports fairness and regulatory oversight with ongoing records.
Builds trust with stakeholders who rely on the model.

Limitations:

Drift is not the same as harm; many shifts do not affect accuracy.
Too many alerts lead to alert fatigue and ignored warnings.
Delayed or missing labels make true performance hard to know.
High-dimensional inputs such as images and text are harder to monitor directly.
Logging inputs can raise privacy and storage concerns.

How it compares with alternatives

Approach	What it covers	Gap
Full model monitoring	Data, predictions, performance, fairness	Takes effort to design well
Application performance monitoring with Prometheus and Grafana	Latency, errors, resources	Blind to quiet accuracy loss
Scheduled retraining only	Keeps models recent	No detection of sudden breaks
Offline evals before release	Quality at deployment time	Nothing after launch
Data quality checks in pipelines	Broken inputs	Not prediction or outcome quality

Practical guidance and best practices

Log inputs, outputs, model version and a request ID for every prediction from day one.
Monitor the most important features and the prediction distribution first, then expand.
Set thresholds per feature, using historical variation rather than a single global rule.
Monitor key segments separately, because averages hide problems in subgroups.
Build the ground-truth join pipeline early, even if labels arrive slowly.
Route alerts to a named owner with a short runbook of likely causes and actions.
Track business metrics next to model metrics, so priorities are clear.
Review alerts monthly and remove those that never lead to action.

Common mistakes

Monitoring only latency and errors, then missing months of accuracy decline.
Alerting on every small drift, until the team ignores all alerts.
Comparing against an outdated reference that no longer represents normal traffic.
Forgetting that a data pipeline bug, not the world, is the most common cause of drift.
Never joining outcomes back, so performance is assumed rather than measured.
Logging raw personal data without a retention and access policy.

Frequently asked questions

What is the difference between data drift and concept drift?

Data drift means the inputs look different, for example customers are younger on average. Concept drift means the same inputs now lead to different outcomes, for example fraudsters change tactics. Data drift can be seen without labels; concept drift usually needs ground truth to confirm.

How often should models be monitored?

It depends on traffic and risk. High-volume, real-time models are often checked hourly or daily. Low-volume batch models may be checked weekly or per scoring run. The window must contain enough data for stable statistics.

Should drift always trigger retraining?

No. Drift should trigger investigation. Retraining helps when the world has genuinely changed; it does not help when a pipeline is broken, and it can even make things worse.

How do you monitor models when labels take months?

Watch input and prediction drift as early signals, track proxy outcomes that arrive sooner, and use methods that estimate performance from model confidence. When real labels arrive, confirm or correct the picture.

Is monitoring different for LLM applications?

Partly. Service health and drift still matter, but quality is usually measured with automated judges, rules, human review and user feedback, and traces show each step of a request. Cost and safety are also monitored closely.

Glossary

Baseline: the reference data used to judge whether current data has changed.
Alert fatigue: when too many alerts cause people to ignore them.
Shadow deployment: running a new model on live traffic without using its outputs.
Proxy metric: an early signal that correlates with the real outcome.
Observability: the ability to understand a system's state from its outputs and logs.
Runbook: a short guide on how to respond to a specific alert.
Rollback: returning to a previous model version.

Summary

Model monitoring watches deployed models across service health, data quality, input and prediction drift, true performance, fairness and business impact, so that quiet failures are caught early. It works by logging every prediction, comparing windows against a reference, joining outcomes when they arrive and routing focused alerts to owners. Done well, it turns retraining and rollback into evidence-based decisions; done badly, it becomes noise, so thresholds, segments and runbooks deserve as much care as the statistics themselves.

It belongs to MLOps in my stack, under MLOps and AI infrastructure. Others in the same group:

MLflow Kubeflow Weights & Biases BentoML KServe Ray Serve SageMaker Vertex AI Feature store Model cards