Anomaly detection

Overview

Anomaly detection is the task of finding data points that do not fit the pattern of the rest of the data. In a table, an anomaly may be a transaction that looks nothing like a customer's usual behavior, a sensor reading that signals a failing machine, or a row with a value that cannot be right because of a data entry error. The main idea is to learn what normal looks like, then flag anything that is far from normal. Because anomalies are rare and often new, most methods do not need many labeled examples of them.

There is no single best method. Statistical rules, distance and density methods, tree based methods such as Isolation Forest, boundary methods such as one-class SVM, and neural methods such as autoencoders all have a place, and the right choice depends on the data type, the size of the data, whether labels exist and how costly mistakes are. Evaluation is also tricky, since anomalies are rare and definitions are task specific. For models that learn from raw tables, such as those described on /model/architecture, anomaly detection is a natural downstream task: a model that has learned the joint structure of rows and columns can tell when a row is unlikely.

What it is

Anomaly detection, also called outlier detection or novelty detection, identifies observations that deviate markedly from expected behavior. The three terms overlap but are not identical.

Terms in common use

In outlier detection, the training data may itself contain the unusual points, and the goal is to find them. In novelty detection, the training data is assumed to be clean, and the goal is to flag new points that differ from it. Many methods serve both purposes.

Kinds of anomalies

1. Point anomalies. A single record is unusual by itself. A purchase amount one hundred times larger than any before is a point anomaly.
2. Contextual anomalies. A record is unusual only in context. A temperature of thirty degrees is normal in summer and strange in winter. Time, place or customer group provide the context.
3. Collective anomalies. A group of records is unusual together, even if each is fine alone. A burst of small transactions in a short period is an example.
4. Structural anomalies. A relationship between columns breaks. Age and birth year no longer agree.
5. Data quality anomalies. Impossible values, wrong units, swapped columns or broken joins.

An everyday analogy

Imagine a security guard who walks the same building every night. After a few weeks the guard knows which lights are on, which doors are closed and what sounds are normal. One night a door is open and a light is on in a room that is usually dark. The guard has never been told "an open door is a threat". The guard simply notices that this does not match what normal looks like. Anomaly detection gives a model the same habit of noticing.

Typical applications

Where it came from

The study of unusual observations goes back to early statistics. In the nineteenth century, scientists working with astronomical and survey measurements asked when a measurement should be thrown out as an error. Rules of thumb based on how many standard deviations a value sits from the mean, and later formal tests such as those for extreme values in a sample, came out of this work.

Through the twentieth century, statisticians developed robust estimators that resist the influence of outliers, such as the median and the median absolute deviation. Distance based thinking, like the Mahalanobis distance, accounted for correlations between variables.

In the 1990s and early 2000s, data mining research introduced methods suited to large databases. Nearest neighbor distance methods scored points by how far they are from their neighbors. Density based methods, notably the local outlier factor published in 2000, compared a point's local density with that of its neighbors. Support vector based methods learned a boundary around normal data. In 2008, the Isolation Forest paper proposed a very different idea: instead of modeling normal data, isolate points with random splits, since anomalies are easier to isolate.

With deep learning, autoencoders, variational models, generative adversarial models and later self-supervised and transformer based methods were applied to anomaly detection, first on images and time series, then on tabular data. Benchmark studies of the last several years have repeatedly shown that, on tabular data, simple and classical methods remain strong, and that no method wins everywhere. This is an important lesson: the choice depends on what kind of anomaly you expect.

More recently, foundation style models that are pre-trained on large numbers of tables have opened a new route: use the model's learned notion of what rows are likely, given the other rows and columns, as an anomaly score.

How it works

Almost every anomaly detection method has the same shape: fit a model of normal data, assign each point a score that says how unusual it is, and choose a threshold above which points are flagged. The methods differ in how they model normal and how they score.

Step 1: understand the data and the anomaly

Ask what kind of anomaly matters and what it costs to miss one or to raise a false alarm. A missed fraud case has a different cost than a false alert that interrupts a customer. Also decide whether you have any labels, even a few.

Step 2: prepare the data

• Handle missing values deliberately. Missingness itself may be a signal. See imputation networks.
• Encode categorical columns. Rare categories can be anomalous by themselves.
• Scale numeric columns, since many methods are distance based.
• Remove or mark exact repeats. See deduplication.
• Add context features, such as time of day, group averages or recent history.

Step 3: choose a family of methods

Statistical methods

Assume a distribution for normal data and flag points in the tails.

• Z-score: how many standard deviations from the mean. Simple, but sensitive to outliers itself.
• Modified z-score with median absolute deviation: more robust.
• Interquartile range rule: flag values far outside the middle half of the data.
• Mahalanobis distance: accounts for correlation between columns.
• Gaussian mixture models: model normal data as a blend of several bell shapes and flag low-likelihood points.

These methods are fast and explainable, and they work well when the assumption roughly holds.

Distance and density methods

Normal points live in dense regions. Anomalies are far from others or in sparse regions.

• k-nearest neighbor distance: score by distance to the k-th neighbor.
• Local outlier factor: compare a point's local density to its neighbors' densities, which handles clusters of different density.
• Clustering based scores: points far from any cluster center or in tiny clusters are suspicious. See clustering.

These suffer in high dimensions, where distances become less informative.

Tree based methods

Isolation Forest builds many random trees. At each node, pick a random column and a random split value. Points that are alone after only a few splits are likely anomalies, since they sit in sparse regions. The score is based on the average path length. The method is fast, scales well and needs little tuning.

An analogy: imagine playing twenty questions to identify a person in a crowd. If they stand out, a few random questions single them out. If they look like everyone else, it takes many.

Boundary methods

One-class SVM learns a boundary that encloses most of the normal data in a transformed feature space. Points outside the boundary are anomalies. It can model complex shapes but is sensitive to scaling and parameter choice, and slows down on large data.

Reconstruction methods

An autoencoder compresses each row into a short code and rebuilds it. Trained on normal data, it learns to rebuild normal rows well. Anomalous rows are rebuilt badly, so the reconstruction error is the anomaly score. Variants include denoising and variational autoencoders. Principal component analysis works the same way in a linear form, scoring the part of a row that the main components cannot explain.

Predictive and self-supervised methods

Train a model to predict each column from the others, or to fill in masked cells. A row whose actual values are far from the predictions is surprising. This idea is closely related to masked cell modeling. It uses the structure of the table directly and can pinpoint which cell is unusual.

Supervised and semi-supervised methods

If labeled anomalies exist, a classifier such as XGBoost or LightGBM can be trained with class weighting. If only a few labels exist, combine them with unsupervised scores. See supervised learning, semi-supervised learning and unsupervised learning.

Step 4: score

Every method yields a number per point. Higher means more unusual. Scores from different methods are on different scales, so convert them to ranks or normalize before combining.

Step 5: set a threshold

Common options:

• a fixed fraction of points, such as the top one percent, when you have a review budget,
• a score cutoff chosen on validation data, balancing false alarms and misses,
• a statistical cutoff, such as a high quantile of scores on clean data.

In production, the threshold often drifts with the data, so recheck it regularly.

Step 6: explain and act

An alert without a reason is hard to use. Methods for explanation include showing which columns contributed most to the score, nearest normal examples, and feature attribution with tools such as SHAP or LIME. Alerts then go to a review queue, an automatic rule, or a human in the loop. See human-in-the-loop.

Step 7: monitor and update

Normal behavior changes over time. New products, seasons and policies shift what normal means. Retrain on a schedule, watch the alert rate, and gather reviewer feedback to refine the system.

Key concepts and terms

A worked example

A numeric walk-through with a simple rule

Suppose the daily number of card transactions for a customer over ten days is 4, 5, 3, 6, 4, 5, 4, 5, 6, 40.

• The median of the first nine values is 5.
• The absolute deviations from 5 are 1, 0, 2, 1, 1, 0, 1, 0, 1. Their median is 1.
• The modified z-score for the value 40 is about 0.6745 × (40 − 5) / 1 ≈ 23.6.

A common cutoff is about 3.5. The value 40 is far beyond it and is flagged. A plain z-score using the mean and standard deviation of all ten values would be much smaller, because the outlier inflates the standard deviation. That is why robust statistics matter.

Isolation Forest in Python

import numpy as np
import pandas as pd
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import OrdinalEncoder

df = pd.read_csv("transactions.csv")

num_cols = ["amount", "hour", "n_items", "days_since_last"]
cat_cols = ["merchant_type", "channel", "country"]

X = df[num_cols].copy()
enc = OrdinalEncoder(handle_unknown="use_encoded_value", unknown_value=-1)
X[cat_cols] = enc.fit_transform(df[cat_cols].astype("string").fillna("missing"))

model = IsolationForest(
    n_estimators=300,
    max_samples=256,
    contamination="auto",
    random_state=0,
)
model.fit(X)

df["anomaly_score"] = -model.score_samples(X)   # higher = more unusual
top = df.sort_values("anomaly_score", ascending=False).head(50)
print(top[["amount", "merchant_type", "anomaly_score"]])

The review team looks at the top fifty rows each day. That review budget sets the threshold.

Reconstruction with an autoencoder

import torch
import torch.nn as nn

class AE(nn.Module):
    def __init__(self, d, k=8):
        super().__init__()
        self.enc = nn.Sequential(nn.Linear(d, 64), nn.ReLU(), nn.Linear(64, k))
        self.dec = nn.Sequential(nn.Linear(k, 64), nn.ReLU(), nn.Linear(64, d))
    def forward(self, x):
        return self.dec(self.enc(x))

d = Xn.shape[1]                      # Xn: scaled numeric matrix, normal data only
ae = AE(d)
opt = torch.optim.Adam(ae.parameters(), lr=1e-3)
X_t = torch.tensor(Xn, dtype=torch.float32)

for epoch in range(50):
    for i in range(0, len(X_t), 256):
        xb = X_t[i:i + 256]
        loss = ((ae(xb) - xb) ** 2).mean()
        opt.zero_grad(); loss.backward(); opt.step()

with torch.no_grad():
    err = ((ae(X_t) - X_t) ** 2).mean(dim=1).numpy()
threshold = np.quantile(err, 0.99)
flags = err > threshold

Training on data that is mostly normal makes the autoencoder rebuild normal rows well, so high error signals something different.

Per-column explanation

with torch.no_grad():
    per_col = (ae(X_t) - X_t) ** 2          # shape: rows x columns
worst = per_col[idx].argsort(descending=True)[:3]
print([feature_names[j] for j in worst])

Reporting the three columns with the highest error tells the reviewer where to look first.

Evaluating with few labels

from sklearn.metrics import roc_auc_score, average_precision_score

print("AUROC:", roc_auc_score(y_true, df["anomaly_score"]))
print("AUPRC:", average_precision_score(y_true, df["anomaly_score"]))

When anomalies are around one in a thousand, AUPRC is more telling than AUROC, since AUROC can look high even when most flagged points are false alarms.

Strengths and limitations

Strengths

• Works with little or no labels: many methods learn only from normal data.
• Catches the unexpected: unlike a classifier trained on known cases, it can flag new types of problems.
• Wide applicability: fraud, faults, data quality, security and more.
• Fast baselines exist: robust statistics and Isolation Forest are quick to run and easy to deploy.
• Pairs with human review: ranked scores focus scarce attention.
• Can run continuously: scoring is cheap, so monitoring in near real time is feasible. See real-time systems.

Limitations

• No ground truth: without labels, it is hard to know if the system works.
• Definitions vary: what counts as anomalous depends on the task and the person.
• High false alarm risk: a low base rate means even a small false positive rate can bury the true cases.
• Drift: normal changes, so models go stale.
• Contamination: if the training data contains many anomalies, the model learns to treat them as normal.
• High dimensions: distance and density methods weaken as columns multiply.
• Mixed types: categorical and numeric columns need careful encoding.
• Explainability: neural scores can be hard to interpret.
• No universal winner: benchmark studies show rankings change from dataset to dataset.

Match method to anomaly type

How it compares with alternatives

Anomaly detection versus classification

If you have many labeled examples of every anomaly type, a classifier will usually do better on those types. If anomalies are rare, changing or poorly labeled, anomaly detection is the practical choice. Many real systems use both: a classifier for known patterns and an unsupervised detector for new ones.

Anomaly detection versus data validation

Validation checks explicit rules, such as ranges and formats. Anomaly detection learns implicit rules from the data. They complement each other. See data contract validation for rule based checks.

Anomaly detection versus drift detection

Anomaly detection looks at individual records. Drift detection looks at whether the whole distribution moves. A sudden wave of anomalies can be the first sign of drift. See schema drift detection.

Anomaly detection versus deduplication

Duplicates are repeated records, not unusual ones. They can mislead anomaly models, since a repeated pattern looks normal. Clean duplicates first with deduplication.

Role in tabular foundation models

Tabular foundation models are pre-trained on large numbers of tables so that they can work on a new table without task specific training. Their inputs have row and column structure, mixed types, missing values and column names that may be absent or unhelpful. Anomaly detection is one of the most natural uses of such a model, and the link is concrete. See /model/architecture, /model/research and /model/benchmarks.

The core link: likelihood of a row given the table

A model that has learned how rows and columns relate can estimate how plausible a row is given the other rows and columns. An anomaly is, by definition, a row that is implausible. There are several ways to turn this into a score.

1. Predict a column from the others. For each column, ask the model to predict the cell from the rest of the row, using the other rows as context. A large gap between the predicted and actual values signals a problem. The columns with the largest gaps show where the row is odd.
2. Masked cell reconstruction. Mask a cell, ask the model to fill it, and compare with the truth. This is the objective used in masked cell modeling, and the same machinery gives an anomaly score for free.
3. Density or likelihood scores. If the model gives a probability for a cell value, the negative log probability is a surprise score. Summing over cells gives a row score.
4. Embedding distance. Map rows to vectors and score by distance to normal rows, in the spirit of nearest neighbor methods.

In-context learning

In-context models, such as TabPFN and TabICL, take a table of examples as input at prediction time and adapt without updating weights. For anomaly detection, the "normal" rows are supplied as context. The model compares a query row with that context. This means no per-table training is needed: give it a few thousand normal rows and score new rows right away. It also adapts to each table's own notion of normal, which is exactly what you want in data quality monitoring, where every table has its own patterns.

Row and column structure

Anomalies in tables are often about relations: a combination of values that never occurs, though each value is fine alone. Models that attend across columns within a row and across rows within a column capture these relations. Column wise attention sees how one column behaves across many rows, which helps flag a value that is extreme for its column. Row wise attention sees how the cells of a single row fit together, which helps flag broken relations.

Mixed types and rare categories

A foundation model has cell encoders for numbers and categories, so one score can combine a surprising number and a surprising category. A rare category value is itself a signal, and a model that has seen many tables knows that rarity within a column is not unusual in every context.

Missing values

Missingness patterns can be normal or anomalous. A column that is usually filled but is suddenly empty for one row, or a row where the missing cells follow a new pattern, can be a data quality signal. A model trained with missing values in its pre-training handles this natively, and an imputation capability gives a natural companion: impute a cell, compare to the observed value, and score the gap.

No column names

Many tables arrive with unclear or absent column names. Methods that rely on names, such as hand written business rules, fail there. A model that works from values and structure can still score rows. It may also be able to infer a column's role from its values, which makes zero-configuration monitoring more realistic.

Pre-training on synthetic and real tables

Pre-training on synthetic tables (see synthetic data generation) can include injected anomalies: generators add outliers, shifted subgroups, swapped columns and broken relations, so that the model sees the patterns it will need to detect. Real tables add realistic structure. A curriculum can start with obvious anomalies and move to subtle ones.

Continual learning

Normal changes over time. A model that can keep learning from new tables, while protecting what it learned before, can stay current. Replay and regularization approaches such as elastic weight consolidation reduce forgetting. Anomaly systems need this because yesterday's anomaly may be today's normal, and a model must not forget old normal patterns that return seasonally.

Sector and semantic classification

Knowing the sector or the meaning of a table helps decide what normal is. A column of amounts in a retail table behaves differently from the same numbers in an industrial table. Sector and semantic classification of tables and columns provides context that improves scores and reduces false alarms. For example, a spike may be normal for a holiday sales table and abnormal for a monthly payroll table.

Cautions specific to foundation models

• Calibration: scores from a foundation model are not automatically comparable across tables. Calibrate on each table, for example with a quantile of scores on a clean part.
• Contamination of the context: if the context rows contain anomalies, the model may treat them as normal. Clean or filter the context.
• Context size limits: very large tables may need sampling to fit the context. Sample in a way that keeps rare but legitimate patterns.
• Duplicates: repeated rows in the context can distort what the model thinks is typical. Use deduplication first.
• Evaluation: use labeled benchmarks and report AUPRC as well as AUROC. See /model/benchmarks.

Practical guidance and best practices

Start simple

1. Add rule based checks for obvious problems.
2. Add a robust statistical score per numeric column.
3. Add Isolation Forest as a multi-column baseline.
4. Add a reconstruction or column-prediction model if the baseline misses subtle cases.
5. Compare all of them on whatever labels you can obtain.

Collect labels from the beginning

Even a few hundred reviewed alerts make evaluation possible. Store reviewer decisions with the alert and use them to tune thresholds and to train a supervised model later.

Use the right metric

Pick the threshold from costs

Estimate the cost of a missed anomaly and the cost of a false alarm, then choose the threshold that minimizes expected cost on validation data. If costs are unknown, fix the review capacity and take the top k.

Provide context features

Add features such as the customer's recent average, the hour of day, the peer group median and the ratio of an amount to its usual level. Contextual anomalies are a large share of real cases and need these features.

Handle categorical data with care

• Encode categories with frequency as well as identity.
• Group very rare categories.
• Treat unseen categories as a signal, not as an error.

Train on clean data when you can

If you can remove known anomalies from the training set, do so. For novelty detection, this is essential. With contaminated data, use robust methods and a small contamination assumption.

Keep models fresh

Schedule retraining. Watch the alert rate and the score distribution. A sudden change in either may signal drift or a pipeline fault, rather than a surge of true anomalies.

Make alerts actionable

alert:
  id: txn-20240101-0001
  score: 0.97
  rank_today: 3
  top_reasons:
    - column: amount
      note: 40x the customer's 30 day median
    - column: country
      note: first time seen for this customer
  nearest_normal_examples: 3
  suggested_action: manual_review

An alert with reasons is far more useful than a bare score.

Combine detectors

Average the ranks of several diverse detectors. Ensembles are usually more stable than any single method, since different methods catch different kinds of anomalies.

Protect against bias

Check alert rates across groups. If one group is flagged much more often, investigate whether the model is picking up on a proxy for something it should not use. Review the features and the training data.

Common mistakes

• Treating rare as wrong. A rare event can be perfectly legitimate. Anomaly means unusual, not bad.
• Judging by accuracy. With a tiny base rate, a model that flags nothing is almost always accurate. Use precision and recall based measures.
• Evaluating with a random split on contaminated data. Anomalies in the training set make the model learn them as normal.
• Ignoring context. The same value can be normal for one group and extreme for another.
• Not scaling features. Distance based methods are dominated by large-scale columns.
• Forgetting that duplicates hide anomalies. Repeated rows look normal because of repetition. Clean them first.
• Setting the threshold once and never revisiting it. Data drifts and so should thresholds.
• Using only one method. Different methods find different anomalies. Combine them.
• Flagging without explaining. Reviewers need reasons or they will ignore the alerts.
• Training on the future. In time-ordered data, a random split leaks future information. Split by time.
• Ignoring missing values. Dropping rows with missing cells removes some of the most interesting cases.
• Overfitting hyperparameters to a few known anomalies. This gives optimistic results that do not hold on new anomaly types.
• Skipping human feedback. Reviewer labels are the best source of improvement.

Frequently asked questions

What is anomaly detection in simple terms?

It is the task of finding data points that do not fit the usual pattern. A model learns what normal looks like, then flags rows, events or readings that are far from it. It is used for things like fraud, machine faults and data quality checks.

What is the difference between an anomaly and an outlier?

An outlier is a point that lies far from the rest of the data. An anomaly is an unusual observation that usually has a real-world meaning, such as a fault or a fraud case. Not every outlier is an anomaly worth acting on, and not every anomaly is an extreme value, since some are unusual only in combination or context.

Which algorithm is best for anomaly detection?

There is no single best algorithm. Robust statistics work well for single columns, Isolation Forest is a strong general baseline for tables, local density methods suit data with clusters of different density, and autoencoders or column prediction models can capture complex relations. Benchmark studies show the ranking changes by dataset, so test several and combine them.

How does Isolation Forest detect anomalies?

It builds many random trees by picking random columns and random split values. Points that are separated from the rest after only a few splits are considered anomalies, because unusual points sit in sparse regions and are easy to isolate. The average path length across trees is turned into a score.

Can I do anomaly detection without labeled data?

Yes. Many methods learn from data that is mostly normal and need no labels. Labels still help for evaluation and tuning, so collect reviewer decisions as you go, and consider a supervised or semi-supervised model once you have enough.

Why is evaluating anomaly detection hard?

Anomalies are rare, definitions vary and ground truth is often missing or incomplete. Accuracy is misleading when almost all points are normal, so use precision recall measures such as AUPRC, precision at a fixed number of alerts, and review based feedback. Evaluate on time-ordered splits to avoid leakage.

How do tabular foundation models help with anomaly detection?

They learn the joint structure of rows and columns from many tables, so they can estimate how plausible a row is given the others. Scores can come from predicting a column from the rest, filling masked cells, or likelihoods. Because they use in-context learning, they can adapt to each table's notion of normal without training from scratch.

How do I set the threshold for flagging anomalies?

Choose it from costs and capacity. If a missed anomaly is costly and reviewers have limited time, pick the cutoff that maximizes value at the number of alerts they can handle. If costs are unknown, flag the top k rows or a fixed quantile of scores, then adjust as you learn from reviewed alerts.

What is a contextual anomaly?

It is a point that is unusual only given its context, such as time, place or group. A high energy use may be normal on a cold day and abnormal on a mild one. Add context features or build separate models per group so the detector can see the context.

How often should an anomaly model be retrained?

It depends on how fast normal behavior changes. Monitor the alert rate and the score distribution, and retrain on a schedule or when they shift. Systems with seasonal patterns should keep older data so that returning patterns are not mistaken for anomalies.

Glossary

• Anomaly: an observation that does not conform to expected behavior.
• Anomaly score: a number expressing how unusual a point is.
• Autoencoder: a neural network that compresses and rebuilds its input; the rebuild error is used as a score.
• AUPRC: area under the precision recall curve.
• AUROC: area under the receiver operating characteristic curve.
• Base rate: the share of anomalies in the data.
• Concept drift: change over time in the data distribution.
• Contamination: anomalies present in the training data.
• Contextual anomaly: a point that is unusual only in its context.
• Collective anomaly: a group of points that is unusual together.
• False negative: a missed anomaly.
• False positive: a normal point flagged as anomalous.
• Isolation Forest: a method that scores points by how quickly random splits isolate them.
• Local outlier factor: a method that compares local densities.
• Mahalanobis distance: a distance that accounts for correlations between variables.
• Masked cell modeling: predicting hidden cells of a table from the rest.
• Novelty detection: flagging new points against a model of clean normal data.
• One-class SVM: a method that learns a boundary around normal data.
• Outlier: a point far from the bulk of the data.
• Reconstruction error: the gap between an input and its rebuilt version.
• Threshold: the score above which points are flagged.

Summary

Anomaly detection finds data that does not fit the pattern of normal behavior. Methods range from robust statistics and distance or density scores to Isolation Forest, one-class SVM and autoencoders, and all follow the same shape: model normal data, score each point, set a threshold, explain and monitor. The hard parts are defining what counts as anomalous, handling rare events with the right metrics, keeping up with drift and avoiding contaminated training data.

For tabular foundation models, anomaly detection is a natural downstream task. A model that has learned row and column structure from many tables can score how plausible a row is by predicting columns, filling masked cells or using likelihoods, and in-context learning lets it adapt to each table without training. Good practice still applies: clean duplicates first, calibrate scores per table, give reviewers reasons, evaluate with AUPRC and refresh models as normal changes. For more, see Isolation Forest, autoencoders, masked cell modeling, /model/architecture, /model/benchmarks and /model/research.

Related

It belongs to Forecasting and tasks in my stack, under Foundation models and applied ML. Others in the same group: