One-class SVM

Overview

A one-class SVM is a kernel method that learns what normal data looks like and flags anything that falls outside. It is trained on examples of a single class, the normal one, and needs no labeled anomalies. After training, it draws a boundary around the dense region of the data. New points inside the boundary are treated as normal. Points outside are flagged as anomalies or novelties.

This makes it useful in situations where anomalies are rare, varied or unknown in advance, such as equipment faults, fraud patterns, unusual sensor readings or odd records in a table. It is a close relative of the standard SVM, but instead of separating two classes it separates the data from the origin, or from empty space, in a kernel-induced feature space. It works best on small to medium datasets with scaled numeric features and needs careful tuning of two settings, nu and gamma. It remains a standard baseline in Anomaly detection studies.

What it is

One-class SVM is an unsupervised, or more precisely a semi-supervised, method. It is unsupervised because it does not need anomaly labels. It is semi-supervised in the common setup because the training data is assumed to be mostly or entirely normal.

The fence analogy

Imagine a flock of sheep on a hillside. You never see wolves, so you have no examples of what a wolf looks like. You can still build a fence around the area where the sheep graze. Anything that wanders outside the fence is suspicious. A one-class SVM builds that fence. The shape of the fence is flexible, because the kernel lets it bend around clusters, curves and gaps.

Two views of the same method

There are two classic formulations:

With the RBF kernel, the two give equivalent solutions. Most libraries implement the first version, and the second is often described in the same breath.

Outliers versus novelties

These words are used in different ways, and it helps to be clear.

• Outlier detection: the training data may already contain anomalies, and you want to find them in that same data.

• Novelty detection: the training data is clean, and you want to judge new points that arrive later.

A one-class SVM is mostly used for novelty detection. It can be used for outlier detection if the share of anomalies in the training data is small and nu is set to match.

Output

The model gives a decision function value for each point. Positive means inside the boundary and normal. Negative means outside and anomalous. The size of the value is a score of how far the point is from the boundary. The method also gives a label of +1 or -1.

Key settings

• kernel: usually RBF.

• nu: a number between 0 and 1 that bounds the fraction of training points allowed outside the boundary, and the fraction of support vectors.

• gamma: the width parameter of the RBF kernel.

Where it came from

The method was proposed in the late 1990s. A paper on estimating the support of a high-dimensional distribution introduced the one-class SVM as a way to find a region that captures most of the probability mass of the data. At about the same time, another line of work proposed support vector data description, based on the smallest enclosing sphere. Both built on the SVM framework of margins and kernels from the 1990s.

The idea of estimating a support

The goal is not to estimate a full probability density, which is hard in many dimensions. It is to estimate a region that holds most of the data. This is an easier problem and is exactly what you need to decide whether a new point is typical. The approach came from a general principle: do not solve a harder problem than you need to.

The nu parameterization

The nu formulation is a neat detail. In the standard SVM, C controls the penalty on errors but has no direct meaning in terms of the data. In the nu version, the parameter has a clear interpretation: it is an upper bound on the fraction of training points that fall outside the boundary and a lower bound on the fraction of support vectors. This makes it easier to set from knowledge of the data.

Position among anomaly methods

Over the years, many other detectors appeared, including density-based, distance-based and tree-based ones. The tree-based Isolation forest became a common default because it scales well and needs little tuning. Deep methods such as autoencoders and deep one-class networks extended the same idea to images and sequences. The one-class SVM stayed in use as a principled kernel baseline and as a teaching example. See also Autoencoders.

How it works

This section explains the one-class SVM step by step.

Step 1: Represent rows as points

Each row becomes a point in feature space. Numeric columns are scaled so that no single column dominates. Categories are encoded, and missing values are imputed. The method sees only numbers.

Analogy: before drawing the fence, you need a map with a common scale. If north-south is in meters and east-west is in kilometers, the fence will be badly shaped.

Step 2: Map to a feature space with a kernel

The RBF kernel measures similarity between two points as exp(-gamma * ||x - z||^2). It equals 1 for identical points and falls toward 0 as they move apart. The kernel trick means the algorithm acts as if the points lived in a very high-dimensional space where the data forms a compact cloud, without computing that space.

Step 3: Separate the data from the origin

In the feature space of the RBF kernel, all points lie on one side of the origin, because every kernel value is positive. The one-class SVM finds the hyperplane that separates the data from the origin with the largest margin. In the original space, this hyperplane becomes a closed, curved contour around the dense areas of the data.

Analogy: pushing a flexible sheet against the data from behind, so that it hugs the cloud as closely as the settings allow.

Step 4: Allow some points outside

Strict enclosure of every point would make the fence stretch to include the farthest training point, even if it is an error. The nu setting lets a fraction of points fall outside, which keeps the boundary tight around the bulk of the data.

If nu is 0.05, about 5 percent of the training points are allowed outside the boundary, and at least 5 percent of points become support vectors.

Step 5: Solve for the support vectors

As in the standard SVM, the solution depends on a small subset of points, the support vectors. They sit on or beyond the boundary. The decision function is:

f(x) = sum_i alpha_i K(x_i, x) - rho

Here alpha_i are non-zero only for support vectors, K is the kernel and rho is an offset. A point is normal if f(x) >= 0.

Step 6: Score new points

For a new row, compute its kernel similarity to each support vector, sum the weighted similarities, and subtract the offset. A positive result means it looks like the training data. A strongly negative result means it is far from anything seen in training.

Step 7: Choose a threshold

The default threshold is zero, set by nu. You can instead rank points by score and choose a cutoff that matches your tolerance for false alarms. Many applications review only the top few percent of the most anomalous points.

The effect of gamma

gamma sets how local the boundary is.

• Large gamma: each support vector affects only a small area, so the boundary hugs the data and may form islands. It risks flagging many normal points that are not close to training examples.

• Small gamma: each support vector affects a wide area, so the boundary becomes smooth and broad. It risks missing anomalies that lie near the data.

Analogy: gamma is the thickness of the pencil. A very thin pencil traces every bump. A very thick one gives a rough blob.

The effect of nu

• Small nu: a loose fence that tries to include nearly everything in training. Few alarms on training data, but possible misses.

• Large nu: a tight fence. More alarms, including on normal data.

For clean training data, a small nu is usually right. For data with some contamination, set nu close to the expected fraction of anomalies.

Cost

Training cost grows quickly with the number of rows, as for other kernel methods. It is practical up to tens of thousands of rows. For larger data, use a linear approximation of the kernel, a stochastic solver on kernel feature maps, or a different detector.

Key concepts and terms

A worked example

A numeric walk-through of the decision function

Suppose training produced three support vectors in two dimensions, with weights as follows. The kernel is RBF with gamma = 1, and the offset rho = 0.5.

Score a new point P = (0.5, 0.5).

The squared distance from P to each support vector is 0.5. The kernel value is exp(-0.5), about 0.607, for all three. The weighted sum is (0.4 + 0.3 + 0.3) * 0.607 = 0.607. Subtract rho: 0.607 - 0.5 = 0.107. This is positive, so P is inside the boundary and normal.

Now score Q = (3, 3). The squared distances are 18 to S1, 13 to S2 and 13 to S3. Kernel values are about 0.0000000152, 0.0000023 and 0.0000023. The weighted sum is essentially 0.0000014. Subtract rho: about minus 0.5. Q is clearly outside and flagged as anomalous.

This shows the key behavior: points far from every support vector get near-zero similarity, so the score falls to -rho.

Code: novelty detection on synthetic data

import numpy as np
from sklearn.svm import OneClassSVM
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

rng = np.random.default_rng(0)

# normal data: two clusters
normal = np.vstack([
    rng.normal([0, 0], 0.5, size=(300, 2)),
    rng.normal([3, 3], 0.5, size=(300, 2)),
])

# new data: some normal, some far away
test_normal = np.vstack([
    rng.normal([0, 0], 0.5, size=(50, 2)),
    rng.normal([3, 3], 0.5, size=(50, 2)),
])
test_anom = rng.uniform(-3, 6, size=(40, 2))

pipe = Pipeline([
    ("scale", StandardScaler()),
    ("ocsvm", OneClassSVM(kernel="rbf", nu=0.05, gamma="scale")),
])
pipe.fit(normal)

pred_normal = pipe.predict(test_normal)    # +1 normal, -1 anomaly
pred_anom = pipe.predict(test_anom)

print("normal flagged as anomaly:", (pred_normal == -1).mean().round(3))
print("anomalies caught:", (pred_anom == -1).mean().round(3))

The uniform points include some that happen to fall inside the clusters, so not all will be flagged. That is expected. The output shows the false alarm rate on normal points and the catch rate on the spread-out points.

Code: scores and a custom threshold

scores = pipe.decision_function(test_anom)     # higher = more normal
order = np.argsort(scores)                     # most anomalous first
top = order[:10]
print(test_anom[top])

In practice you rank by score, review the top part, and adjust the cutoff to the number of alerts your team can handle.

Code: tuning without anomaly labels

When you have no anomaly labels, you cannot compute a normal accuracy. Use these checks:

from sklearn.model_selection import KFold

def false_alarm_rate(nu, gamma, X, folds=5):
    rates = []
    for tr, va in KFold(folds, shuffle=True, random_state=0).split(X):
        m = Pipeline([("s", StandardScaler()),
                      ("o", OneClassSVM(nu=nu, gamma=gamma))]).fit(X[tr])
        rates.append((m.predict(X[va]) == -1).mean())
    return np.mean(rates)

for nu in [0.01, 0.05, 0.1]:
    for g in [0.01, 0.1, 1.0]:
        print(nu, g, round(false_alarm_rate(nu, g, normal), 3))

The held-out false alarm rate should be near nu. If it is much higher, gamma is too large and the model overfits the training points. If you have even a small set of known anomalies, use them to pick the setting with the best recall at an acceptable false alarm rate.

Code: semi-synthetic evaluation

A practical trick for tuning is to inject synthetic anomalies, for example by shuffling one column or by adding noise, and then check how well the detector finds them. This does not replace real labels, but it can help compare settings and models in a controlled way.

def inject(X, frac=0.05, seed=0):
    r = np.random.default_rng(seed)
    X2 = X.copy()
    idx = r.choice(len(X), int(frac * len(X)), replace=False)
    X2[idx, 0] = r.permutation(X2[:, 0])[: len(idx)] + r.normal(0, 3, len(idx))
    y = np.zeros(len(X), dtype=int)
    y[idx] = 1
    return X2, y

Strengths and limitations

Strengths

• Needs only normal data to train.

• Flexible, curved boundaries through kernels.

• Few settings: nu, gamma and the kernel.

• nu has a clear meaning as a bound on training errors.

• Works well in low to medium dimensions with scaled numeric features.

• Compact model: only support vectors are stored.

• Fits into standard pipelines and tooling.

Limitations

• Sensitive to feature scale and to the choice of gamma.

• Slow on large datasets.

• Struggles in very high dimensions, where distances lose meaning.

• Does not handle mixed types, categories or missing values natively.

• Assumes the training data is mostly clean. Contamination degrades the boundary.

• Scores are not calibrated probabilities.

• Hard to explain why a given row is flagged.

• Tuning without labels is difficult.

• A single global boundary may fit data with several regimes poorly.

How it compares with alternatives

A common practice is to run two or three detectors from different families, such as Isolation forest, local outlier factor and a one-class SVM, and look at the points that several of them flag. Agreement across different methods raises confidence. For the toolkit that offers all of them, see scikit-learn.

Role in tabular foundation models

Tabular foundation models learn from rows and columns of many tables and can make predictions on a new table with little or no extra training. Anomaly detection on tables is one of the tasks they may be used for, and the one-class SVM is a classical reference point for it.

Same input shape

A one-class SVM reads a matrix of rows and columns, just like a tabular foundation model. Each row is a point, and each column is an unnamed dimension. It works without column names, which suits the schema-agnostic setting. This makes it easy to compare with a foundation model on the same table and split.

Mixed types and missing values

The one-class SVM needs numeric, scaled, complete input. Real tables have categories, dates, text and gaps. A preprocessing pipeline must fill the gaps and encode the categories, and these choices change what counts as unusual. A tabular foundation model can read mixed types and missing cells more directly, so it can judge a row on its whole content. When comparing, give the SVM a careful pipeline so the comparison is fair. See Imputation networks.

Pre-training versus fitting from scratch

A one-class SVM is fitted from scratch on each dataset, using only that table's normal rows. It has no prior knowledge. A foundation model is pre-trained on many generated or real tables and brings general knowledge about how columns relate to each other, how values are distributed and what typical dependence looks like. For anomaly detection, this prior can help on small tables, where a from-scratch method has too few rows to define the normal region well. See Synthetic data generation for how pre-training tables can be produced.

In-context learning for anomaly scoring

A foundation model can use the normal rows as context at inference time, with no weight updates. A row is then scored by how well the model can predict or reconstruct it, given the context. Rows that are hard to predict from the others are suspicious. This resembles the one-class SVM in spirit, since both judge a new row by its relationship to known normal rows. The SVM uses a fixed sparse set of support vectors and a kernel. The foundation model uses a learned procedure that weighs context rows by relevance. See TabPFN and /model/architecture.

Column-level anomalies and imputation

A related idea is to mask a cell and predict it from the rest of the row. If the observed value is far from the prediction, the cell is suspicious. This ties anomaly detection to imputation. A model good at imputing is also a model good at spotting cells that do not fit. The one-class SVM works at row level only and does not give cell-level explanations.

Continual learning

Standard one-class SVMs have no cheap update. When normal behavior drifts, you retrain on recent data. With a foundation model, you can refresh the context set, and for deeper changes, update the model through further training. See Continued pre-training. Either way, a monitoring plan should track how the share of flagged rows changes over time.

Semantic and sector classification

One-class models can also help with semantic checks: for example, learn the typical profile of rows in one sector or category, and flag rows that do not match that profile. This is a form of one-class classification used as a data quality check. For comparisons across detectors and models, see /model/benchmarks and /model/research.

Practical guidance and best practices

Prepare the training set

• Use data that is as clean and representative as possible.

• Remove known anomalies and obvious errors.

• Include all normal regimes, such as weekday and weekend patterns, or the different machine settings.

• Scale numeric columns inside a pipeline.

• Encode categories carefully, and avoid huge one-hot expansions.

• Impute missing values, and consider adding missing indicators if missingness is informative.

Set nu from knowledge

If you expect about one percent of rows to be unusual, start near nu = 0.01. For clean training data, a small value such as 0.01 to 0.05 is typical. Remember that nu also limits the share of support vectors, so large nu slows prediction.

Set gamma by checking held-out behavior

• Start with gamma="scale" as a default.

• Check the false alarm rate on held-out normal data. It should be close to nu.

• Lower gamma if held-out normal data is flagged too often.

• Raise gamma if known anomalies are not caught and the boundary looks too loose.

Reduce dimensions first when needed

With dozens or hundreds of columns, distances become less informative. Apply feature selection or a projection such as PCA before fitting. Check that the reduced space still separates the known anomalies.

Use more than one detector

Combine rankings from different methods, for example by averaging ranks. Review the points flagged by several detectors first.

Think about action

An anomaly score is only useful if someone acts on it. Decide in advance who reviews flagged rows, how many alerts per day are manageable, and what happens next. Pick the threshold from those limits, not from the default of zero.

Monitor over time

• Track the share of flagged rows per day.

• Plot the score distribution and watch for shifts.

• Retrain on a schedule, or when drift is detected.

• Keep a log of confirmed anomalies and false alarms, and use it to tune.

Explain the flags

Because the kernel model is a black box, add simple explanations. For each flagged row, list the columns with the largest standardized deviation from the median, or compare the row to its nearest normal neighbors. This helps reviewers decide quickly.

A short rollout plan

1. Collect a window of data that you believe is normal.

2. Split it by time into a fit part and a check part.

3. Fit the pipeline on the fit part with a small nu.

4. Measure the false alarm rate on the check part.

5. Inject a few synthetic anomalies and measure the catch rate.

6. Run the model in shadow mode, where it scores rows but triggers no action.

7. Review a sample of the flags with domain experts.

8. Turn on alerts at a threshold that matches the team's capacity.

9. Review the results monthly and retrain when needed.

Common mistakes

1. Training on data that contains many anomalies and expecting a clean boundary.

2. Forgetting to scale features.

3. Using the default gamma without checking held-out false alarms.

4. Setting nu far above the real anomaly rate and drowning in alerts.

5. Applying the method to very high-dimensional data without reduction.

6. Using it on a large table and waiting a long time.

7. Treating the sign of the score as proof of an anomaly, not as a lead to review.

8. Ignoring categorical and missing data issues in preprocessing.

9. Measuring accuracy on a dataset where anomalies are 0.1 percent of rows. Use precision, recall and ranking metrics instead.

10. Ignoring drift, so that the boundary becomes stale.

11. Tuning on the same labeled anomalies used for the final report.

12. Mixing several normal regimes in one model without checking that the boundary covers them well.

Frequently asked questions

What is a one-class SVM used for?

A one-class SVM is used to detect anomalies and novelties when only normal examples are available. It learns a boundary around normal data and flags new points that fall outside. Typical uses include equipment monitoring, fraud screening, quality control and data quality checks on tables.

How is a one-class SVM different from a regular SVM?

A regular SVM needs labeled examples from at least two classes and learns to separate them. A one-class SVM sees only one class and learns a boundary that encloses it. Both use kernels and support vectors, but the one-class version separates the data from the origin in feature space instead of one class from another.

What does the nu parameter mean?

Nu is a number between 0 and 1. It sets an upper bound on the fraction of training points that may lie outside the boundary, and a lower bound on the fraction of support vectors. Setting nu to 0.05 means about five percent of training points are allowed outside. It is the main way to control how tight the boundary is.

How do I choose gamma?

Start with the library default, which scales gamma by the number and variance of the features. Then check the false alarm rate on held-out normal data. If it is much higher than nu, gamma is too large. If known anomalies are missed, gamma may be too small. Tune on a log scale and use cross-validation.

Do I need to scale the data?

Yes. The RBF kernel depends on distances, so columns with large ranges dominate. Standardize numeric columns inside a pipeline so that the scaler learns only from training data. Without scaling, the boundary will mainly reflect the largest-scale column.

Can a one-class SVM handle categorical data or missing values?

Not directly. You need to encode categories as numbers and impute missing values before fitting. These choices matter, because they change distances. Many practitioners prefer tree-based detectors or tabular foundation models when the table has many categorical columns or gaps.

Is a one-class SVM better than isolation forest?

Neither is always better. Isolation forest scales better, needs less tuning and often works well on mixed tabular data. A one-class SVM can give smoother, more compact boundaries on small, well-scaled numeric data. It is wise to try both and compare on a small labeled sample or on injected anomalies.

How do I evaluate it without anomaly labels?

Use indirect checks. Measure the false alarm rate on held-out normal data and compare it with nu. Inject synthetic anomalies and see whether they are found. Review the top-scoring rows by hand. If any labeled anomalies exist, even a few, use ranking metrics such as average precision.

How does a one-class SVM relate to tabular foundation models?

It is a classical baseline for the same task. Both read rows and columns, and both can score how unusual a row is. The one-class SVM is fitted from scratch on one table. A tabular foundation model is pre-trained on many tables and can use normal rows as context, handle mixed types and missing values, and bring prior knowledge to small tables.

How big a dataset can a one-class SVM handle?

It is comfortable with up to tens of thousands of rows, since training time grows quickly with the number of rows. Beyond that, use a linear or approximate-kernel version, sample the training set, or switch to a detector that scales better, such as isolation forest.

Glossary

• Anomaly score: a number that says how unusual a row is.

• Contamination: the share of anomalies in training data.

• Decision function: the function whose sign tells normal from anomalous.

• Kernel: a similarity function between two points.

• Novelty detection: judging new points against a clean reference set.

• nu: a parameter that bounds the fraction of training errors.

• Outlier: a point that lies far from the others.

• RBF kernel: a kernel with similarity that decays with distance.

• SVDD: a sphere-based formulation of one-class learning.

• Support vector: a training point that defines the boundary.

Summary

A one-class SVM learns a flexible boundary around normal data and flags points outside it. It needs no anomaly labels, uses kernels to bend around the data, and exposes a clear control through nu. It is best on small to medium, scaled numeric data, and it is sensitive to gamma, feature scale and contamination in the training set. It is a standard baseline for anomaly detection.

For tabular foundation models, the one-class SVM is a reference method on the same row and column input. It highlights what pre-trained models add: built-in handling of mixed types and missing values, prior knowledge from many tables, and in-context use of normal rows without retraining. Use it alongside isolation forest and other detectors, tune with held-out false alarms and injected anomalies, and plan for review and drift. For related reading see SVM, Isolation forest, Anomaly detection, scikit-learn, Autoencoders, TabPFN, /model/architecture, /model/research and /model/benchmarks.

Related

It belongs to Classical ML in my stack, under Foundation models and applied ML. Others in the same group: