UMAP

Overview

UMAP, short for Uniform Manifold Approximation and Projection, is a method that squeezes high-dimensional data into two or three dimensions so that people can look at it. It tries to keep points that are close in the original space close in the low-dimensional picture, and it often keeps some of the larger shape of the data as well. It is widely used to visualize embeddings, to explore clusters and as a preprocessing step before clustering.

UMAP is fast, works on large datasets, and can transform new points after it has been fitted. It rests on ideas from topology and graph theory, but you can use it without the math. The key warning is that a UMAP plot is a view, not a measurement. Distances between far-apart groups and the sizes of groups can mislead. Use it to explore, and confirm findings with other methods such as Clustering and PCA-based checks.

What it is

UMAP is a nonlinear dimensionality reduction algorithm. Given a table of n rows and d columns, it returns n points in a space with just a few dimensions, usually two. It belongs to the family of manifold learning methods, which assume that the data lie on or near a lower-dimensional curved surface (a manifold) inside the high-dimensional space.

Analogy: a crumpled sheet of paper in a room. The sheet is flat, but it sits in 3D space as a crumpled ball. Two points that look far apart in the room (on opposite sides of a fold) may be close along the sheet. UMAP tries to flatten the paper back out so that neighbors along the paper are neighbors in the picture.

What UMAP is used for

• Visualization: plot embeddings from text, images, audio or tabular rows.
• Exploration: see whether classes separate, whether batches differ, whether outliers exist.
• Preprocessing: reduce dimensions before clustering with k-means or HDBSCAN.
• Quality checks: look for label noise, duplicates and drift.
• Feature generation: use the low-dimensional coordinates as extra features, with care.

What UMAP is not

• It is not a clustering algorithm. It does not output labels.
• It is not a faithful map of global distances.
• It is not deterministic unless you fix the random seed, and even then small changes in data can change the layout.
• It is not a replacement for a trained model on the original features.

Inputs and outputs

Where it came from

Dimensionality reduction has a long history. Principal component analysis, from the early twentieth century, finds linear directions of maximum variance. Multidimensional scaling and Isomap in the early 2000s tried to preserve distances along a curved manifold. Locally linear embedding and Laplacian eigenmaps used neighborhood graphs. In 2008, t-SNE popularized nonlinear embeddings for visualization, especially in biology and machine learning, by matching neighbor probabilities between the high-dimensional and low-dimensional spaces.

UMAP was introduced in a paper published in 2018 by Leland McInnes, John Healy and James Melville. It was presented as a method grounded in Riemannian geometry and algebraic topology, and also described in practical graph terms. In practice it proved faster than t-SNE on large datasets, kept more of the global structure in many cases, and supported general embedding dimensions and transforming new data. A Python package, commonly called umap-learn, made it easy to use, and it became a standard tool in single-cell biology, natural language processing and general data exploration. Variants include parametric UMAP, which learns a neural network that maps data to the embedding, and aligned UMAP, which compares embeddings across time.

How it works

UMAP has two main phases: build a weighted graph in the original space, then lay out a low-dimensional graph that matches it.

Phase 1: build a fuzzy neighbor graph

Step 1: find nearest neighbors. For every point, find its n_neighbors closest points under the chosen metric. For large data UMAP uses an approximate method (nearest neighbor descent) so this step stays fast. Approximate search is the same idea behind HNSW and FAISS.

Step 2: set a local scale. In dense regions, neighbors are very close. In sparse regions, they are far. If you used one fixed radius everywhere, sparse regions would look disconnected. UMAP adapts: for each point it measures the distance to its nearest neighbor and builds a local scale so that every point is strongly connected to at least its nearest neighbor.

Analogy: measuring distances in a city and a village with a flexible ruler. In the city, a short walk reaches many people. In the village, you walk farther. The ruler stretches so that each person has about the same number of "close" friends.

Step 3: turn distances into weights. For each point, the edge weight to a neighbor falls off as the distance (beyond the nearest-neighbor distance) grows. Weights lie between 0 and 1 and are read as the strength of a connection, a "fuzzy" membership.

Step 4: symmetrize. Point A may see B as a close neighbor while B does not see A that way. UMAP combines the two directed weights into one undirected weight using a fuzzy union: if either side believes in the edge, the edge stays fairly strong.

The result is a weighted graph that captures the local shape of the data.

Phase 2: optimize a low-dimensional layout

Step 5: initialize. Place points in the low-dimensional space. A common default is a spectral initialization, which uses eigenvectors of the graph. This gives a good starting layout that already reflects global structure, and it makes results more stable than random starts.

Step 6: define the target graph. In the low-dimensional space, UMAP defines weights from distances between points with a smooth curve. The parameter min_dist controls how tightly points may pack together.

Step 7: minimize a cross-entropy. UMAP compares the high-dimensional weights with the low-dimensional weights using a fuzzy set cross-entropy. The loss has two parts.

• An attractive part pulls together points that are connected in the original graph.
• A repulsive part pushes apart pairs that are not connected. UMAP samples random non-neighbors for this, which keeps computation low.

Step 8: stochastic gradient descent. The positions are updated over a few hundred epochs, with the step size shrinking over time. The process is like a physical simulation with springs and magnets, where connected points are joined by springs and others repel each other.

Analogy: tying beads with strings of different strengths and gently shaking the board. Strongly connected beads cluster together, and unconnected ones drift apart.

Transforming new data

After fitting, UMAP can place a new point by finding its nearest neighbors among the training points and positioning it relative to their embedded coordinates. This is useful for production pipelines, though the placement is approximate and the layout of the training data stays fixed.

Supervised and semi-supervised UMAP

If you pass labels, UMAP adds label information to the graph, which pulls points of the same class together. You can mark unlabeled points with a special value for semi-supervised use. Supervised UMAP is a useful tool for feature extraction, but it can create apparent class separation that does not generalize, so evaluate on held-out data.

The main parameters

A comparison of parameter effects in practice

The table below describes what a typical user sees when moving the two main settings. The exact picture depends on the data.

A good habit is to make a small grid of plots, three values of n_neighbors by three values of min_dist, and look for structure that holds across the grid.

Handling very different feature types

Real tables rarely contain only one kind of column. Here is a short guide.

• Continuous columns: scale them. Use a log or rank transform for heavy tails.
• Binary flags: keep them as 0 and 1, but limit how many you include so they do not drown out other signals.
• Low-cardinality categories: one-hot encode, or use target-free embeddings.
• High-cardinality categories: hash, group rare values, or replace by aggregate statistics.
• Dates: extract parts such as day of week and age, or use cyclical encodings.
• Text: embed with a sentence encoder and use the cosine metric.
• Images: embed with a pretrained vision model such as ViT or DINO.

You can also combine several UMAP graphs, one per feature group, using the set operations that UMAP objects support (union, intersection and difference of fuzzy graphs). This lets you give each type of data its own metric.

Speed and memory notes

The neighbor search dominates run time for large data. For a million points with fifty dimensions, expect minutes rather than seconds on a modern multi-core machine. The optimization step scales with the number of edges, not with the square of the number of points. Memory use grows with the number of points times n_neighbors. If memory is tight, reduce n_neighbors, reduce the input dimension first with PCA, or fit on a sample.

Key concepts and terms

A worked example

This example uses the umap-learn package on a small image dataset of digits that ships with scikit-learn, then clusters the embedding. See scikit-learn for the library.

import numpy as np
import matplotlib.pyplot as plt
import umap
from sklearn.datasets import load_digits
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score
from sklearn.manifold import trustworthiness

digits = load_digits()
X = StandardScaler().fit_transform(digits.data)   # 1797 rows, 64 columns
y = digits.target

reducer = umap.UMAP(
    n_neighbors=15,
    min_dist=0.1,
    n_components=2,
    metric="euclidean",
    random_state=42,
)
Z = reducer.fit_transform(X)

print("embedding shape:", Z.shape)
print("trustworthiness:", trustworthiness(X, Z, n_neighbors=15))

# Plot the layout colored by the true digit
plt.figure(figsize=(7, 6))
plt.scatter(Z[:, 0], Z[:, 1], c=y, s=6, cmap="tab10")
plt.title("UMAP of 8x8 digit images")
plt.savefig("digits_umap.png", dpi=150)

# Use UMAP before clustering: more output dimensions, tighter packing
reducer10 = umap.UMAP(n_neighbors=30, min_dist=0.0, n_components=10, random_state=42)
Z10 = reducer10.fit_transform(X)
labels = KMeans(n_clusters=10, n_init=10, random_state=0).fit_predict(Z10)
print("ARI of clusters vs digits:", adjusted_rand_score(y, labels))

# Transform new points
new_points = X[:5] + np.random.default_rng(0).normal(0, 0.05, size=(5, 64))
print(reducer.transform(new_points))

What you should see:

• The 2D plot shows ten groups that largely match the digit classes, with a few confusable digits such as 4 and 9 sitting close together.
• Trustworthiness is high when local neighborhoods are preserved.
• Clustering in a 10-dimensional UMAP space with min_dist near zero typically recovers the classes well.
• Transformed new points land near the training points they resemble.

A tabular example with mixed types

For a table with numeric and categorical columns, you need a numeric representation first.

import pandas as pd
import umap
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import Pipeline

df = pd.read_csv("example.internal/exports/customers.csv")

num_cols = ["age", "tenure_months", "monthly_spend"]
cat_cols = ["region", "device", "plan_type"]

prep = ColumnTransformer([
    ("num", StandardScaler(), num_cols),
    ("cat", OneHotEncoder(handle_unknown="ignore", sparse_output=False), cat_cols),
])

pipe = Pipeline([
    ("prep", prep),
    ("umap", umap.UMAP(n_neighbors=25, min_dist=0.05, random_state=1)),
])

Z = pipe.fit_transform(df[num_cols + cat_cols].fillna(df.median(numeric_only=True)))
df["u1"], df["u2"] = Z[:, 0], Z[:, 1]

This sketch has two weaknesses that are worth noting. One-hot columns can dominate distance when there are many of them, and median imputation can create artificial clumps. Better options are to use a mixed-type metric, or to feed UMAP with learned row embeddings, as discussed later.

Reading a UMAP plot

When you look at a plot, ask these questions.

1. Do known labels (classes, batches, dates) separate or mix? Separation by batch is often a warning.
2. Are there small isolated islands? They may be outliers, duplicates or a rare class.
3. Do groups look similar in size because of the method, not the data? UMAP can expand dense groups and shrink sparse ones.
4. Is the picture stable under different seeds and different n_neighbors? If not, the structure is weak.

A numeric intuition for the local scale

Suppose point A has nearest neighbors at distances 1.0, 1.2 and 1.5, and point B in a sparse area has neighbors at distances 4.0, 4.3 and 5.0. Without adaptation, B would be only weakly linked. UMAP subtracts each point's nearest-neighbor distance and rescales. So A's edges get weights based on 0.0, 0.2 and 0.5, and B's on 0.0, 0.3 and 1.0. Both points end up strongly tied to their nearest neighbor. That is how UMAP treats dense and sparse regions even-handedly.

Strengths and limitations

Strengths

• Fast on large datasets compared with many older nonlinear methods.
• Preserves local neighborhoods well and often keeps more global shape than t-SNE.
• Supports any output dimension, not only two.
• Can transform new data after fitting.
• Works with many metrics, including cosine, Hamming and custom ones.
• Has supervised and semi-supervised modes.
• Practical for pre-clustering.
• Open source, with a large user community.

Limitations

• Distances between clusters in the plot are not reliable.
• Cluster sizes and densities in the plot may not reflect the data.
• Results depend on parameters and on the random seed.
• Can show structure in noise, especially with small n_neighbors.
• Hard to interpret axes; they have no meaning.
• Not designed for mixed categorical data without preparation.
• New points are placed approximately, and large shifts in the data distribution break the mapping.
• Poor for statistical inference; it is an exploration tool.
• Without care, supervised UMAP leaks label information and looks better than it is.

How it compares with alternatives

When to pick which:

• Use PCA when you need a linear, invertible, explainable reduction, or as a first step to reduce hundreds of columns to fifty before UMAP.
• Use t-SNE when you only need a small, careful 2D picture and are aware of its limits.
• Use UMAP when you have lots of data, want to cluster afterwards, or need to transform new points.
• Use an autoencoder when you need a model you can train on your own objective and deploy as part of a network.

UMAP pairs naturally with Clustering and with embedding spaces from Contrastive learning, CLIP and DINO. It is also used to inspect features of models built with PyTorch.

Role in tabular foundation models

Tabular foundation models learn from raw tables: rows with numbers and categories, missing cells and sometimes anonymous columns. UMAP plays supporting roles around such models rather than inside them, but those roles are practical and common.

Visualizing row embeddings

A tabular foundation model can produce a vector for each row, or for each column, through its internal layers. A 64-dimensional or 512-dimensional vector is hard to inspect. UMAP projects those vectors into a plot. Colored by a known label, the plot shows whether the model has separated classes. Colored by data source, it can show batch effects. Colored by time, it can show drift. Architecture details of such models are in /model/architecture.

Mixed types and no column names

Raw tables mix numeric and categorical columns, and the columns may have no usable names. Plotting raw columns with UMAP then requires hand-made encodings, which distort distance. Learned representations from models such as TabPFN, TabICL, TabSTAR, FT-Transformer and SAINT handle mixed types inside the network. Feeding UMAP these learned vectors usually gives a more meaningful picture than feeding it one-hot encoded columns. The picture needs no column names, since the representation depends on values and their relationships.

Missing values

UMAP cannot handle missing values directly, so tables must be imputed first. Naive imputation (mean or median) produces artificial clumps of "imputed" rows. Pre-trained models that have seen masked cells, as in Masked cell modeling, or learned imputers from Imputation networks, fill gaps using the joint distribution of each row. A good test is to color the UMAP by a missingness flag. If missing rows form their own island, the imputation is influencing the picture.

Pre-training on synthetic or real tables

When tabular models are pre-trained on large collections of synthetic or real tables, researchers use projections to study what they learned. A UMAP of the representations of many different tables can show whether the model groups tables by type, size or noise level. These are exploratory checks. They do not replace quantitative benchmarks such as those described on /model/benchmarks, and the research context is on /model/research.

Continual learning and drift

When a model is updated with new data, you want to know whether its representation space moved. Embedding the same fixed probe rows before and after an update, and projecting both with the same fitted UMAP, shows displacement. Aligned UMAP or a shared transform makes the comparison fair. This is a way to see forgetting in Experience replay or EWC style training, but quantitative metrics should back up any claim.

Anomaly detection

Points far from all others in the embedding may be anomalies. UMAP makes such points visible, and a density or distance score computed in the learned space can rank them. Use this as a review aid together with Anomaly detection and Isolation forest, not as the only detector, since UMAP can pull some outliers into groups and push some normal points out.

Sector and semantic classification

Text descriptions, product names and category labels are often embedded and then examined with UMAP. Groups that correspond to industries or sectors become visible, and mislabeled items appear inside the wrong group. This supports building and checking semantic classifiers such as those in Semantic sector heads, and it helps label cleaning before training.

In-context learning

In-context learning relies on choosing good example rows. A UMAP of the training table helps you check whether the chosen context rows cover the space or leave a region uncovered. It also helps you spot near-duplicate rows that waste context.

Preprocessing for tree and tabular models

Adding UMAP coordinates as extra features to a supervised model is possible, but be careful. Fit UMAP on training data only and transform validation and test data, or you leak information. Gains are usually small for boosted trees such as XGBoost and LightGBM. UMAP is better used for understanding data than for adding features.

Practical guidance and best practices

Prepare the data

• Scale numeric features. Standardization is a good default.
• Remove constant and duplicate columns.
• Reduce very wide data first with PCA to 30 to 50 components. This speeds up the neighbor search and removes noise.
• Handle missing values deliberately.
• For text and images, use a good embedding and the cosine metric.

Choose parameters

• Start with n_neighbors 15 and min_dist 0.1 for visualization.
• For clustering, try n_neighbors 30 or more, min_dist 0.0, and 5 to 20 components.
• Try several values of n_neighbors, such as 5, 15, 50 and 200. Structure that survives all of them is more likely to be real.
• Try several random seeds.

Interpret carefully

• Treat the picture as hypotheses, not facts.
• Verify findings in the original space with statistics.
• Do not compare distances between clusters in the plot.
• Do not compare cluster areas as if they were sizes.
• Do not use the axes as variables.

Evaluate

• Compute trustworthiness or neighbor preservation for a numeric check.
• Check whether known labels separate.
• Compare to PCA to see whether UMAP adds information.
• Use downstream tasks, such as clustering quality, as a practical test.

Work with large data

• Use the low-memory option and approximate neighbors.
• Fit on a sample, then transform the rest.
• Use parallelism, noting that setting a random seed can disable it.
• Consider parametric UMAP if you need very fast transformation of streaming data.

Share results

• Provide the parameters, seed, library version and preprocessing steps.
• Include a note that the plot is exploratory.
• Provide an interactive plot if possible, with hover text for each point.

A short end-to-end checklist

1. Write down the question the plot should answer.
2. Build a clean numeric matrix, with scaling and deliberate imputation.
3. Reduce very wide data with PCA to a few dozen components.
4. Choose the metric that matches the data type.
5. Run a small grid of n_neighbors and min_dist values and several seeds.
6. Keep only the structure that is stable across runs.
7. Color the plot by labels, batches, time and missingness.
8. Verify any conclusion with a statistic in the original space.
9. Record the parameters, the seed and the library version.
10. If you will reuse the mapping, save the fitted object and test the transform on held-out rows.

What to do when the plot looks like a blob

A single featureless cloud can have several causes. The features may carry little signal, n_neighbors may be too large, scaling may be wrong, or the data may truly have no cluster structure. Try a smaller n_neighbors, remove noisy columns, and compare with a PCA plot. If PCA shows structure that UMAP hides, the settings are probably too smooth. If neither shows structure, accept that the data may simply be continuous.

What to do when the plot shows too many islands

Many small islands often mean that n_neighbors is too small or that the graph is disconnected. Increase n_neighbors, raise min_dist slightly, check for duplicates, and look at whether the islands match batches or data sources.

Common mistakes

1. Reading cluster distances literally. The gap between two blobs does not measure how different they are.
2. Reading cluster size literally. UMAP can inflate sparse groups and shrink dense ones.
3. One run, one setting. Structure that appears only for one parameter set may be an artifact.
4. No scaling. A large-scale feature dominates distance and the picture.
5. Using raw one-hot data. Many binary columns skew distances.
6. Imputing with a constant. Creates artificial clusters of imputed rows.
7. Leakage from supervised mode. Labels pull classes apart, which looks great and fails on new data.
8. Fitting on all data before splitting. Test information leaks into features.
9. Calling UMAP a clustering method. It gives positions, not groups.
10. Over-trusting small clusters. Tiny islands can be noise or duplicates.
11. Ignoring the metric. Embeddings from language models usually need cosine, not Euclidean.
12. Forgetting versions and seeds. Without them, the plot cannot be reproduced.

Variants and related tools

Several extensions of UMAP exist, and they are worth knowing.

• Parametric UMAP: trains a neural network to produce the embedding. It transforms new data very quickly and can be fine-tuned, but needs more setup than the standard version.
• Aligned UMAP: fits several embeddings that stay consistent with one another, useful for data that changes over time.
• DensMAP: adds a term that tries to keep local density information in the layout.
• Inverse transform: approximates the mapping from low-dimensional points back to the original space. It is rough and best used for illustration.
• Graph-based combination: unions or intersections of UMAP graphs let you mix data types.

Related tools in the same space include PCA, t-SNE, TriMap and PaCMAP. If a result matters, reproduce it with at least one other method to see whether the finding holds.

Reporting guidelines

When you include a UMAP figure in a report, add the following to the caption: the data source and size, the preprocessing, the metric, n_neighbors, min_dist, the seed and the library version. State plainly that the figure is exploratory. This small habit prevents readers from drawing strong conclusions from the picture alone.

Frequently asked questions

What does UMAP stand for?

UMAP stands for Uniform Manifold Approximation and Projection. It is a nonlinear dimensionality reduction algorithm that builds a weighted graph of neighbors in the original data and then finds a low-dimensional layout with a similar graph.

What is UMAP used for?

UMAP is mostly used to visualize high-dimensional data such as embeddings, to explore clusters and batches, and to reduce dimensions before clustering. It is common in biology, language processing, image analysis and general data exploration.

How is UMAP different from t-SNE?

Both are nonlinear methods that preserve local neighborhoods. UMAP is usually faster on large datasets, supports output dimensions other than two, can transform new points, and often keeps more of the broad structure of the data. t-SNE is still popular for careful two-dimensional pictures but is harder to apply to new data.

What do n_neighbors and min_dist do?

The n_neighbors setting controls how large a neighborhood UMAP considers around each point. Small values emphasize fine local detail, and large values emphasize broader structure. The min_dist setting controls how tightly points may pack in the layout. Low values give tight clumps that suit clustering, and higher values spread points out.

Can I trust the distances between clusters in a UMAP plot?

Not as a measurement. UMAP preserves local neighborhoods much better than distances between far-apart groups. Two clusters that look near each other may not be similar, and two that look far apart may be related. Use the plot to form hypotheses and check them with statistics in the original space.

Can I use UMAP before clustering?

Yes, this is a common workflow. Use more output dimensions, such as 5 to 20, a larger n_neighbors and min_dist close to zero, then run a density method such as HDBSCAN or k-means on the result. Validate the clusters in the original space too, because UMAP can create tight groups from noise.

Does UMAP work on categorical or mixed data?

Not directly. UMAP needs a numeric representation or a suitable distance. You can encode the columns, use a mixed-type distance, or feed UMAP with learned row embeddings from a model that handles mixed types. Be careful, because naive one-hot encoding can distort distances.

Is UMAP deterministic?

Not by default. The optimization uses random sampling, so different runs can give different layouts. Setting a random seed makes the result reproducible, though it can reduce parallel speed, and even then small changes in the data or library version may change the picture.

How do tabular foundation models relate to UMAP?

UMAP is a companion tool rather than a part of these models. It is used to inspect the row or column embeddings they produce, to check imputation effects, to explore clusters and anomalies, and to monitor representation drift after updates. The projection is an exploratory aid and should be backed by quantitative checks.

Can I add UMAP coordinates as features for a model?

You can, but fit UMAP only on training data and transform the other splits, otherwise you leak information. The gain is often small for tree ensembles, so test it against a baseline. UMAP is usually more valuable for understanding the data than for boosting accuracy.

Glossary

• Approximate nearest neighbors: Fast search that finds close points without checking all pairs.
• Cosine metric: A distance based on the angle between vectors.
• Cross-entropy: A loss that measures disagreement between two distributions or weight sets.
• Curse of dimensionality: The loss of contrast in distances as dimensions grow.
• Dimensionality reduction: Mapping data to fewer dimensions while keeping useful structure.
• Embedding: A vector representation of an item or row.
• Fuzzy set: A set where membership has a degree between 0 and 1.
• Manifold: A curved low-dimensional surface in a higher-dimensional space.
• min_dist: The parameter that sets how tightly points may pack.
• n_neighbors: The parameter that sets neighborhood size.
• PCA: Principal component analysis, a linear dimensionality reduction method.
• Spectral initialization: Starting the layout from graph eigenvectors.
• t-SNE: A nonlinear method for 2D visualization based on neighbor probabilities.
• Transform: Placing new points into an existing embedding.
• Trustworthiness: A score of how well neighborhoods in the embedding match the original.

Summary

UMAP reduces high-dimensional data to a few dimensions by building a weighted neighbor graph and then optimizing a low-dimensional layout that matches it. It is fast, supports any output size, can transform new points and often keeps both local and some global structure. The key settings are n_neighbors, min_dist, the metric and the seed.

Its pictures are exploratory. Cluster distances, sizes and axes should not be over-read, and the result should be checked across settings and against the original data. Around tabular foundation models, UMAP is most useful for inspecting learned row embeddings, spotting imputation artifacts, exploring clusters and anomalies, checking semantic groupings and watching representation drift after updates. Prepare and scale the data, reduce very wide tables first, try several parameters and seeds, and back every claim with a numeric check. For more background see /model/research, /model/architecture and /model/benchmarks.

Related

It belongs to Forecasting and tasks in my stack, under Foundation models and applied ML. Others in the same group: