Unsupervised learning

Overview

Unsupervised learning finds structure in data that has no labels. There is no answer key. The algorithm looks only at the features and tries to discover groups, patterns, unusual rows, or a simpler description of the data. The output is not a prediction of a known target. It is a new view of the table.

On tabular data this covers several jobs. Clustering groups similar rows, such as customer segments. Dimensionality reduction squeezes many columns into a few. Anomaly detection finds rows that do not fit. Density models describe how likely a row is. These ideas also sit inside modern tabular foundation models, where learning from raw, unlabeled tables is a key step before any prediction task.

What it is

Unsupervised learning is learning from data without target values. Picture a person who walks into a large warehouse with no labels on any box. They cannot name the contents, but they can sort the boxes by size, weight, and shape. They notice a few odd boxes that look different from all the rest. That sorting and noticing is unsupervised learning.

In table terms, you have rows and columns, and no column is marked as the answer. The algorithm uses the relations among columns and among rows.

The main families

• Clustering: group rows so that rows in one group are alike and rows in different groups differ.
• Dimensionality reduction: replace many columns with a few that keep most of the information.
• Density estimation: model how the data is spread, so rare rows get low probability.
• Anomaly detection: flag rows that are far from the normal pattern.
• Association mining: find items or values that often appear together.
• Representation learning: learn compact numeric codes for rows or columns.

What it is not

• It is not supervised learning, because there is no target to predict.
• It is not magic. The algorithm cannot know which structure you care about. Different settings give different groups.
• It is not always self-checking. With no answer key, judging quality takes care.

Close cousins

• Self-supervised learning is often counted as a kind of unsupervised learning. It builds a task from the data itself, such as hiding a cell and predicting it, and uses the result to learn good representations.
• Semi-supervised learning mixes a few labels with many unlabeled rows.
• Contrastive learning learns by pulling similar rows together and pushing different rows apart.

Where it came from

The earliest ideas come from statistics and the study of data structure. Principal component analysis was developed in the early twentieth century to find the main directions of variation in a set of measurements. Factor analysis, from psychology, tried to explain many test scores with a few hidden traits.

Clustering grew through the mid twentieth century. Hierarchical clustering, which builds a tree of merges, and k-means, which assigns points to the nearest center, became standard methods in the 1950s and 1960s. Density based methods such as DBSCAN arrived in the 1990s and handled odd shaped groups and noise.

Mixture models, fitted with the expectation maximization algorithm in the 1970s, gave a probabilistic view of clusters. Self-organizing maps and early neural methods followed in the 1980s.

Autoencoders appeared as neural versions of dimensionality reduction. Later work added variational autoencoders and generative adversarial networks, which learn full models of the data. Methods such as t-SNE and UMAP, from the 2000s and 2010s, became popular for visualizing high dimensional data in two dimensions.

From the late 2010s, large scale self-supervised pre-training on text and images showed that unlabeled data can teach powerful models. The same idea now reaches tables. See the research overview for how this carries into models trained on raw tabular data.

How it works

Each family works differently. The steps below show the common flow, then the specific methods.

The common flow

1. Prepare the table. Handle missing values, encode categories, and scale numeric columns.
2. Choose a notion of similarity or likelihood. Distance, density, and reconstruction error are the usual choices.
3. Run the algorithm to find groups, axes, or scores.
4. Inspect the result. Look at group sizes, profiles, and examples.
5. Name and use the structure, or feed it to another system.

Clustering with k-means

K-means is like placing k flags on a map and letting every house join its closest flag. Then each flag moves to the middle of its houses. The process repeats until the flags stop moving.

• Pick k starting centers.
• Assign each row to the nearest center.
• Move each center to the mean of its rows.
• Repeat until assignments stop changing.

It is fast and simple. It assumes round, similar sized groups, and it needs k up front.

Hierarchical clustering

Think of a family tree built from the bottom. Start with each row alone. Merge the two closest rows, then the next closest, and so on. The result is a tree, called a dendrogram. Cutting the tree at some height gives clusters. You need not choose k in advance.

Density based clustering

DBSCAN treats crowded regions as clusters and sparse points as noise. Imagine islands in a sea. Dense land is a cluster. Isolated rocks are outliers. It finds shapes that are not round and flags noise by design.

Gaussian mixture models

A mixture model assumes the data comes from several bell shaped groups. It estimates each group's center, spread, and weight. Each row gets a probability of belonging to every group. This is a soft version of clustering.

Principal component analysis

PCA finds new axes along which the data varies most. Imagine photographing a cigar-shaped cloud of points. The best single photo angle shows the long side. PCA picks that angle first, then the next best angle at a right angle to the first, and so on. Keeping the first few axes gives a compact version of the table.

Autoencoders

An autoencoder is a neural network with a narrow middle. It squeezes a row into a short code and then tries to rebuild the row. If the rebuild is good, the code holds the essence of the row. Rows that rebuild badly may be anomalies.

Anomaly detection

Several unsupervised ideas flag odd rows.

• Distance: a row far from its neighbors is unusual.
• Density: a row in a sparse region is unusual.
• Isolation: odd rows are easy to isolate with random splits, so they need fewer splits.
• Reconstruction: a row that a model cannot rebuild is unusual.

See anomaly detection for details.

Association rules

Association mining finds sets of items that appear together. Basket analysis is the classic case: customers who buy A often buy B. Rules are scored by support, confidence, and lift.

Distance and similarity choices

The word "similar" needs a definition before any method can run.

• Euclidean distance is the straight line gap. It suits scaled numeric data.
• Manhattan distance adds up per column gaps. It is less sensitive to a single large gap.
• Cosine similarity compares direction, not size. It suits sparse or count data.
• Hamming distance counts differing categories.
• Gower distance mixes numeric and categorical columns in one measure.
• Learned distance from an embedding can capture meaning that raw columns miss.

Soft versus hard answers

A hard method says "this row is in group 2". A soft method says "this row is 70 percent group 2 and 30 percent group 1". Soft answers are more honest when groups overlap. They also give an uncertainty signal that hard methods hide.

Evaluating without labels

Key concepts and terms

Choosing the number of groups

• Elbow method: plot within-group spread against k and look for a bend.
• Silhouette: pick the k with the highest average silhouette.
• Stability: run on resamples and keep the k that gives similar results.
• Business sense: groups must be few enough to act on.

A worked example

This example clusters a small synthetic customer table, picks k with the silhouette score, and then reduces the data to two dimensions for inspection.

import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.decomposition import PCA

rng = np.random.default_rng(1)

# Three hidden customer types, but we pretend we do not know that.
a = rng.normal([20, 5, 100], [3, 1, 20], size=(300, 3))
b = rng.normal([45, 15, 400], [5, 2, 60], size=(300, 3))
c = rng.normal([65, 2, 900], [4, 1, 120], size=(300, 3))
X = pd.DataFrame(np.vstack([a, b, c]),
                 columns=["age", "visits_per_month", "spend"])

Xs = StandardScaler().fit_transform(X)

scores = {}
for k in range(2, 8):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(Xs)
    scores[k] = silhouette_score(Xs, km.labels_)
print({k: round(v, 3) for k, v in scores.items()})

best_k = max(scores, key=scores.get)
labels = KMeans(n_clusters=best_k, n_init=10, random_state=0).fit_predict(Xs)

X["cluster"] = labels
print(X.groupby("cluster").mean().round(1))

pca = PCA(n_components=2).fit(Xs)
print("explained variance:", pca.explained_variance_ratio_.round(3))

What to expect

• The silhouette score should peak at k equal to 3, since three groups were built in.
• The group means show the profile of each segment, such as young low spenders and older high spenders.
• PCA shows that two axes keep most of the variance, so a scatter plot of them is a fair picture.

A numeric walk-through of k-means

Take four points on a line: 1, 2, 9, and 10. Start with centers at 1 and 10. Points 1 and 2 are closer to the first center. Points 9 and 10 are closer to the second. New centers are 1.5 and 9.5. Assignments do not change, so the algorithm stops. The total within-group squared distance is 0.25 plus 0.25 plus 0.25 plus 0.25, which is 1.0. A single group would have had a far larger spread, so two groups describe the data well.

Why scaling matters

If income is in thousands and age is in tens, income dominates every distance. Scaling puts columns on a comparable footing. Without it, the clusters just reflect the column with the biggest numbers.

Applications on tables

• Customer segmentation for marketing and service design.
• Fraud and error screening by flagging unusual transactions.
• Data quality checks, such as finding duplicated or mistyped records.
• Feature creation, using cluster IDs or components as inputs to a supervised model.
• Product catalog grouping and taxonomy building.
• Sensor monitoring, where normal operation is learned and drift is flagged.
• Survey analysis, grouping respondents by answer patterns.
• Exploratory work before any model is chosen.

Variants worth knowing

• Mini-batch k-means handles very large tables by using small random batches.
• K-medoids uses real rows as centers and is more robust to outliers.
• Spectral clustering uses a graph of similarities and finds non-round groups.
• Agglomerative clustering with different linkage rules gives chains or compact balls.
• Kernel PCA captures nonlinear shape.
• Non-negative matrix factorization finds additive parts, useful for counts.
• Topic models treat rows as mixes of hidden themes.

A second numeric walk-through: PCA by hand

Take three rows with two columns: (2, 4), (4, 8), and (6, 12). The second column is exactly twice the first. The data lies on a straight line.

• The mean is (4, 8). Centering gives (-2, -4), (0, 0), and (2, 4).
• All the variation runs along one direction, roughly (1, 2) scaled to unit length.
• The first principal component explains 100 percent of the variance.
• The second explains 0 percent, so it can be dropped with no loss.

Real data is never this clean, but the idea holds. When columns move together, a few axes can stand in for many.

Reading cluster profiles

After clustering, build a profile table with the mean or median of each column per cluster. Compare each cluster with the overall average.

• A column far above average marks what defines that cluster.
• Columns close to average are not useful for naming it.
• Share of total rows tells you whether the cluster is big enough to act on.
• Check categorical columns by the most common value.

A profile like "young, frequent visits, low spend" is a segment a team can use. A profile that no one can describe is probably noise.

Strengths and limitations

Strengths

• No labels are needed, so it works wherever data exists.
• It can reveal groups and patterns that nobody thought to look for.
• It gives compact features that help later supervised models.
• It supports data exploration, cleaning, and quality checks.
• It scales to large tables, especially for k-means and PCA.
• It finds rare events when labeled examples do not exist.

Limitations

• There is no single right answer. Results depend on the distance, the scaling, and the settings.
• Quality is hard to judge without labels. Internal scores only give hints.
• Clusters can be unstable under small changes in the data.
• Mixed type columns make distances awkward.
• Distances lose meaning in very high dimensions.
• Groups found may not match any useful business idea.
• Anomaly scores say "unusual", not "bad". Many rare rows are fine.

How it compares with alternatives

Choosing among unsupervised methods

Role in tabular foundation models

A tabular foundation model learns from raw tables. Much of what makes this possible is unsupervised or self-supervised in spirit. This section links the ideas to concrete behavior.

Learning structure without labels

Real organizations hold huge numbers of tables with no target column. Unsupervised and self-supervised objectives let a model learn from all of them. A common objective hides some cells and asks the model to fill them in. This forces it to learn how columns relate, which values are typical, and how rows resemble each other. See self-supervised learning.

Row and column structure

A table has two axes of structure. Rows are examples and columns are fields. Classic unsupervised methods mostly treat rows as points and columns as axes. A table model goes further. It uses attention across rows to find similar examples and attention across columns to find related fields. This design is covered on the architecture page and builds on transformers.

Mixed types and missing values

Unsupervised methods struggle with mixed types, since distance between a number and a category is unclear. A foundation model learns separate treatment for numeric, categorical, and date cells and projects them into a shared space. Missing cells are not a defect. They are a signal and a training target. Predicting a hidden cell is imputation, and it ties to imputation networks.

Embeddings of rows and columns

Autoencoders and contrastive objectives produce vectors that capture meaning. In a table model, a row embedding can describe the whole record, and a column embedding can describe what a field is about. These vectors support:

• Similar row search and deduplication.
• Grouping of rows or tables by theme.
• Semantic and sector classification, such as deciding that a table holds retail transactions or sensor logs.
• Matching columns across tables even when the names differ.

No column names

If headers are missing or cryptic, a model can still infer the role of a column from its value distribution and its relations to other columns. This is an unsupervised skill. It reads shape, range, repetition, and correlation. When names exist, they can add extra meaning.

Pre-training on synthetic and real tables

Models such as TabPFN are pre-trained on huge numbers of generated tables, built with synthetic data generation. Generators draw structure from random causal graphs and mixtures, so the model sees a wide range of patterns. Real tables can add realism through self-supervised cell masking.

Anomaly detection and density

A model that learns what a normal row looks like can score how unlikely a new row is. That gives anomaly detection from the same backbone. The model compares each row with the learned pattern of the table and the rest of the rows. See anomaly detection.

In-context learning and continual learning

At use time, a table model can take a whole table as context and answer questions about it without training a new model. This is in-context learning. For long lived systems, continual learning adds new tables over time while keeping old skills. Unsupervised signals are cheap to gather, so they are a natural fuel for this process. Evidence on how such models are scored is on the benchmarks page.

From unsupervised pre-training to prediction

A simple view helps. Phase one is learning what tables look like in general, using no labels. Phase two is using that knowledge for a specific job.

1. Pre-train on many tables with cell masking and similar objectives.
2. The model learns column types, typical ranges, correlations, and missing value patterns.
3. At use time, give it a new table as context.
4. Ask it to fill gaps, score rows, group rows, or predict a target if labels exist.

The same knowledge serves all four requests, which is why one backbone can power several features.

What a human gets from the learned structure

• A fast first look at a table: types found, relations, odd columns.
• A ranked list of unusual rows with a reason, such as which columns drove the score.
• Suggested fills for missing cells, with uncertainty.
• Suggested labels for what a table is about, such as logistics, retail, or finance.

Practical guidance and best practices

Prepare the data first

• Remove identifiers such as row IDs. They add noise.
• Handle missing values on purpose. Decide whether to fill, flag, or drop.
• Scale numeric columns.
• Encode categories in a way that fits the method. One-hot works for small sets.
• Consider log transforms for skewed columns like income.

Pick a method that fits the shape

• Round, even groups: k-means.
• Odd shapes and noise: DBSCAN.
• Need probabilities: mixture models.
• Need a tree view: hierarchical clustering.
• Need visuals: PCA first, then UMAP or t-SNE.

Validate without labels

• Check stability on resamples.
• Compare against a simple baseline such as random groups.
• Inspect cluster profiles with domain experts.
• Use a downstream task. If clusters improve a later model, they carry signal.
• Plot examples from each cluster and read them.

Use the result well

• Name each segment in plain words.
• Track segment sizes over time.
• Re-fit on a schedule, since the world drifts.
• Keep the scaling and settings stored so new rows get assigned the same way.

A short checklist

• Are columns scaled and encoded?
• Is the number of groups justified?
• Are results stable across seeds?
• Do experts agree the groups make sense?
• Is there a plan to refresh the model?

Working with very large tables

• Sample rows for exploration, then fit on all rows once settings are chosen.
• Use mini-batch methods, randomized PCA, or approximate nearest neighbors.
• Reduce columns first if there are hundreds.
• Run several seeds and compare results.

Handling time

• If rows have timestamps, cluster on behavior features, not on the timestamp itself.
• Re-fit on a rolling window to follow drift.
• Compare cluster shares across periods to spot change.

Governance and privacy notes

• Groups built from personal data can reveal sensitive traits. Review them before use.
• Anomaly lists can cause real harm if acted on without a human check.
• Keep a record of the data used, the settings, and the date of each fit.
• Aggregate or de-identify rows when sharing group results.

Common mistakes

• Skipping scaling, so one big column controls everything.
• Choosing k by habit instead of evidence.
• Treating clusters as ground truth. They are a view of the data, not a fact.
• Reading t-SNE or UMAP plots too literally. Gaps and sizes in the picture can mislead.
• Including ID or timestamp columns that create fake structure.
• Using Euclidean distance on mixed or sparse data without thought.
• Clustering on too many columns at once. Many irrelevant columns blur the groups.
• Flagging every outlier as fraud. Rare is not the same as wrong.
• Never checking stability. A result that changes with every seed is not reliable.
• Failing to store the fitted pipeline, so new rows cannot be scored the same way.

Frequently asked questions

What is unsupervised learning in simple words?

Unsupervised learning is finding patterns in data that has no answer key. The algorithm groups similar rows, compresses many columns into a few, or spots rows that look odd. A person then reads the result and decides what it means.

What are the most common unsupervised learning methods?

The most common are k-means and hierarchical clustering for grouping, PCA for reducing columns, and isolation or density methods for finding odd rows. Autoencoders and mixture models are also widely used. Each fits a different shape of problem.

How do I know if my clusters are good?

Without labels you rely on indirect checks. Look at silhouette scores, test stability on resampled data, and ask domain experts whether the groups make sense. The strongest test is practical: if the groups help a later task or decision, they are useful.

What is the difference between unsupervised and self-supervised learning?

Both use unlabeled data. Classic unsupervised learning looks for structure such as groups. Self-supervised learning builds a prediction task from the data, such as hiding a cell and predicting it, and uses that task to learn representations. The line between them is often blurry.

Can unsupervised learning work on tabular data with mixed types?

Yes, but it needs care. Numeric columns are scaled, categories are encoded, and distances may be replaced by measures built for mixed data. Learned embeddings from neural models and table foundation models handle mixed types more naturally than raw distance does.

Is anomaly detection unsupervised?

Often it is. Many detectors learn what normal rows look like from unlabeled data and score how far a new row departs from it. When some labeled anomalies exist, detectors can be tuned or combined with supervised models for better results.

Why is PCA used so often?

PCA is fast, simple, and has a clear meaning. It finds the directions of greatest variation and lets you keep a few of them. It helps with visualization, noise removal, and removing correlated columns before other methods.

How does unsupervised learning relate to tabular foundation models?

Tabular foundation models rely on unlabeled or self-supervised signals to learn how tables behave. Hiding and predicting cells teaches column relations, missing value patterns, and row similarity. The same learned knowledge then supports imputation, anomaly scoring, and semantic classification.

How much data do I need?

It depends on the method and the number of columns. Simple clustering in a few dimensions can work with hundreds of rows. Methods that learn embeddings or densities in many dimensions need far more. More columns need more rows to keep structure reliable.

Can I combine unsupervised and supervised learning?

Yes, and it is common. Cluster labels, PCA components, or embeddings can become features for a supervised model. Unsupervised pre-training on all rows, followed by supervised fitting on the labeled ones, is the basic idea behind semi-supervised and transfer learning.

Glossary

• Anomaly: A row that departs from the normal pattern.
• Association rule: A statement that some items tend to occur together.
• Autoencoder: A network that compresses and rebuilds its input.
• Centroid: The center of a cluster.
• Cluster: A group of similar rows.
• Dendrogram: A tree showing nested merges of clusters.
• Dimensionality reduction: Representing data with fewer variables.
• Embedding: A learned vector that represents a row, column, or category.
• Expectation maximization: An algorithm for fitting mixture models.
• Isolation: Separating a row from the rest by random splits.
• Latent variable: A hidden factor behind the visible data.
• Mixture model: A model that assumes data comes from several groups.
• Principal component: An axis along which variation is greatest.
• Representation learning: Learning useful forms of data without explicit labels.
• Silhouette score: A measure of cluster quality from -1 to 1.
• Soft clustering: Assigning each row a probability for each cluster.
• Standardization: Rescaling a column to zero mean and unit spread.

Summary

Unsupervised learning discovers structure in tables without labels. Its main tools are clustering, dimensionality reduction, density models, and anomaly detection. Its results are views of the data, so judging them takes domain knowledge, stability checks, and downstream tests.

Good practice starts with clean, scaled data and a method matched to the shape of the problem. Avoid over-reading plots, avoid treating clusters as facts, and keep the pipeline stored so new rows can be scored the same way.

For tabular foundation models, unsupervised and self-supervised signals are the fuel for learning from raw tables at scale. They teach row and column structure, mixed types, missing values, and column meaning without names or labels. They also enable imputation, anomaly detection, and semantic classification from one backbone. See the research, architecture, and benchmarks pages for more.

Related

It belongs to Learning paradigms in my stack, under AI and machine learning. Others in the same group: