Tabular foundation models: prediction from context on any table
A tabular foundation model is pre-trained once on many tables and then predicts on a new table in one forward pass, reading the labelled rows as context. This page explains how, and when to use one.
Overview
Classical models are trained from scratch on every new table. A tabular foundation model learns, during pre-training on many real or synthetic tables, how to read a table it has never seen: the labelled rows go in as context, the rows to predict go in as queries, and predictions come out with no training step. The idea mirrors how language models answer from examples in the prompt.
How it works
| Step | What happens | Read more | Try it |
|---|---|---|---|
| Encode cells | Numbers, categories and text become vectors | Cell encoding | Any table |
| Read context | Attention across rows and columns of the labelled data | In-context learning | Classification |
| Predict | A distribution for each query row | In-context prediction | Probabilistic regression |
| Other tasks | The same model imputes, flags anomalies | Imputation, anomalies |
Compared with gradient boosting
| Gradient boosting | Large tabular foundation model | |
|---|---|---|
| Training per table | Yes, with tuning | None |
| Small data (hundreds of rows) | Often overfits | Strong |
| Probabilities | Need calibration | Usually well calibrated |
| Very large tables | Scales well | Needs context selection or fine-tuning |
| Speed at prediction | Very fast | Slower per row |
The models
TabPFN popularised the approach and reached version 3.5 in September 2026; TabICL, TabSTAR and others extend it. Data language models broaden it beyond prediction to imputation, schema matching and anomaly detection; see what is a data language model and this site's large tabular foundation model.
Limits
Context size limits how many rows are read at once, very wide tables need column selection, and latency per prediction is higher than a tree's. The large tables simulation shows how row selection handles big data.
Frequently asked questions
Do I still need feature engineering?
Much less. The model reads raw columns, though clean types and sensible units still help.
Can it be fine-tuned?
Yes. Fine-tuning on a domain's tables improves accuracy further while keeping the no-training option for new tables.
Summary
Tabular foundation models replace per-table training with in-context prediction, are strongest on small and medium tables, give calibrated probabilities and handle several tasks with one model. Start with what is a tabular foundation model or the model lab.
Back to the topic
This page is part of Machine learning, which maps every concept, tool, simulation and article on the subject.