Özer Cem KELAHMETKELAHMET
Topics · Machine learning

Tabular foundation models: prediction from context on any table

A tabular foundation model is pre-trained once on many tables and then predicts on a new table in one forward pass, reading the labelled rows as context. This page explains how, and when to use one.

Overview

Classical models are trained from scratch on every new table. A tabular foundation model learns, during pre-training on many real or synthetic tables, how to read a table it has never seen: the labelled rows go in as context, the rows to predict go in as queries, and predictions come out with no training step. The idea mirrors how language models answer from examples in the prompt.

How it works

Step What happens Read more Try it
Encode cells Numbers, categories and text become vectors Cell encoding Any table
Read context Attention across rows and columns of the labelled data In-context learning Classification
Predict A distribution for each query row In-context prediction Probabilistic regression
Other tasks The same model imputes, flags anomalies Imputation, anomalies

Compared with gradient boosting

Gradient boosting Large tabular foundation model
Training per table Yes, with tuning None
Small data (hundreds of rows) Often overfits Strong
Probabilities Need calibration Usually well calibrated
Very large tables Scales well Needs context selection or fine-tuning
Speed at prediction Very fast Slower per row

The models

TabPFN popularised the approach and reached version 3.5 in September 2026; TabICL, TabSTAR and others extend it. Data language models broaden it beyond prediction to imputation, schema matching and anomaly detection; see what is a data language model and this site's large tabular foundation model.

Limits

Context size limits how many rows are read at once, very wide tables need column selection, and latency per prediction is higher than a tree's. The large tables simulation shows how row selection handles big data.

Frequently asked questions

Do I still need feature engineering?

Much less. The model reads raw columns, though clean types and sensible units still help.

Can it be fine-tuned?

Yes. Fine-tuning on a domain's tables improves accuracy further while keeping the no-training option for new tables.

Summary

Tabular foundation models replace per-table training with in-context prediction, are strongest on small and medium tables, give calibrated probabilities and handle several tasks with one model. Start with what is a tabular foundation model or the model lab.

Back to the topic

This page is part of Machine learning, which maps every concept, tool, simulation and article on the subject.

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open