Large tabular foundation models: how in-context learning works

The problem: one model per table

Most useful business data lives in tables. Customers and whether they churned. Machines and their failures. Loans and whether they were repaid. Each row is one thing, each column is one property, and one column is the thing you want to predict.

For about two decades the standard recipe has been the same. Split the table into training and test sets and train a model on it. Very often that model is gradient boosting: libraries such as XGBoost, LightGBM and CatBoost build hundreds of small decision trees, each correcting the mistakes of the ones before. You tune, validate and ship.

This works very well, but it has a hidden cost. Every new table means a new training run. The churn model knows nothing about loan defaults. A thousand small tables means a thousand models to tune and maintain.

In-context learning (ICL) changes the arrangement. You train one large model once, in advance, on a huge variety of tables. When a new table arrives, you do not update the model at all. You hand it the labelled rows as context and ask it to predict the labels of the unlabelled rows, in a single forward pass.

An everyday analogy

Think of an experienced house appraiser. Asked to price a house, they do not go back to school. They look at a few comparable sales nearby: three bedrooms, similar garden, sold last spring for this much. Then they look at the house in front of them and give a price, usually with a range.

Their years of experience are the pretraining. The comparable sales are the context. The new house is the query. Nothing in the appraiser's head is "retrained" for this one job. A tabular foundation model is built to behave the same way. Its experience comes from millions of practice tables; its comparables are your training rows.

In-context learning: from language models to tables

ICL in a language model

The phrase became famous with large language models. Give one the prompt "cat → chat, dog → chien, house → ?" and it answers "maison". It was never trained on that exact task; it picked up the pattern from the examples. The weights did not change. The "learning" happened inside one forward pass, as the model's attention layers related the question to the examples. The general idea is covered on the in-context learning page.

ICL on a table

On a table, the prompt is made of rows instead of words.

• The context is your training set: every training row with its features and its label.
• The queries are your test rows: features only, label missing.
• The output is a predictive distribution: a probability for each class, or, for regression, a full distribution over possible values.

When you call fit on such a model, very little happens. It stores the training rows and perhaps precomputes some internal state. When you call predict, it reads the stored rows and the new rows together. No gradient is computed and no weight changes. The same frozen network handles your churn table on Monday and your loan table on Tuesday.

That is why these are called tabular foundation models: one set of weights serves many downstream tasks, applied directly to numbers and categories rather than sentences.

The theory in plain words: prior-data fitted networks

Why should a network that has never seen your table predict well on it? The clearest answer comes from "Transformers Can Do Bayesian Inference" by Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka and Frank Hutter, published at ICLR 2022. It introduced prior-data fitted networks, or PFNs.

The Bayesian ideal

Bayesian statistics has an ideal way to predict. Start with a prior: a belief about which data-generating processes are plausible before seeing any data. Look at the data and update that belief into a posterior. Then average the predictions of every plausible process, weighted by its posterior plausibility. The result is the posterior predictive distribution.

The trouble is that this average is almost never computable exactly for realistic priors. Sampling methods can approximate it, but slowly, and from scratch for every dataset.

The PFN trick

The PFN idea is to learn that averaging instead of computing it each time.

1. Choose a prior you can sample from.
2. Draw a random data-generating process from it and produce a small synthetic dataset.
3. Hide the labels of some points.
4. Train a network to predict the hidden labels from the rest.
5. Repeat with fresh synthetic datasets, millions of times.

After enough practice, the network does in one forward pass roughly what Bayesian inference under that prior would do. The paper frames this as turning posterior approximation into ordinary supervised learning.

Posterior predictive distribution (the ideal answer):

  p(y* | x*, D)  =  ∫ p(y* | x*, φ) · p(φ | D) dφ

What a PFN computes in one forward pass:

  p(y* | x*, D)  ≈  q_θ(y* | x*, D)

How the PFN is trained (expected negative log-likelihood over the prior):

  L(θ)  =  E over (D, x*, y*) drawn from the prior [ − log q_θ(y* | x*, D) ]

Symbols
  D        the context: n labelled rows {(x_1, y_1), ..., (x_n, y_n)}
  x*       the features of a query row whose label is unknown
  y*       the unknown label of that query row
  φ        one possible data-generating process (one "explanation" of the data)
  p(φ | D) how plausible that process is after seeing D (the posterior)
  p(y* | x*, φ)  what that process predicts for x*
  q_θ      the neural network with weights θ; it reads D and x* and outputs
           a distribution over y*
  E[...]   the average over many synthetic datasets sampled from the prior
  − log q  the loss: small when the network gave high probability to the
           true hidden label

In plain English: the top line says the best prediction averages over every plausible explanation of the data. The middle line says the network's output stands in for that average. The bottom line says we train the network on vast numbers of synthetic datasets and penalise it whenever it gives low probability to the label we hid.

The paper shows that minimising this loss pushes the network's output towards the true posterior predictive distribution, on average over the prior. That is the formal reason a PFN can behave sensibly on a new dataset: if your table looks like something the prior could have produced, the network has, in effect, practised on tables like yours.

Why this is not magic

The guarantee holds only relative to the prior. If the prior never produces anything like your data, the network has no reason to do well. This is the single most important point in the field: a tabular foundation model is only as good as the range of worlds it practised on.

Synthetic priors: what the model practised on

Real tabular datasets are scarce, messy and often private, so the TabPFN line pretrains entirely on synthetic data. The design of the generator is the prior, and it encodes almost all of the model's assumptions.

Structural causal models

The main ingredient is the structural causal model (SCM), a recipe for generating data with cause-and-effect structure, familiar from causal inference.

1. Draw a random directed acyclic graph. Nodes are variables; arrows run from causes to effects.
2. Define how each node is computed from its parents. In the TabPFN v2 Nature paper these mappings include small neural networks, discretisation steps and decision trees.
3. Add random noise at each step.
4. Pick some nodes as features and one as the target, and sample rows.

Each synthetic dataset is a new random world: rainfall affects yield, yield affects price, and a few noisy sensors measure some of it. Because every world differs, the model cannot memorise any of them. It must learn general habits: which columns matter, how smooth relationships tend to be, what noise looks like. According to the Nature paper, each final TabPFN v2 model was trained on roughly 130 million synthetic datasets (the main text rounds this to about 100 million).

Other priors

• Bayesian neural networks and Gaussian processes. The 2022 PFN paper showed PFNs closely approximating these posteriors, which made them natural early priors.
• Tree-based priors. Amazon's Mitra (July 2025 blog post, NeurIPS 2025 paper) mixes SCM priors with priors built from gradient boosting, random forests, decision trees and extra trees. Its authors argue that prior design matters more than architecture, and that a good mixture is diverse and distinctive, not just individually strong.
• Real data. TabDPT, from Layer 6 AI, pretrains on real OpenML tables with a self-supervised objective: hide a column and predict it from the others.

What the prior encodes

Think of the prior as a curriculum of everything the model should expect.

• Inductive biases. The original TabPFN prior was described by its authors as having "a bias for simplicity", in the spirit of Occam's razor.
• Noise. How noisy labels and features typically are.
• Missing values. In TabPFN v2's prior some values are masked completely at random, so the model learns to predict with holes in the data.
• Categorical features. These arise by discretising continuous variables, so the model learns that some columns are labels rather than quantities.
• Shapes and sizes. Whatever ranges of rows, features and class balance the prior samples are the ranges the model is comfortable with.

If your data has a property the prior never produced, such as strong time ordering or free-text columns, treat the output with extra care.

Architecture: how a table becomes tokens

The networks are transformers, but arranged differently from language models because a table is two-dimensional.

From cells to tokens

TabPFN v2 gives each cell its own representation: a small learned encoder maps each value into a vector. Each training row's label is encoded and attached to that row. Query rows carry no label, so the model knows which rows it must predict.

Two-way attention

The model then alternates between two kinds of axial attention.

• Across features, within a row. Each cell looks at the other cells of its row. This is how the model sees that, for this customer, high usage and a recent complaint go together.
• Across rows, within a column. Each cell looks at the same feature in other rows. This is how a query customer is compared with training customers, like the appraiser comparing houses.

The Nature paper notes that test rows attend only to training rows, not to each other. A prediction therefore does not depend on which other queries share the batch, and the training rows' internal state can be computed once and reused.

Permutation invariance

Rows have no natural order, and usually neither do columns. Shuffling them should not change the answer. Two-way attention without positional encodings provides this by design, treating rows and features as sets. Implementations still average over several feature orders and preprocessings to smooth out small differences.

Mixed types and missing values

Numeric values are normalised inside the model's own preprocessing, categorical columns are encoded as categories, and missing values are encoded explicitly. Because the prior contained all three, the model has learned what to do with each.

Regression with a bar distribution

For regression the model splits the range of target values into many small bins and outputs a probability for each. The result is a piecewise-constant density, often called a bar distribution: a histogram drawn for each query row. It can express skewed or two-peaked beliefs, such as "about 10 or about 50, but not 30", which a single mean and variance cannot.

The flow at a glance

 training rows (features + labels)         query rows (features only)
              |                                       |
              v                                       v
     +--------------------------------------------------------+
     |  cell encoders: numbers, categories, missing flags     |
     |  label tokens attached to training rows only           |
     +--------------------------------------------------------+
                               |
                               v
     +--------------------------------------------------------+
     |  repeat L times:                                       |
     |    attention across features  (inside each row)        |
     |    attention across rows      (inside each column;     |
     |                                queries read training   |
     |                                rows, not each other)   |
     +--------------------------------------------------------+
                               |
                               v
     +--------------------------------------------------------+
     |  output head per query row                             |
     |    classification: probability for each class          |
     |    regression: probability for each bin (bar density)  |
     +--------------------------------------------------------+
                               |
                               v
             predictive distribution for every query row

Variations on the theme

TabICL uses two stages: column-then-row attention compresses each row into a fixed-size vector, then a transformer performs in-context learning over those vectors. Because the expensive grid attention only builds row vectors, it scales to far more rows. TabDPT retrieves the nearest training rows for each query and uses them as context. TabuLa-8B turns each row into text for a language model.

The models, verified

Facts below were checked against papers or official documentation as of November 2026. Limits are the officially stated design or recommended limits, not hard walls. Vendor technical reports are self-reported.

Notes on the table

TabPFN (2022). The first TabPFN was deliberately narrow: up to 1,000 training examples, 100 features and 10 classes, mostly numeric data without missing values. Within that range, its authors reported results competitive with AutoML systems given an hour, in under a second.

TabPFN v2 (2025). The Nature paper, "Accurate predictions on small data with a tabular foundation model", extended the range to 10,000 samples and 500 features and added regression, categorical features and missing values. Its headline claim, on datasets up to 10,000 samples: "in 2.8 s, TabPFN outperforms an ensemble of the strongest baselines tuned for 4 h in a classification setting". The authors are candid about limits: inference can be slower than optimised tree libraries, memory grows with dataset size, and for larger or highly non-smooth regression tasks CatBoost, XGBoost or AutoGluon may do better.

Later TabPFN versions. Prior Labs has released versions quickly since. Its documentation lists TabPFN-3.5 as the open-source package's default. The code is Apache 2.0, but newer weights carry licences that restrict production use. Check before you build.

TabICL and TabICLv2. TabICL was the first open model to show ICL stretching well past 10,000 rows; on 53 datasets with more than 10,000 samples its paper reports beating both TabPFN v2 and CatBoost. The v2 paper reports beating a tuned, ensembled and fine-tuned RealTabPFN-2.5 on TabArena and TALENT without tuning, and releases pretraining and prior-generation code as well as weights.

LLM-based approaches. TabuLa-8B serialises rows as text and fine-tunes Llama 3-8B on more than 2.1 billion rows from over 4 million tables. It can use column names and text values, but its 8,192-token window fits only a few dozen examples. Its authors report it was 5 to 15 percentage points more accurate than XGBoost and TabPFN in the 1 to 32 shot range.

CARTE, as a contrast. CARTE is not an in-context learner. It pretrains a graph-attentional network that represents rows as small graphs of column names and values, with string embeddings so any vocabulary works, then fine-tunes on your table. It can transfer between tables whose columns do not match. It is a useful reminder that "pretrained" and "in-context" are different ideas.

Scaling the context

Why rows and features are expensive

Attention compares every token with every other token in the dimension it attends over. On a grid, each cell attends to its row-mates and its column-mates, so cost grows faster than the table does: double the rows and the cross-row work per cell roughly doubles too. Memory grows because every cell's representation must sit on the GPU. That is why the first model stopped at a thousand rows. These strategies push the limit outward.

Subsampling

Use a random subset of training rows as context: twenty comparable sales instead of two thousand. Simple and often good, but it discards information.

Ensembling over permutations and subsets

Run the model several times with different feature orders, preprocessings or row subsets, and average. Like asking several appraisers who each saw different comparables, it reduces variance at the price of more compute. The TabPFN and TabICL packages do this by default.

Retrieval of local context

For each query, find its nearest neighbours in the training set and use only those as context: the appraiser looking only at houses on the same street. LoCalPFN (Thomas et al., NeurIPS 2024) and TabDPT use this. It scales because each query gets a small, relevant context, but it depends on a sensible distance between rows.

Compressing rows first

TabICL turns each row into a single vector, then performs ICL over vectors instead of cells: each comparable sale summarised on one index card.

Caching and chunking: fit once, predict many

The training rows do not change between predictions, so their internal representations can be computed once and stored, like the KV cache in language models. The TabPFN package exposes this as fit_mode="fit_with_cache", TabICL as kv_cache=True. The TabPFN-2.5 report notes the trade-off: the cache costs memory per training cell, so it suits smaller training sets or many repeated predict calls. The TabPFN-3 report combines a reduced KV cache with row chunking, processing rows in pieces so peak memory does not grow with the whole table, to reach a million rows on one H100.

Fine-tuning on the table

Finally, you can fine-tune the pretrained model on your table. Mitra reports fine-tuned variants, LoCalPFN fine-tunes with retrieved neighbours in context, and RealTabPFN variants are fine-tuned on real data. This often helps on larger tables but brings back training cost, validation and the risk of overfitting: the appraiser spending a week studying one unusual neighbourhood.

ICL versus fine-tuning versus gradient boosting

What the benchmarks say, with their conditions

The TabPFN v2 Nature paper. Strong classification and regression results on datasets up to 10,000 samples and 500 features against baselines tuned for four hours. Its own limitations section points to larger datasets as where trees may still win.

TabArena (Erickson, Purucker, Tschalzev, Holzmüller, Desai, Salinas, Hutter; NeurIPS 2025 Datasets and Benchmarks). A curated, continuously maintained benchmark. Its main findings: gradient-boosted trees "are still strong contenders on practical tabular datasets"; deep learning catches up with larger time budgets and ensembling; foundation models excel on smaller datasets; and validation and ensembling choices strongly affect every method's ranking. Since then, Prior Labs and the TabICL team have each reported their newest models at the top of the leaderboard. Those are builders' claims on a benchmark that keeps changing.

TALENT (Ye, Liu, Cai, Zhou, Zhan; Nanjing University). A toolbox and benchmark of more than 300 datasets. Its authors conclude that gradient-boosted trees remain very strong baselines and that pretrained tabular models are narrowing, but not eliminating, the advantage of tree ensembles. The mix of categorical and numeric features strongly influences which family wins.

When each tends to win

In-context learning tends to win on small and medium tables (hundreds to tens of thousands of rows, depending on the model), when you have many different tables and no time to tune each, when you want good probabilities without a calibration step, and when you need a strong baseline in seconds.

Gradient boosting tends to win on very large tables, where it is cheap to train and fast to serve; when you need microsecond predictions on a CPU; when mature interpretability tooling and regulatory familiarity matter; and when the data is far from any synthetic prior.

Fine-tuning a foundation model tends to win when the table exceeds the comfortable context range, when one table matters enough to justify the compute, and when you have validation data to guard against overfitting.

Latency, cost and interpretability

ICL moves cost from training to prediction. A boosted model is expensive to tune but almost free to query; an ICL model is free to "train" but every prediction pays to read the context, usually on a GPU. If you serve millions of predictions from one fixed table, that matters. Caching and distillation exist to close the gap: the TabPFN-2.5 report describes converting the model into a compact MLP or tree ensemble for low-latency serving.

Tree models also come with mature explanation tools. Model-agnostic methods such as SHAP work on foundation models too, but need many forward passes, so budget for them.

Uncertainty and calibration

Why a distribution beats a number

"The house is worth 400,000" is a guess. "Most likely 400,000, plausibly 360,000 to 450,000, very unlikely below 300,000" is a basis for a decision. A bank approving a loan or a factory scheduling maintenance needs to know how sure the model is, not just its best guess.

Because PFN-style models are trained to approximate the posterior predictive distribution, uncertainty is what they output, not an add-on. Classification gives class probabilities; regression gives the full bar distribution, from which you can read a median, quantiles or an interval.

Calibration

A model is calibrated if its probabilities mean what they say: of all the cases where it says 70 per cent, about 70 per cent turn out positive. Many models are over-confident and need a separate calibration step. PFNs have a structural advantage, because their loss is a log-likelihood averaged over many datasets, which rewards honest probabilities. When your data matches the prior, calibration tends to be good without extra work. When it does not, calibration can degrade, so check it with a reliability plot or expected calibration error on held-out data.

Know what the uncertainty covers

The model's uncertainty is uncertainty under its prior and your context. If your test data comes from a different distribution, for example next year's customers, the model cannot know. No predictive distribution protects you from a shifted world you never showed it.

A practical guide

A minimal example

This uses the official tabpfn package and its scikit-learn style interface, following the current README. On first use the package downloads the weights and asks you to log in to Prior Labs and accept the weights licence; in headless environments you accept the licence on their site and set the TABPFN_TOKEN environment variable. A GPU is recommended. The default model at the time of writing is TabPFN-3.5, whose licence restricts production use.

from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score, roc_auc_score
from sklearn.model_selection import train_test_split
from tabpfn import TabPFNClassifier

# A small public dataset: 569 rows, 30 numeric features, 2 classes
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42, stratify=y
)

clf = TabPFNClassifier()
clf.fit(X_train, y_train)            # stores the context; no gradient training

proba = clf.predict_proba(X_test)    # one probability per class, per row
pred = clf.predict(X_test)

print("ROC AUC :", roc_auc_score(y_test, proba[:, 1]))
print("Accuracy:", accuracy_score(y_test, pred))

Notice what is missing: no learning rate, no number of trees, no early stopping. TabICL's package has the same shape, with TabICLClassifier().fit(X_train, y_train) followed by predict.

A decision checklist

1. Size. Are your rows and features inside the stated range of the model and version?
2. Task. Classification or regression? How many classes? Older versions support 10 natively.
3. Data types. Mostly numbers and categories, or free text that needs a text-aware model?
4. Order. Are rows independent, or is there time or group structure that the split must respect?
5. Licence. Can you use these weights for your purpose, including production?
6. Hardware and latency. Do you have a GPU? How many predictions per second do you need?
7. Baseline. Train a default gradient-boosting model on the same split. Never judge a foundation model without one.
8. Calibration. Check it on held-out data, not just accuracy.
9. Decide. If the foundation model clearly wins within your constraints, use it. If not, try retrieval, fine-tuning, or an ensemble of both families.

Common mistakes

Leakage through the context. The context is training data. If a test row or a near-duplicate sits in it, the model will copy its label and your score will look wonderful. Split first, then build the context only from the training split.

Wrong assumptions about scaling. These models do their own preprocessing. Extra scaling rarely hurts, but one-hot encoding hundreds of categories can push you past the feature limit and remove structure the model would have used. Pass categories as categories where supported.

Too many classes. Beyond the native class limit, check whether the library offers a many-class extension, and expect slower predictions.

Time-ordered data. A random split lets the model see the future in its context. Use a time-based split, with context strictly before queries.

Privacy of context rows. Every prediction reads the context. With a hosted API, your training rows travel with each request; a saved fitted model may contain them. Treat a fitted ICL model like the raw training data.

Trusting one leaderboard number. Leaderboards change monthly. Test on your own data with your own metric.

Where the field is going

This section is an outlook: directions visible in recent papers, not settled results.

Longer contexts

In under four years, stated limits moved from 1,000 rows to a million. Reduced KV caches, chunking, grouped-query attention and faster kernels, mostly borrowed from language models, are doing much of that work.

Multi-table and relational inputs

Business data is rarely one flat table; it is customers, orders and payments linked by keys. Today you flatten them by hand. Models that read relational structure directly, in the spirit of CARTE's graph view or LimiX's joint modelling, are an active area, and the TabPFN-3 report lists relational and temporal domains among its open questions.

Time series

Forecasting can be framed as tabular prediction with time-aware context. Prior Labs describes TabPFN-3.5 as stronger on temporal and grouped data, and priors with temporal structure are being built.

Test-time compute and memory

Some recent versions spend extra compute at prediction time for accuracy, as reasoning language models do. A related research line gives models memory that updates during inference, explored for sequence models in the article on Titans and MIRAS. For tables, that could mean a model that improves as it sees more of your data without a fine-tuning step.

Agents using tabular models as tools

Agents are good at reading questions, writing queries and explaining results, but poor at precise numerical prediction from their own weights. A natural pattern is for the agent to call a tabular foundation model as a tool: assemble the context table, call fit and predict, and report the distribution in plain words. Because ICL needs no training step, it fits the request-and-response rhythm of tool calls well. The large tabular foundation model documented on this site is one example of this family.

Frequently asked questions

Does a tabular foundation model really learn nothing from my data?

It learns nothing in the sense of changing its weights. It uses your data intensely at prediction time: every prediction is computed by attending to your labelled rows. The learning happens inside the forward pass, the way a person applies experience to new examples without being rewired. If you want the weights to adapt as well, that is fine-tuning, a separate optional step.

Is TabPFN better than XGBoost?

It depends on the table. On small and medium tables, the TabPFN v2 Nature paper and TabArena show tabular foundation models often matching or beating tuned gradient boosting with no tuning. On very large tables, or where CPU latency matters, gradient boosting often remains the better choice, and independent benchmarks such as TALENT still find it very strong. Run both on your data.

Why train on synthetic data instead of real tables?

Synthetic data is unlimited, free of privacy problems and fully controllable. You can generate a hundred million datasets with known structure, far more variety than any public collection. The cost is that the model only knows what the generator produces, which is why some models, such as TabDPT, pretrain on real data, and others fine-tune on it afterwards.

What is a prior-data fitted network in one sentence?

A network trained on many datasets sampled from a prior, so that, given a new dataset as input, it outputs an approximation of the Bayesian posterior predictive distribution in a single forward pass.

Can I use these models on large datasets?

Increasingly, yes. TabICLv2 and TabPFN-3 report handling up to around a million rows through compression, caching and chunking. Cost per prediction still grows with the context and a GPU becomes necessary. For very large tables, subsampling, retrieval or fine-tuning are common, and gradient boosting remains strong and cheap.

Do I need to scale or encode my features first?

Usually not much. The models normalise values and handle missing values and categorical columns themselves. Clean obvious errors, pass categorical columns in the form the library expects, and avoid creating hundreds of one-hot columns. Read the documentation for your exact version, because preprocessing changes between releases.

Are the probabilities trustworthy?

Often more than those of an untuned model, because the training objective rewards honest probabilities. Trust them only after checking calibration on held-out data from the distribution you will predict on. Under drift, or on data far from the prior, calibration suffers like any model's.

How do large language models fit into tabular prediction?

Models such as TabuLa-8B turn rows into text and use a language model's in-context learning. They understand column names and text values, but have short context windows and are costly to run. Dedicated tabular models are usually stronger and cheaper on numeric tables; text-aware models help when meaning lives in words.

Summary

A large tabular foundation model replaces "train one model per table" with "train one model once, then read each new table as context". The idea rests on prior-data fitted networks: a transformer trained on millions of synthetic datasets drawn from a prior, such as structural causal models, learns to approximate Bayesian posterior predictive inference in one forward pass. These models treat a table as a grid of cell tokens, attend across rows and across features, ignore row and column order, and output full predictive distributions. The family has grown from TabPFN's 1,000-row limit in 2022 to models reporting a million rows in 2026, alongside open models such as TabICL, real-data approaches such as TabDPT, prior mixtures such as Mitra, and LLM-based ones such as TabuLa-8B. Independent benchmarks show them strongest on small and medium tables, while gradient-boosted trees stay hard to beat at scale and on latency. Use them with a clean split, a gradient-boosting baseline, a calibration check and a careful reading of the weights licence.

Sources

• Müller, Hollmann, Pineda Arango, Grabocka, Hutter: Transformers Can Do Bayesian Inference (ICLR 2022)
• Hollmann, Müller, Eggensperger, Hutter: TabPFN, A Transformer That Solves Small Tabular Classification Problems in a Second (ICLR 2023)
• Hollmann et al.: Accurate predictions on small data with a tabular foundation model (Nature, 2025)
• Prior Labs: TabPFN-2.5, Advancing the State of the Art in Tabular Foundation Models
• Prior Labs: TabPFN-3.5 Technical Report
• Prior Labs documentation: TabPFN models, limits and licences
• Qu, Holzmüller, Varoquaux, Le Morvan: TabICL, A Tabular Foundation Model for In-Context Learning on Large Data (ICML 2025)
• Qu, Holzmüller, Varoquaux, Le Morvan: TabICLv2, A Better, Faster, Scalable, and Open Tabular Foundation Model (ICML 2026)
• Ma et al., Layer 6 AI: TabDPT, Scaling Tabular Foundation Models
• Zhang, Maddix et al., Amazon: Mitra, Mixed synthetic priors for enhancing tabular foundation models
• LimiX team: LimiX, Unleashing Structured-Data Modeling Capability for Generalist Intelligence
• Gardner, Perdomo, Schmidt: Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024)
• Kim, Grinsztajn, Varoquaux: CARTE, Pretraining and Transfer for Tabular Learning (ICML 2024)
• Erickson et al.: TabArena, A Living Benchmark for Machine Learning on Tabular Data (NeurIPS 2025)