In-context prediction

In-context prediction means the model makes a prediction by reading the labelled rows that are already in your table, with no training run and no change to its weights. The labelled rows are the support set. A new row is compared with them, and an answer is read off.

This is the reason a prediction takes seconds and not hours of tuning. It is also the reason a prediction can only be as good as the labelled rows it reads.

This page explains the idea with everyday pictures, walks through what goes in and what happens inside, covers classification, regression and anomaly scoring, and spends real time on the part that decides quality: the support set. It closes with the lessons I learned the hard way, where the method fails, and practical guidance. It builds on Cell encoding and Architecture.

The idea in plain words

Imagine two ways of getting help with a new filing cabinet. You could hire a clerk and spend weeks training them on that cabinet. Or you could bring in an experienced clerk, hand them a few solved examples, and ask for the next one.

In-context prediction is the second way. The model is the experienced clerk. It was trained earlier, on a very large and varied collection of tables, so it already knows how tables behave. Your table supplies the solved examples.

The word in-context comes from language models, which can pick up a task from examples placed in their input. The same trick works on tables, where the examples are labelled rows. See few-shot learning for the general idea.

What is not happening

Several things that people expect do not happen.

• No training on your table. Weights are not updated for a prediction.

• No hyperparameter search. There are no settings to tune per table.

• No per-table model. There is one model, and it serves every table.

• No memorizing. The model reads your data at use time and does not keep it in its weights.

What is happening

The model encodes the labelled rows and the rows to predict into vectors. It lets them attend to each other along rows and columns. It then compares each new row with the labelled rows and returns an answer. All of that is a forward pass through fixed weights.

The skill of doing this well was learned during pre-training, when the model practised on a huge number of tables. See Pre-training.

Where it sits in the bigger picture

In-context prediction is one of several kinds of prediction a tabular foundation model can make. It is the one that is most different from the usual workflow.

For the wider landscape see Tabular foundation models. Prior-fitted models such as TabPFN and TabICL use the same principle, and the model on this site applies it to raw cells.

What goes in

A prediction needs three things.

1. A table with labelled rows, in which the target column is filled.

2. Rows to predict, in which the target is unknown or withheld.

3. A choice of task, which can be made for you.

Support rows and query rows

The labelled rows are called the support set, or the context. The rows to be predicted are the query rows.

They go into the model together. The support rows show their targets, and the query rows do not. Nothing else distinguishes the roles.

support set (targets visible)         query rows (targets withheld)
r1  34  gold  2024-11-03  -> churned   q1  41  free  2024-12-01  -> ?
r2  51  free  2024-11-05  -> kept      q2  29  gold  2024-12-02  -> ?
r3  29  gold  2024-11-09  -> kept
r4  46  free  2024-11-10  -> churned

The target and the task

The target is the column to predict. The task is the kind of prediction: classification, regression or anomaly scoring. Both can be set automatically.

The response states what was chosen and why, and lists the candidate columns with the reason each was selected or ruled out. You see the decision and its rationale. See Capabilities.

Rows with a missing target

In classification, rows whose target is missing are counted as dropped and never become a predictable label. A row cannot be a support example if it has no answer to show.

What happens inside

The path of a prediction is the same as the path of any other pass through the model, followed by a small head that does the comparing.

Step one: encode the cells

Every cell, in the support rows and the query rows, becomes a vector that combines value, type and position. Missing cells are encoded explicitly, and column identity is never used. See Cell encoding.

Step two: read along rows and columns

Axial attention alternates between row layers and column layers. In a row layer, each cell looks at the other cells of its own row, which is how the model learns that a high balance and a long tenure go together in one customer. In a column layer, each cell looks at the other values of its own column, which is how it learns what a typical value looks like for that attribute. See axial attention.

Step three: a view of the whole table

A Perceiver-style block lets a small set of latent vectors read the table and write back, so each cell carries a table-level view. See Perceiver IO.

Step four: compare with the supports

The prediction head works by retrieval over the supports. For a query row, it looks at how the row relates to the labelled rows and reads off an answer from the ones that are most informative.

You can think of it as a careful nearest-neighbour judgement, with one important difference. The notion of near is not a fixed distance in the raw columns. It is learned, in the shared representation, and shaped by everything the model has read in the table.

Step five: return the answer

For classification, the answer is a label with a probability for every class. For regression, it is a value with a distribution. For anomaly scoring, it is a score. Each comes with the confidence described below.

Why it works: learning to learn from examples

A reasonable question is how a model with fixed weights can adapt to a table at all.

The answer is that the adaptation is not in the weights. It is in the computation. During pre-training the model was shown an enormous number of tables, each with some rows to read as context and other rows to answer. It was rewarded when its answers were right.

Over many tables it learned a general skill: given a handful of labelled examples, work out the pattern and apply it. That skill lives in the weights, and the particular pattern of your table lives in the context.

The pre-training shape matches the use

The model is asked to work in the shape it will meet at use time, with some rows to read as context and other rows to answer. The objective and the use fit each other, in the same way the masked cell objective and the row and column attention fit each other. See Pre-training.

The prior supplies the variety

Tables in the training mix come from real data and from a synthetic generator. The generator encodes beliefs about how columns relate and how targets depend on features, so the model has practised on a huge variety of relationships. That set of beliefs is the prior, and the model absorbs it. See synthetic data generation.

An analogy is a driving instructor who has taught thousands of students in thousands of cars. A new student in a new car is not a new problem.

A Bayesian way to say it

Prior-fitted networks can be seen as approximating Bayesian prediction. In that view the best prediction for a new point averages over all the explanations that fit the labelled data, weighted by how well each fits. Doing that exactly is too expensive. A network trained on data from a prior can learn to produce a good approximation in one forward pass.

You do not need that framing to use the model. It explains why a single pass can give calibrated-looking answers on small data.

Classification

For classification, each query row gets a label, a confidence and the full probability vector. The classes come from the target column.

{
  "row": 0,
  "label": "churned",
  "confidence": 0.91,
  "probabilities": { "churned": 0.91, "kept": 0.09 }
}

The confidence is the top class probability. The probability vector lets you set your own threshold.

Multiclass

With more than two classes, the vector has one entry per class. Multiclass is one of the boards where the model is behind the leaders, so compare on your own split if the choice matters. See Row-level prediction.

What the benchmarks say

On the row-level prediction boards the model runs one default configuration. It is first on the classification and binary boards at every board size I tested, against rivals that were tuned for hours per dataset.

The overall board only became valid after I found a harness error. A regression run filed under a different method name made the harness fill missing datasets with a penalty. I merged the metadata and re-checked that no row was imputed.

Regression

Regression returns a value, a median, a spread and quantiles, so you can read a range and not only a single number.

{
  "row": 0,
  "value": 12480.0,
  "median": 12010.0,
  "std": 3120.0,
  "quantiles": { "0.1": 8400.0, "0.5": 12010.0, "0.9": 17900.0 }
}

The distribution head

Underneath is a 999-quantile head trained with a CRPS loss. The benchmark paper's own finding is that you win on the scoring rule you train for, and training for CRPS gave first place on CRPS.

A quantile head says, in effect, that 10 percent of the time the answer is below this number, 50 percent below that one, and 90 percent below the next. With 999 of them, you have a fine picture of the whole distribution.

Why standardizing by the standard deviation failed

CRLS lagged at first, and the diagnosis was precise. The target was standardized by its standard deviation, which for heavy-tailed targets such as house prices, runtimes and car prices made the predicted distribution too narrow. CRLS punishes a narrow distribution hard.

The fixes were, in order, a log transform for heavy tails, a CRLS fine-tune that I later dropped when both metrics got worse, and finally split-conformal calibration with an isotonic correction.

Conformal calibration

Conformal calibration is a way to make intervals cover as often as they claim. Part of the labelled rows is held aside as a calibration set. The model predicts those rows, the gaps between predicted and true values are measured, and the intervals are widened or narrowed so that they cover the truth at the stated rate. An isotonic correction then smooths the mapping.

It is an in-context-friendly idea, because it needs only labelled rows that are already in the table. See Probabilistic regression.

What the benchmark says

The model has the lowest median CRPS and the best mean rank on the merged board of 102 regression datasets, with one sealed configuration and no per-dataset tuning. Interval coverage was the weak metric before calibration. Mean absolute error and RMSE were not the target of this head.

Anomaly scoring

Anomaly scoring runs unsupervised, with no target. It returns an anomaly score, a flag and the threshold that was used.

This is in-context in a looser sense. The rows of the table, or the normal rows in the semi-supervised setting, are the context, and a new row is judged by how well it fits them.

Three model-based channels

The detector uses what the model has learned about the table.

1. The residual of the imputation head: how surprising a cell is given everything else in its row.

2. Nearest neighbours in the model's embedding space.

3. Nearest neighbours in the raw space.

An isolation forest joins as an auxiliary channel. See isolation forest.

Choosing channels without labels

Which channel to trust is decided by a pseudo-anomaly selector. Fake anomalies are built from the training rows by shuffling features and injecting uniform values. Each channel is measured on them, and the channels are weighted with a softmax. The ranks are then fused.

If the data has no missing values, the imputation head would have nothing to do, so 15 percent random missingness is injected before fitting.

What the benchmark says

The setting is semi-supervised ADBench: trained on normal rows only and scored on a test set. The detector is first among thirteen methods by both average rank and AUROC. See Anomaly detection.

The support set decides everything

The comparison is only as good as the examples. This is the most important section on the page, because it is where users have the most control.

What a good support set looks like

• Enough rows. The more informative examples there are, the more neighbours a new row has.

• Coverage of the classes. Every class you care about has examples, and rare classes have enough that a new rare row can find a similar one.

• Coverage of the space. The examples span the kinds of rows you expect, not just the common ones.

• Clean labels. Wrong labels mislead the comparison.

• Honest timing. If you will predict the future, the support rows come from the past.

The lesson of 24 supports

My own work gave a vivid example. When I first tried to build row-level prediction, I ran nine similarity-based decoders, and every one hit the same wall. Part of the cause was that the support sets were physically too small, 24 supports, where 128 or more were needed.

No decoder can find a good neighbour that is not in the set. The cure was to give the method enough supports, and it was not a cleverer comparison.

The practical translation is that a handful of labelled rows is enough to start, and more labelled rows keep helping, up to the limits of what the model can read.

Too many rows

A very large table is expensive to read all at once. Attention along the long axis, the rows, grows with the number of rows. This is one reason the model is designed to read a support set of rows and not every row of a large table.

So for big tables, the support set is a sample. A good sample is stratified so that classes are represented, spread across time if time matters, and free of duplicates. A bad sample, for example the most recent thousand rows only, biases the answer.

Class balance

Lopsided classes are common. If 99 percent of rows are one class, a random support set contains very few of the other class.

Stratified sampling that keeps rare classes well represented is the usual fix. Per-class results in the report show whether the fix was enough. See Evaluation.

Duplicates and leakage

If a query row also appears among the supports, the model will recognize it, and the prediction will look brilliant and mean nothing. Remove duplicates between the support set and the rows you want to judge.

Leaky columns work the same way. A column that contains the answer in disguise will be found and used, and it will not exist at the real moment of prediction. Review the columns for leakage before you trust a score.

Label noise

A mislabelled support row is a bad example. The comparison can be misled by it when a query is near that row. A small audit of labels, even a few dozen sampled by hand, is cheap and valuable.

The prior and the XOR wall

In-context prediction inherits the beliefs of the prior it was trained on. My nine failed decoders show what that means in practice.

The training prior turned quantile-band targets into XOR-like patterns. An XOR pattern is one in which neither input alone predicts the output, only the combination does. A decoder that looks for similar neighbours cannot learn that, because similar rows can carry opposite labels.

Changing the decoder did nothing, because the problem was upstream, in what the model had been shown. A cluster-style prior removed the wall.

The lesson is general. When many different heads fail in the same way, look at the data the body was trained on. See Pre-training.

Retrieval, and what I tried instead

The prediction head works by direct retrieval over the supports. I tested alternatives, and the record is worth stating.

Error-correcting output codes

Error-correcting output codes turn a multiclass problem into several binary ones, in the way a communication code adds redundancy to survive noise. They are a classic tool.

Here they added only bit noise. Direct retrieval over the supports stayed.

A leaner retrieval mode

After that I made the retrieval mode leaner and tested each change on the whole board, not on one dataset. A gain on one dataset can hide a loss on ten others, so the full board is the judge.

Where retrieval is weak

Retrieval works when similar rows have similar outcomes. It is weaker when the outcome follows a rule that no neighbour reveals, such as a parity-like combination of columns. Gradient-boosted trees can learn some rules of that kind from a large table.

That is one reason a tuned tree ensemble can still win on a large table with many labels. A comparison on your own split settles it.

The held-out score uses the same mechanism

To measure a prediction honestly, the model must be scored on rows it did not see. The held-out evaluation uses in-context prediction itself.

A stratified test partition is held out, 80/20 by default. The model reads the other 80 percent as the support set, predicts the held-out 20 percent, and the predictions are scored against the true values. The report records the method, the exact partition sizes and the seed. See Evaluation.

Because no weights are trained on your table, nothing can leak through the weights. What must not leak is the answer, and the held-out rows have their targets withheld.

The score claims one thing, performance on unseen rows of the same table. It does not claim performance on a different distribution.

Keeping predictions current

Because there is no training, there is nothing to retrain when the data changes. There are two things to do instead.

Update the context

Add the new labelled rows to the support set, drop the rows that no longer describe the world, and predict again. The model reads whatever you give it.

A refresh does exactly this on the renewed snapshot, and re-measures on a fresh split so that the score reflects the current data and not the old one.

Watch for drift

If the world changes, the old support rows go stale, and the model does not notice. Track live accuracy from labels that arrive after launch, and compare it with the held-out number. A widening gap is an early warning.

In-context prediction and continual learning

These are different things. In-context prediction adapts through the context, per request, and forgets nothing because nothing was learned. Continual learning changes the model itself, through adapters, replay and regularization, so that it can take on new tasks. See Continual learning for tables.

Most users need only the first. The second matters when a platform accumulates many tasks over time.

In-context prediction versus other kinds of in-context learning

The term is borrowed, so it is worth being clear about how it differs from prompting a large language model.

The tabular version keeps the structure of the table. That is the point of reading the grid as a grid. See Tabular foundation models.

The long axis is the row axis, so large tables are read through a support set. The Perceiver-style block keeps the whole-table step linear in the number of cells.

In practice, small and medium tables are fast, large tables are sampled, and very wide tables are expensive on the column side. Dropping columns that are constant, duplicated or irrelevant helps.

Compressed weights

The model can be stored at half precision, 8-bit and 4-bit, and fidelity was measured against the original. A smaller file makes serving cheaper. Because in-context prediction is a forward pass, it benefits directly. See Compression and serving.

A worked example

Take a small, invented case. A shop has 5,000 past orders, of which 4,200 are labelled returned or kept. It wants to predict returns for 800 new orders.

1. The 4,200 labelled rows form the support set, or a stratified sample of it if the table is large.

2. The 800 new orders are the query rows.

3. Every cell is encoded, with value, type and position. The few blank cells are encoded as missing.

4. Axial attention lets each order relate to its own fields and to the typical values of each attribute. The latent block adds a table-level view.

5. The prediction head compares each new order with the labelled ones that resemble it, and returns a label, a confidence and the class probabilities.

6. For the held-out check, a stratified 20 percent of the 4,200 is hidden first, predicted from the other 80 percent, and scored against a majority-class baseline.

No model was trained. If the shop adds a month of new labelled orders, it simply adds them to the support set and predicts again.

Practical guidance

A short list of habits that make in-context prediction work well.

• Give it as many clean, representative labelled rows as you can.

• Sample large tables with stratification, and keep duplicates out.

• Review columns for leakage before trusting a score.

• Split by time when you will predict the future, and by group when rows cluster.

• Write down the baseline before you look at the score.

• Read per-class results and the weakest slice.

• Send low-confidence rows to a person.

• Refresh the context when the world changes, and track live accuracy.

Common mistakes

These are the errors I would expect to see first, and each has a simple remedy.

Judging on the rows you gave as context

If you score the model on rows that were also in the support set, the answer is inflated. The model has seen their targets. Always score on rows whose targets were withheld.

Using a random split for a time problem

A random split lets the model peek at the future through neighbouring rows. If you will predict next month, make the test rows later than the support rows.

Ignoring the baseline

A score of 96 percent is meaningless without knowing the majority-class share. Write the baseline down first, and read the lift.

Trusting the average

An average hides a weak slice. Read per-class results and the weakest slice, because that is where the first real failure will appear.

Feeding in a column that will not exist later

Check that every column is available at the moment you will predict. A column filled in after the event is a leak, and the model cannot tell.

Expecting the model to know what it was not shown

If a class never appears in the support set, it cannot be predicted as a class. If the table over-represents one group, the predictions lean the same way. Refresh the context to cover what you care about.

Reading the confidence

Every prediction comes with a confidence. For classification it is the top class probability. For regression it is carried by the spread and the quantiles. For anomaly scoring it is the score against its threshold.

Confidences are probabilities, and they are the model's own estimate of how much it knows. They are useful, and they are not guarantees.

A simple calibration check

Bin the predictions by confidence, for example everything between 0.8 and 0.9, and compare with the share that turned out right. If the bin says 85 percent and the truth is 60 percent, the model is over-confident for your data, and you should set thresholds accordingly.

Low-confidence rows

Rows with low confidence deserve review. A sensible routine is to let high-confidence predictions through automatically and send the rest to a person.

Table-level confidence

A summary of how typical or unusual the table is feeds into how confident the outputs are. A table that looks very unlike anything the model has seen yields lower confidence across the board. That is a useful warning in its own right.

Why not train a model for each table

It is fair to ask whether in-context prediction is simply a convenience. For some uses it is more than that.

• Speed to first answer. A new table gives an answer in a forward pass.

• No tuning burden. There are no settings to search, and no risk of overfitting a search to a small validation set.

• Consistency. The same model reads every table, so the results have the same shape.

• Small data. With few labels, a model trained from scratch has little to learn from. A pre-trained model brings knowledge.

• A fair test. With nothing trained on your table, the held-out score is a clean measure.

For other uses, per-table training remains the better tool, mainly on large tables with abundant labels and carefully engineered features. The honest position is that the two are complements.

Open questions

The method is young, and some questions are unsettled.

• How to pick the support rows? Stratified sampling is the usual answer, and smarter selection is possible.

• How to keep calibration under drift? Conformal calibration assumes new rows resemble calibration rows.

• How to improve regression and multiclass? These boards are where the model is behind the leaders.

• How to detect that the context has gone stale? The model cannot notice drift unaided.

I list these because they are real, and because a reader deciding whether to rely on the method should know where the edges are.

Where it is strong, and where it is not

See Limitations for the full list and Metadata independence for the name-free results.

Frequently asked questions

What does in-context mean?

It means the model uses the labelled rows in your table as examples, and reads off an answer for a new row by comparing it with them. No weights are updated and no training run happens. The skill of doing this was learned earlier, during pre-training.

Is any training done on my table?

No. The model is pre-trained on other tables, and at use time it reads your table in context. There is no fitting step and no new model per table. Because nothing is trained on your data, nothing can leak into the weights.

How many labelled rows do I need?

There is no single number. A handful is enough to start, and more clean, representative rows keep helping. In my own early work the support sets were too small at 24 supports where 128 or more were needed, and no decoder could make up for it. Rare classes need examples of their own.

How does it handle a huge table?

The row axis is the expensive one, so the model reads a support set of rows and not every row at once. A good sample is stratified by class, spread across time where time matters, and free of duplicates. The whole-table step stays linear in the number of cells.

What does the model return for regression?

A value, a median, a spread and quantiles, so you can read a range and not only a single number. Underneath is a 999-quantile head trained with a CRPS loss, followed by split-conformal calibration with an isotonic correction so that intervals cover as often as they claim.

Can it detect anomalies without labels?

Yes. Anomaly scoring runs unsupervised and returns a score, a flag and the threshold used. It combines three model-based channels with an isolation forest, and a pseudo-anomaly selector weights the channels without labels by building fake anomalies from the training rows.

How is the held-out score computed?

A stratified test partition is held out, 80/20 by default. The model reads the rest as the support set, predicts the held-out rows, and the predictions are scored against the truth, against a baseline. The report records the method, the exact partition sizes and the seed.

What happens when my data changes?

There is nothing to retrain. Update the labelled rows in the context, drop stale ones, and predict again. A refresh re-measures on a fresh split. Track live accuracy from new labels, because the model cannot notice drift by itself.

Is it the same as prompting a language model with examples?

The idea is the same and the mechanics are not. A language model reads examples as a string of tokens. This model reads a grid of encoded cells, with no order imposed on rows or columns, and returns labels, values, distributions or scores. That keeps the structure of the table.

Will it always beat a trained model?

No. It leads on the classification and binary boards under one sealed configuration, and it is behind the leaders on regression and multiclass boards. On a large, clean table with many labels a tuned tree ensemble may win. Compare on your own split.

Summary

In-context prediction lets a pre-trained model predict on your table without any training. The labelled rows are the support set, the rows to predict are the queries, and the prediction head compares each query with the supports and reads off an answer. The skill was learned in pre-training and the pattern comes from your context.

It returns labels with probabilities, values with distributions, or anomaly scores, each with a confidence, and the held-out score uses the same mechanism on rows kept out of context. Quality depends on the support set: enough rows, clean labels, coverage of the classes, no duplicates, no leaky columns and honest timing.

The method has limits, from regression and multiclass boards behind the leaders to drift that the model cannot see. For the encoder that feeds it read Cell encoding, for the network read Architecture, and for how to read its scores read Evaluation and Limitations.