# Missing-value imputation benchmark: filling the gaps in real tables

How a tabular foundation model fills missing numeric and categorical cells, and how a hunt for fifteen measurement errors produced a fair first place.

> Informational documentation, based on the paper Data Language Models: A New Foundation Model Class for Tabular Data (arXiv:2605.06290, https://arxiv.org/abs/2605.06290). Reference material, not a hosted service; hosts, keys, and ids are placeholders.

Imputation means filling in the cells that are blank. This benchmark hides cells in real tables, asks a method to reconstruct them, and measures how close it gets. The tabular foundation model described here came first on the real-data board, ahead of language models and the classical imputers in every setting, with the widest margin under not-at-random missingness.

This page explains the task, the three ways data goes missing, how I built the imputer, the fifteen errors I found in my own harness and chain, what made the difference, and what stayed open. Scores are left out on purpose. The model card carries them with their protocols. For all boards, see Benchmarks.

## The short version

- Numeric imputation hides cells and measures reconstruction error (NRMSE) on 20 real datasets under three missingness mechanisms.
- Categorical imputation measures macro-F1 on masked categorical cells and has no published competitor.
- This was the longest hunt, and most of it was fixing my own measurement. I found fifteen errors in the harness and the model chain.
- The final design is a chain of three arms: ridge, in-context, and neighbours. The arm for each column is chosen on masked holdouts.
- For categories I replaced argmax with the expected value, because NRMSE measures squared error.
- A plain ridge chain matched MICE exactly, which gave me a verified floor.
- On two synthetic settings the best published MICE figure stayed ahead of my own runs, and I left those rows as reported.

## What imputation is

Real tables have holes. A sensor drops out. A customer skips a field. A lab test was never ordered. Most pipelines mean-fill the holes and hope.

The model fills gaps from the joint structure of your own data, not from generic priors, and scores every fill so a pipeline can gate on it. Missing values are treated as signal, not as errors. See Solutions, question six, "what about the gaps?".

### Why the mean is a poor fill

Filling every blank in a column with that column's average is simple and almost always wrong. It ignores everything else in the row. If a customer is young and has a short tenure, their missing income is probably not the average income of all customers.

An everyday analogy: you are completing a jigsaw puzzle with a few missing pieces. You do not guess the missing piece from the average colour of the whole puzzle. You look at the edges of the hole and fill it to match what surrounds it.

### Numeric and categorical

- **Numeric cells** get a number. The error is measured with NRMSE.
- **Categorical cells** get a class label. The score is macro-F1 on masked cells.

Both are covered by the same machinery. See Imputation networks for the general idea.

## What the benchmark measures

| Part | Setting |
| --- | --- |
| Numeric task | Hide cells, measure reconstruction error with NRMSE |
| Datasets | 20 real datasets |
| Mechanisms | Completely at random, at random, not at random |
| Categorical task | Macro-F1 on masked categorical cells |
| Categorical competitor | None published |
| Rivals | Language models and classical imputers |

### NRMSE in plain words

RMSE is the typical size of an error, with big errors counting extra because they are squared. NRMSE divides it by the scale of the column, so a column measured in millions and a column measured in single digits can be compared. Lower is better.

Squared error has a useful property: the best single guess under squared error is the mean of the possible answers, not the most likely one. This matters for the categorical story below.

### Macro-F1 in plain words

F1 balances two questions: of the cells I filled with a class, how many were right, and of the cells that truly had that class, how many did I find. Macro means the score is computed for each class and then averaged, so rare classes count as much as common ones. A method cannot win by always guessing the biggest class.

## The three ways data goes missing

The mechanism of missingness changes how hard imputation is. These are the standard mechanisms, and the pre-training mix covers all of them. See Pre-training.

| Mechanism | Plain meaning | Example |
| --- | --- | --- |
| Missing completely at random (MCAR) | The gap has nothing to do with any value | A sensor drops a packet |
| Missing at random (MAR) | The gap depends on other observed values | Older customers skip an online field |
| Missing not at random (MNAR) | The gap depends on the hidden value itself | High earners leave income blank |

### Why MNAR is the hardest

When the reason for a gap is the very value you cannot see, the observed data is a biased sample. Simple methods that assume gaps are harmless fail here. A method that models how the columns relate, and that treats the pattern of blanks as information, has a chance.

This is where the widest margin over the other methods appeared.

## How I built it

The honest summary is that this was the longest hunt on the timeline, and most of it was fixing my own measurement. The model was not the first suspect, but the harness was.

### The first design

The imputation network reads the shared representation of a masked or empty cell and returns a value. It was trained with masked cell modeling, which hides cells and asks the model to reconstruct them. See Masked cell modeling and Architecture.

For numbers, the output is a number. For categories, the output is a distribution over class labels. Confidence is reported per row.

### The final design: a chain of three arms

On the hardest imputation benchmark, the final system was not a single head. It was a chain of three arms.

| Arm | What it is | Role |
| --- | --- | --- |
| Ridge | A plain linear model | Hard to beat, and a verified floor |
| In-context | The model's own reading of the table | Captures joint structure |
| Neighbours | Borrows values from similar rows | Helps when similar rows exist |

The chain is explained in the same words on the Architecture page.

### Choosing an arm per column

The arm used for each column is chosen on masked holdouts. The method hides some known cells in that column, tries each arm, and keeps the arm that reconstructs them best.

### Margins against ridge

There is a statistical trap here. When only 30 holdout cells decide among 77 candidates, the winner is partly luck. If you test 77 options on 30 cells, one will look best by chance.

So I set fixed proportional margins against the plain ridge arm. A candidate must beat the baseline by a set margin, not by a hair. The margin protects against picking noise.

```text
for each column:
    hide ~30 known cells
    score ridge on the holdout            -> baseline
    score each candidate arm and setting  -> up to 77 candidates
    keep a candidate only if it beats the baseline by a fixed proportion
    otherwise use ridge
```

### Expected value for categories

For categories I replaced argmax with the expected value. NRMSE measures squared error, and the mean of the distribution is optimal for squared error. Picking the single most likely class throws away the shape of the distribution.

An everyday analogy: if a coin flip pays you by how close your guess is to the outcome coded as 0 or 1, saying 0.5 is better on average than always saying 1. Argmax says 1 every time the odds tip even slightly.

## Fifteen errors in my own harness and chain

An imputation benchmark is easy to get wrong because the cheats are quiet. I found fifteen errors in the harness and the model chain. Several of them stand for the whole class.

### Error: the imputer saw a fifth of each table

The imputer was given only a fifth of each table, the test fold. A model that sees 20 percent of the data has far less context than one that sees all of it. I let it see train and test together, with both masked. The masking applies to both, so nothing known about the test cells leaks. The model simply has the whole table to reason about, as it would in use.

### Error: placeholder column names

The semantic channel had been fed placeholder column names. It was reading noise as if it were meaning. After the fix it gets what it should, in line with the rest of the model, which never relies on a column's identity. See Metadata independence.

### Error: the neighbours arm cheated

The nearest-neighbour arm read real values on its own validation cells. When it was scored on those cells, it was copying answers. Its validation looked excellent, and the test did not. The fix was to hide the validation cells from the arm while it works.

### The class of errors

| Class | Symptom | Cure |
| --- | --- | --- |
| Too little context | Imputer sees a fifth of the table | Show train and test together, both masked |
| Wrong input | Placeholder names in the semantic channel | Feed what the model is meant to see |
| Leakage | Arm reads the very cells it is scored on | Hide validation cells during the fit |
| Wrong estimator | Argmax used for a squared-error metric | Use the expected value |
| Selection by luck | 77 candidates on 30 cells | Fixed proportional margins |

### Why I list them

Most of the gain on this board came from fixing the ruler, not the model. This echoes the rule on every page of the timeline. See Pre-training, section "Keeping the ruler honest".

## The verified floor

A good habit is to build a floor you trust before you build a ceiling. A plain ridge chain matched MICE exactly. MICE is a classical method that fills each column in turn from the others, over several rounds.

Matching it exactly was reassuring in two ways.

- The harness reproduced a known method's behaviour, so the harness was sound.
- The new arms were measured against a floor that was already as good as a respected baseline.

Anything above that floor is then a real gain, not a fault in how the floor was measured.

## What made the difference

- **Letting the imputer see the whole table.** The context mattered.
- **Removing the quiet cheats.** Placeholder names and the leaking neighbours arm.
- **A chain, not a single head.** Three arms with different strengths.
- **Per-column choice with margins.** Chosen on masked holdouts, protected from luck.
- **Expected value for categories.** The right answer for a squared-error metric.
- **A verified floor.** Ridge matching MICE.

The result is first place on the real-data board, ahead of language models and the classical imputers in every setting, with the widest margin under not-at-random missingness.

### Why language models fall behind

Language models serialize the table into text. That flattens the joint structure of columns, which is exactly what imputation needs. They also do not natively handle the numeric precision that NRMSE rewards. See Research for the argument against serialization.

## The categorical side

Categorical imputation measures macro-F1 on masked categorical cells. There is no published competitor, so the claim is bounded: the model fills categorical cells and the score is reported with its protocol. It is a capability that other imputers on the numeric board do not cover.

### Why this matters in practice

Real tables are full of categories: a product type, a diagnosis code, a country. Leaving those blank blocks downstream work as surely as a missing number does.

- A class label comes back with a confidence per row.
- A pipeline can gate on that confidence and send low-confidence rows to a person.
- The filled value is a class label, not a distribution, when you ask for a fill.

## What stayed open

- **Two synthetic settings.** The best published MICE figure stayed ahead of my own runs on two synthetic settings. I left those rows as reported and did not retune to catch up.
- **Few columns or few rows.** On a few datasets with few columns or few rows the chain has little context, and the gain is small.
- **Categorical has no external yardstick.** With no published competitor, there is no third-party board.
- **Confidence is not certainty.** A low-confidence fill deserves review. See Limitations.

### Why I did not tune to fix the synthetic rows

Retuning on the synthetic rows would break the standing rule: one sealed configuration per benchmark, with no per-dataset tuning. A number that comes from a configuration chosen after seeing the answer is not comparable to the others.

## Protocol

- Numeric imputation: hidden cells, NRMSE, 20 real datasets, three missingness mechanisms.
- Categorical imputation: macro-F1 on masked cells.
- The imputer sees train and test together, both masked.
- Arms chosen per column on masked holdouts, with fixed proportional margins against ridge.
- One sealed configuration, with no per-dataset tuning.
- Competitor figures are third-party published values cited to their primary source, not re-run by me.
- Results under different datasets, metrics or masking schemes are never blended into one number.
- Each run is checked for contamination against the training corpus.
- The protocol follows arXiv:2603.22332.

## How the output looks

A fill comes with a confidence per row. The shape is the same for every table.

```json
{
  "imputation": {
    "column": "income",
    "type": "numeric",
    "fills": [
      { "row": 18, "value": 51200.0, "confidence": 0.81 },
      { "row": 77, "value": 38900.0, "confidence": 0.44 }
    ]
  }
}
```

The numbers are invented to show the shape. See Capabilities for the full output bundle.

### Gating on confidence

- Accept fills above a confidence you choose.
- Send the rest to review.
- Track the share of low-confidence fills as a data-quality signal.

## How this connects to other boards

Imputation is related to several other results on this site.

- **Anomaly detection.** The residual of the imputation head is one of the three model-based channels of the detector. See Anomaly detection benchmark.
- **Missing-data robustness.** Predictions under injected gaps use an imputation family chosen per dataset. See Missing-data robustness.
- **Matching.** Both rely on the model reading relations between columns. See Entity and schema matching.
- **Packaging.** The imputation head ships in the merged checkpoint. See One checkpoint.

## Lessons from this board

- Build a floor you trust before you build a ceiling.
- Check what the method is allowed to see.
- Be suspicious of a component that looks too good on its own validation.
- Match the estimator to the metric.
- When many candidates compete on few cells, demand a margin.
- Do not retune to rescue a row that is behind.

## Fact sheet

| Item | Value |
| --- | --- |
| Task | Fill hidden cells |
| Numeric metric | NRMSE |
| Categorical metric | Macro-F1 |
| Real datasets (numeric) | 20 |
| Mechanisms | MCAR, MAR, MNAR |
| Harness and chain errors found | Fifteen |
| Chain arms | Ridge, in-context, neighbours |
| Holdout cells deciding a choice | About 30 |
| Candidates compared | 77 |
| Verified floor | Plain ridge chain equal to MICE |
| Widest margin | Not at random |
| Behind on | Two synthetic settings |
| Categorical competitor | None published |

## Common misreadings

### "It is just a better average"

No. The fill uses the joint structure of the row and the table, not a column average.

### "The fifteen errors invalidate the result"

They are the reason the result can be trusted. Each was found, fixed and checked before the final run.

### "First place means first on every row"

It means ahead in every setting of the real-data board. Two synthetic settings are the exception, and they are stated.

### "Expected value means the model is unsure"

It means the output is optimal for a squared-error metric. Uncertainty is reported separately as a confidence per row.

### "Imputation changes the original data"

Fills are proposals with confidences. Gating and review are the user's choice.

## The work in order

| Step | What I did | Why |
| --- | --- | --- |
| 1 | Set up the real-data board: 20 datasets, three mechanisms | Cover the standard ways data goes missing |
| 2 | Ran a first imputer and compared with rivals | See where the gap was |
| 3 | Noticed the imputer saw a fifth of each table | Too little context |
| 4 | Let it see train and test together, both masked | Match how it is used |
| 5 | Found placeholder names in the semantic channel | Feed what the model should see |
| 6 | Found the neighbours arm reading its own validation cells | Remove the cheat |
| 7 | Replaced argmax with the expected value for categories | Match the squared-error metric |
| 8 | Built the chain: ridge, in-context, neighbours | Combine strengths |
| 9 | Chose arms per column on masked holdouts | Fit each column |
| 10 | Added fixed proportional margins against ridge | Protect against luck |
| 11 | Confirmed ridge matched MICE exactly | Verified floor |
| 12 | Reported the two synthetic rows as they stood | No retuning |

## How to test imputation on your own table

You can run a fair imputation test on any table with no missing cells. The code is a sketch of the idea.

```python
import numpy as np

def mask_cells(df, rate, mechanism, rng):
    """Hide a share of known cells. mechanism: 'mcar', 'mar' or 'mnar'."""
    mask = np.zeros(df.shape, dtype=bool)
    if mechanism == "mcar":
        mask = rng.random(df.shape) < rate
    # for 'mar', make the chance depend on another observed column
    # for 'mnar', make the chance depend on the value being hidden
    return mask

hidden = mask_cells(df, 0.2, "mcar", np.random.default_rng(0))
truth = df.values[hidden]
filled = impute(df.mask(hidden))      # any method
error = np.sqrt(np.mean((filled.values[hidden] - truth) ** 2))
nrmse = error / column_scale
```

Four habits keep the test honest.

1. Hide cells yourself, so you know the true answers.
2. Give every method the same masked table.
3. Keep the cells used to choose a setting separate from the cells used to score it.
4. Test all three mechanisms, not just the easy one.

## Reading the three mechanisms

| Mechanism | What a good method must do | Risk for a weak method |
| --- | --- | --- |
| Completely at random | Use the other columns well | Falls back to the average |
| At random | Notice that gaps follow observed values | Ignores the dependence |
| Not at random | Treat the pattern of blanks as information | Biased fills |

### Why a chain helps under each

- Ridge is strong when relations are close to linear.
- The in-context arm captures nonlinear joint structure.
- The neighbours arm helps when similar rows exist.
- Choosing per column lets each column use what suits it.

## Do and do not

Do:

- Let the imputer see the whole table, with the cells to score masked.
- Keep validation cells hidden from every arm.
- Match the fill to the metric: expected value for squared error.
- Demand a margin when many candidates compete on few cells.
- Build a floor you trust first.

Do not:

- Score a method on cells it has read.
- Feed a channel placeholder text and call it meaning.
- Retune to rescue a row that is behind.
- Blend results from different masking schemes into one number.
- Call a lucky pick among 77 a finding.

## Questions to ask about any imputation claim

- How much of the table did the imputer see?
- Were the scored cells hidden from every component?
- Which missingness mechanisms were tested?
- Is the estimator matched to the metric?
- How were settings chosen, and on how many cells?
- Is there a simple baseline that was reproduced, such as ridge or MICE?
- Which rows are behind, and are they shown?

## The result in six lines

- Real-data board: first, in every setting.
- Widest margin: not at random.
- Floor: ridge equals MICE.
- Errors found: fifteen, all in my own pipeline.
- Behind: two synthetic settings.
- Thin tables: small gain.

## A walk through one column

The column below is invented. It shows the logic of the per-column choice.

1. A column of ages has 12 percent of cells missing.
2. I hide about 30 known ages as a holdout.
3. Ridge, the in-context arm and the neighbours arm each try to fill them.
4. The in-context arm looks best, but only by a hair.
5. The fixed margin says a hair is not enough, so ridge is kept for this column.
6. In another column, the in-context arm wins by a wide gap and is chosen.

The same procedure makes a different, justified choice for each column.

## Quick answers

- Does it use column names? No.
- Does it need a clean table first? No.
- Does it fill numbers and categories? Yes, both.
- Does it return a confidence? Yes, per row.
- Does it always beat MICE? No, two synthetic settings are the exception.
- Where are the scores? In the model card, with their protocols.

## What a reader can take away

- Gaps can be filled from the structure of your own table.
- A strong linear floor is worth building before anything else.
- Most of the gain on a hard board can come from fixing the ruler.
- A fair claim states where it is behind.

## Where to read next

- Missing-data robustness for predictions under injected gaps.
- Anomaly detection benchmark for the detector that reuses the imputation head.
- Pre-training for how masked cell modeling teaches the model to read around holes.
- Evaluation for what a score claims.

## The standing rules, restated

- Competitor figures are never re-run by me.
- They are cited to their primary source.
- My figures come from one sealed configuration.
- There is no per-dataset tuning.
- Different datasets, metrics or masking schemes are never blended.
- Each run is checked for contamination.
- Weak rows are shown next to strong ones.

## Mechanisms in one line each

- MCAR: the gap is a coin flip.
- MAR: the gap follows something you can see.
- MNAR: the gap follows the value you cannot see.

## Terms used on this page

- **Imputation.** Filling in missing cells.
- **NRMSE.** Root mean squared error divided by the scale of the column. Lower is better.
- **Macro-F1.** An F1 score computed per class and averaged.
- **MCAR, MAR, MNAR.** Missing completely at random, at random, and not at random.
- **MICE.** A classical method that fills each column in turn from the others.
- **Ridge.** A linear model with a penalty on large weights.
- **Holdout.** Known cells hidden on purpose to test a choice.
- **Argmax.** Picking the single most likely class.

## Frequently asked questions

### What does the imputation benchmark measure?

For numbers, it hides cells in 20 real datasets and measures reconstruction error with NRMSE under three missingness mechanisms. For categories, it measures macro-F1 on masked categorical cells. The categorical task has no published competitor.

### How did the model do against the other methods?

It came first on the real-data board, ahead of language models and the classical imputers in every setting. The widest margin was under not-at-random missingness. Scores are in the model card with their protocols.

### Why did it take so long?

Most of the work was fixing my own measurement. I found fifteen errors in the harness and the model chain, including an imputer that saw only a fifth of each table and a neighbours arm that read the very cells it was scored on.

### What is the chain of three arms?

A ridge arm, an in-context arm that uses the model's own reading of the table, and a neighbours arm that borrows from similar rows. The arm for each column is chosen on masked holdouts, with fixed proportional margins against the plain ridge arm.

### Why use fixed margins against ridge?

When only about 30 holdout cells decide among 77 candidates, the winner is partly luck. A candidate must beat the baseline by a set proportion, so a lucky pick cannot replace a solid floor.

### Why expected value instead of the most likely class?

NRMSE measures squared error, and the mean is optimal for squared error.

### What does it mean that ridge matched MICE exactly?

It gave a verified floor. The harness reproduced a respected classical method, so gains above it are real. It also showed that a plain linear chain is a strong baseline that is hard to beat.

### Where does the model fall short?

On two synthetic settings, the best published MICE figure stayed ahead of my runs. On a few datasets with few columns or few rows the chain has little context, and the gain is small.

### Is the filled value certain?

No. Each fill carries a confidence per row, and low-confidence rows deserve review. A pipeline can gate on the score.

### Does it need column names?

No. The model does not use a column's identity, and a placeholder name in the semantic channel was one of the errors I fixed.

## Summary

Imputation fills hidden cells. On 20 real datasets and three missingness mechanisms, the model came first, ahead of language models and classical imputers in every setting, with its widest margin under not-at-random missingness. Categorical imputation, scored by macro-F1, has no published competitor.

The path to that result went through fifteen errors in my own harness and chain: too little context, placeholder names, a cheating neighbours arm, the wrong estimator for the metric, and selection by luck. The final design is a chain of ridge, in-context and neighbours arms chosen per column on masked holdouts, with fixed margins against ridge. A plain ridge chain matching MICE gave a verified floor. What stayed open is two synthetic settings where MICE stayed ahead, and thin tables where context is scarce. Read next: Missing-data robustness, Architecture, Capabilities and Limitations.