Data Language Models: the paper, explorable
Text, images and audio each have foundation models that read them natively. The paper "Data Language Models: A New Foundation Model Class for Tabular Data" argues that tables have lacked one, and defines what such a model must do. This page turns the paper into something you can work with: its definition, its inputs on a table you type, and its results, read with their limits.
Level AdvancedAbout 15 minRead first: In-context classification, Imputation1 challenge in the Lab
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
The first tab is the paper's definition. A Data Language Model must read all of a table's signals jointly (C1), say what kind of dataset it is looking at (C2) and keep working when column names are removed (C3). The table compares the approaches the paper discusses against those three conditions.
The second tab computes, in your browser, three of the four signals the paper names for every column: the column's identifier, a distribution summary (centre, spread, range, share of numbers, unique-value density, observed count) and the pattern of missing cells. Switching column names off zeroes the identifiers, which is the situation C3 is about.
The third tab draws the paper's six evaluations with its own numbers, and the fourth lists the limits the paper itself makes visible.
Things to try
- In the first tab, select each row and read why it does or does not meet each condition.
- In the second tab, paste a table of your own with a few empty cells, then switch the column names off.
- In the third tab, compare the imputation table by condition: the gap between MCAR and MNAR is where most methods struggle.
How it works in real systems
Tables are the most common form of business data, and most production models for them are still trained one table at a time with gradient-boosted trees. Tabular foundation models change that by learning, once, how tables behave and then predicting a new table in context. A Data Language Model, as the paper defines it, goes further: it reads column meaning, statistics, values and missingness together, tells you what kind of data it is, and does not depend on clean column names. For a builder this means fewer manual steps before a model is useful.
Key formulas
$$\mathrm{NRMSE}=\frac{\sqrt{\tfrac1n\sum_i(\hat x_i-x_i)^2}}{\sigma_x}$$
The imputation score: error on the hidden cells, divided by the column's spread. Lower is better.
$$\text{relative improvement}=\frac{e_{\text{baseline}}-e_{\text{model}}}{e_{\text{baseline}}}$$
How the paper's 'about 31% lower than the best LLM' is computed.
Common mistakes
- Quoting a headline number without its benchmark: 0.9849, 0.9418 and 0.9435 come from three different dataset subsets.
- Reading 'first on every benchmark' into the results: imputation is first on 8 of 9 conditions.
- Assuming the model's internals from its inputs: the paper does not describe the layers or the training objective.
Real-world numbers
- 140 million parameters, trained on 2,307,000 datasets (2,000,000 synthetic, 307,000 real) across 10,000 sectors.
- OpenML-CC18: mean ROC-AUC 0.9849, first on 18 of 18 datasets.
- Imputation: mean NRMSE 0.163 against 0.235 for the best LLM and 0.302 for missForest.
- Blind sector identification: 91.4% top-1 and 97.0% top-5 over 10,000 sectors.
Further reading
- Data Language Models: A New Foundation Model Class for Tabular Data (arXiv:2605.06290)
- TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second
Code lab
The per-column summary and the missing-value pattern from the second tab, in pandas.
Step 1 · Per-column distribution summary
import pandas as pd, io
csv = "age,income,city,visits\n34,52000,Izmir,3\n41,,Ankara,5\n29,38000,Izmir,\n55,91000,Istanbul,9\n,47000,Ankara,2"
df = pd.read_csv(io.StringIO(csv))
summary = pd.DataFrame({
"centre": df.mean(numeric_only=True), "spread": df.std(numeric_only=True, ddof=0),
"min": df.min(numeric_only=True), "max": df.max(numeric_only=True),
"numeric_share": df.apply(lambda c: pd.to_numeric(c, errors="coerce").notna().sum() / max(1, c.notna().sum())),
"unique_density": df.nunique() / df.notna().sum(), "observed": df.notna().sum()})
print(summary)
Step 2 · The missing-value pattern and NRMSE
import numpy as np
mask = df.isna().to_numpy() # True where a value is missing
print(mask.astype(int))
def nrmse(pred, true):
return np.sqrt(np.mean((pred - true) ** 2)) / np.std(true)
true = np.array([52000, 38000, 47000.]); guess = np.full(3, true.mean())
print("mean imputation NRMSE", round(nrmse(guess, true), 3))
More simulations
All AI Simulations · Mini tabular model → · Tabular model: imputation →