# Why build a large tabular foundation model?

A large tabular foundation model is a tabular foundation model scaled up in parameters, pre-training data, and the range of tasks one checkpoint can serve. Scale matters for tables for the same reason it does for text: a bigger model can hold more general knowledge about how data behaves, and one network can then replace many task-specific ones.

## What scale buys

- **One checkpoint, many tasks.** Prediction, imputation, anomaly detection, probabilistic regression, and matching are different tasks on the same underlying structure. A single large model can carry all of them instead of a separate model per task.
- **Less dataset-specific work.** The model arrives with a broad prior. You do not tune it for each dataset.
- **Independence from metadata.** A model that has seen enough tables can rely on values and relations, so it keeps working when column names are missing or meaningless. See [column name independence](https://www.ozercemkelahmet.com/model/column-name-independence).
- **Room for continual learning.** A large model can take on new tasks while keeping old ones, which is hard for small models. See [continual learning for tables](https://www.ozercemkelahmet.com/model/continual-learning-for-tables).

## An example: Schema-2

Schema-2 is the large tabular foundation model I built. It has more than 1.5 billion parameters. It uses axial attention over rows and columns, a Perceiver-style global path, and a schema path, and one checkpoint of about 2.18 billion parameters carries every capability. It reads a raw table with no serialization, no preprocessing, and no column names. The [architecture](https://www.ozercemkelahmet.com/model/architecture) page explains each part, and the [capabilities](https://www.ozercemkelahmet.com/model/capabilities) page lists what it does.

## What I measured

Each capability has its own benchmark story, with the protocol, what the model achieved, and the limits:

- [Row-level prediction](https://www.ozercemkelahmet.com/model/bench-row-prediction)
- [Missing-value imputation](https://www.ozercemkelahmet.com/model/bench-imputation)
- [Anomaly detection](https://www.ozercemkelahmet.com/model/bench-anomaly-detection)
- [Probabilistic regression](https://www.ozercemkelahmet.com/model/bench-probabilistic-regression)
- [Continual learning](https://www.ozercemkelahmet.com/model/bench-continual-learning)
- [Metadata independence](https://www.ozercemkelahmet.com/model/bench-metadata-independence)
- [Blind sector identification](https://www.ozercemkelahmet.com/model/bench-sector-identification)
- [Entity and schema matching](https://www.ozercemkelahmet.com/model/bench-entity-schema-matching)
- [Missing-data robustness](https://www.ozercemkelahmet.com/model/bench-missing-data-robustness)
- [One checkpoint, then compression](https://www.ozercemkelahmet.com/model/bench-unified-checkpoint)

Scores are left out of the stories on purpose, because they depend on the harness and the snapshot date. Each story states its protocol. Regression and multiclass prediction stayed behind the leaders, and the stories say so.

## The costs of scale

A large model is expensive to host. Serving it needs memory and an accelerator, and latency is higher than for a tree ensemble. Compression is the main answer: quantizing the checkpoint reduces its size and memory use with a measured loss in fidelity. See [compression and serving](https://www.ozercemkelahmet.com/model/compression-and-serving). Scale also raises the bar for evaluation, because a model trained on a wide range of tables can overlap with public benchmarks, so every run needs a contamination check.

## When it is the right choice

Choose a large tabular foundation model when you have many different tables, little time to tune, columns that are unnamed or inconsistent, or a need for several capabilities from one model. Choose gradient-boosted trees when you have one stable dataset, strict latency or memory limits, and time to tune. Often the best answer is to use both and compare them on your own held-out data.

## Frequently asked questions

What makes a tabular foundation model large?
There is no fixed threshold. In this work it means more than a billion parameters, pre-training on a very wide range of tables, and one checkpoint that serves many tasks.

Is bigger always better for tables?
No. Scale helps breadth and robustness, but trees remain strong on single, stable datasets, and large models cost more to serve.

How large is Schema-2?
More than 1.5 billion parameters, with one checkpoint of about 2.18 billion parameters carrying every capability.

Where is the research behind it?
In the arXiv paper 2605.06290, Data Language Models: A New Foundation Model Class for Tabular Data, and in the [research](https://www.ozercemkelahmet.com/model/research) page.