Research

The idea in one paragraph

Every major data modality has a foundation model that understands it natively: text has language models, images have vision models, audio has audio models. Tabular data, the modality on which many consequential real-world decisions are made, does not. Every approach to tabular AI today, from gradient-boosted trees to the latest tabular foundation models, needs a preprocessing pipeline before any model can read the data. The paper introduces the Data Language Model (DLM): a foundation model that reads tables the way a language model reads sentences, directly from raw cell values, with no serialization and no preprocessing. The first model of this class was a 140M parameter model trained on more than 2.3 million synthetic and real-world tabular datasets. It outperformed gradient-boosted ensembles, AutoML stacks and tabular foundation models on established row-level prediction benchmarks, and it could name the industry sector of an unseen dataset from raw cell values alone, a task no earlier tabular model performs. The model described on this site is its successor.

Why tabular AI is difficult

Tables look easy: columns have types and values sit in rows. Five properties make them hard for neural networks.

• Heterogeneous types. One table mixes continuous numbers, categories, dates, currencies, identifiers and free text, each with different statistics and meaning.
• Semantic ambiguity. A column called value can be money, a sensor reading or an abstract score. Without outside knowledge the meaning is underspecified.
• Variable structure. Images have fixed pixel grids and text has one axis. Tables vary in both rows and columns.
• Missing data. Values go missing through collection errors, privacy rules or inapplicability, and the pattern of missingness often carries information.

The preprocessing bottleneck

The standard workflow cleans, imputes, normalizes, encodes and engineers features before any model trains, and a human specifies the domain first. Every new data source needs that pipeline rebuilt from scratch.

Two newer routes keep a bottleneck of their own. Tabular foundation models cut the training cold start, but a human still names the target column, a normalization step is still needed, and the output carries no identification of the domain. Language models on tables replace preprocessing with serialization (flattening rows into text), which discards the joint distributional structure between columns.

This is not a tooling problem that better libraries solve. It is a mismatch: structured data is forced through transformations designed around the limits of models that never understood structure.

Tables as language

Language models work because they learn the structure of language. Tables have an analogous structure.

The implication

If tables have learnable structure like language, the language-modelling paradigm should transfer: train a model on diverse tables and it learns tabular grammar, the patterns of composition, relationship and meaning that make a table intelligible. Such a model needs no preprocessing, because it reads the raw structure. Missing values need no separate imputation step, because it reasons about uncertainty. Feature engineering becomes unnecessary, because it learns the representations. Format conversion disappears, because tables are the native input. That is the motivation for the Data Language Model.

Where to read next

• Architecture and Pre-training describe how the model is built and trained.
• Solutions lists the six questions a model like this answers.
• Benchmarks tells how each evaluation was developed.