Learning paths
From tabular machine learning to large tabular foundation models
Read these in order to see how machine learning on tables evolved, from boosted trees to models that predict on a new table without training on it.
Stage 1: Machine learning on tables
- AI and machine learning for tabular data: a practical guide
How AI, machine learning, and deep learning relate, which learning paradigms apply to tables, and when to choose trees or a foundation model. - Gradient boosting
Gradient boosting builds an ensemble of small trees, each correcting the errors of the previous ones. Learn how it works and why it dominates tabular data. - XGBoost
XGBoost is a regularized gradient-boosting library for tabular data. Learn how it works, how to tune it, and how it relates to tabular foundation models. - FT-Transformer
FT-Transformer turns every table feature into a token and runs a transformer over them. Learn how it works, where it wins, and how it fits tabular…
Stage 2: In-context learning
- In-context learning
In-context learning lets a model pick up a task from examples in its input, with no weight updates. How it works in language and tabular models, and limits. - Large tabular foundation models
A technical guide to tabular foundation models: how they encode cells, learn from synthetic priors or real tables, predict in context and compare with GBDTs. - TabPFN
TabPFN is a transformer pre-trained on synthetic tables that predicts in one forward pass. Learn how it works, where it shines, and where it falls short. - TabICL
TabICL is a tabular foundation model that uses column-then-row attention to scale in-context learning to large tables. See how it works and when to use it. - What is a tabular foundation model?
A tabular foundation model is pre-trained on many tables and predicts, imputes, and flags anomalies on a new table in a forward pass, with no per-dataset…
Stage 3: Large tabular foundation models
- Why build a large tabular foundation model?
Why build a large tabular foundation model? Scaling parameters, pre-training data and tasks into one checkpoint that reads raw tables across every domain. - Large tabular foundation models: how in-context learning works
How large tabular foundation models predict on a new table without training on it, by reading labelled rows as context: theory, models, limits and practice. - Architecture
How a large tabular foundation model is built: cell encoding, axial attention, Perceiver-style global reasoning, task heads, adapters and one checkpoint. - Benchmarks
What each benchmark measures, how I built the capability that answers it, and what made the difference, in the order the work was done. - A score is only as good as its report
A score is only as good as its report: what a held-out evaluation report should contain, and how to read one without being fooled by a headline number.
How to use this path
Read the steps in order; each guide links onward to related topics, so you can go deeper wherever you need to. If you already know a stage, skip to the next one. All learning paths are listed on Learning paths.