# From tabular machine learning to large tabular foundation models

Read these in order to see how machine learning on tables evolved, from boosted trees to models that predict on a new table without training on it.

## Stage 1: Machine learning on tables

1. **[AI and machine learning for tabular data: a practical guide](/articles/ai-and-machine-learning-for-tables)**  
   How AI, machine learning, and deep learning relate, which learning paradigms apply to tables, and when to choose trees or a foundation model.
2. **[Gradient boosting](/tools/gradient-boosting)**  
   Gradient boosting builds an ensemble of small trees, each correcting the errors of the previous ones. Learn how it works and why it dominates tabular data.
3. **[XGBoost](/tools/xgboost)**  
   XGBoost is a regularized gradient-boosting library for tabular data. Learn how it works, how to tune it, and how it relates to tabular foundation models.
4. **[FT-Transformer](/tools/ft-transformer)**  
   FT-Transformer turns every table feature into a token and runs a transformer over them. Learn how it works, where it wins, and how it fits tabular…

## Stage 2: In-context learning

5. **[In-context learning](/tools/in-context-learning)**  
   In-context learning lets a model pick up a task from examples in its input, with no weight updates. How it works in language and tabular models, and limits.
6. **[Large tabular foundation models](/tools/tabular-foundation-models)**  
   A technical guide to tabular foundation models: how they encode cells, learn from synthetic priors or real tables, predict in context and compare with GBDTs.
7. **[TabPFN](/tools/tabpfn)**  
   TabPFN is a transformer pre-trained on synthetic tables that predicts in one forward pass. Learn how it works, where it shines, and where it falls short.
8. **[TabICL](/tools/tabicl)**  
   TabICL is a tabular foundation model that uses column-then-row attention to scale in-context learning to large tables. See how it works and when to use it.
9. **[What is a tabular foundation model?](/articles/what-is-a-tabular-foundation-model)**  
   A tabular foundation model is pre-trained on many tables and predicts, imputes, and flags anomalies on a new table in a forward pass, with no per-dataset…

## Stage 3: Large tabular foundation models

10. **[Why build a large tabular foundation model?](/articles/large-tabular-foundation-model)**  
   Why build a large tabular foundation model? Scaling parameters, pre-training data and tasks into one checkpoint that reads raw tables across every domain.
11. **[Large tabular foundation models: how in-context learning works](/articles/large-tabular-foundation-models-in-context-learning)**  
   How large tabular foundation models predict on a new table without training on it, by reading labelled rows as context: theory, models, limits and practice.
12. **[Architecture](/model/architecture)**  
   How a large tabular foundation model is built: cell encoding, axial attention, Perceiver-style global reasoning, task heads, adapters and one checkpoint.
13. **[Benchmarks](/model/benchmarks)**  
   What each benchmark measures, how I built the capability that answers it, and what made the difference, in the order the work was done.
14. **[A score is only as good as its report](/articles/a-score-is-only-as-good-as-its-report)**  
   A score is only as good as its report: what a held-out evaluation report should contain, and how to read one without being fooled by a headline number.

## How to use this path

Read the steps in order; each guide links onward to related topics, so you can go deeper wherever you need to. If you already know a stage, skip to the next one. All learning paths are listed on [Learning paths](/paths).
