Glossary · Deep learning
What is FT-Transformer? Definition and meaning
Turns each feature into an embedding and processes them with a transformer.
Definition
FT-Transformer: Turns each feature into an embedding and processes them with a transformer.
In short: Feature tokenizer plus transformer for tables.
When it is used
A deep-learning baseline for tabular prediction.
Where it fits
FT-Transformer belongs to Large tabular foundation models in the Deep learning topic. These are the other terms in the same group:
- Tabular foundation models: A large neural network pretrained across many tables so it can predict, impute or score new tables with little or no task-specific training.
- In-context learning: The ability of a trained model to perform a new task from examples or instructions in its input, without changing its weights.
- TabPFN: A transformer pre-trained on synthetic datasets that predicts in a single forward pass with in-context learning.
- TabICL: A tabular foundation model that adds column-then-row attention to scale in-context learning to bigger datasets.
- TabSTAR: A tabular model that reads textual features through a pretrained language encoder together with numbers.
- XTab: A framework that pre-trains shared transformer layers across many tables with table-specific embeddings.
- SAINT: A transformer that attends across features and across rows, with contrastive pre-training.
- TabNet: A deep model that selects features step by step with sequential attention.
- Masked cell modeling: The tabular counterpart of masked language modelling: random cells are masked and the model reconstructs them.
- Imputation networks: Models trained to predict missing cells from the observed cells of the same row and table.
Learn more
The full guide to FT-Transformer explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: FSDP · Next term: Full fine-tuning