Semantic sector heads

Overview

Semantic sector heads are small output modules that read a representation of a whole table and name the industry sector it belongs to. Given a table of raw cell values, such as amounts, codes, dates and short labels, the head answers questions like "is this retail, banking, healthcare or logistics data?" It does this with a coarse-to-fine hierarchy: first a broad sector, then a narrower industry group, then, if the evidence allows, a specific industry.

The idea matters because many tables arrive without context. Column names may be missing, cryptic or in another language. A file name tells you little. Yet a good system should still be able to say what kind of business data it is looking at, so that it can choose the right cleaning rules, the right checks and the right models. Semantic sector heads supply that table-level meaning on top of a model that learns from raw tables, such as the TabPFN, TabICL and TabSTAR family.

What it is

A "head" in deep learning is the last part of a network that turns an internal representation into a task answer. A classification head takes a vector and produces class scores. A sector head is a classification head whose classes are industry sectors and whose input is a representation of a table, not of an image or a sentence.

The word "semantic" signals that the label is about meaning. The head does not predict a column type such as "integer" or "date". It predicts what the data is about. A table with columns of invoice totals, tax amounts and supplier codes means accounting or procurement. A table with columns of timestamps, vehicle identifiers and route distances means transport or logistics. The head learns these associations from many examples.

Key traits:

• Table-level. The prediction is for the whole table, not for a row or a cell.
• Hierarchical. Labels form a tree: sector, industry group, industry.
• Content-driven. It can work from cell values alone, and it can use column names and text when they exist.
• Calibrated and cautious. A good head can say "unsure" instead of forcing a label.
• Attachable. It sits on top of an encoder and can be added, swapped or retrained without changing the encoder.

Everyday analogy

Think of a librarian who must shelve a book from its inside pages alone, with no cover. She skims a few pages, sees the vocabulary and the layout, and decides: "this is a cookbook". If she is confident, she goes on to the sub-shelf: "baking". If the evidence is thin, she stops at "cookbook" or places the book in a "to be reviewed" pile. A sector head behaves like this librarian: broad decision first, finer decision only when justified, and a polite refusal when the evidence is weak.

Where it came from

Semantic sector heads draw on several lines of work. There is no single founding paper under this exact name, so this section describes the ideas it combines, with named references only where they are well known.

• Semantic column typing. Systems such as Sherlock (2019) and Sato (2020) learned to predict the semantic type of a column, for example "city", "name" or "year", from its values and its neighbors. They showed that cell values alone carry strong clues about meaning. The sector head lifts this idea from one column to the whole table.
• Table pre-training with language models. TaBERT (2020) and TURL (2020) pre-trained models on large collections of web tables to understand cell content, column names and entities. They showed that table representations can be learned and reused.
• Hierarchical classification. Text categorization, product categorization and biological taxonomies have long used tree-structured labels, with methods such as per-level classifiers, hierarchical softmax and consistency losses.
• Industry classification standards. National and international systems classify economic activity into sectors, divisions and classes. Examples include the North American Industry Classification System, the International Standard Industrial Classification and the Global Industry Classification Standard. They provide ready-made taxonomies that a sector head can follow.
• Foundation models for tables. The prior-fitted network line, including "Transformers Can Do Bayesian Inference" (2021) and "TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second" (2022), plus TabICL and TabSTAR later, made reusable table representations practical. See /model/research for the wider research picture.
• Open-set and selective classification. Work on rejecting inputs that do not belong to any known class gave the head its ability to abstain.

The sector head is therefore a recombination: take a table encoder, add a hierarchical label space from industry taxonomies, and train with methods that handle uncertainty.

How it works

The pipeline has five parts: encode, pool, classify by level, enforce consistency, and decide. Each is explained here.

Step 1: Encode the table

A table encoder converts cells, rows and columns into vectors. Depending on the design, it may be a transformer that attends across columns and rows, as in TabICL, or a model that also reads text, as in TabSTAR. The encoder is usually pre-trained on many tables, so it already knows about value ranges, repeated codes, dates, currencies and text.

Analogy: the librarian's reading skills. She can read a page quickly and notice what kind of words appear.

Step 2: Pool into one table vector

Rows and columns are numerous and vary in number between tables, so the head needs a fixed-size summary. Pooling does this. Common choices:

• Mean pooling. Average the row or column vectors. Simple and stable.
• Attention pooling. Use a learned query vector that attends to rows or columns and weights them by relevance. This lets a few informative columns matter more than many bland ones.
• Set-based pooling. Use a set transformer or latent tokens, as in Perceiver IO, to compress a large set into a few vectors.
• Two-level pooling. First pool cells into column vectors, then pool column vectors into a table vector. This respects the table structure.

The result is one vector that stands for the table. It should not depend on the order of rows or the order of columns. Permutation-invariant pooling takes care of that.

Analogy: the librarian forms an overall impression of the book after skimming many pages.

Step 3: Classify at each level

The label space is a tree. For example:

The head can be built in several ways:

1. Flat softmax over leaves. One classifier over all fine labels. Simple, but ignores the tree and struggles with rare leaves.
2. One classifier per level. Separate softmax layers for sector, group and industry, sharing the same table vector. The hierarchy is used through the loss and the decoding rules.
3. Local classifiers per parent. A small classifier for the children of each node. Prediction proceeds top down.
4. Hierarchical softmax. The probability of a leaf is the product of the probabilities along its path.
5. Label embeddings. Each label is represented as a vector, built from its name and description, and the head scores the table vector against label vectors. This enables adding labels without a new output layer, and it connects naturally to text-aware models.

Analogy: the librarian first picks the aisle, then the shelf, then the position on the shelf.

Step 4: Enforce consistency across levels

Predictions must agree. A table cannot be "Healthcare" at the top level and "Retail banking" at the bottom. Consistency can be enforced in different ways:

• By construction. With hierarchical softmax, leaf probabilities already sum correctly along paths.
• By decoding. Choose the best leaf, then read the parent labels off its path. Or choose the best path by the product of level probabilities.
• By loss. Add a penalty when a child probability is higher than that of its parent.
• By masking. At each level, allow only the children of the chosen parent.

Step 5: Decide, or abstain

Not all tables deserve a fine label. The decision rule can use confidence at each level:

• If the top-level confidence is high but the next level is low, return only the sector.
• If even the top level is low, return "uncertain" and, if useful, the top two candidates.
• If the table looks unlike any known sector, flag it as out of distribution.

This is selective classification. It is the difference between a system that sounds sure and a system that is useful when it is sure.

Analogy: the librarian puts the book on the "to be reviewed" cart when she cannot tell.

Training

Training data is a collection of labeled tables. Labels may come from the source system, from catalogs, from human annotation or from a weak method such as keyword rules on metadata. A few practical points:

• Many tables, many views. A single table can yield many training examples through sampling rows and columns. This works as data augmentation, and it teaches the head to work with partial views.
• Column dropout and name dropout. Randomly hide column names, or hide whole columns, so that the head learns not to depend on any single clue. This is key for robustness.
• Loss. Cross-entropy at each level, with weights that emphasize coarse levels first. Label smoothing helps with noisy labels. A consistency term enforces the tree.
• Class imbalance. Use class weights or balanced sampling, since some sectors have far more tables than others.
• Frozen or tuned encoder. Training the head only is cheap and safe. Tuning the encoder as well can improve accuracy but risks forgetting its general skills. Parameter-efficient tuning is a good middle path.
• Calibration. After training, fit a calibration step, such as temperature scaling, on a held-out set, so that confidence means what it says.

Inference

At inference, the table is sampled if it is large: pick a set of rows and use all columns, or all rows up to a limit. The encoder produces the table vector. The head produces probabilities per level. A decision rule returns the deepest label that passes its confidence threshold, along with the probabilities.

A useful trick is test-time voting over several random row samples. Average the probabilities over, say, ten samples. This reduces variance and gives a more stable answer.

Key concepts and terms

Related methods are described in Transfer learning, for reusing an encoder, Contrastive learning, for learning table embeddings without labels, and Self-supervised learning, for the general idea.

A worked example

Building a small sector head

The following example builds a two-level head on top of a table embedding. It assumes you already have an encoder that returns a fixed-size vector for each table. Here random data stands in for those vectors so that the code runs on its own.

import torch
import torch.nn as nn
import torch.nn.functional as F

# A tiny taxonomy: 3 sectors, each with 2 industries
sectors = ["retail", "finance", "transport"]
industries = {
    "retail": ["grocery", "apparel"],
    "finance": ["banking", "insurance"],
    "transport": ["freight", "ride_hailing"],
}
leaf_names = [f"{s}/{i}" for s in sectors for i in industries[s]]
leaf_to_sector = [sectors.index(n.split("/")[0]) for n in leaf_names]

class SectorHead(nn.Module):
    def __init__(self, dim, n_sectors, n_leaves):
        super().__init__()
        self.sector = nn.Linear(dim, n_sectors)
        self.leaf = nn.Linear(dim, n_leaves)

    def forward(self, z):
        return self.sector(z), self.leaf(z)

dim = 64
head = SectorHead(dim, len(sectors), len(leaf_names))
opt = torch.optim.AdamW(head.parameters(), lr=1e-3)

# Fake table embeddings and labels, for illustration only
torch.manual_seed(0)
z = torch.randn(256, dim)
y_leaf = torch.randint(0, len(leaf_names), (256,))
y_sector = torch.tensor([leaf_to_sector[i] for i in y_leaf])
leaf_to_sector_t = torch.tensor(leaf_to_sector)

for step in range(200):
    s_logits, l_logits = head(z)
    loss_s = F.cross_entropy(s_logits, y_sector, label_smoothing=0.05)
    loss_l = F.cross_entropy(l_logits, y_leaf, label_smoothing=0.05)

    # Consistency: leaf probabilities summed per sector should match sector probabilities
    p_leaf = F.softmax(l_logits, dim=-1)
    p_sec_from_leaf = torch.zeros(z.size(0), len(sectors)).index_add_(
        1, leaf_to_sector_t, p_leaf
    )
    p_sec = F.softmax(s_logits, dim=-1)
    loss_c = F.kl_div(p_sec_from_leaf.clamp_min(1e-8).log(), p_sec, reduction="batchmean")

    loss = loss_s + loss_l + 0.5 * loss_c
    opt.zero_grad()
    loss.backward()
    opt.step()

Notes on the code:

• Two linear layers share the same table vector z. One predicts the sector, the other the leaf.
• The consistency term compares the sector probability implied by the leaf head with the sector probability from the sector head. If they disagree, the loss rises.
• Label smoothing softens hard labels, which helps when some labels are noisy.

Decoding with abstention

def predict(head, z, t_sector=0.7, t_leaf=0.6):
    with torch.no_grad():
        s_logits, l_logits = head(z)
        p_s = F.softmax(s_logits, -1)
        p_l = F.softmax(l_logits, -1)
    out = []
    for i in range(z.size(0)):
        s_conf, s_idx = p_s[i].max(0)
        if s_conf < t_sector:
            out.append(("uncertain", float(s_conf)))
            continue
        # Restrict leaf choice to the children of the chosen sector
        mask = (leaf_to_sector_t == s_idx).float()
        p_child = p_l[i] * mask
        p_child = p_child / p_child.sum().clamp_min(1e-8)
        l_conf, l_idx = p_child.max(0)
        if l_conf < t_leaf:
            out.append((sectors[int(s_idx)], float(s_conf)))
        else:
            out.append((leaf_names[int(l_idx)], float(l_conf)))
    return out

print(predict(head, z[:5]))

The decoder first checks the sector. It then masks the leaf probabilities to the children of that sector, renormalizes, and returns the leaf only if confidence passes a threshold. Otherwise it stops at the sector, or returns "uncertain".

Where the table vector comes from

In practice, the vector z comes from a table encoder. A typical recipe:

1. Sample up to a few thousand rows from the table.
2. Pass the sampled table through the encoder to get one vector per row, or per column.
3. Pool them with attention pooling into a single vector.
4. Optionally concatenate summary statistics, such as the share of numeric columns, the share of missing cells, the number of distinct values per column, and the share of date-like columns.

Summary statistics are cheap and useful. A table with many columns of money amounts and a few identifiers looks different from a table full of timestamps and sensor readings, even before any deep model is applied.

A numeric walk-through of the decision rule

Suppose the sector probabilities for a table are retail 0.10, finance 0.78, transport 0.12. With a sector threshold of 0.70, finance passes. Now the leaf probabilities, after masking to finance children, are banking 0.52 and insurance 0.48. With a leaf threshold of 0.60, neither passes. The system returns "finance" with confidence 0.78 and does not name an industry. That is the correct, honest behavior: the evidence supports a sector, but not a finer label.

Evaluating at every level

Report metrics per level, not only at the leaves.

Plotting selective accuracy against coverage gives a risk-coverage curve. A good head keeps accuracy high even as coverage grows.

Strengths and limitations

Strengths

• Provides table-level meaning when column names are missing or unreliable.
• Hierarchical output gives partial answers: a correct sector is useful even when the industry is unclear.
• Can abstain, which prevents confident wrong labels.
• Cheap to train and to run on top of an existing encoder.
• Works with different tables of different sizes through pooling.
• Modular: the encoder and head can be updated separately.
• Supports routing: the predicted sector can select cleaning rules, validation checks and specialist models.
• Label embeddings allow adding new labels without changing the architecture.

Limitations

• Depends on labeled tables, which are hard to collect and may be noisy. Labels from source systems often reflect where data came from, not what it means.
• Sectors overlap. A table from an insurance broker's accounting system could be finance, insurance or accounting, and labels depend on a taxonomy choice.
• Cell values alone can be ambiguous. Amounts, dates and ids look similar across industries.
• Distribution shift. A head trained on tables from some organizations may fail on tables from others with different habits and formats.
• Rare sectors have few examples, so the fine level is often weak.
• Pooling can wash out small but decisive clues, such as a single column of diagnosis codes.
• Taxonomies change over time. Labels may need to be remapped when the standard is revised.
• Privacy and compliance. Tables may contain sensitive values, so the input to the head must be handled under the same rules as the data itself.

How it compares with alternatives

Practical advice: combine them. Use metadata when it exists, keyword rules as weak signals, and the sector head as the general fallback. Let the head see the output of the others as extra features if you can.

Role in tabular foundation models

This is the central section for this page. A sector head is not a stand-alone model. It is an add-on that gives a tabular foundation model a way to say what a table is about. Read it with /model/architecture, which shows how encoders and heads fit together, and /model/benchmarks, which discusses how to evaluate them.

Row and column structure

A sector head needs a table-level view, and tables are two-dimensional. Row order and column order carry no meaning, yet both rows and columns carry information. A good encoder attends along columns and along rows, as in Axial attention, and the pooling step then removes ordering. The sector head therefore sits at the top of a stack that has already respected the table structure. If the head were fed a flat bag of cells, it would lose the fact that some cells belong together in a column, such as all the values of "amount".

Mixed types

Real tables mix numbers, categories, dates, flags and text. The head profits from an encoder that represents all of these in a shared space. Type itself is a clue: a table with many date columns and many amounts leans toward finance or logistics, while one with many free-text fields leans toward support or content. Summary features about types, such as the share of numeric versus text columns, can be concatenated to the table vector.

No column names

This is the hardest and most valuable setting. In-context models like TabPFN and TabICL work without names, and a sector head built on them must read value patterns: ranges, number of distinct values, skew, units, formats such as currency-like decimals, code lengths and date spans. Such patterns are surprisingly informative. For example, a column of integers between 1 and 5 with strong skew toward 5 looks like a rating, and a column of ten-digit strings looks like an account or phone-like code. When names exist, as in the text-aware setting of TabSTAR, they add strong evidence. The best practice is to train with name dropout so the head works in both settings and degrades gracefully.

Missing values

Missingness is a signal about the sector. Medical records, survey tables and sensor logs have different missing patterns. The head can use the missing rate per column, and the correlation of missingness across columns, as features. The encoder should treat missing cells explicitly and not as zeros. A head that is robust to missing values avoids being confused by incomplete extracts. See Imputation networks for how encoders deal with holes.

Pre-training on synthetic or real tables

Sector labels need meaning that synthetic structural-causal-model tables do not carry. The encoder can still be pre-trained on synthetic tables to learn general structure, as in the prior-fitted family, but the head needs real tables with real sector labels. There is an interesting middle path: generate synthetic tables that imitate sector-typical structure, such as invoice-like or telemetry-like tables, and use them to augment rare sectors. See Synthetic data generation. This must be validated carefully, since synthetic tables that are too clean can teach the head shortcuts. Real tables with sector labels are needed to anchor the head.

In-context learning

A sector head can use in-context ideas. Instead of a fixed set of output weights, one can give the model a few labeled example tables per sector as context, and let it classify a new table by comparing its embedding with theirs. This is closely related to Few-shot learning and to prototype-based classification. The advantage is flexibility: you can add a new sector by adding a few example tables, without retraining. The risk is that the quality depends on how representative the examples are.

Continual learning

Taxonomies grow and customers bring new kinds of tables. A sector head must therefore be extended over time without forgetting old sectors. Several tools help. Label embeddings let you add classes without changing the output layer. Experience replay keeps a small buffer of old tables to rehearse. Elastic weight consolidation protects important weights. Keeping the encoder frozen and updating only the head is the simplest strategy and often enough. It is also wise to keep a fixed regression set of tables with known labels, and to rerun it after every update.

Imputation

Sector knowledge helps imputation. If the model knows that the table is a retail transaction table, it can narrow the range of plausible values for a missing cell: a quantity is a small positive integer, a price is positive with two decimals. Conversely, imputation helps sector detection by making incomplete tables more complete before pooling. The two tasks can share an encoder and be trained together. For techniques see Masked cell modeling, MIDAS and Denoising autoencoders.

Anomaly detection

The sector label gives context for anomalies. A value that is normal in one sector may be unusual in another. A head's confidence is itself a signal: a table that does not match any known sector with confidence may be corrupted, mislabeled or from a new domain. This ties to out-of-distribution detection. Further, within a known sector, per-sector models of normal behavior can be used to flag unusual rows. See Anomaly detection and Isolation forest for standard methods that can be conditioned on the predicted sector.

Sector and semantic classification

This is the direct purpose of the head. It turns the encoder's table-level representation into labels that people can act on. A well-built head gives a ranked list of sectors, a confidence at each level and a decision about how deep to go. It complements column-level semantic typing: column types say what each column is, and the sector head says what the whole table is. Together they provide an understanding that lets the rest of the system choose sensible defaults.

A quick sanity test for any sector head

Before trusting a head, run three simple checks on a handful of tables you know well:

1. Shuffle test. Shuffle rows and columns. The prediction should not change.
2. Rename test. Replace column names with neutral labels. The sector should stay the same or degrade gently.
3. Subsample test. Keep only five percent of the rows. The sector should stay the same, with somewhat lower confidence.

If any of these checks flips the answer, the head is leaning on an accidental clue and needs more robust training.

How it fits in a larger system

In a platform that ingests tables from many sources, the sector head can sit at the front door:

1. A new table is uploaded.
2. The encoder produces row, column and table embeddings.
3. The sector head predicts the sector with confidence.
4. The sector selects validation rules, default imputation settings and candidate models.
5. Low confidence sends the table to human review and stores the result as new labeled data.

This closes a loop: the review step produces labeled tables that improve the head over time, which is a form of Active learning.

Practical guidance and best practices

Design the taxonomy first

• Start from a recognized industry standard rather than inventing your own, then trim it to what your data actually covers.
• Keep the number of top-level sectors small, around a dozen or two. Too many top-level labels make the coarse decision hard.
• Write a short definition for each label, with examples of typical tables. These definitions help annotators and can feed label embeddings.
• Add an explicit "other" or "unknown" label at each level.
• Version the taxonomy and keep a mapping when it changes.

Collect labels carefully

• Prefer labels from people who understand the data over labels inferred from file names.
• Have two annotators label a sample and measure agreement. Low agreement means the taxonomy is unclear.
• Keep tables from the same source organization in the same data split, to avoid leakage.
• Record where each label came from, so that weak labels can be down-weighted.

Train for robustness

• Use name dropout, column dropout and row sampling.
• Vary the number of rows per sample, from a few dozen to thousands.
• Add noise to numeric values and shuffle row and column order.
• Use class weights for rare sectors.
• Calibrate on a separate validation set.

Evaluate honestly

• Split by source organization, not by table, to test generalization to new organizations.
• Report metrics at every level plus coverage and calibration.
• Test with names removed, and with names replaced by misleading ones.
• Test on tables with heavy missingness and on very small tables.
• Include out-of-distribution tables to test abstention.

Deploy safely

• Log predicted label, confidence and the decision rule outcome.
• Treat the label as a suggestion where the cost of a mistake is high, and let users correct it.
• Monitor the share of "uncertain" outputs. A rise suggests drift.
• Keep a regression set and rerun it at every update.
• Do not send sensitive raw values to outside services. Compute embeddings inside your own environment.

Tips for fine-level labels

• Train the fine level only for sectors with enough tables.
• Use the parent label as a feature or as a mask for the child classifier.
• Allow the system to stop at a coarse level. A correct coarse label beats a wrong fine label.
• Use label embeddings so that rare leaves can borrow strength from their names and descriptions.

Common mistakes

• Treating sector detection as a flat classification with dozens of leaves and then wondering why rare leaves fail.
• Splitting train and test at the table level when many tables come from the same organization, which inflates accuracy.
• Training with column names always present, then deploying on tables without names.
• Forcing a label for every table, with no abstention path.
• Not calibrating, so that a confidence of 0.9 does not mean 90 percent.
• Using labels that reflect the source system rather than the meaning of the data.
• Pooling by simple averaging when a few columns are decisive, and losing them.
• Ignoring class imbalance, so that the head always predicts the most common sector.
• Using a taxonomy that is too detailed for the available data.
• Not versioning the taxonomy, so old predictions cannot be compared with new ones.
• Sending sensitive table contents to an outside service for classification.
• Forgetting to test on out-of-distribution tables, so that failures only show up in production.

Frequently asked questions

What are semantic sector heads?

Semantic sector heads are classification modules placed on top of a table encoder. They read a single vector that summarizes a table and output the industry sector of the data, using a coarse-to-fine hierarchy of labels. They are used to identify what kind of business data a table contains, even when column names are missing.

Why use a hierarchy instead of one flat label?

A hierarchy gives partial credit and safer answers. If the evidence supports "finance" but not "insurance", the system can still return the sector. It also helps rare fine labels, which share a parent with better-represented siblings, and it lets downstream systems pick rules at the level they need.

Can the head work without column names?

Yes. When names are missing, the encoder relies on the values: ranges, formats, repeated codes, the share of missing cells and relationships between columns. Training with name dropout, where names are randomly hidden, prepares the head for this situation. Names, when present, still add valuable evidence.

How does a sector head decide when to abstain?

It compares its confidence at each level with a threshold. If the top-level probability is low, it returns "uncertain". If the top level is confident but the fine level is not, it stops at the sector. Thresholds are chosen on validation data, using a risk-coverage curve to balance accuracy against the share of tables that get a label.

What data do I need to train one?

You need a collection of tables with sector labels, ideally from many different sources. A few dozen examples per sector is a rough lower bound for useful coarse prediction, and more is needed for fine labels. Each table can be sampled many times to create training examples, but diversity of sources matters more than the number of samples from one source.

How is it different from semantic column typing?

Column typing predicts the meaning of one column, such as "city" or "price". A sector head predicts the meaning of the whole table, such as "retail" or "logistics". The two are complementary. Column types can be used as input features for the sector head, and the sector can help disambiguate column types.

Can it be used with TabPFN, TabICL or TabSTAR?

Yes, in principle. The head needs a table-level vector, which can be obtained by pooling the internal representations of an encoder in the style of TabICL or TabSTAR. Models like TabPFN predict row labels, so using them for table-level tasks requires extracting and pooling internal features or training a separate encoder.

How do I add a new sector later?

With a flat output layer you would have to resize it and retrain. With label embeddings, you add the new label's name and description and train briefly with a few examples, while a replay buffer protects older sectors. Prototype-based in-context classification is another route: store a few example tables for the new sector as references.

What are the main risks?

The main risks are label noise, overlapping sectors, distribution shift between organizations, and false confidence. Careful taxonomy design, source-level data splits, calibration and abstention reduce these risks. Privacy is also a concern, so process tables inside a secure environment.

Is a large language model a better choice?

A language model can classify a table zero-shot from a sample of rows and names, and it is flexible. But it is more expensive, less consistent, and sends data through a large model that may be hard to run privately. A trained sector head is cheaper, faster, calibrated and consistent. A common pattern is to use the head by default and a language model as a second opinion for low-confidence cases.

Glossary

• Abstention: Choosing not to give a label when confidence is low.
• Attention pooling: Combining vectors with weights learned by attention.
• Calibration: Alignment of predicted confidence with observed accuracy.
• Coarse-to-fine: Moving from broad labels to specific ones.
• Embedding: A vector that represents an item such as a cell, row, column or table.
• Head: The final module of a network that produces the task output.
• Hierarchical softmax: A way to compute leaf probabilities as products along a tree path.
• Label embedding: A vector that represents a class, built from its name or description.
• Name dropout: Randomly hiding column names during training.
• Open-set recognition: Detecting inputs that belong to no known class.
• Out-of-distribution: Unlike the data seen in training.
• Pooling: Reducing many vectors to one fixed-size vector.
• Prototype: A representative vector for a class, such as the mean of its examples.
• Risk-coverage curve: A plot of error rate against the share of inputs that receive an answer.
• Taxonomy: A structured list of categories, often arranged as a tree.

Summary

Semantic sector heads give a table-aware model the ability to name the industry sector of a table from its content. They sit on top of an encoder, pool rows and columns into one table vector, classify through a coarse-to-fine hierarchy, keep levels consistent, and abstain when evidence is weak. They are cheap to train, work when column names are missing, and make partial answers possible.

Within tabular foundation models, they connect the structural skills of encoders such as TabPFN and TabICL and the semantic skills of TabSTAR to a practical task: knowing what a table is about, and routing it to the right rules and models. They interact with missing-value handling, imputation, anomaly detection, in-context learning and continual learning. For broader background see /model/research, /model/architecture and /model/benchmarks.

Related

It belongs to Tabular foundation models in my stack, under Foundation models and applied ML. Others in the same group: