GraphSAGE

Overview

GraphSAGE is a graph neural network method that learns a function for building a node's embedding from a sample of its neighbours' features. Because it learns the function and not a fixed embedding per node, it can produce vectors for nodes that were never seen in training. The name comes from "SAmple and aggreGatE".

This inductive property is the main reason GraphSAGE is used on large and changing graphs, such as social networks, transaction networks and product graphs where new nodes arrive every day. Neighbour sampling also keeps memory use under control. For tabular work, GraphSAGE is a practical way to add relational context to rows that are linked by keys, and it sits beside GCN and GAT in the family of graph neural networks.

What it is

GraphSAGE is a framework with three parts: a way to sample a fixed number of neighbours per node, a way to aggregate their features, and a way to combine the result with the node's own features. By choosing different aggregators you get different variants.

Transductive versus inductive

A transductive method learns an embedding table with one vector per node. If a new node arrives, there is no vector for it, and you must retrain. An inductive method learns a function from features and neighbourhood to an embedding. A new node gets its embedding by running the function on its features and neighbours. GraphSAGE is inductive.

What it needs

• Node features. These are essential, since the function maps features to embeddings.

• A graph, given as an edge list or adjacency lists.

• A training signal. This can be labels for supervised learning, or a graph-based loss for unsupervised learning, where nearby nodes should have similar embeddings.

What it outputs

For each node, a dense vector. You can feed the vector to a classifier, to a nearest neighbour search, or to a flat-table model as extra columns.

Where it came from

Early node embedding methods based on random walks learned one vector per node and worked only on the graph they were trained on. Spectral graph networks and the GCN learned from features but were typically trained with the full graph in memory.

The paper "Inductive Representation Learning on Large Graphs" appeared in 2017. It introduced GraphSAGE with sampling and several aggregators, and showed it could embed unseen nodes in citation, community and protein interaction graphs. It also made mini-batch training of graph networks practical, because each batch needs only a small sampled neighbourhood.

The approach influenced large-scale industrial systems that run graph learning on graphs with very many nodes and edges. It also inspired later sampling methods and the use of graph networks for recommendation, where new items and users constantly appear. See /model/research for related research directions on structured data.

How it works

Imagine you move to a new city and need a first impression of a neighbourhood. You cannot meet everyone. You pick a handful of residents at random, listen to what they say, summarise it, and combine it with what you already know about the area. GraphSAGE does exactly that for each node.

Step 1: Pick a target batch

Training starts with a batch of target nodes, for example 512 accounts. Only these need final embeddings in this step.

Step 2: Sample neighbours

For each target node, sample a fixed number of neighbours, for example 25. If a node has fewer, sample with replacement or take all. For a second layer, sample again for each of those neighbours, for example 10. The result is a small tree of nodes around each target.

Fixed sample sizes keep the cost per batch predictable, no matter how large or skewed the graph is. A node with a million neighbours costs the same as one with thirty.

Step 3: Aggregate neighbour features

Take the features of the sampled neighbours and combine them with an aggregator function. Common choices:

• Mean aggregator. Take the element-wise average. This is close to the GCN idea.

• Pooling aggregator. Pass each neighbour through a small network, then take an element-wise maximum.

• LSTM aggregator. Run a recurrent network over a random permutation of neighbours. It is expressive but not order invariant by design.

• Sum aggregator. Add neighbour vectors, which keeps degree information.

Step 4: Combine with the node's own vector

Concatenate the node's current vector with the aggregated neighbour vector. Multiply by a weight matrix and apply a non-linearity such as ReLU. Then normalise the result to unit length.

h_neigh = AGG( { h_u : u in sampled N(v) } )
h_v_new = ReLU( W * concat( h_v, h_neigh ) )
h_v_new = h_v_new / ||h_v_new||

Keeping the node's own vector in a separate slot, not mixed into the average, is a key difference from the GCN rule.

Step 5: Repeat for K layers

Layer 1 uses raw features of the second-hop samples to build one-hop embeddings. Layer 2 uses those to build the target's final embedding. With K layers, the target sees K hops of context, but only through the sampled paths.

Step 6: Compute the loss

For supervised training, a classification head and cross-entropy on labelled targets are used. For unsupervised training, a loss pushes embeddings of nodes that co-occur on short random walks to be close, and pushes random pairs apart:

loss = - log( sigmoid( z_u . z_v ) ) - Q * E_neg[ log( sigmoid( - z_u . z_vn ) ) ]

Here v is a node near u, vn is a random negative node, and Q is the number of negatives.

Step 7: Inference on new nodes

For a node that was not in training, run the same sampling and aggregation with the learned weights. No retraining is needed. For production, many systems precompute embeddings in batch and refresh them on a schedule, plus compute on demand for brand-new nodes.

Why sampling works

Averages of random samples are good estimates of averages of the whole set. Taking 25 of 10,000 neighbours gives a noisy but unbiased estimate of the mean. Noise acts as a regulariser. The trade-off is variance: for very important neighbours, a random sample may miss them, so some systems use importance sampling.

Comparing the aggregators

Why keep self and neighbours apart

Mixing a node into its own neighbour average, as the basic GCN rule does, blends two different kinds of information. Concatenation lets the weight matrix decide how much to trust the node itself and how much to trust the neighbourhood. For a node with few neighbours, the model can lean on its own features. For a node with a clear neighbourhood pattern, it can lean on the neighbours.

Unsupervised training in plain words

Without labels, the model is taught a simple idea: friends should look alike. Pairs of nodes that are close in the graph are pulled together in embedding space. Random pairs are pushed apart. After training, the embeddings capture the shape of the graph and the features together, and a downstream classifier can use them with only a few labels.

Layers and reach

The exponential growth is the reason most deployments stay at two layers.

Key concepts and terms

A worked example

A numeric walk-through of one layer

Target node v has own vector [1, 0]. It has three sampled neighbours with vectors [0, 2], [2, 2] and [1, 0].

1. Mean aggregation: the average is [(0+2+1)/3, (2+2+0)/3] = [1.0, 1.33].

2. Concatenate with the own vector: [1, 0, 1.0, 1.33].

3. Suppose the weight matrix W has two output rows: [1, 0, 0.5, 0] and [0, 1, 0, 0.5]. The raw outputs are [1 + 0.5, 0 + 0.67] = [1.5, 0.67].

4. ReLU keeps both values, since they are positive.

5. Normalise to unit length: the length is sqrt(1.5^2 + 0.67^2) = about 1.64. The embedding is about [0.91, 0.41].

Notice that the node's own vector and the neighbour summary enter through different columns of W, so the model can weigh them differently.

Fan-out and cost

With fan-outs of 25 and 10 for two layers, one target needs at most 25 first-hop nodes plus 25 times 10 = 250 second-hop nodes, so 275 sampled nodes. A batch of 512 targets needs at most about 140 thousand node lookups. This number is independent of the size of the graph, which is why GraphSAGE scales.

A GraphSAGE model in code

import torch
import torch.nn.functional as F
from torch_geometric.nn import SAGEConv
from torch_geometric.loader import NeighborLoader

class SAGE(torch.nn.Module):
    def __init__(self, in_dim, hid, out_dim):
        super().__init__()
        self.c1 = SAGEConv(in_dim, hid, aggr="mean")
        self.c2 = SAGEConv(hid, out_dim, aggr="mean")

    def forward(self, x, edge_index):
        x = F.relu(self.c1(x, edge_index))
        x = F.dropout(x, 0.5, self.training)
        return self.c2(x, edge_index)

loader = NeighborLoader(
    data,                       # a graph object with x, y, edge_index
    num_neighbors=[25, 10],     # fan-out per layer
    batch_size=512,
    input_nodes=data.train_mask,
    shuffle=True,
)

model = SAGE(data.num_features, 64, 5)
opt = torch.optim.Adam(model.parameters(), lr=0.003)

for batch in loader:
    opt.zero_grad()
    out = model(batch.x, batch.edge_index)
    n = batch.batch_size              # the first n nodes are the targets
    loss = F.cross_entropy(out[:n], batch.y[:n])
    loss.backward()
    opt.step()

The loader builds the small sampled sub-graph for each batch. Only the first rows of the output belong to the target nodes. The rest are helper nodes.

Turning a transactions table into a graph

-- Node features: one row per account
SELECT account_id, age_days, avg_amount_30d, n_tx_30d
FROM account_features;

-- Edges: who paid whom in the last 90 days
SELECT sender_id AS src, receiver_id AS dst
FROM transfers
WHERE created_at >= CURRENT_DATE - INTERVAL '90' DAY;

After training, the embeddings for the accounts can be exported as new columns, such as emb_0 to emb_63, and joined back to the account table.

Embedding a brand-new node

# A new account appears with a few links already.
new_x = torch.tensor([[12.0, 0.8, 3.0]])           # raw features
# Build a small graph that includes the new node and its neighbours,
# then run the trained model on that sub-graph.
with torch.no_grad():
    emb = model(sub_x, sub_edge_index)[new_index]

No retraining is needed. The function learned in training is applied to the new neighbourhood.

Exporting embeddings as table columns

import numpy as np
import pandas as pd

model.eval()
with torch.no_grad():
    emb = model.embed_all(data)          # a placeholder helper returning (n, 64)

cols = [f"emb_{i}" for i in range(emb.shape[1])]
df_emb = pd.DataFrame(emb.numpy(), columns=cols)
df_emb["account_id"] = account_ids
df_emb.to_parquet("account_embeddings.parquet")

The resulting file can be joined to the account table on account_id. A flat-table model then treats the 64 columns like any other numeric features.

A feature-only comparison

A fair test trains three models on the same split:

1. A boosted tree model on the original node columns only.

2. The same model with hand-built neighbour aggregates.

3. The same model with GraphSAGE embeddings added.

If model 3 beats model 2 by more than the spread across seeds, the learned multi-hop signal is adding value. If not, the simpler aggregates are enough, and the pipeline can stay simple.

Checking for leakage

• Rebuild the graph as it looked at the prediction date, not as it looks today.

• Remove edges that exist only because of the outcome, such as a link created by a fraud review.

• Make sure the labels of validation nodes are not used as features of their neighbours.

• Check that embeddings for validation nodes are computed only from training-time information.

Strengths and limitations

Strengths

• Inductive: works on new nodes and new graphs that share the feature space.

• Scalable: mini-batches with fixed fan-out keep memory bounded.

• Flexible: several aggregators and both supervised and unsupervised training.

• Handles skewed degree distributions, since hubs are sampled.

• Practical for production, as embeddings can be refreshed in batch.

• Separates self and neighbour information through concatenation.

Limitations

• Sampling adds variance, so results can differ between runs and at inference.

• Fixed fan-out may miss rare but important neighbours.

• Depth is limited, because the sampled tree grows exponentially with layers.

• Features are required. A graph with no node features gives weak results.

• Mean aggregation weights neighbours equally, unlike GAT.

• Heavy hubs and duplicates can distort the sample.

• Feature drift hurts. If new nodes look different from training nodes, the function may not generalise.

How it compares with alternatives

Hand-built neighbour aggregates, such as counts and averages over a window, are the simple cousin of GraphSAGE. They are easy to explain and often strong. GraphSAGE adds learned, multi-hop and non-linear aggregation on top.

Role in tabular foundation models

Tabular foundation models learn from raw tables with mixed column types, missing values and often no column names. They are pre-trained on large numbers of synthetic or real tables, and at inference they read labelled rows as context, which is a kind of few-shot learning. GraphSAGE is not such a model, but it fills a gap they leave: most of them read one flat table at a time, while real data often spans linked tables.

Rows linked by keys

A database is a graph of rows joined by keys. GraphSAGE can turn a row and its linked rows into an embedding. Two practical patterns are common:

• Compute node embeddings with GraphSAGE and add them as columns to a flat table, which a model such as TabPFN, TabICL or XGBoost then reads.

• Use the same idea by hand: aggregate related rows with counts, means and maxima, which is one mean-aggregator layer written in SQL.

Mixed types and missing cells

GraphSAGE needs a numeric feature matrix. You must encode categories and dates, scale numbers and decide how to treat missing cells. A tabular foundation model handles these inside the network using per-cell encoders and missing-value flags. When using both, give GraphSAGE clean features with missing indicators, and keep the encoding consistent between training and inference, otherwise embeddings for new nodes will drift.

No column names

GraphSAGE treats features as anonymous channels. It never looks at column names. This is similar to how tabular foundation models identify columns by position and by data statistics. Both approaches stay useful when headers are missing or unreliable.

Cold start and inductive behaviour

The inductive property mirrors in-context learning in tabular models. A new node can be embedded with no retraining, and a new table can be handled with no gradient update. In both cases the model applies a general rule learned in advance to new data. See /model/architecture for how in-context prediction is arranged for tables, and /model/research for related work.

Imputation

A missing attribute can often be predicted from linked rows. A customer's missing region is likely to match the region of connected customers. A GraphSAGE embedding contains this neighbour signal, so it can improve imputation. It complements masked cell modeling and imputation networks, which learn to restore hidden cells from the rest of the row.

Anomaly detection

Embeddings from GraphSAGE can reveal accounts that behave unlike their neighbours. A simple recipe trains an unsupervised loss, then flags nodes whose embeddings are far from their neighbours' average, or from any cluster. See anomaly detection and isolation forest for models that can score such embeddings.

Sector and semantic classification

Pooling node or column embeddings gives a vector for a group, such as a table or a customer segment. A head on top can predict the domain, a task related to semantic sector heads.

Continual learning

Graphs grow all the time. GraphSAGE handles new nodes without retraining, but the function itself can drift as the data changes. Periodic fine-tuning with experience replay or regularisation such as EWC limits forgetting. For measuring such systems, see /model/benchmarks.

Practical guidance and best practices

Graph and feature preparation

• Build features only from information available at prediction time.

• Standardise numeric features and encode categories consistently.

• Add missing-value indicators.

• Remove duplicate edges, or weight them deliberately.

• Decide whether edges are directed and whether to include reverse edges.

Sampling choices

• Use fan-outs such as 25 and 10 as a starting point.

• Use more samples at inference than in training to reduce variance.

• Consider importance sampling if a few neighbours matter much more.

• Cap very large hubs.

Model choices

• Start with two layers and the mean aggregator.

• Try the pooling aggregator if the mean underperforms.

• Add dropout and weight decay.

• Normalise outputs when embeddings will be used in distance search.

Training and evaluation

• Use time-based splits when the task is forecasting.

• Hold out nodes, and also check performance on nodes added after the training cut-off.

• Compare with a feature-only model and with hand-built aggregates.

• Repeat with several seeds, since sampling adds noise.

Production

• Precompute embeddings in batch and store them in a feature store or a vector index.

• Compute embeddings on demand for new nodes with a small sub-graph query.

• Version the model and the feature definitions together.

• Monitor embedding drift, for example the mean and spread of each dimension over time.

A decision guide

Sizing a first experiment

• Hidden width: 64 to 256 is typical.

• Fan-outs: 25 and 10, then tune.

• Batch size: 256 to 2048 targets.

• Learning rate: around 0.001 to 0.005 with Adam.

• Epochs: until validation loss stops improving, often tens of epochs.

• Seeds: at least three.

Common mistakes

• Leaking the label through features or through edges that are created after the outcome.

• Splitting nodes at random when new nodes arrive over time.

• Using test-time neighbours that include labelled test nodes in a way not possible in production.

• Using a graph with no node features and expecting good results.

• Ignoring variance from sampling and reporting a single run.

• Choosing very deep stacks, which blow up the sampled tree and over-smooth.

• Changing feature encodings between training and serving.

• Treating embeddings as stable forever, without checking drift.

Frequently asked questions

What does GraphSAGE stand for?

The name stands for SAmple and aggreGatE. The method samples a fixed number of neighbours for each node, aggregates their features with a learned or simple function, and combines the result with the node's own features to produce an embedding.

Why is GraphSAGE called inductive?

It learns a function that maps node features and neighbourhoods to embeddings, rather than a lookup table of embeddings. A new node can therefore be embedded by applying the same function to its own features and neighbours, without retraining the whole model.

How does neighbour sampling help?

Sampling fixes the number of neighbours per node, so the cost of a batch does not depend on how large the graph is or how many links a hub has. It lets you train on graphs that do not fit in memory, and it adds a small amount of noise that acts as regularisation.

Which aggregator should I use?

Start with the mean aggregator, since it is simple, fast and strong. Try the pooling aggregator if you need more expressive power. The LSTM aggregator can work well but depends on neighbour order, so it requires random permutation and is used less often.

How is GraphSAGE different from GCN?

GCN uses all neighbours with a fixed normalised average and is usually trained on the full graph. GraphSAGE samples neighbours, supports several aggregators, keeps the node's own vector separate through concatenation, and is designed to embed unseen nodes. In practice, the mean version of GraphSAGE is close to GCN with sampling.

Can GraphSAGE be trained without labels?

Yes. An unsupervised loss encourages nodes that are close in the graph, for example nodes that co-occur on short random walks, to have similar embeddings, and pushes random pairs apart. The embeddings can then be used for clustering, search or as features for another model.

Does GraphSAGE need node features?

Yes. Since it learns a function from features to embeddings, features are essential. If a graph has no natural features, people sometimes use degree statistics or learned ids, but this removes much of the inductive benefit.

Can GraphSAGE help with tabular data?

It can, when rows are linked by keys such as customer, account or product ids. Embeddings computed by GraphSAGE can be added as columns to a flat table. If rows are independent, a graph adds nothing, and a model built for tables is the better tool.

How many layers should I use?

Two layers are the usual choice. Each additional layer multiplies the number of sampled nodes and tends to blur node differences. If you need a wider view, consider skip connections, different aggregators or precomputed multi-hop features.

How do I deploy GraphSAGE in production?

A common approach is to compute embeddings for all known nodes in a batch job and store them for fast lookup, then compute embeddings on demand for new nodes using a small sampled sub-graph. Version the feature definitions with the model and monitor drift in the embedding distribution.

Glossary

• Aggregator: A function that combines neighbour vectors.

• Cold start: Making predictions for a node with no history.

• Embedding: A dense vector that represents a node.

• Fan-out: The number of neighbours sampled per layer.

• Inductive learning: Learning that generalises to unseen nodes or graphs.

• Mini-batch: A small group of target nodes processed at once.

• Negative sampling: Drawing random unrelated pairs to contrast with related ones.

• Neighbour sampling: Picking a fixed number of neighbours per node.

• Random walk: A path that moves to a random neighbour at each step.

• Receptive field: The nodes that can affect a target's embedding.

• Transductive learning: Learning embeddings only for nodes present at training time.

Summary

GraphSAGE learns a function that builds a node's embedding from its own features and a sample of its neighbours, using an aggregator such as the mean, a pooling operation or a recurrent network. Because it learns a function and not a table of vectors, it embeds unseen nodes and scales to large graphs through fixed fan-out sampling. Its costs are sampling variance, the need for node features and limited depth. Compared with GCN it is built for change and scale, and compared with GAT it trades learned neighbour weights for efficiency. For tabular foundation models, it supplies relational context for rows linked by keys, supports imputation and anomaly detection, and shares the idea of applying a pre-learned rule to new data. See /model/architecture, /model/research and /model/benchmarks for further reading.

Related

It belongs to Architectures in my stack, under AI and machine learning. Others in the same group: