GAN

Overview

A GAN, or generative adversarial network, is a pair of neural networks trained as opponents. The generator makes fake samples from random noise. The discriminator looks at a sample and guesses whether it is real or fake. As each network improves, the other has to improve too, and the generator ends up producing samples that look like the training data.

GANs became famous for realistic images, and they are still used for image synthesis, super-resolution, style transfer and data augmentation. They are also used to make synthetic tables, where the goal is to create rows that look like real records without copying them. This page explains how GANs work, why they are hard to train, how they compare with Diffusion Models and VAE, and how they connect to models that learn from raw tables.

What it is

A GAN is a generative model. A generative model learns the distribution of a dataset so it can produce new samples that could have come from that dataset. Many generative models do this by computing a likelihood. A GAN avoids that. It learns by a game.

The two players

• The generator takes a vector of random numbers, called the latent vector, and turns it into a sample such as an image or a table row.
• The discriminator takes a sample, either real or generated, and outputs the probability that it is real.

Neither network is told what a good sample looks like. The generator learns only from the discriminator's feedback. The discriminator learns only from the labels "real" and "fake", which come for free from where the sample was taken.

The forger and the inspector

The classic analogy is a forger and an art inspector. The forger paints copies and tries to pass them off as originals. The inspector studies real paintings and the forger's copies, and learns to spot the differences. Each time the inspector catches a flaw, the forger learns what to fix. Each time the forger improves, the inspector must look harder. In the ideal end state, the forger's work is so close to the real thing that the inspector can only guess, with a fifty percent success rate.

What a GAN is not

A GAN does not store and replay training samples by design, although it can memorise them if it overfits. It does not give you an exact probability for a sample. It does not come with a built-in way to encode a real sample back into the latent space. These traits shape how GANs are used and evaluated.

Main families

Where it came from

The GAN was introduced in 2014 in a paper titled "Generative Adversarial Nets". The authors described the minimax game between a generator and a discriminator and showed that, in theory, the game has a solution where the generator matches the data distribution.

The early samples were small and blurry. In 2015 the DCGAN recipe gave practical rules for stable convolutional GANs, such as using strided convolutions, batch normalisation and certain activations. In 2017 the Wasserstein GAN replaced the original loss with a distance that gives smoother gradients, and a gradient penalty variant followed. Progressive growing, which trains at low resolution and adds layers over time, led to sharp images in 2017 and 2018. StyleGAN arrived in 2018 and set a new standard for image quality and control.

Conditional GANs and image-to-image models such as pix2pix and CycleGAN, from 2016 and 2017, showed that GANs could translate sketches to photos and horses to zebras. For tables, a conditional tabular GAN design appeared in 2019, with mode-specific normalisation for numeric columns and conditional sampling for categories. It remains a widely cited baseline for synthetic tables.

From 2020 onward, diffusion models, covered in DDPM and Diffusion Models, overtook GANs on image benchmarks because they train more stably and cover the data better. GANs remain valuable when one forward pass must be fast, for example in real-time rendering. See /model/research for how generative ideas feed into table-focused work.

How it works

Step 1: sample noise

Draw a latent vector z from a simple distribution, usually a standard normal. A typical size is 64 to 512 numbers. This vector is the raw material. Different draws of z lead to different outputs.

Step 2: generate a sample

The generator G maps z to a sample G(z). For images this is a stack of upsampling layers. For tables it is a stack of dense layers that output numbers for numeric columns and probabilities for categorical columns.

Step 3: judge real and fake

The discriminator D receives a batch with real samples from the dataset and fake samples from G. It outputs a score for each. In the original form, the score is the probability that the sample is real.

Step 4: update the discriminator

The discriminator is trained as a standard classifier. It is rewarded for scoring real samples high and fake samples low. Only D's weights change in this step.

Step 5: update the generator

The generator is trained to fool D. It is rewarded when D scores its fakes as real. The gradient flows through D into G, so G learns which changes to its output would raise D's score. Only G's weights change in this step.

Step 6: repeat

The two updates alternate for many thousands of iterations. Often D is updated several times per G update, especially in Wasserstein variants.

The objective

The original game can be written as a minimax problem.

min over G, max over D of
    E_x~data [ log D(x) ] + E_z~noise [ log(1 - D(G(z))) ]

In practice the generator uses a non-saturating loss, which maximises log D(G(z)) instead. It gives stronger gradients early in training, when D easily rejects the fakes.

What the game means

If D were perfect, it would measure the gap between the real and generated distributions. Training G against this D pushes the generated distribution towards the real one. In theory the equilibrium is reached when the two match and D outputs one half everywhere. In practice, networks are finite and optimisation is noisy, so the game rarely reaches a clean equilibrium. That is the root of most GAN training problems.

A tiny numeric walk-through

Suppose D sees two real samples and two fake ones, and outputs these probabilities of "real":

• Real: 0.9 and 0.8.
• Fake: 0.3 and 0.2.

D is doing well. Its loss is low, because it scores real samples high and fake samples low. The generator's non-saturating loss is the average of the negative log of D's scores on fakes: negative log of 0.3 is about 1.2, and negative log of 0.2 is about 1.6, so the mean is about 1.4. That is a large loss, so G gets a strong push to improve.

After some updates, suppose D's scores on fakes rise to 0.5 and 0.45. The mean loss drops to about 0.74. D can no longer tell fakes apart with confidence, and the game is near balance. If the scores on fakes keep rising above the scores on real samples, D would then retrain to correct itself.

Common training problems

• Mode collapse: the generator produces only a few kinds of samples, because they fool D well enough.
• Vanishing gradients: if D wins too easily, G receives little useful signal.
• Oscillation: the two networks chase each other without settling.
• Non-convergence: the loss values do not tell you how good the samples are.
• Sensitivity to hyperparameters: small changes in learning rate or architecture can break training.

Tricks that help

• Use the non-saturating generator loss.
• Use a Wasserstein loss with a gradient penalty, or spectral normalisation on D.
• Use two time-scale updates, with a different learning rate for each network.
• Add small noise to D's inputs, or use label smoothing.
• Use data augmentation on both real and fake samples for small datasets.
• Keep a moving average of G's weights for sampling.
• Monitor sample quality visually or with metrics, not just loss curves.

Loss functions at a glance

Image architectures in brief

For images, the generator starts with a small grid of features made from the latent vector. It then upsamples again and again, using transposed convolutions or resize-then-convolve blocks, until it reaches full size. The discriminator mirrors this: it downsamples with strided convolutions until a single score remains. Residual blocks, attention layers and normalisation are added in larger systems. The ideas in CNN and ResNet carry over directly.

Latent space structure

A well-trained generator organises its latent space smoothly. Nearby latent vectors give similar outputs. Walking in a straight line between two vectors gives a smooth morph. Some directions in latent space correspond to clear changes, such as pose or lighting. This is why latent editing is popular in image GANs. For tables, smoothness means that small changes to z give small changes in the row, which is a useful sanity check.

Designing a tabular GAN

A good tabular GAN has several parts that differ from an image GAN.

• A transformer for each column that turns raw values into model-friendly numbers. Numeric columns are often split into several Gaussian modes, so a multi-peaked column becomes a mode choice plus a position within that mode.
• A one-hot encoding for each categorical column, with rare categories grouped if needed.
• A generator output layer with a separate head per column type: tanh or linear for numbers, softmax or Gumbel-softmax for categories.
• A conditional vector that picks one categorical value at random during training, so rare categories get enough attention.
• A discriminator that sees groups of rows together, so it can notice when a batch lacks diversity.
• An inverse transform that turns the output back into the original column formats.

Everyday picture of training

Imagine a street artist and a passer-by who points out which drawings look off. At first the artist draws stick figures and the passer-by laughs. The artist learns that heads should be round and begins to add faces. The passer-by now looks for finer errors, such as odd hands. Over many rounds the portraits become convincing. If the passer-by only ever comments on one kind of portrait, the artist may learn only that kind. That is the human version of mode collapse.

Conditional GANs

In a conditional GAN, both players receive a condition c, such as a class label or a text embedding. The generator makes G(z, c), and the discriminator judges D(x, c). The condition steers the output. For tables, the condition may be a category value, which lets you control how often each category appears in the synthetic data.

Key concepts and terms

A worked example

This example builds a small GAN for synthetic tabular rows with two numeric columns and one categorical column. All data is made up for illustration.

The data

Each row has an age-like number, an income-like number, and a category with three values. The numbers are correlated, and the category shifts their averages.

import numpy as np
import torch
import torch.nn as nn

rng = np.random.default_rng(0)
n = 5000
cat = rng.integers(0, 3, size=n)
base = np.array([30.0, 45.0, 60.0])[cat]
age = base + rng.normal(0, 5, size=n)
income = 1.5 * age + rng.normal(0, 8, size=n)

num = np.stack([age, income], axis=1)
mu, sd = num.mean(0), num.std(0)
num_n = (num - mu) / sd
cat_oh = np.eye(3)[cat]
real = torch.tensor(np.concatenate([num_n, cat_oh], axis=1), dtype=torch.float32)

Networks

Z = 16

G = nn.Sequential(
    nn.Linear(Z, 64), nn.ReLU(),
    nn.Linear(64, 64), nn.ReLU(),
    nn.Linear(64, 5),            # 2 numeric + 3 category logits
)

D = nn.Sequential(
    nn.Linear(5, 64), nn.LeakyReLU(0.2),
    nn.Linear(64, 64), nn.LeakyReLU(0.2),
    nn.Linear(64, 1),
)

def make_sample(raw):
    num = raw[:, :2]
    cat = torch.softmax(raw[:, 2:], dim=1)   # soft one-hot keeps gradients
    return torch.cat([num, cat], dim=1)

Training loop

optG = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.9))
optD = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.9))
bce = nn.BCEWithLogitsLoss()

for step in range(5000):
    idx = torch.randint(0, real.shape[0], (128,))
    xr = real[idx]
    z = torch.randn(128, Z)
    xf = make_sample(G(z))

    # discriminator step
    optD.zero_grad()
    lossD = bce(D(xr), torch.ones(128, 1)) + bce(D(xf.detach()), torch.zeros(128, 1))
    lossD.backward()
    optD.step()

    # generator step
    optG.zero_grad()
    lossG = bce(D(make_sample(G(torch.randn(128, Z)))), torch.ones(128, 1))
    lossG.backward()
    optG.step()

Checking the output

Do not trust the loss. Check the data.

1. Generate 5000 synthetic rows and undo the scaling.
2. Compare the mean and spread of each numeric column to the real data.
3. Compare the share of each category.
4. Compare the correlation between age and income.
5. Train a simple classifier to tell real from synthetic rows. If it reaches near fifty percent accuracy, the synthetic data is hard to tell apart.
6. Train a model on synthetic rows and test it on real rows, then compare with a model trained on real rows. This is called train on synthetic, test on real.

Checking privacy

Synthetic does not automatically mean private. Measure the distance from each synthetic row to its nearest real row. If many synthetic rows are almost exact copies, the model has memorised. Reduce training time, add regularisation, or use a privacy-preserving training method.

Common surprises in this example

• The category column may drift, with one category becoming too frequent. Conditional sampling fixes this.
• The correlation may be weaker than in the real data. A larger model or longer training may help.
• Numeric tails may be cut off. Mode-specific normalisation helps with multi-peaked columns.

Strengths and limitations

Strengths

• Sharp, realistic samples, especially for images.
• Fast generation, since a sample needs one forward pass.
• No need for an explicit likelihood.
• Flexible: works for images, audio, tables and more.
• Conditional versions give control over the output.
• Useful for data augmentation when real data is rare.

Limitations

• Training is unstable and sensitive to settings.
• Mode collapse can leave parts of the data uncovered.
• Loss curves do not reliably show sample quality.
• No exact likelihood, so comparing models is harder.
• Hard to encode a real sample into the latent space without extra networks.
• Can memorise training data, which raises privacy concerns.
• Mixed-type tables with rare categories and heavy-tailed numbers are difficult.
• Recent diffusion models often beat GANs on coverage and quality.

How it compares with alternatives

When to pick a GAN

Choose a GAN when you need many samples quickly, when sharpness matters more than perfect coverage, and when you can afford careful tuning. Choose a VAE or diffusion model when stability and coverage matter more. For small tables, a statistical model such as a copula can be both simpler and more faithful.

GAN and StyleGAN

StyleGAN is a GAN architecture that injects style vectors at each layer of the generator. It gives fine control over image features and is the best-known high-resolution GAN. The general GAN training game stays the same.

Role in tabular foundation models

Tabular foundation models learn from raw tables. Tables have rows and columns, mixed numeric and categorical types, missing cells, and often no useful column names. See /model/architecture for how a foundation model for tables is built. GANs relate to this field in several concrete ways.

Synthetic tables for pre-training and testing

Foundation models for tables need large amounts of varied training tables. Real tables are limited and often private. Generative models, including GANs, can produce extra tables that mimic the structure of real ones. More often, though, tabular foundation models are pre-trained on tables drawn from random programs or structural causal models, since these give wide variety without needing real data. See Synthetic Data Generation and TabPFN for an example of this approach. A GAN is one tool for making synthetic data. It is rarely the main tool for broad pre-training.

Privacy and data sharing

Synthetic rows can let teams share data patterns without sharing the original records. A GAN trained on a sensitive table can output rows with similar statistics. The risk is leakage, so use nearest-neighbour checks and consider differentially private training. Never assume synthetic data is anonymous without testing.

Class imbalance and augmentation

In fraud, fault detection and rare-event tables, positives are scarce. A conditional GAN can create more minority-class rows. Simple methods like oversampling or SMOTE often work as well on small tables, so compare before adding complexity. Evaluate on untouched real data only.

Mixed types and missing values

A tabular GAN must handle numbers, categories and missing cells in one output vector. Typical solutions are:

• Mode-specific normalisation for numeric columns with several peaks.
• One-hot or soft one-hot outputs for categories, often with Gumbel-softmax.
• A mask output for missing cells, so the generator learns where values are absent.
• Conditional sampling so rare categories appear often enough in training.

Tabular foundation models solve the same problems from the encoder side. Each cell is embedded by its type, and missing values carry their own marker. Compare this with Imputation Networks and Masked Cell Modeling.

No column names

A GAN for tables usually treats columns by position. If column order changes, the model must be retrained. Foundation models aim for order-independent processing across columns and rows, using attention in the style of SAINT. This lets them accept a table they have never seen, even with unnamed columns.

In-context learning versus per-table training

A GAN must be trained from scratch for each table. A tabular foundation model reads a new table at prediction time and adapts in context, with no new training. This difference is large for small tables, where training a GAN well is hard.

Imputation

A GAN can be set up for imputation. In the GAIN design, the generator fills missing entries and the discriminator guesses which entries were imputed. Pre-trained tabular models now do imputation by masked cell prediction, which is simpler to train.

Anomaly detection

GAN-based anomaly detectors learn what normal data looks like. A sample that the generator cannot reproduce, or that the discriminator scores as unusual, is flagged. See Anomaly Detection. In practice, simpler detectors often match these methods on tables.

Continual learning and drift

When data drifts, a GAN trained on old data produces old patterns. Retraining can cause forgetting of earlier modes. Techniques like Experience Replay can be used, where a generator itself replays old samples to keep earlier knowledge, an idea known as generative replay.

Sector and semantic classification

A GAN is not a classifier, but synthetic tables for different sectors can help build labelled training sets for table-type classification. The caveat is that synthetic examples must stay faithful to real sector patterns, or the classifier will learn artefacts.

Practical guidance and best practices

Before you start

• State the purpose: augmentation, sharing, testing or research. It sets your quality bar.
• Try simple baselines first, including resampling and statistical models.
• Hold out a real test set that no generator ever sees.

Architecture

• For images, start from a proven recipe such as a DCGAN or a StyleGAN implementation.
• For tables, use conditional sampling and type-aware encoding.
• Keep the discriminator and generator balanced in size.
• Use normalisation layers carefully; some work best only in one network.

Training

• Use Adam with a low learning rate and a low first momentum value.
• Try a Wasserstein loss with gradient penalty for stability.
• Update the discriminator more often than the generator when using a critic.
• Save checkpoints often and compare samples across them.
• Watch for mode collapse by checking sample diversity every few thousand steps.

Evaluation

Hyperparameters that matter

A simple experiment plan

1. Train a baseline with resampling or a statistical model and record its scores.
2. Train a small GAN and record the same scores.
3. Change one setting at a time and keep a log of each change and its effect.
4. Pick the best by downstream usefulness and privacy checks, not by loss.
5. Repeat with three random seeds to see how stable the result is.

Responsible use

• Label synthetic data clearly so it is not mistaken for real records.
• Do not use generated media to mislead people.
• Document the training data and the checks you ran.
• Keep a human review step for any use that affects people.

Common mistakes

• Trusting the loss curve as a quality measure.
• Evaluating only on a few cherry-picked samples.
• Training and testing downstream models on the same synthetic data.
• Letting the discriminator get too strong too early.
• Ignoring mode collapse because the few samples look good.
• Assuming synthetic data is private without checking for memorised rows.
• Applying a GAN to a tiny table where a simple statistical model would do better.
• Forgetting to preprocess skewed numeric columns.
• Dropping rare categories, which then never appear in the output.
• Mixing up train and test data when scaling.
• Leaving out a baseline such as resampling.

Frequently asked questions

What does GAN stand for?

GAN stands for generative adversarial network. It is a framework in which two neural networks, a generator and a discriminator, are trained in competition so that the generator learns to produce realistic samples from random noise.

How does a GAN learn without labels?

The only labels it needs are "real" and "fake", and those come from the source of each sample. Real samples come from the dataset and fake ones come from the generator. The discriminator learns to separate them, and the generator learns from the discriminator's feedback, so no human labelling is required.

What is mode collapse?

Mode collapse is when the generator produces only a small set of outputs instead of the full variety in the data. It happens because a few outputs can fool the discriminator reliably. Remedies include Wasserstein losses, minibatch-based diversity features, and larger or better-regularised networks.

Why are GANs hard to train?

Training is a two-player game, not a single minimisation problem. If one network gets too strong, the other receives poor feedback, and the pair can oscillate instead of settling. Small changes to learning rates, architecture or data can tip the balance, so careful tuning and monitoring are needed.

Are GANs still used after diffusion models?

Yes. Diffusion models lead many image benchmarks, but GANs produce samples in one pass, which is much faster. They are still used in real-time graphics, video effects, super-resolution and some synthetic data work where speed matters.

Can a GAN create synthetic tabular data?

Yes. Tabular GANs handle mixed numeric and categorical columns using special encoding and conditional sampling. They can work well on mid-sized tables, but simpler statistical methods or a VAE can be as good or better, so test several approaches.

Is GAN-generated data private?

Not automatically. A GAN can memorise and reproduce training rows, especially when it overfits. Measure nearest-neighbour distances, run membership tests, and consider differentially private training if the source data is sensitive.

How do I evaluate a GAN?

Use several checks. Compare statistics of real and synthetic data, train a classifier to tell them apart, run downstream models trained on synthetic data and tested on real data, and inspect samples. For images, feature-based scores such as FID are common.

What is a conditional GAN?

A conditional GAN gives both networks an extra input, such as a class label. The generator makes samples that match the label, and the discriminator checks both realism and match. This allows control over what kind of sample is generated, for example a chosen category in a table.

How is a GAN different from a tabular foundation model?

A GAN is trained for one dataset and generates new samples. A tabular foundation model is pre-trained on many tables and makes predictions on a new table by reading it in context. They can be combined, for example by using synthetic data in pre-training, as described in /model/architecture.

Glossary

• Adversarial training: training two models against each other.
• Critic: the name for the discriminator in Wasserstein GANs, which outputs a score, not a probability.
• Discriminator: the network that judges whether a sample is real.
• FID: Frechet Inception Distance, a feature-based score comparing real and generated images.
• Generator: the network that turns noise into samples.
• Gradient penalty: a regulariser that keeps the critic's gradient norm near one.
• Latent space: the space of input noise vectors for the generator.
• Mode: a cluster or type of sample in the data distribution.
• Mode collapse: failure where the generator covers too few modes.
• Non-saturating loss: a generator loss that gives strong gradients early in training.
• Spectral normalisation: scaling weights so a layer cannot amplify too much.
• Synthetic data: artificial data that mimics the statistics of real data.
• Train on synthetic, test on real: an evaluation where a model learns from generated data and is scored on real data.
• Wasserstein distance: a distance between distributions based on the cost of moving mass.

Summary

A GAN trains a generator and a discriminator in a game. The generator turns noise into samples. The discriminator tries to tell them from real data. Over time the generator learns the data distribution and produces realistic outputs in a single forward pass. This makes GANs fast and sharp, but also hard to train, prone to mode collapse, and difficult to evaluate with loss curves alone.

For tabular work, GANs offer a way to create synthetic rows for augmentation, testing and data sharing, as long as privacy and fidelity are checked. Tabular foundation models take a different route: they pre-train on many tables, often synthetic ones, and learn in context without per-table training. See /model/architecture, /model/research and /model/benchmarks for more. Choose a GAN when speed and sharpness matter, compare it with a VAE or diffusion model, and always validate on untouched real data.

Related

It belongs to Architectures in my stack, under AI and machine learning. Others in the same group: