SpikingJelly

Overview

SpikingJelly is an open-source framework for spiking neural networks built on PyTorch. It provides neuron layers, surrogate gradients, spike encoders, event-based datasets, optimized kernels and tools to convert conventional networks into spiking ones. It is designed to be a full workbench: you can load a neuromorphic dataset, build a spiking model, train it on a GPU and measure the result, all from one package.

The framework is popular for benchmarking on event data, such as recordings from event cameras. Its multi-step mode and optional compiled kernels make spiking models train faster than naive loops. It sits beside snnTorch and Norse, which are lighter research libraries, and beside Lava, which targets event-driven hardware. If you are new to the field, the background on spiking neural networks helps.

What it is

SpikingJelly is an infrastructure platform for spike-based machine learning. It is a Python package with several layers of functionality.

Main components

• Neuron nodes. Modules such as integrate-and-fire and leaky integrate-and-fire nodes, plus parametric and adaptive variants.
• Surrogate functions. A collection of smooth backward-pass stand-ins for the spike function, with settable steepness.
• Layers and containers. Wrappers that apply ordinary layers, such as convolution or normalization, across the time dimension.
• Encoders. Tools that convert numbers or images into spike trains.
• Datasets. Loaders and preprocessing for event-based and static datasets.
• Conversion tools. Routines for turning a trained conventional network into a spiking network.
• Kernel backends. Optional compiled kernels that speed up neuron updates and time loops.
• Utilities. Functions for resetting state, monitoring activity and plotting.

Single-step and multi-step modes

This is a central idea in the library.

In single-step mode, a layer processes one time step at a time. You write the time loop yourself.

In multi-step mode, a layer takes a tensor of all time steps at once and processes the whole sequence in one call.

Analogy: single-step is like reading a book one page at a time and turning each page yourself. Multi-step is like handing the whole book to a reader who flips pages for you.

Multi-step mode lets the library fuse operations and run stateless layers, such as convolutions, on all steps together by folding time into the batch dimension. That is a big speed gain.

Memory and speed options

• Choose a plain PyTorch backend for easy debugging.
• Choose a compiled backend for faster training when you have a supported GPU setup.
• Use mixed precision where it is safe.

Where it came from

Spiking neural networks come from computational neuroscience. The leaky integrate-and-fire neuron is the simple model most people use. For a long time, training was hard because a spike is a hard threshold with no useful gradient.

Surrogate gradient methods changed that. They keep the spike in the forward pass and use a smooth curve in the backward pass. After this, deep spiking networks with convolution layers and residual connections became trainable on image and event data.

Event cameras, which report only changes in brightness, created a natural data source for spiking models. Datasets recorded with these sensors, such as gesture and object recordings, became common benchmarks. Researchers needed a tool to load them and train on them without writing custom code each time.

SpikingJelly grew out of this need. It was released as an open-source project by a research group and later described in a journal paper as an infrastructure platform for spike-based intelligence. Over time it added more neuron types, a multi-step mode, compiled kernels and conversion tools.

Other libraries chose different focuses. snnTorch concentrates on teaching and simple APIs. Norse concentrates on bio-inspired components. Lava concentrates on deployment to hardware such as Loihi. For research context on this site see /model/research.

How it works

This section follows a typical SpikingJelly training pipeline.

Step 1: Load the data

For event-based datasets, raw data is a list of events with position, time and polarity. The dataset loader integrates events into frames over a chosen number of time slices.

You choose how to split time. One option is a fixed number of frames per sample. Another is a fixed duration per frame.

The output is a tensor of shape time, channels, height, width for each sample.

For static data such as images, you either repeat the image over time steps or use an encoder.

Step 2: Encode when needed

Encoders convert real values into spikes.

• Poisson encoder. Each value is the probability of a spike at each step.
• Latency encoder. Larger values spike earlier.
• Weighted phase and other encoders. Spread information across steps in patterns.
• Direct encoding. Feed the analog values straight into the first layer and let that layer's neurons produce spikes.

Analogy: Poisson encoding is rain, where heavier rain means more drops per second. Latency encoding is a race, where faster runners arrive first. Direct encoding is giving the first layer the actual weather report and letting it decide when to ring the bell.

Step 3: Build the network

A network is a stack of ordinary layers and spiking nodes.

A common block is convolution, batch normalization, then a spiking node. Blocks are stacked, followed by pooling and a final linear layer.

Because convolution and normalization have no state, they can process all time steps at once in multi-step mode.

Only the neuron nodes need to loop over time, since they hold the membrane state.

Step 4: The neuron update

For a leaky integrate-and-fire node, each time step does the following:

1. Charge: the membrane moves toward the input, with a decay set by a time constant.
2. Fire: if the membrane is above the threshold, output a spike.
3. Reset: lower the membrane, either to a fixed value or by subtraction.

The same computation is carried out on whole tensors for many neurons at once.

Step 5: Read the output

Average the output of the last layer over time. The mean firing rate of each class neuron acts as a score. A standard cross entropy loss then compares scores with the label.

Step 6: Train with surrogate gradients

In the backward pass, the spike function has a surrogate derivative. SpikingJelly supplies several shapes, including arctangent, sigmoid and piecewise linear.

The loss is back-propagated through time and layers. The optimizer updates weights as usual.

Step 7: Reset state

After each batch you must reset all neuron states. A single helper call walks the network and clears them.

If you forget, the membrane potential of one sample leaks into the next.

ANN to SNN conversion

There is a second route to a spiking network.

1. Train a conventional network with ReLU activations.
2. Normalize activations so that they fit in a fixed range.
3. Replace ReLU units with spiking nodes.
4. Run for enough time steps that firing rates approximate the original activations.

The idea is that a rate-coded integrate-and-fire neuron approximates a ReLU. The more time steps you allow, the better the match.

The cost is latency. Conversion often needs many steps. Direct training can reach good accuracy with fewer steps.

Common neuron nodes and when to use them

A learned decay adds a few parameters and often improves accuracy for little cost.

Reset choices

After a spike the membrane can be set to a fixed value (hard reset) or lowered by the threshold (soft reset).

• Hard reset throws away extra charge. It is simple and common in direct training.
• Soft reset keeps extra charge. It fits conversion, since it preserves the rate relationship.

Detaching the reset path

In the backward pass, the gradient can flow through the reset or be cut there.

Cutting it (detaching) often makes training more stable, because it removes a feedback path that can blow up gradients.

Try both when training is unstable.

Key concepts and terms

A worked example

Building and training a small network

The sketch below shows the typical shape of a multi-step spiking classifier. Names follow the public API in spirit. Always check the current documentation, since arguments and module paths change across versions.

import torch
import torch.nn as nn
from spikingjelly.activation_based import neuron, functional, surrogate, layer

T = 8  # number of time steps

net = nn.Sequential(
    layer.Flatten(),
    layer.Linear(20, 64),
    neuron.LIFNode(tau=2.0, surrogate_function=surrogate.ATan()),
    layer.Linear(64, 3),
    neuron.LIFNode(tau=2.0, surrogate_function=surrogate.ATan()),
)
functional.set_step_mode(net, step_mode="m")   # multi-step mode

opt = torch.optim.Adam(net.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()

x = torch.rand(16, 20)                  # batch of 16 rows with 20 features
y = torch.randint(0, 3, (16,))

for epoch in range(5):
    x_seq = x.unsqueeze(0).repeat(T, 1, 1)   # repeat over time: T x B x F
    out = net(x_seq).mean(dim=0)             # average firing rate over time
    loss = loss_fn(out, y)
    opt.zero_grad()
    loss.backward()
    opt.step()
    functional.reset_net(net)                # clear neuron states
    print(epoch, float(loss))

Reading the code

• The input is repeated over T steps, which is direct encoding.
• The network is a plain sequence of layers and spiking nodes.
• Multi-step mode is switched on for the whole network at once.
• The output is the mean spike rate per class over time.
• The state is reset after each batch.

A numeric walk-through of a LIF node

Take tau = 2, threshold 1.0, reset to zero, and constant input 1.2. A common charge rule is v = v + (x - v) / tau.

The neuron needs three steps to build up and fire. A larger input or a smaller tau makes it fire sooner. Over a long run, this constant input turns into a regular spike rate.

Loading an event dataset

from spikingjelly.datasets.n_mnist import NMNIST
from torch.utils.data import DataLoader

train_set = NMNIST(root="./data", train=True, data_type="frame",
                   frames_number=10, split_by="number")
loader = DataLoader(train_set, batch_size=32, shuffle=True)

frames, label = next(iter(loader))
# frames shape: batch x T x channels x height x width
frames = frames.transpose(0, 1)    # to T x batch x C x H x W for multi-step mode

The key choice is frames_number. Ten frames give coarse timing and fast training. Many frames give fine timing and slower training.

What a good run looks like

• The training loss falls steadily.
• Validation accuracy tracks training accuracy with a modest gap.
• Hidden layers fire at a moderate rate, neither silent nor saturated.
• The total number of spikes per sample is stable from epoch to epoch.

Choosing frames for event data

Event data is a list, not an image. The way you cut it into frames shapes everything.

For most benchmarks the fixed number rule is the easiest to batch.

Data augmentation for events

• Random shifts and flips of the spatial grid.
• Random cropping.
• Time jitter, shifting event times by small amounts.
• Event dropout, removing a fraction of events.

These help because event datasets are often small.

Caching processed data

Converting raw events to frames takes time. Do it once and store the frames on disk, then load the cached version for training.

Record the settings used, so you do not mix caches built with different frame counts.

Strengths and limitations

Strengths

• Complete workbench. Datasets, neurons, encoders, conversion and kernels in one package.
• Speed. Multi-step mode and compiled backends reduce training time.
• Event data focus. Strong support for event camera benchmarks.
• Many neuron types. Easy to compare different dynamics.
• Familiar to PyTorch users. Standard optimizers, losses and GPU workflow.
• Conversion route. Lets you start from an existing conventional model.

Limitations

• Larger surface area. More modules and options than a minimal library, so the learning curve is steeper.
• API changes. Moving between versions or between older and newer module layouts can break code.
• Hardware gains are not automatic. Sparse spikes save energy only on suitable chips.
• Memory. Backpropagation through time keeps activations for every step.
• Compiled kernels need setup. A matching GPU toolkit and compiler are required for the fast backend.
• Not a deployment framework. Export to chips needs extra steps, for example through Lava.

How it compares with alternatives

A short rule: pick SpikingJelly when you want to train and benchmark on event data quickly, snnTorch when you want the gentlest start, Norse when you want to change the neuron equations, and Lava when your goal is a deployed event-driven system.

Role in tabular foundation models

Tabular foundation models learn from raw tables. A table has rows and columns, mixed types such as numbers and categories, missing values and sometimes no helpful column names. These models are often pre-trained on many synthetic or real tables and then predict on a new table through in-context learning, without updating weights at prediction time. See TabPFN and FT-Transformer, and /model/architecture for how such a model is structured.

SpikingJelly is not a tabular model. It belongs to a different line of research. Its connection to tabular work is limited, but there are concrete places where the two meet.

Points of contact

• Rows as input. A row is a numeric vector. With direct encoding, you feed it repeatedly over time steps. With Poisson or latency encoders you turn each feature into spikes. This is the counterpart of the embedding step in tabular transformers, where each cell becomes a vector.
• Missing values. An unobserved cell can be encoded as silence, but under rate coding a true zero also looks silent. Adding a binary mask input keeps these cases apart. For model-based filling see imputation networks.
• Anomaly detection. Spiking autoencoders and prediction-error networks can flag odd rows or windows in sensor tables, and their sparse activity suits streaming at the edge. See Anomaly Detection.
• Time-stamped tables. Event logs are tables where each row is an event with a time. Converting such logs into frames resembles what the dataset loaders do for event cameras: bin events by time, then feed bins as steps.
• Continual learning. Local plasticity and small recurrent state can support models that update as new rows arrive. This parallels the continual learning goals of tabular foundation models, even though the methods differ.

Where it does not fit

• Static mixed-type tables. For tables with categories, skewed numbers and modest size, XGBoost, random forests or TabPFN are far better starting points.
• In-context learning. Predicting from labelled example rows placed in the input is a property of attention in transformers. Spiking networks lack it.
• Column semantics. Using names and descriptions of columns requires text understanding.
• Sector and semantic classification. Deciding the domain of a table from its headers and values is a language task.
• Synthetic pre-training. Training one network on millions of artificial tables is a feature of the foundation model, not of the spiking framework.

A realistic role

SpikingJelly is best seen as a tool for building low-power detectors and scorers that live near data sources. It can turn a stream of sensor rows into spikes, learn a compact anomaly score, and pass the result on to a larger tabular model that does the heavy analysis. For how tabular models are measured, see /model/benchmarks.

Practical guidance and best practices

Start in single-step mode, then move to multi-step

Single-step mode is easier to debug because you control the loop and can print states.

Once the model works, switch to multi-step mode for speed.

Check that outputs match between the two modes on a small batch.

Choose time steps wisely

• For direct training, 4 to 16 steps is a common starting range.
• For conversion, expect to need many more.
• More steps improve accuracy up to a point and increase memory and time linearly.

Keep an eye on firing rates

Log the mean firing rate of every spiking layer.

A rate of zero means a dead layer with no gradient.

A rate near one means saturation and little information.

Adjust initialization, thresholds or normalization to bring rates into a moderate range.

Use normalization

Batch normalization before the spiking node keeps the input distribution stable and speeds up training. Apply it over the channel dimension and fold time into the batch in multi-step mode.

Pick the surrogate with a quick test

Run short trainings with two or three surrogate shapes and steepness values. Keep the one with the smoothest loss curve.

Reset every batch

for x, y in loader:
    out = net(x)
    loss = loss_fn(out, y)
    opt.zero_grad()
    loss.backward()
    opt.step()
    functional.reset_net(net)   # never skip this line

Record configuration

run: sj-lif-baseline
dataset: event-gesture
frames_number: 16
neuron: lif
tau: 2.0
surrogate: atan
step_mode: multi
backend: torch
optimizer: adam
lr: 0.001
seed: 7

Compare with a dense baseline

Train a conventional network of equal size on the same input frames. Report accuracy, spikes per sample and training time for both. This makes claims about efficiency honest.

Plan deployment early

If you target a chip, use bounded weights, simple neurons and limited precision from the start. Consider quantization-aware training, and test the exported model in a simulator such as Lava before relying on it.

Profiling a training run

If training is slow, check these in order:

1. Is multi-step mode on for all layers?
2. Is the data loader the bottleneck? Try more workers or cached frames.
3. Is the number of time steps larger than needed?
4. Is mixed precision enabled where safe?
5. Is the compiled backend available and in use?

Measure time per epoch after each change, and keep notes.

A model card for your run

Write a short record next to every trained model.

• Task and dataset, with the split used.
• Encoding and number of steps.
• Architecture and neuron settings.
• Accuracy and spikes per sample.
• Training time and hardware.
• Known limits, such as sensitivity to input scale.

This makes comparisons across projects fair and saves time months later.

When to stop tuning

Set a budget. If three well-chosen changes do not move the metric, the limit is probably the data or the task, not the settings. Move on to a different question.

Common mistakes

• Not resetting state. The most frequent error. Membrane values leak between samples.
• Wrong tensor layout. Multi-step mode expects time first. A swapped axis trains silently on nonsense.
• Mixing step modes. Some layers in single-step mode and others in multi-step mode produce shape errors.
• Too few frames. Event data binned into very few frames loses timing information.
• Too many frames. Many frames slow training and use a lot of memory with little gain.
• Ignoring input scale. Unnormalized inputs produce silent or saturated neurons.
• Judging by wall-clock on a GPU. Spiking models can be slower than dense ones on a GPU. Efficiency is a hardware question.
• Using it for plain tables. For ordinary tabular prediction start with scikit-learn pipelines and boosting.
• Treating missing values as zeros. This changes the meaning of the data under most encoders.
• Forgetting version pins. Module paths and defaults have changed across releases. Pin the version.

Frequently asked questions

What is SpikingJelly used for?

SpikingJelly is used to build, train and benchmark spiking neural networks. It is especially common for work on event-based data from dynamic vision sensors, and for research on neuron models, surrogate gradients and conversion from conventional networks.

What is multi-step mode?

Multi-step mode lets a layer process all time steps in one call instead of one step at a time. Stateless layers such as convolutions run on all steps together, and only the spiking nodes loop over time. This makes training noticeably faster than a hand-written time loop.

How does it differ from snnTorch?

Both are PyTorch based. SpikingJelly is a larger platform with datasets, many neuron types, conversion tools and optional compiled kernels. snnTorch is lighter and focuses on a simple API and teaching material. Researchers often pick based on whether they need breadth or simplicity.

Can SpikingJelly convert a regular network into a spiking one?

Yes. It includes conversion routines that replace ReLU activations with spiking nodes after normalizing activations. The result approximates the original network when run for enough time steps, though it can need many steps and may lose some accuracy.

Do I need a GPU?

A GPU is strongly recommended for anything beyond small experiments, because spiking models loop over time and need more compute than a conventional network of the same size. Small examples run fine on a CPU.

Does it save energy?

Not on a GPU. Sparse spiking activity can save energy on neuromorphic hardware that skips work for silent units. On a standard GPU, the extra time steps usually cost more than a conventional network would.

Can it handle tabular data?

You can feed a table row as a vector, using direct, Poisson or latency encoding. This is mainly a research exercise. For ordinary tabular prediction, gradient boosting or a tabular foundation model is usually simpler and stronger.

Is SpikingJelly related to tabular foundation models?

Only loosely. Tabular foundation models are pre-trained on many tables and use in-context learning on new ones. SpikingJelly could help prototype sparse anomaly detectors or adaptive scorers beside such a model, but it does not provide in-context learning or column understanding.

How should I handle missing values?

The library has no built-in handling. Silence can represent a missing cell, but a true zero under rate coding also looks silent. The usual fix is to add a binary mask input that marks which cells were observed.

Can I deploy a trained model to neuromorphic hardware?

Not directly. You would export the weights and neuron parameters, quantize them, and map them with a hardware toolchain or a framework such as Lava. Expect some change in accuracy and plan to validate in simulation first.

Glossary

• ANN to SNN conversion: turning a trained conventional network into a spiking network by replacing activations with spiking nodes.
• Arctangent surrogate: a smooth curve shaped like the derivative of arctangent, used in the backward pass.
• Charge, fire, reset: the three phases of a spiking neuron update.
• DVS: dynamic vision sensor, a camera that outputs events when brightness changes.
• Direct encoding: feeding analog values into the first layer repeatedly over time.
• Event data: a stream of changes with position, time and polarity.
• Frame integration: accumulating events into frames over time windows.
• Firing rate: the average number of spikes per neuron per step.
• LIF node: a leaky integrate-and-fire neuron layer.
• Multi-step mode: processing all time steps in one layer call.
• Poisson encoding: generating spikes at random with probability set by a value.
• Reset net: a helper that clears all neuron states.
• Single-step mode: processing one time step per layer call.
• Surrogate function: a smooth stand-in for the spike derivative.
• Time constant: the speed at which the membrane decays.

Summary

SpikingJelly is a PyTorch based platform for spiking neural networks. It combines neuron nodes, surrogate gradients, encoders, event dataset loaders, conversion tools and fast multi-step kernels so that you can train and benchmark spiking models on event data with a single package.

For tabular foundation models its role is supportive. It can encode rows as spikes, handle missing cells through silence plus a mask, bin time-stamped logs into frames, and prototype sparse anomaly detectors that sit next to a larger model. It does not supply in-context learning, column understanding or synthetic table pre-training, which are features of models such as TabPFN. To go further, read Spiking Neural Networks, compare snnTorch, Norse and Lava, and visit /model/research.

Related

It belongs to Neuromorphic in my stack, under AI and machine learning. Others in the same group: