Norse

Overview

Norse is an open-source library that brings bio-inspired spiking neural networks to PyTorch. It provides neuron models, spike encoders, decoders, plasticity rules and utilities for event-based data. You use it much like any other PyTorch extension: define modules, run a forward pass, compute a loss and call backward.

What sets Norse apart is its design. Neurons are written as functions that take an input and a state and return an output and a new state. The state is explicit and is passed around by the caller. This functional style makes the code easy to test, easy to compose with other transformations and close to the equations found in neuroscience papers. Norse is aimed at research on spiking models and neuromorphic learning rules, and it sits beside snnTorch, SpikingJelly and the deployment framework Lava.

What it is

Norse is a Python package that extends PyTorch with the building blocks of spiking neural networks.

The main parts

• Neuron models. Leaky integrate-and-fire neurons in several flavors, plus models that capture adaptation, bursting and other behaviors seen in biology.
• Cells and modules. Each neuron model comes as a low level cell that takes one step at a time, and as a module that loops over a whole sequence.
• Encoders. Tools that turn real values or images into spike trains.
• Decoders. Tools that turn spike outputs back into values, such as the class with the highest activity.
• Plasticity rules. Learning rules based on spike timing, which can be used alongside or instead of gradient descent.
• Functional building blocks. Plain functions for the neuron equations, the threshold and the surrogate gradient.
• Dataset helpers. Support for event-based data and examples that use it.

The functional design

Many libraries hide the hidden state inside the layer object. Norse makes it visible.

A step looks like this: you give the neuron an input and its current state. You get back spikes and a new state.

Analogy: a normal layer is like a calculator with a memory button that you cannot see. A Norse neuron is like a calculator that shows the memory value on screen and asks you to type it back in on the next step. It is a little more work, and it is much easier to inspect.

Why this matters for research

• You can swap a neuron model without touching the rest of the network.
• You can copy, save or modify states between steps.
• You can test a neuron with a handmade state.
• You can apply PyTorch transformations to the functions with few surprises.

Where it came from

Spiking neural networks come from neuroscience. The leaky integrate-and-fire neuron is a simple model of how a nerve cell collects input, leaks charge and fires. Richer models add adaptation, conductances and bursting.

Early machine learning with spikes relied on local rules, such as spike-timing-dependent plasticity, or on converting trained conventional networks. In the 2010s, surrogate gradient methods made direct training with backpropagation through time practical. Deep learning libraries meanwhile made automatic differentiation common.

Norse was created in the early 2020s by researchers working at the border of neuroscience and machine learning. The goal was a library that treats spiking neurons as first class citizens in PyTorch, with clear equations and a clean functional core. A paper describing the library presented it as a deep learning toolkit for spiking networks and emphasized its composable design.

Other libraries made different choices. snnTorch focuses on a simple layer API and teaching material. SpikingJelly bundles datasets and fast kernels. Lava targets deployment on Loihi and similar hardware. Together they form the working toolbox of the spiking neural networks community. For wider research context see /model/research.

How it works

This section follows the life of one spiking layer in Norse.

Step 1: Describe the neuron

A neuron model is defined by a set of parameters and a state.

For a leaky integrate-and-fire neuron the state holds the membrane voltage and the synaptic current. The parameters include time constants, the threshold and the reset voltage.

Parameters are bundled in a small object so that you can reuse and vary them easily.

Step 2: Encode the data

Numbers must become spikes. Norse offers a few routes.

• Constant current encoding. Each value is fed as a steady current into an input neuron. A larger value makes it fire faster.
• Poisson encoding. Each value sets the probability of a spike at each step. This gives a noisy spike train.
• Latency encoding. A larger value gives an earlier spike.

Analogy: constant current is like holding a faucet open at a set rate. Poisson encoding is like rain, where the chance of a drop in any second depends on the weather. Latency encoding is like a starter pistol with a delay that depends on the runner.

Step 3: Integrate over time

For each time step the neuron does the following.

1. The synaptic current decays and receives new input.
2. The membrane voltage moves toward the current, with a leak.
3. If the voltage passes the threshold, the neuron emits a spike.
4. The voltage is reset, and the new state is returned.

The step is a small function of the old state, the input and the parameters. The same function runs in a loop over time.

Step 4: Stack layers

You can stack neurons with ordinary linear layers in between. Norse offers a sequential container that passes states along automatically, so that you do not have to track every state by hand.

Step 5: Decode the output

For classification, the last layer has one neuron per class. A decoder turns the spike trains into a decision. Common choices:

• the class with the largest total spike count,
• the class with the largest maximum membrane voltage,
• the class that fires first.

Step 6: Learn

There are two broad ways to learn.

Gradient-based learning. The hard spike is replaced in the backward pass by a smooth surrogate. Backpropagation through time then updates the weights.

Local plasticity. Weights change based on the timing of pre and post spikes. This does not need a global loss, which is attractive for on-chip learning.

You can also mix them, for example by training with gradients and then adapting with a local rule.

Surrogate gradients in Norse

The threshold is implemented as a function with a custom backward pass. In the forward pass it outputs 0 or 1. In the backward pass it returns a smooth curve, such as a super-spike shape or a sigmoid derivative.

The surrogate has a steepness parameter. A higher value gives a sharper, more local learning signal. A lower value gives a smoother one that reaches farther from the threshold.

Plasticity rules in more detail

Spike-timing-dependent plasticity changes a weight based on the order of two spikes.

• If the sending neuron fires shortly before the receiving neuron, the weight goes up.
• If it fires shortly after, the weight goes down.
• The size of the change shrinks as the gap in time grows.

Analogy: two colleagues who often speak in a helpful order learn to trust each other. If one keeps speaking after the decision is made, the link weakens.

Implementations keep short-lived traces of recent spikes at each neuron. The weight update then needs only the traces, not a stored history.

This is local, cheap and well suited to chips. Its weakness is that it has no direct notion of a task goal, so it is often combined with a gradient signal or a reward.

Time stepping and units

Norse neurons are written in physical-like units, such as milliseconds for time constants.

The step size dt converts between simulated time and steps.

A small dt such as 1 ms gives smooth dynamics and needs many steps per second of data.

A large dt is faster but can make the update unstable if it nears the time constant.

A safe rule is to keep dt several times smaller than the shortest time constant.

Batching and shapes

Norse follows the PyTorch habit of keeping a batch dimension.

For sequences the common layout is time first, then batch, then features.

A module that loops over time expects this layout and returns outputs with the same leading dimension.

When something fails, print the shapes first. Most early errors come from a swapped time and batch axis.

Beyond leaky neurons

Norse includes models that behave in richer ways.

Key concepts and terms

A worked example

A small sequence classifier

This sketch shows the usual pattern: an encoder, a linear layer, spiking neurons and a decoder. Check the current documentation for exact names and arguments, since the API evolves.

import torch
import torch.nn as nn
import norse.torch as snn

class Classifier(nn.Module):
    def __init__(self, n_in, n_hidden, n_out, seq_length=32):
        super().__init__()
        self.seq_length = seq_length
        self.encoder = snn.ConstantCurrentLIFEncoder(seq_length)
        self.fc1 = nn.Linear(n_in, n_hidden)
        self.lif1 = snn.LIF()
        self.fc2 = nn.Linear(n_hidden, n_out)
        self.readout = snn.LILinearCell(n_hidden, n_out)

    def forward(self, x):
        spikes = self.encoder(x)           # time x batch x features
        s1 = None
        v = None
        outputs = []
        for t in range(self.seq_length):
            z = self.fc1(spikes[t])
            z, s1 = self.lif1(z, s1)
            vout, v = self.readout(z, v)
            outputs.append(vout)
        return torch.stack(outputs).max(dim=0).values

model = Classifier(20, 64, 3)
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
loss_fn = nn.CrossEntropyLoss()

x = torch.rand(16, 20)
y = torch.randint(0, 3, (16,))
for step in range(5):
    logits = model(x)
    loss = loss_fn(logits, y)
    opt.zero_grad()
    loss.backward()
    opt.step()
    print(step, float(loss))

Reading the code

• The encoder turns each feature into a spike train using a constant current into a neuron.
• The LIF module takes a state input, here s1, and returns a new one. The first call uses None to start from the default state.
• The read-out is a leaky integrator with no spike. It tracks a smooth voltage per class.
• The decision uses the maximum voltage over time, then the usual cross entropy loss.

A numeric walk-through of leaky integration

Take a read-out neuron with decay 0.8 and input 1.0 for three steps, then 0.

The voltage rises while input is present and decays after it stops. The maximum over time (2.44) is what the decoder reads. This is how an output neuron summarizes a whole sequence.

A functional step by hand

The functional style lets you write one step explicitly.

import torch

def lif_step(v, i, x, dt=0.001, tau_mem=0.02, tau_syn=0.005, v_th=1.0):
    # synaptic current decays and receives input
    i = i - dt / tau_syn * i + x
    # membrane moves toward current
    v = v + dt / tau_mem * (i - v)
    z = (v > v_th).float()
    v = v * (1 - z)      # reset to zero
    return z, v, i

v = torch.zeros(4)
i = torch.zeros(4)
for t in range(100):
    z, v, i = lif_step(v, i, x=torch.full((4,), 2.0))

This tiny function is the heart of a LIF neuron. A library version adds a surrogate gradient on the threshold and handles batches and parameters.

Extending the example to a recurrent layer

A recurrent spiking layer feeds its own spikes back as input at the next step.

This gives the network a longer memory than a single leak can offer.

class RecurrentNet(nn.Module):
    def __init__(self, n_in, n_hidden, n_out):
        super().__init__()
        self.rec = snn.LIFRecurrent(n_in, n_hidden)
        self.out = nn.Linear(n_hidden, n_out)

    def forward(self, x):
        # x: time x batch x features
        z, _ = self.rec(x)          # module loops over time
        return self.out(z.mean(dim=0))

Here a recurrent module handles the time loop itself, and the output layer reads the average spike activity over time.

This shorter form is handy when you do not need to touch the step by step state.

Checking the result

After a few epochs, compare three numbers:

1. training loss,
2. validation accuracy,
3. mean spike rate of the hidden layer.

If the loss falls and the spike rate is neither zero nor near one, the model is learning in a healthy way.

Strengths and limitations

Strengths

• Clean functional core. Easy to read, test and extend.
• Rich neuron zoo. Many models beyond the plain LIF neuron.
• Bio-inspired learning. Plasticity rules are available for research.
• Works with PyTorch tooling. Optimizers, data loaders and GPUs behave as usual.
• Good for research prototypes. Explicit states make it easy to try unusual ideas.
• Event data support. Examples cover event-based datasets.

Limitations

• Smaller community than mainstream libraries. Fewer tutorials and answers.
• Moving API. Names and defaults have changed across versions.
• Speed. Time loops are costly on a GPU. Spiking models can be slower than conventional ones.
• Explicit state is more verbose. Beginners may find it harder than layer-only libraries.
• No direct deployment path. Moving a trained model to hardware needs other tools.
• Energy gains are hardware dependent. Sparse activity only saves energy on suitable chips.

How it compares with alternatives

If you like pure functions and want to modify the neuron equations, Norse is a natural fit. If you want the shortest path to a trained classifier, snnTorch may feel easier.

Role in tabular foundation models

Tabular foundation models learn from raw tables. They read rows and columns, handle mixed types such as numbers, categories and dates, tolerate missing values and often cannot rely on informative column names. Many are pre-trained on large collections of synthetic or real tables so that they can predict on a new table through in-context learning. See TabPFN and FT-Transformer, and /model/architecture for how such a model is organized.

Norse is not a tabular model. It is a research library for a different model family. The links are real but modest, and it helps to say clearly where they stop.

Where Norse touches tabular work

• Row encoding. A row is a vector. Norse encoders can turn it into spike trains by constant current, Poisson sampling or latency. This mirrors the embedding step in tabular transformers, where each cell is mapped into a vector. In both cases the choice of encoding sets what the model can learn.
• Missing values. Silence on an input line can mean "not observed". Because Poisson and current encoders also map zero to silence, a separate mask input is needed to tell the cases apart. For model-driven filling of gaps, see imputation networks.
• Anomaly detection. Adaptive neurons respond strongly to sudden change and settle on steady input. That behavior suits detectors for streams of rows or windows of sensor tables. See Anomaly Detection.
• Continual learning. Local plasticity rules update weights from recent activity alone, without replaying old data. This is related in spirit to the aims of continual learning for tabular models, where new rows arrive and old knowledge must survive.
• Temporal tables. Tables with a time column, such as logs from machines, have the sequence structure that recurrent spiking networks can use.

Where it does not fit

• Static, mixed-type tables. A table with skewed numbers, categories and a few thousand rows is handled better by XGBoost, random forests or a pre-trained model such as TabPFN.
• In-context learning. The ability to read labelled example rows in the input and predict new rows without weight updates comes from attention in transformers. A spiking network has no such mechanism built in.
• Column meaning. Using a header like "postcode" or "invoice total" to guide the model requires text understanding. Norse offers none.
• Sector and semantic classification. Deciding what industry a table belongs to from headers and values is a language task.
• Pre-training on synthetic tables. The recipe of sampling millions of artificial tables from random causal models and training a single network on them is a property of the tabular foundation model, not of Norse.

A realistic role

Norse can serve as a laboratory for the low-power side of a tabular pipeline. For example, it can prototype a detector that flags unusual sensor rows with sparse computation, or test a plasticity rule that adapts a small scorer as new data arrives. The result, a flag or a compact score, can then feed a larger table model. See /model/benchmarks for how tabular models are measured.

Practical guidance and best practices

Choose the neuron for the task

• Start with plain LIF neurons for classification.
• Add adaptation when you need longer memory, or when input has slow drifts.
• Use a non-spiking leaky integrator for the read-out layer. It gives a smooth signal for the loss.

Keep time constants sensible

The ratio between the time step and the time constants decides behavior. If the membrane time constant is much shorter than the step, the neuron forgets at once. If it is much longer, it never forgets.

Match the constants to the timescale of your data.

Manage state carefully

• Start each sequence from a fresh state.
• Detach states across truncated segments when you use truncated backpropagation.
• Never reuse a state across unrelated samples.

Tune the surrogate

Try a couple of steepness values, such as a gentle one and a sharp one, and compare the loss curves. Stay with the one that trains stably.

Monitor activity

Record the average spike rate for each layer.

A dead layer has no spikes. A saturated layer fires at every step.

Both are signs that weights or thresholds need adjustment.

Encode with care

# constant current: deterministic, fast, good first try
encoder = snn.ConstantCurrentLIFEncoder(seq_length=32)

# Poisson: noisy but closer to biology, needs more steps
poisson = snn.PoissonEncoder(seq_length=32, f_max=100)

Normalize features to a bounded range before encoding, such as 0 to 1.

Log configuration

run: norse-lif-baseline
neuron: lif
tau_mem_ms: 20
tau_syn_ms: 5
threshold: 1.0
surrogate: superspike
encoder: constant_current
seq_length: 32
optimizer: adam
lr: 0.001
seed: 7

Keep a dense baseline

Train a conventional network of similar size on the same data. If it matches or beats the spiking model with less cost, you have learned something useful about the task.

A short experiment plan

1. Reproduce a published small result with fixed seeds.
2. Replace the neuron model and note the change in accuracy and spike rate.
3. Replace the encoder and repeat.
4. Vary the surrogate steepness over three values.
5. Add a plasticity rule only after the gradient baseline works.
6. Write down what changed and what did not.

Changing one thing at a time keeps results readable.

Reading papers with the library in hand

Neuron papers give equations with time constants and thresholds.

Map each symbol to a parameter in the library.

Run the neuron on a constant input and compare its voltage trace with the figure in the paper.

If they match, you can trust the setup for larger experiments.

Testing

Write small numeric tests.

• A neuron with no input must stay at rest.
• A neuron with strong constant input must fire at a steady rate.
• A reset must bring voltage below the threshold.
• Two identical runs with the same seed must give identical spikes.

These checks take seconds and catch many regressions after a library update.

Common mistakes

• Dropping the state. Passing None at every step resets the neuron each time and removes all memory.
• Carrying state across samples. The leftover voltage from one sample biases the next.
• Unscaled inputs. Large inputs saturate neurons. Tiny inputs never trigger a spike.
• Mismatched time constants. Constants chosen without regard to the time step produce odd dynamics.
• Wrong read-out. Using spike counts where a smooth voltage would train better.
• Expecting speed. Spiking loops are not fast on GPUs. Do not judge the idea by wall-clock time on a laptop.
• Assuming energy savings. Savings depend on hardware that skips silent units.
• Using it for plain tables. For ordinary tabular prediction, start with scikit-learn baselines and boosting.
• Treating missing as zero. Under most encoders this changes the meaning of the data.
• Skipping version pins. The API has moved. Pin the library version for repeatable results.

Frequently asked questions

What is Norse used for?

Norse is used to build, train and study spiking neural networks inside PyTorch. Researchers use it to test neuron models, encoders and learning rules, and to prepare models for event-based and neuromorphic applications.

Why does Norse pass state explicitly?

Explicit state makes each neuron step a pure function of its input, parameters and previous state. That makes the code easy to read, test and compose, and it matches the equations in neuroscience papers. The cost is a little extra bookkeeping.

Is Norse different from snnTorch?

Both are PyTorch libraries for spiking networks. Norse leans on a functional style with explicit state and a wide set of neuron and plasticity models. snnTorch offers a simple layer API and extensive tutorials. Many researchers try both and keep what suits their workflow.

Can Norse train with backpropagation?

Yes. It uses surrogate gradients, which replace the derivative of the hard spike in the backward pass, so backpropagation through time works. It also provides plasticity rules for local learning when a global loss is not wanted.

Does Norse run on a GPU?

Yes, because it is built on PyTorch tensors. Spiking models loop over time steps, so the speed advantage over a CPU is smaller than for conventional networks, and memory use grows with sequence length.

Can it handle tabular data?

It can accept a table row as a vector and encode it into spikes, but this is mainly a research exercise. For standard tabular prediction, gradient boosting or a tabular foundation model is usually simpler and more accurate.

How does it treat missing values?

It has no special treatment. Silence can stand for a missing value, but under common encoders a zero also produces silence. Adding a mask input that marks observed cells is the usual remedy.

Is it related to tabular foundation models?

Only loosely. Tabular foundation models are pre-trained on many tables and use in-context learning on new ones. Norse could prototype low-power anomaly detectors or adaptive scorers that sit next to such a model, but it does not provide in-context learning or column understanding.

Can I deploy a Norse model on neuromorphic hardware?

Not directly. You would export weights and parameters, check how they behave under quantization, and use a deployment framework such as Lava or a hardware toolchain. Expect to adjust the model to meet chip limits.

Where should a beginner start?

Learn the leaky integrate-and-fire neuron on paper first. Then run a single Norse neuron with a constant input and plot its voltage and spikes. After that, build a small classifier and compare it with a plain network of the same size.

Glossary

• Adaptive neuron: a neuron whose threshold or current changes with its own activity.
• Cell: a module that advances a neuron by one time step.
• Decoder: a function that converts spikes into a value or class.
• Encoder: a function that converts values into spikes.
• Functional style: programming with functions that take and return all state explicitly.
• Leaky integrator: a unit that sums input and decays over time without spiking.
• LIF: leaky integrate-and-fire, a basic spiking neuron model.
• Membrane time constant: the time scale of voltage decay.
• Plasticity: change in connection strength driven by activity.
• Poisson encoding: sampling spikes randomly with a probability set by the value.
• Recurrent layer: a layer that feeds its own output back as input.
• STDP: spike-timing-dependent plasticity.
• Surrogate gradient: a smooth stand-in used in the backward pass of the spike function.
• Synaptic time constant: the time scale of input current decay.
• Truncated BPTT: training over short windows of a long sequence to save memory.

Summary

Norse is a PyTorch library for spiking neural networks with a functional design in which neuron state is explicit. It supplies LIF and richer neuron models, encoders and decoders, surrogate gradients and plasticity rules, and it fits research on bio-inspired learning and event-based data.

For tabular foundation models, its role is supportive and experimental. It can encode rows as spikes, treat silence as a possible missing signal when paired with a mask, and prototype sparse anomaly detectors and adaptive learners for streaming tables. It does not offer in-context learning, column understanding or pre-training on synthetic tables, which belong to transformer style models such as TabPFN. To continue, read Spiking Neural Networks, compare snnTorch and SpikingJelly, and see /model/research.

Related

It belongs to Neuromorphic in my stack, under AI and machine learning. Others in the same group: