# Real-time AI: from event streams to decisions

Real-time AI is AI that acts on data while it is still arriving. Events go in, a model scores them, and a decision comes out in milliseconds to seconds, with the data and often the model kept current. It is a systems problem as much as a modeling problem.

## What real-time means

People use the word for very different latencies, and the design changes with each:

- **Hard real time:** a response in single-digit milliseconds, for example fraud checks on a payment.
- **Interactive:** tens to hundreds of milliseconds, for example a recommendation on a page.
- **Near real time:** seconds to a few minutes, for example a dashboard or an alert.

The first step is to decide which tier a use case needs, because each step toward hard real time costs more in engineering and compute.

## The streaming backbone

[Apache Kafka](https://www.ozercemkelahmet.com/tools/apache-kafka) is a distributed log. Producers append events to partitioned topics, and consumer groups read them independently and at their own pace, which decouples the systems that create data from those that use it. [Kafka Connect](https://www.ozercemkelahmet.com/tools/kafka-connect) moves data in and out of external systems.

[Apache Flink](https://www.ozercemkelahmet.com/tools/apache-flink) processes those streams with state. It supports event-time processing with watermarks, so late and out-of-order events are handled correctly, and it takes consistent checkpoints so a job can recover without losing or duplicating results. [Spark Streaming](https://www.ozercemkelahmet.com/tools/spark-streaming) covers similar ground with micro-batches.

## Fast inference

A model on the hot path needs low and predictable latency. The usual levers are:

- keeping features pre-computed or computed in the stream, so the model call does not wait for a database;
- compressing the model, for example with quantization, to cut memory and time per call. See [compression and serving](https://www.ozercemkelahmet.com/model/compression-and-serving);
- batching requests without breaking the latency budget;
- keeping a simple fallback for when the model is unavailable.

## Staying current

Data drifts. A model trained once will slowly get worse as customers, prices, or sensors change. Real-time systems therefore monitor inputs and outputs, and update models either on a schedule or continuously. Updating on new tasks without forgetting old ones is the subject of [continual learning for tables](https://www.ozercemkelahmet.com/model/continual-learning-for-tables), and the [benchmark story](https://www.ozercemkelahmet.com/model/bench-continual-learning) shows how one model took on new tasks without a full retrain.

## Where tabular models fit

Most event streams are structured: rows with a schema. A [tabular foundation model](https://www.ozercemkelahmet.com/articles/what-is-a-tabular-foundation-model) scores such rows directly, and in-context prediction lets it adapt to a new table without a training run. That makes it a natural fit for streams where tables and schemas change often.

## Frequently asked questions

What is real-time AI?
AI that processes streaming data and returns results with low latency, often with models that keep learning online.

Which tools do you use for streaming?
Apache Kafka for the event log and Apache Flink for stateful stream processing.

Is real-time AI the same as online learning?
No. Real-time describes how fast a result is produced. Online learning describes how a model is updated. A system can have one without the other.