# Circuit breaker pattern

A circuit breaker stops calls to a failing service for a while, returning fast errors or fallbacks, so one failure does not take down the whole system.

## Overview

The circuit breaker is a resilience pattern for calls between software components. It wraps a call to a dependency, such as another service, a database or a model API, and watches for failures. When failures pass a threshold, the breaker "opens" and stops sending calls for a while, returning an immediate error or a fallback instead. After a cooling-off period it lets a few trial calls through. If they succeed, normal traffic resumes; if not, it stays open.

The name comes from the electrical circuit breaker in a house. When a faulty appliance draws too much current, the breaker trips and cuts power to that circuit. That protects the wiring and the rest of the house from a fire. You fix the fault, then reset the switch. The software version protects a system in the same way: one failing dependency should not drag down every caller with it.

Without a breaker, a slow or broken dependency causes callers to wait on timeouts, threads and connections pile up, queues fill and the caller fails too. That failure then spreads upstream, a cascading failure. In [microservices](/tools/microservices) and AI systems, where one request may fan out to a retrieval service, a reranker and an external model provider, circuit breakers are a basic safety mechanism alongside timeouts, retries and rate limits.

## What it is

A circuit breaker is a small state machine around a call.

| State | Behaviour | Moves to |
|---|---|---|
| Closed | Calls pass through normally; results are recorded in a sliding window | Open, when the failure rate or slow-call rate exceeds a threshold |
| Open | Calls fail fast without contacting the dependency; a fallback may run | Half-open, after a wait duration |
| Half-open | A limited number of trial calls are allowed through | Closed if trials succeed; Open if they fail |

Typical settings include the failure-rate threshold (for example 50%), the minimum number of calls before the rate is evaluated (for example 20), the window type (last N calls or last N seconds), the slow-call threshold (for example calls over 2 seconds count as slow), the wait duration in open state (for example 30 seconds) and the number of trial calls in half-open state.

A breaker usually works together with other patterns: a timeout on each call, retries with backoff for transient errors, a bulkhead to cap concurrent calls, and a fallback such as a cached answer or a degraded feature.

## Where it came from

Michael Nygard described the circuit breaker as a stability pattern in his book "Release It! Design and Deploy Production-Ready Software", published in 2007, with a second edition in 2018. The book collected patterns from real production failures, including cascading failures caused by integration points that hang.

Martin Fowler's short 2014 article helped spread the pattern among developers. Netflix released Hystrix, a Java latency and fault-tolerance library with circuit breakers and bulkheads, as open source in 2012, and it became widely used. In 2018 Netflix put Hystrix into maintenance mode and pointed users towards alternatives such as Resilience4j. Today breakers are available in many libraries, such as Resilience4j for Java and Polly for .NET, and in proxies and service meshes such as Envoy and Istio, which offer outlier detection and connection limits at the network layer.

## How it works

### 1. Wrap the call

Code calls the dependency through the breaker rather than directly. The breaker is usually per dependency, sometimes per endpoint, so that one bad endpoint does not block healthy ones.

### 2. Record outcomes

In closed state, each call is classified as success, failure or slow. Exceptions that indicate a client error, such as an invalid request, are often excluded, because they say nothing about the dependency's health.

### 3. Evaluate the window

Once the minimum number of calls is reached, the breaker computes the failure and slow-call rates over the window. If either exceeds its threshold, it trips to open.

### 4. Fail fast

In open state, calls return immediately with an error such as "circuit open", or run a fallback. No threads wait and the struggling dependency gets breathing room to recover.

### 5. Probe

After the wait duration, the breaker moves to half-open and allows a few trial calls. Their results decide the next state.

### 6. Report

State changes and rejected calls are exported as metrics and events, so dashboards show which breakers are open and alerts can fire when one stays open.

A short example using Resilience4j's Java API:

```java
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .failureRateThreshold(50)
    .slowCallDurationThreshold(Duration.ofSeconds(2))
    .waitDurationInOpenState(Duration.ofSeconds(30))
    .minimumNumberOfCalls(20)
    .build();
CircuitBreaker breaker = CircuitBreaker.of("reranker", config);
Supplier<List<Doc>> guarded =
    CircuitBreaker.decorateSupplier(breaker, () -> rerankerClient.rerank(query));
```

## Key concepts and terms

| Term | Meaning |
|---|---|
| Closed | Normal state; calls flow and are measured. |
| Open | Tripped state; calls are rejected immediately. |
| Half-open | Trial state; a few calls test whether the dependency recovered. |
| Failure-rate threshold | Percentage of failed calls that trips the breaker. |
| Slow-call threshold | Duration above which a call counts as slow. |
| Sliding window | The recent calls or time period used to compute rates. |
| Fallback | Alternative response used when the call fails or is rejected. |
| Fail fast | Returning an error at once instead of waiting. |
| Cascading failure | A failure spreading from one component to its callers. |
| Bulkhead | Limiting concurrent calls so one dependency cannot use all resources. |
| Outlier detection | Proxy feature that ejects unhealthy instances from a load-balancing pool. |
| Exponential backoff | Increasing delays between retries. |

## A worked example

This scenario is illustrative. A shopping assistant answers product questions. Each request calls a retrieval service, then a reranker service, then an external language model. Normal latencies are 80 ms, 120 ms and 1.5 seconds. The web tier has 200 worker threads.

One evening the reranker's GPU node degrades and responses take 20 seconds, the client timeout. Within a minute all 200 threads are waiting on the reranker. The assistant stops answering entirely, and the product search page, which shares the same workers, fails too.

After the incident, the team adds three things. First, a timeout of 800 ms on reranker calls. Second, a circuit breaker per dependency: it opens when 50% of the last 20 calls fail or exceed 500 ms, stays open for 30 seconds, then allows 3 trial calls. Third, a fallback: when the reranker breaker is open, the assistant uses retrieval order without reranking and logs that answers are degraded.

In the next reranker incident, the breaker opens after about 25 calls. Requests skip the reranker within a millisecond, end-to-end latency goes from 1.7 seconds to 1.6 seconds, and answer quality drops slightly but the assistant stays up. An alert fires because the breaker has been open for 5 minutes. When the node is replaced, the half-open trials succeed and the breaker closes on its own.

## Strengths and limitations

**Strengths:**

- Stops cascading failures by isolating a failing dependency.
- Frees threads and connections quickly instead of waiting on timeouts.
- Gives struggling services time to recover by reducing load.
- Enables graceful degradation through fallbacks.
- Makes dependency health visible through state metrics.

**Limitations:**

- Wrong thresholds cause flapping, or trip too late to help.
- A shared breaker across many instances or endpoints can block healthy paths.
- Fallbacks must be designed and tested; without them users still see errors.
- Each client instance has its own view, so breakers may disagree.
- Does not replace capacity planning, timeouts or fixing root causes.

## How it compares with alternatives

| Pattern | What it does | Relationship to circuit breakers |
|---|---|---|
| Timeout | Limits how long one call can wait | Essential companion; breakers act on timeout results |
| Retry with backoff | Repeats failed calls after delays | Helps with brief glitches; without a breaker, retries add load during outages |
| Bulkhead | Caps concurrent calls per dependency | Limits damage even before a breaker trips |
| Rate limiting | Caps request rate, often on the server side | Protects a service from too much traffic; a breaker protects the caller |
| Service mesh outlier detection | Removes failing instances from load balancing | Network-level protection; complements application breakers |

See [guardrails](/tools/guardrails) and [cost and latency budgets](/tools/cost-and-latency-budgets) for related controls in LLM applications.

## Practical guidance and best practices

1. Always set explicit timeouts; a breaker without timeouts reacts too slowly.
2. Use one breaker per dependency, or per endpoint when endpoints fail independently.
3. Count slow calls as well as errors, since slowness causes most cascades.
4. Exclude client errors, such as validation failures, from failure counts.
5. Define a useful fallback for each critical call: cached data, a simpler model, or a clear degraded message.
6. Combine retries with a breaker and limit the total retry budget.
7. Export breaker states to [Prometheus](/tools/prometheus) and alert when breakers stay open.
8. Test breakers with fault injection, such as adding latency in staging.
9. For LLM providers, use breakers to fail over to a second model or region when one provider degrades.

## Common mistakes

- Retrying aggressively without a breaker, multiplying load on a failing service.
- Setting the minimum call count too low, so two failures in a quiet period trip the breaker.
- Sharing one breaker across unrelated endpoints or tenants.
- Returning fallbacks silently, so nobody notices the system is degraded.
- Never testing the open and half-open paths until a real outage.
- Treating a breaker as a fix rather than a symptom to investigate.

## Frequently asked questions

### What is the difference between a circuit breaker and a retry?

A retry tries the same call again, hoping a transient fault has passed. A circuit breaker stops calling when failures are persistent. They work best together: retry briefly, and let the breaker stop retries during real outages.

### Where should circuit breakers live: in code or in the network?

Both are common. Libraries give fine control and application-level fallbacks. Proxies and service meshes provide outlier detection and connection limits without code changes. Many systems use both layers.

### Is Hystrix still recommended?

No. Netflix moved Hystrix into maintenance mode in 2018 and pointed users to alternatives such as Resilience4j. Use an actively maintained library for your language.

### How do I choose thresholds?

Start from observed behaviour: normal error rates, latency percentiles and traffic volume. Set the slow-call threshold above normal p99 latency, require enough calls to be statistically meaningful, and adjust after load tests and incidents.

### Do circuit breakers help with AI model APIs?

Yes. External model providers can slow down or return errors under load. A breaker with a fallback to a second provider, a smaller model or a cached response keeps an application responsive.

### What happens in half-open state?

The breaker allows a small, fixed number of calls through. If enough succeed, it closes; if they fail, it opens again for another wait period.

## Glossary

- Circuit breaker: a state machine that stops calls to a failing dependency.
- Trip: the transition from closed to open.
- Fallback: an alternative response.
- Cascading failure: failure spreading through dependent components.
- Bulkhead: isolation of resources per dependency.
- Timeout: a maximum wait for a response.
- Backoff: increasing delay between retries.
- Fault injection: deliberately introducing failures to test resilience.
- Graceful degradation: reduced but working functionality during faults.

## Summary

A circuit breaker watches calls to a dependency and, when failures or slow calls cross a threshold, stops sending calls for a time and fails fast, then probes for recovery. Described by Michael Nygard in "Release It!" in 2007 and spread by libraries such as Hystrix and its successors, it is now standard in resilient distributed systems. It works best with timeouts, bounded retries, bulkheads and meaningful fallbacks, with breakers scoped per dependency, thresholds based on real measurements, open states monitored and failure paths tested before real incidents.