Microservices

Overview

Microservices are an architectural style in which one application is built as a set of small services. Each service does one business job, runs as its own process, owns its own data, and talks to the others over the network through well-defined APIs or messages. Each can be built, deployed and scaled on its own, usually by one team. The opposite style is the monolith, where all features live in one deployable unit sharing one database.

An everyday analogy is a food court compared with a single restaurant kitchen. In one large kitchen, every cook shares the same stoves, fridge and menu; a change to the menu affects everyone, and if the kitchen floods, nothing is served. In a food court, each stall has its own kitchen, staff and stock. The noodle stall can add a dish or hire more cooks without asking the pizza stall. But the food court needs shared seating, payment and cleaning, and a customer ordering from three stalls has to wait for three separate orders. Microservices bring the same freedoms and the same coordination costs.

For AI products the style is common because model serving, feature computation, retrieval, user management and billing have very different scaling and release needs. A GPU-backed inference service may need to scale on its own schedule, while the account service changes rarely. Microservices let those parts evolve separately, at the cost of running a distributed system.

What it is

Microservice architecture is a set of characteristics rather than one technology.

Supporting pieces usually appear around the services: an API gateway at the edge, service discovery, centralised configuration, observability with OpenTelemetry, and sometimes a service mesh that handles mTLS, retries and traffic routing.

Where it came from

The ideas grew from service-oriented architecture, Unix philosophy and the experience of large web companies in the 2000s. The term "microservices" was discussed at a software architects' workshop near Venice in 2011 and adopted by that group in 2012. James Lewis and Martin Fowler published a widely cited article defining the style in March 2014, listing characteristics such as componentisation via services, organisation around business capabilities, decentralised data management and infrastructure automation. Sam Newman's book "Building Microservices" followed in 2015, with a second edition in 2021.

The rise of containers with Docker from 2013 and of Kubernetes from 2014 made it practical to run many small services. In later years many practitioners stressed caution: start with a well-structured monolith and split only where the benefits are clear. Domain-driven design became the usual method for finding service boundaries.

How it works

1. Find boundaries

Teams identify business capabilities and bounded contexts, areas where a model and language are consistent. Each becomes a candidate service. Good boundaries mean most changes touch one service.

2. Give each service its data

A service owns its storage. The order service does not read the payment database directly; it calls the payment API or reacts to payment events. This keeps services independently changeable but means no cross-service joins or simple transactions.

3. Choose communication styles

Synchronous calls (REST or gRPC) suit queries that need an immediate answer. Asynchronous events suit notifications and workflows where the caller does not need to wait. Many systems use both. See event-driven architecture.

4. Handle distributed workflows

Business operations spanning services use patterns such as the saga pattern, with compensating steps instead of database-wide rollbacks. Read-heavy views that combine data from several services may use CQRS.

5. Deploy and operate

Each service has its own pipeline, container image and runtime configuration. Orchestration schedules instances, and tools such as Helm and Argo CD manage releases.

6. Observe and protect

Distributed tracing links calls across services. Each call has a timeout. Retries use backoff. Circuit breakers stop calling failing dependencies. Authentication between services uses tokens or mutual TLS under a zero trust approach.

Key concepts and terms

A worked example

This scenario is illustrative. An online learning platform started as one monolith with a single PostgreSQL database. By year three, 40 engineers in five teams share one release train every two weeks. A bug in the quiz module delays a recommendation-model update. The recommendation workload needs GPUs at peak hours, but scaling it means scaling the whole application.

The architects map the domain and choose to extract two services first, using the strangler fig approach. The recommendation service gets its own feature store and model server on GPU nodes; it subscribes to "lesson completed" events published to Kafka by the monolith. The billing service moves next, with its own database, because payment rules change for legal reasons on a different cadence.

After six months the recommendation team deploys several times a week, independently. GPU nodes scale from 2 to 8 during evening peaks and back down overnight. New costs appear: an on-call rota for Kafka, tracing to debug a slow page that now involves three services, and a saga for course purchase that must refund if enrolment fails. The rest of the monolith stays as it is, now as a modular monolith with clearer internal boundaries, because splitting it further would not pay off.

Strengths and limitations

Strengths:

• Teams can release independently and more often.
• Each part can scale on its own, which helps for uneven loads like GPU inference.
• Failures can be contained to one service when resilience patterns are in place.
• Technology choices can fit each service's job.
• Smaller codebases are easier for a team to understand.

Limitations:

• Distributed systems are harder: network failures, latency, partial outages.
• No simple cross-service transactions; consistency needs sagas and careful design.
• Much more operational work: pipelines, monitoring, security and on-call per service.
• Debugging needs distributed tracing and good logs.
• Wrong boundaries create a distributed monolith, with the costs of both styles.
• Data spread across services complicates reporting and analytics.

How it compares with alternatives

Practical guidance and best practices

1. Start with a modular monolith unless you already have clear boundaries and several teams.
2. Draw boundaries around business capabilities using domain-driven design, not around technical layers.
3. Give each service its own data and forbid direct database access from other services.
4. Prefer asynchronous events for workflows and notifications to reduce temporal coupling.
5. Set timeouts on every network call and add retries with backoff and circuit breakers.
6. Invest in tracing, metrics, logs and automated deployment before splitting many services.
7. Version APIs and keep changes backward compatible; use contract tests between services.
8. Align services with team ownership; one team should own each service end to end.
9. Migrate gradually with the strangler fig pattern rather than a big-bang rewrite.

Common mistakes

• Splitting into many tiny services before the domain is understood.
• Sharing one database across services, which reintroduces tight coupling.
• Chains of synchronous calls, so one slow service slows everything.
• Ignoring observability and then being unable to debug cross-service failures.
• Copying the architecture of very large companies without their scale or staff.
• Requiring coordinated releases of several services for most features.

Frequently asked questions

When should I use microservices?

When several teams need to release independently, when parts of the system have very different scaling needs, and when the domain is understood well enough to draw stable boundaries. For a small team or a new product, a modular monolith is usually a better start.

How small should a microservice be?

There is no line count. A service should cover one coherent business capability that one team can own and change without coordinating with others.

How do services share data?

Through APIs and events, not through each other's databases. A service that needs another's data either asks for it or keeps its own copy updated from events.

What is a distributed monolith?

A system split into services that still depend so tightly on each other that they must change and deploy together. It has the complexity of microservices without the benefits.

Do microservices need Kubernetes?

No, but they need automated deployment and operations. Kubernetes is a common choice because it handles scheduling, scaling and service discovery for many containers.

How do transactions work across services?

Usually with sagas: a sequence of local transactions, each with a compensating action if a later step fails, combined with the outbox pattern for reliable event publishing.

Glossary

• Microservice: a small, independently deployable service owning one business capability.
• Monolith: an application built and deployed as one unit.
• API gateway: an edge component that routes and secures external traffic.
• Saga: a sequence of local transactions with compensations.
• Outbox pattern: writing events to a local table in the same transaction, then publishing them.
• Strangler fig: incremental replacement of a legacy system.
• Service mesh: a layer managing service-to-service networking.
• Contract test: a test that checks a service still meets what its consumers expect.
• Bounded context: a domain boundary within which terms have one meaning.

Summary

Microservices split an application into independently deployable services that each own a business capability and its data, communicating through APIs and events. Named around 2011 and defined in Lewis and Fowler's 2014 article, the style became practical with containers and Kubernetes. It brings independent releases, targeted scaling and team autonomy, but also distributed-systems complexity, consistency challenges and heavy operational needs. Start modular, find boundaries with domain-driven design, keep data private to each service, and invest in observability and resilience before splitting further.

Related

It belongs to Architecture in my stack, under MLOps and AI infrastructure. Others in the same group: