Microservices and queues
Lesson 3 of 3
3Microservices and queues
The big pictureMicroservices trade code complexity for network complexity: every call can be slow or fail. Queues decouple: a service sends the message and moves on, and the other processes it when it can.
What the picture shows: Asynchronous communication: the sender does not wait, and the message stays in the queue until it is processed successfully.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
A small restaurant: the cook takes the order, cooks and washes the dishes. As it grows you separate the kitchen from washing, with a shelf between them: the cook puts dishes on the shelf and carries on, and the washer takes from it when free. The shelf is the queue. But do not put each person in their own room on day one; coordinating between rooms is costly.
A microservice owns its data and deploys independently, with boundaries following business domains (bounded contexts), not technical layers. The cost: an unreliable network, distributed tracing, eventual consistency and harder deploys. Synchronous calls (HTTP/gRPC) need timeouts, retries with exponential backoff and a circuit breaker. Asynchronous messaging (RabbitMQ, SQS, Kafka) usually guarantees "at least once", so the consumer must be idempotent (a unique key or a processed-messages table). Messages that keep failing go to a dead-letter queue (DLQ). To avoid writing to the database without publishing the event: the outbox pattern.
Before You Start
⚖️Scaling
The scaling and load-balancing lesson.
⏳Async programming
The web-async track.
The Core Idea
1A modular monolith first
One app deployed once, but split inside into modules with clear boundaries (orders, payments, notifications) that never touch each other's tables directly. When you truly need to split a module out, its boundaries are ready.
2A queue with retries
The consumer processes; on failure the message goes back to the queue up to a limit, then to the dead-letter queue:
from collections import deque
queue = deque([{"id": "m1", "order": 1}, {"id": "m2", "order": 2}, {"id": "m3", "order": 3}])
dead_letters, attempts = [], {}
flaky = {"m2": 1} # m2 fails once, then works
broken = {"m3"} # m3 always fails
MAX_ATTEMPTS = 3
def handle(msg):
if msg["id"] in broken:
raise RuntimeError("bad payload")
if flaky.get(msg["id"], 0) > 0:
flaky[msg["id"]] -= 1
raise TimeoutError("email service slow")
print("sent receipt for order", msg["order"])
while queue:
msg = queue.popleft()
attempts[msg["id"]] = attempts.get(msg["id"], 0) + 1
try:
handle(msg)
except Exception as e:
if attempts[msg["id"]] >= MAX_ATTEMPTS:
dead_letters.append((msg["id"], str(e)))
else:
queue.append(msg)
print("dead letters:", dead_letters)3Idempotency: the same message twice
A queue guarantees "at least once": if the consumer dies after processing but before acknowledging, the message arrives again. If it says "charge 50 riyals", it gets charged twice. The fix: record the id of every processed message and skip repeats:
balance = {"sara": 100}
processed = set()
def charge(msg):
if msg["id"] in processed:
print("duplicate", msg["id"], "skipped")
return
balance[msg["user"]] -= msg["amount"]
processed.add(msg["id"])
payment = {"id": "pay-7f3", "user": "sara", "amount": 50}
charge(payment)
charge(payment) # redelivered after a crash
print("balance:", balance["sara"])Worked example
Problem: checkout calls the email service directly, email is sometimes 8 seconds slow, so the user sees a spinner and sometimes an error even though payment succeeded. Fix: the order is saved, publishes an order.paid event to the queue and replies at once. The email consumer processes the event with retries, using the order id as an idempotency key so two emails never go out. Result: checkout in 200ms, and the email arrives even if late.
Myth vs Reality
| Myth | Reality |
|---|---|
| Microservices are faster. | Every network call is slower than a function call. Their benefit is independent teams, deploys and scaling. |
| A queue guarantees exactly-once delivery. | Most are "at least once", and the responsibility lies with the consumer. |
Where you meet it at work
📧Background jobs
Email, reports and images outside the request path.
📦Events between teams
The inventory team listens to order events with no direct call.
Test your understanding
When do you split the system into services?
Splitting solves organisational and scaling problems, at a cost in complexity.
Why must a consumer be idempotent?
"At least once" means duplicates are possible.
A message always fails. Where does it go?
The DLQ keeps it for inspection without blocking the queue.
Quick review cards
Check Your Understanding
What is a "distributed monolith"?
Services split on paper that share a database and deploy together, combining the downsides of both.
What do you add to a synchronous call between services?
A timeout, retries with exponential backoff, and a circuit breaker.
What is wrong with saving, then publishing, in two steps?
If it crashes between them, data is saved and no event is published. The outbox pattern fixes it.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Done with this lesson?
Sign in to save your progress.