System design · interactive
Message queues: at-least-once delivery
A producer drops work on a queue and moves on. Consumers pull it, do it, and delete it. Make the consumers slow, or send a message that always fails, and watch what “at least once” really means.
Producer
1 of 8 sent
Queue
0 visible
Consumer 1
working on m1
Consumer 2
idle
Dead-letter queue
0 parked
m1
in-flight
try 1
m2
not sent
·
m3
not sent
·
m4
not sent
·
m5
not sent
·
m6
not sent
·
m7
not sent
·
m8
not sent
·
0 done · 0 dead-lettered · 0 duplicate charges
Tick 0: Consumer 1 receives m1.
1 2 The message stays in the queue but is invisible for the visibility timeout (4 ticks here, 30 seconds by default on SQS).
1 of 2 consumers busy.
3 4 5
In production
Tools and technologies
Queues decouple producers from consumers in every stack; the delivery guarantees are what differ.
Not tied to one language — use these names when you search docs or design a system.
- Amazon SQSService
Visibility timeout, at-least-once delivery, dead-letter queues.
Official site ↗ - RabbitMQSoftware
Acks, redelivery and dead-letter exchanges.
GitHub ↗ - Apache KafkaSoftware
A replicated log; consumers commit offsets after processing.
GitHub ↗ - Redis StreamsSoftware
Consumer groups with pending entries and claims.
GitHub ↗