Choose the message bus for the bad day

AWS’s application integration guide covers nine services. On a good day they are interchangeable: a producer puts something in, a consumer takes it out, the dashboard is green. Every meaningful difference between them shows up only on the bad day.

So that is the day to select for.

The four properties that actually differ

Strip away the marketing categories — “messaging”, “events”, “orchestration” — and there are four properties. Every one of the nine services is a particular combination of them, and the combination is the product.

Delivery. Almost everything here is at-least-once. That is not a limitation to be worked around; it is a consequence of the only failure model that survives a network. It means duplicates are normal traffic, and it means your consumer must be idempotent — which is a design decision about your data, not a setting on the queue.

The number of teams who write “exactly once” in a design document and then build a consumer that cannot tolerate seeing a message twice is remarkable. The queue was never the problem.

Ordering. Sequence is expensive and most services do not promise it. Where it is offered it comes with a partition key, which means ordering is guaranteed within something and not globally — and choosing that key is choosing your concurrency limit at the same time. One key means one lane.

Retention and replay. This is the property that most changes what you can do in an incident. A queue forgets: a consumed message is gone, and if you processed a thousand of them incorrectly on Thursday, they are not there to process again. A log retains: you move the read position back and the same events go through the corrected code.

The difference between those two is the difference between an incident that costs an afternoon and one that costs a data-reconstruction project.

Fanout. Whether a second consumer can arrive without the producer knowing. The moment a producer has to be changed to add a subscriber, the decoupling that justified the bus is gone.

A worked example: the retry storm

An agentic workflow calls a tool that writes to a downstream system. The downstream system slows down. Calls start timing out — but timing out is not the same as failing, and some of those writes landed.

The agent retries. The queue, correctly, redelivers what was not acknowledged. Both mechanisms are behaving as designed and the downstream system now has more load than it started with, some of it duplicate work it cannot distinguish from new work.

Nothing here is a bug in any component. It is the interaction, and it is decided by three choices made much earlier:

• Whether the consumer is idempotent, which decides whether the duplicates are harmless or are a second set of writes.
• Whether there is a dead-letter destination, which decides whether the poison message loops forever or steps aside after a bounded number of attempts.
• Whether retries are bounded and backed off at both layers, because an agent that retries a queue that also retries produces a multiplication nobody intended.

The third one is the interesting failure, and it is specific to agentic systems. A retrying model and a retrying transport are two independent loops around the same call. Design one of them to be the only one.

Run the sieve

Which of these are true for you?

• Order mattersMessage two must not be processed before message one.
• We will need to reprocess historyA bug is found on Thursday and Monday’s events must go through again.
• A duplicate would be a real incidentCharging twice, emailing twice, approving twice.
• Several unrelated consumers want the same eventAnd a new one must be able to arrive without touching the producer.
• A single flow spans hours or daysWaiting on a human, an external system, or a scheduled step.

Still standing7 of 7

• Amazon SQS (standard)

  A queue that decouples a producer from a consumer and absorbs the difference between their rates.

• Amazon SQS (FIFO)

  Ordering within a group, with duplicate suppression inside a bounded window.

• Amazon SNS

  Publish once, deliver to many subscribers, without the publisher knowing who they are.

• Amazon EventBridge

  Routing by event content, with rules that live outside the producer and an archive worth replaying.

• Amazon Kinesis Data Streams

  An ordered, retained log that several consumers can read at their own pace and re-read from a point in time.

• Amazon MSK

  Kafka, with the operational load of the brokers moved but the semantics unchanged.

• AWS Step Functions

  Explicit state for a flow that has to survive waiting — with the retry, timeout and compensation written down rather than implied.

Our reading, not a specification. Service behaviour changes — check any elimination that decides something against the service page before you design around it.

Turn on a duplicate would be a real incident first. Watch how little survives, and read the reasons. The lesson in that column is the one worth taking: the transport is not where you solve duplicates. You solve them in the consumer, by making the second delivery a no-op, and then you are free to choose the transport on the other three properties.

The question to ask about any of them

For each service on the shortlist, answer this out loud:

A consumer crashes after doing the work and before acknowledging it. What happens next, and who finds out?

Every service has an answer. The answers differ, and they are the actual selection criteria. If nobody on the team can state the answer for the service already in production, that is worth an afternoon before the next design meeting — it is usually the cheapest incident review available, and it happens before the incident.

Next: analytics, where the buried constraint is not latency or cost but who is allowed to see which rows.