Skip to main content
Techdome
All Insights/System Design

Message queues vs event streams: SQS, RabbitMQ, Kafka

Use SQS or RabbitMQ for work distribution; use Kafka when you need retained event logs, replay, fan-out, and per-key ordering.

Techdome Engineering
Techdome EngineeringEngineering Team
Oct 26, 20263 min read
Verified Architecture
A technical illustration comparing queued task delivery with retained event streaming.

Message queues vs event streams is mainly a question of intent. SQS and RabbitMQ move work to consumers and remove it after success; Kafka stores events in an append-only log so consumers can read and replay them independently.

Message queues vs event streams: the core model

A queue is best when one unit of work should be handled by one worker: send an email, resize an image, charge a payment step, run a background job. SQS gives a managed AWS queue; RabbitMQ adds flexible routing with exchanges and per-message ack/nack control.

An event stream is best when an event becomes shared system history: order placed, sensor reading received, ticket scanned, account updated. Kafka is designed around retained topics, partitions, offsets, and many independent consumers. That is why it fits analytics, CDC, audit pipelines, and real-time dashboards. Techdome’s Catalyst product page is one example of a stack that includes Apache Kafka for real-time manufacturing and mobility analytics.

A comparison diagram showing messages removed after acknowledgement in queues and retained for replay in event streams.
A comparison diagram showing messages removed after acknowledgement in queues and retained for replay in event streams.

Delivery, ordering, and replay

Queues usually give at-least-once delivery. SQS Standard can deliver duplicates. SQS FIFO adds ordering and deduplication behavior within a MessageGroupId. RabbitMQ uses acknowledgements; messages can be redelivered when consumers fail or nack.

Kafka is also at-least-once by default, but idempotent producers and transactions support exactly-once processing patterns for producer-to-topic-to-consumer workflows. The important constraint is partition scope: Kafka ordering and processing guarantees are tied to partitions, not a single global order.

Replay is the biggest practical split. In SQS and classic RabbitMQ queue usage, an acknowledged message is removed. Kafka retains events for a configured retention or compaction policy, and consumers manage offsets, so they can rewind and replay. If retries and duplicate delivery are your main concern, the design discipline overlaps with webhook retries and idempotency keys: consumers still need safe reprocessing.

A simple decision guide

Use SQS when:

  • You want low-ops managed work distribution on AWS.
  • Each message represents a task, not durable business history.
  • At-least-once delivery with idempotent consumers is acceptable.
  • FIFO per message group is enough when ordering matters.

Use RabbitMQ when:

  • You need richer routing, exchanges, topics, or request-reply patterns.
  • You want explicit ack, nack, retry, and dead-letter handling.
  • Ordering is simple enough to manage with queue and consumer design.

Use Kafka when:

  • Multiple teams or services need the same event independently.
  • Replay, retention, CDC, analytics, or auditability are first-class needs.
  • You need high-throughput fan-out with per-key ordering.
  • Consumers should move at their own pace using offsets.

Whatever you choose, test the contract at the boundary. The same thinking behind API testing applies to async systems: schema, error cases, retries, and consumer behavior need automated coverage.

Trade-offs and when not to use it

Kafka is powerful, but it is not a better queue by default. It adds operational and design weight: partition strategy, retention, consumer lag, rebalancing, schema discipline, and storage planning. If you only need background jobs, Kafka is usually too much.

SQS is simple, but less expressive. It is a good default for AWS task queues, not for long-lived event history. RabbitMQ is flexible, but clustered operation and ordering under multiple consumers need care.

Do not use a queue when the business needs replayable history. Do not use Kafka when the system only needs short-lived work dispatch. For workflow automation, queue mode can matter too; our article on n8n back-office automation discusses staged workflow thinking without making every step an event stream.

Takeaways

  • Queues distribute work; streams preserve event history.
  • SQS is the low-ops AWS queue choice; RabbitMQ is stronger for routing and ack control.
  • Kafka fits replay, fan-out, analytics, CDC, and per-key ordering at scale.
  • Ordering is scoped: SQS FIFO MessageGroupId, RabbitMQ queue design, Kafka partition.
  • If you are choosing the backbone for an async system, document the delivery, ordering, replay, and failure requirements first.

Further reading

References

FAQ

Questions readers ask

What is the main difference between SQS, RabbitMQ, and Kafka?

SQS and RabbitMQ are message queues for delivering work to consumers. Kafka is an event stream: it stores events in retained topics that many consumers can read independently.

Should we use Kafka for every asynchronous workload?

No. Use Kafka when replay, retention, high-throughput fan-out, or event history matter. For simple background jobs, SQS or RabbitMQ is usually simpler.

Can SQS or RabbitMQ provide exactly-once delivery?

Not natively end to end. SQS FIFO provides deduplication behavior within its model, and RabbitMQ relies on acknowledgements, but consumers should still be idempotent.

When does ordering actually hold?

SQS FIFO orders within a MessageGroupId. RabbitMQ ordering is clearest with a single queue and careful consumer setup. Kafka orders within a partition, not across all partitions.

Continue reading

More foundry teardowns

View all
+
+
+
+
Healthcare AIOct 28, 2026·2 min read

Asynchronous vs synchronous telehealth: which to build first

Asynchronous telehealth means a clinician reviews a prepared case; synchronous means a live visit. How they differ, what each changes in the build, and why most platforms end up with both.

Techdome Engineering
Techdome EngineeringEngineering Team
+
+
+
+
Healthcare AIOct 28, 2026·4 min read

Telehealth compliance checklist for the US and UK

An engineering checklist for a telehealth platform serving US and UK patients: contracts, access and audit, clinic isolation, licensing by jurisdiction, prescribing, signatures, payments and incidents.

Techdome Engineering
Techdome EngineeringEngineering Team
+
+
+
+
Healthcare AIOct 28, 2026·3 min read

How to build a GLP-1 telehealth platform

A builder's guide to a GLP-1 telehealth platform: compliance, multi-tenant clinics, eligibility checks, licensing by jurisdiction, chart review, pharmacy routing, cold chain and billing.

Techdome Engineering
Techdome EngineeringEngineering Team
+
+
+
+
Healthcare AIOct 28, 2026·2 min read

Telehealth metrics and KPIs: the numbers that matter

The 15 telehealth metrics an operator should track, from intake completion to time to physician decision and cold-chain excursions, with real figures from a US and UK GLP-1 platform.

Techdome Engineering
Techdome EngineeringEngineering Team
Illustration of webhook retry flow using a stable idempotency key, de-duplication store, and dead-letter queue.
System DesignOct 26, 2026·3 min read

Webhook retries and idempotency keys, together

Webhook retries need stable idempotency keys so repeated deliveries are safe, deduplicated, observable, and recoverable through DLQs.

Techdome Engineering
Techdome EngineeringEngineering Team
Blueprint-style illustration of a server rack behind a layered firewall shield, with a PHP version badge being upgraded
Engineering PracticeOct 22, 2026·6 min read

PHP2Shell: how to check and protect your PHP site

PHP2Shell (CVE-2026-109618 and CVE-2026-109619): how to check your PHP version, patch it or shield it with a WAF, with two real cases.

Techdome Engineering
Techdome EngineeringEngineering Team
Computer screen filled with code
Engineering Practice8 min read

API testing explained: types, what to automate, and tools

API testing checks an interface directly, below the UI. Learn the five main types, how mocking and test data fit in, and which tests to automate first.

Techdome Engineering
Techdome EngineeringEngineering Team
Computer screen with data
n8n Automation7 min read

"Automating mortgage back-office work with n8n"

Mortgage back offices drown in documents. A staged n8n workflow can handle intake, classification, extraction and validation, and route exceptions to people with a full audit trail.

Techdome Engineering
Techdome EngineeringEngineering Team
Interface illustration of a browser window with three gauge dials, a heavy block swapped for a light placeholder, and images compressed
Engineering PracticeSeptember 2026·6 min read

Making a Next.js site fast: the fixes that moved Core Web Vitals on techdome.io

Hero images that wait for JavaScript, an embedded scheduler that blocked the page, oversized images and bloated page data: the real fixes behind our site's performance work, and a checklist for yours.

Rahul Joshi
Rahul JoshiChief Executive Officer
3D illustration of an AI agent connected to calendar, database and phone tools, with a sound wave and a small knowledge graph
Studio ArchitectureSeptember 2026·7 min read

Building AI agents that hold up in production: what we learned from voice, interviews and GraphRAG

Narrow jobs, grounded answers, deterministic scoring, a latency budget, full logs and a human handoff: the patterns behind our voice agents, AI interviewer and GraphRAG platform.

Rahul Joshi
Rahul JoshiChief Executive Officer
Pipeline illustration of an automation workflow with a queue feeding two workers, a retry loop, an alert bell and a review tray
n8n AutomationAugust 2026·7 min read

Production n8n: the patterns that keep automation from failing quietly

Idempotent triggers, retries with limits, failure alerts, human review and workflows kept in Git: the patterns we use to run n8n automation in production, with the builds they come from.

Rahul Joshi
Rahul JoshiChief Executive Officer
Isometric illustration of one platform split into separate clinic compartments, with a shield, an access gate and an audit log
Healthcare AIAugust 2026·8 min read

Engineering HIPAA-ready telehealth platforms: the decisions we make first

How tenant isolation, access control, audit logging, identity and vendor agreements shape a HIPAA-ready telehealth system, with what each looked like on three clinical platforms we built.

Rahul Joshi
Rahul JoshiChief Executive Officer
Blueprint-style illustration of five phases from a drawing board to a launched product, rising along a path
Studio ArchitectureAugust 2026·7 min read

The Techdome Blueprint: how we take a product from a blank repo to production in eight weeks

The five steps of our Blueprint, what each one delivers, how fixed-scope sprints and engineering pods are priced, and how recent builds ran from kickoff to production.

Rahul Joshi
Rahul JoshiChief Executive Officer