Pub/Sub Pattern

What is the pub/sub pattern?

Publish/Subscribe (Pub/Sub) is a messaging pattern where producers publish messages to named topics without knowing who will receive them, and consumers subscribe to topics to receive messages. The broker handles routing and fan-out.

This decouples producers from consumers: producers don't call consumers directly and don't know how many consumers exist or what they do with the messages.

Topics vs Queues

	Queue	Topic (Pub/Sub)
Consumers	Competing - one consumer gets each message	All subscribers get a copy
Model	Work distribution	Event fan-out
Example	Image resize job -> any available worker	OrderPlaced -> inventory AND notifications AND analytics

Fan-Out

A single published event is delivered to N subscribers independently:

OrderService publishes: OrderPlaced { order_id: 123 }
    ├── InventoryService (subscriber) receives it
    ├── NotificationService (subscriber) receives it
    └── AnalyticsService (subscriber) receives it

Adding a new subscriber doesn't require changing the publisher. This is the core value: loose coupling via indirection.

Message Filtering

Subscribers can filter to receive only messages matching certain criteria:

• Topic-based: subscribe to orders.shipped only
• Content-based: subscribe to all orders but filter WHERE amount > 1000
• Attribute-based: AWS SNS supports attribute filtering on message metadata

Filtering reduces subscriber load - consumers only process relevant messages.

Delivery Semantics

At-Most-Once

Broker fires and forgets. Message may be lost if subscriber is down. Subscriber never sees duplicates.

• Use when: metrics, analytics events where occasional loss is acceptable

At-Least-Once

Broker retries until subscriber acknowledges. Subscriber may receive the same message multiple times if acknowledgement is lost.

• Use when: business-critical events that must not be lost
• Requirement: consumers must be idempotent (handle duplicate delivery safely)

Exactly-Once

Each message delivered exactly once. Very hard to implement at scale. Kafka transactions support it in specific configurations. AWS SQS FIFO approximates it.

• Use when: financial transactions, inventory updates where duplicates cause double-counting

AWS SNS + SQS Pattern

The classic AWS pattern for durable pub/sub fan-out:

SNS Topic (fan-out)
├── SQS Queue A (InventoryService)
│     └── Lambda/ECS (processes messages, retries on failure)
├── SQS Queue B (NotificationService)
│     └── Lambda/ECS
└── SQS Queue C (AnalyticsService)
      └── Lambda/ECS

• SNS fans out to multiple SQS queues
• Each SQS queue buffers messages independently
• If InventoryService is down, its queue absorbs messages until it recovers
• At-least-once delivery with SQS; failed messages go to DLQ after N retries

Push vs Pull

• Push (SNS, webhook-based): Broker pushes messages to subscribers. Low latency. Broker must know subscriber endpoints.
• Pull (Kafka, SQS): Consumers poll the broker. Consumer controls rate. Works well when consumers may be temporarily down.

Interview Angle

For event-driven architecture questions: describe pub/sub as the fan-out mechanism. Then address delivery semantics - "at-least-once means I need idempotent consumers." Explain the SNS+SQS pattern for AWS-native setups: SNS for fan-out, SQS for buffering and retry, DLQ for poison message handling. This demonstrates end-to-end understanding, not just the pattern name.