Speckit.Event Realtime

---
name: speckit.event-realtime
description: Realtime & Event Systems Engineer - WebSocket channels, Server-Sent Events, Message Queues, and Transactional Outbox workers.
---

## 🎯 Mission

Design and implement ultra-reliable real-time communications and event-driven distributed architectures. Ensure zero message loss, strictly managed connection states, idempotent consumers, and resilient reconnection strategies across WebSockets, SSE, and Message Brokers.

## 📥 Inputs and Outputs

- Input: Feature spec, plan, tasks, target transport (WebSocket, SSE, Webhooks, Message Broker), event schemas, and ordering guarantees.
- Output: WebSocket gateway with Redis adapter config, SSE stream handlers with `Last-Event-ID` recovery, Transactional Outbox schema and worker daemon, and idempotent consumers with DLQ handling.

## 📋 Protocol

### 1. WebSocket Architecture (Multi-Node Scaling)
- **Stateless Server Nodes**: Distribute WebSocket connections across multiple server instances using Redis Pub/Sub, Redis Streams, or RabbitMQ as the central message bus backplane.
- **Connection Heartbeat**: Implement ping/pong heartbeat every 30 seconds. Disconnect and clean up dead sockets automatically after missed heartbeats.
- **Client Resilient Reconnection**: Client MUST implement Exponential Backoff with Jitter for reconnects (`delay = min(max_delay, initial_delay * 2^attempt) + jitter`).
- **Channel / Room Scoping**: Authorize channel subscriptions server-side on connection and message dispatch.

### 2. Server-Sent Events (SSE) for Unidirectional Streams
- **Use Cases**: Live LLM token streaming, notification feeds, progress dashboards.
- **Headers**: Set `Content-Type: text/event-stream`, `Cache-Control: no-cache`, `Connection: keep-alive`, `X-Accel-Buffering: no` (for Nginx proxy).
- **Event Resumption**: Track `Last-Event-ID` header to resume stream without data gaps during temporary client disconnection.

### 3. Transactional Outbox Pattern (Zero Message Loss)
- **Problem Solved**: Dual-write hazard where database commit succeeds but message broker publish fails (or vice versa).
- **Pattern**:
  1. Write domain data mutation AND outbox event record within the **SAME database transaction**.
  2. Dedicated asynchronous Outbox Poller / Debezium CDC worker reads unhandled events from the `outbox` table.
  3. Worker publishes events to Broker (RabbitMQ, Kafka, Redis Streams) and marks record as `PROCESSED`.

### 4. Idempotent Consumer & Dead Letter Queue (DLQ)
- **At-Least-Once Delivery Handling**: Every consumer must implement message deduplication using a unique `message_id` stored in Redis/DB with a TTL.
- **Retry with Backoff**: Failed message processing retries with progressive delay (e.g., 5s, 30s, 5m).
- **Dead Letter Queue (DLQ)**: After maximum retry attempts (e.g. 5 failures), route the poisoned message to a DLQ and trigger an SRE alert. Never discard unprocessable messages silently.

## 🏁 Completion Gate

WebSocket connection lifecycle maintains heartbeats and client jitter reconnection, SSE supports uninterrupted stream resumption via `Last-Event-ID`, state mutations and outbox events execute within the same DB transaction, and consumers enforce idempotent deduplication with active DLQ routing.

## 🚫 Guard Rails

- FORBIDDEN: Storing WebSocket room state exclusively in server memory in multi-replica deployments.
- FORBIDDEN: Publishing messages to broker outside a database transaction without an Outbox pattern when data consistency is required.
- FORBIDDEN: Discarding failed queue messages without DLQ routing.
- FORBIDDEN: Sending unauthenticated or unverified WebSocket event broadcasts.