Queue Saturation & Backpressure

When consumers fall behind producers, queues grow unbounded — and then everything crashes at once.

How It Works

A message queue (see Queues for the underlying decoupling pattern — think Kafka, RabbitMQ, SQS) acts as a shock absorber, not an infinite well. When the rate of producers exceeds the rate of consumers over time, the queue grows until something gives: the broker runs out of memory and crashes (OOM, out-of-memory error), messages start getting dropped, or upstream services block waiting for their messages to be accepted. Warning signs: growing queue lag, rising broker memory, and consumer offset drift (consumers reading from a point far behind the producer's latest write). The fix is not bigger queues — it is backpressure: consumers signal upstream "I am falling behind, slow down your input," usually via bounded-size queues, rate limits at the ingress, or explicit flow-control protocols.

Real-World Example

Uber's 2016 surge outage started with a traffic spike that overwhelmed their matching service queue. Without backpressure, the queue grew until the Kafka brokers ran out of memory, which cascaded into the dispatcher service going down. Now they rate-limit at the API gateway with adaptive thresholds — when downstream queue depth exceeds a high-water mark, the gateway starts rejecting requests with 503 errors before the queue can saturate.