Guides

Backpressure

Protect your process from overload by bounding the number of requests that can execute or queue concurrently. Backpressure is separate from rate limiting: rate limiting controls request frequency per identity, while backpressure bounds the total in-flight work inside one process at a time.

Why backpressure matters

A sudden traffic spike can saturate your process with concurrent requests. Without backpressure, each new request adds more work to the event loop, degrading latency for everyone until the process becomes unresponsive or runs out of memory. The backpressure controller acts as an admission gate: it allows at most maxConcurrent requests to execute at once and queues up to maxQueue additional ones. When the queue is full, new requests are rejected immediately with a 503 Service Unavailable — a clean failure the caller can retry.

Wrap a handler with withBackpressure

The simplest way to add backpressure is to wrap your request handler with withBackpressure. It returns a handler with the same signature, so it composes with other middleware:

wrapping the fetch handler

import { withBackpressure } from "@thexjs/core";const app = createApp({ ... });Bun.serve({  fetch: withBackpressure((req) => app.fetch(req), {    maxConcurrent: 50,   // process up to 50 requests at once    maxQueue: 20,        // let 20 more wait for a slot    retryAfterSeconds: 2, // Retry-After header when saturated  }),  port: 3000,});

saturated response (503)

HTTP/1.1 503 Service Unavailable
Content-Type: text/plain; charset=utf-8
Retry-After: 2
Service Unavailable

When the queue is full, new requests are rejected with 503 Service Unavailable (not 429 Too Many Requests): the caller is not being rate-limited; this specific process has no capacity left.

Using the controller directly

For more control — e.g. to apply different limits per route or release a lease after a subtask — create a BackpressureController with createBackpressureController and call acquire / release manually:

manual backpressure — per-route limits

import { createBackpressureController } from "@thexjs/core";// Separate controller for a heavyweight CPU-bound routeconst heavyController = createBackpressureController({  maxConcurrent: 5,  maxQueue: 2,});// Separate controller for a fast I/O routeconst lightController = createBackpressureController({  maxConcurrent: 100,  maxQueue: 50,});async function handleHeavyRoute(req: Request): Promise<Response> {  const lease = await heavyController.acquire(req.signal);  try {    return await processHeavyTask(req);  } finally {    lease.release();  }}async function handleLightRoute(req: Request): Promise<Response> {  const lease = await lightController.acquire(req.signal);  try {    return await processLightTask(req);  } finally {    lease.release();  }}

Handling saturated errors

When acquire rejects with BackpressureSaturatedError, the error includes a retryAfterSeconds property so you can produce a compliant response:

custom error handling

import { createBackpressureController, BackpressureSaturatedError } from "@thexjs/core";const controller = createBackpressureController({ maxConcurrent: 10, maxQueue: 5 });async function handler(req: Request): Promise<Response> {  try {    const lease = await controller.acquire(req.signal);    try {      return await doWork(req);    } finally {      lease.release();    }  } catch (error) {    if (error instanceof BackpressureSaturatedError) {      return new Response("Too many requests in-flight, please wait", {        status: 503,        headers: { "Retry-After": String(error.retryAfterSeconds) },      });    }    throw error; // re-throw abort errors, etc.  }}

Monitoring the controller

Call snapshot() at any time to inspect the controller's state. This is useful for metrics endpoints or health checks:

metrics snapshot

import { createBackpressureController } from "@thexjs/core";const controller = createBackpressureController({ maxConcurrent: 50, maxQueue: 20 });// Expose as a /metrics endpoint or log periodicallyfunction getBackpressureMetrics() {  const s = controller.snapshot();  return {    active_requests: s.active,    queued_requests: s.queued,    max_concurrent: s.maxConcurrent,    max_queue: s.maxQueue,    utilization_pct: Math.round((s.active / s.maxConcurrent) * 100),  };}

AbortSignal support

If the request's AbortSignal fires while the request is waiting in the queue, it is removed from the queue and its promise rejects with the abort reason — it never invokes the upstream handler. This prevents queued work from executing for clients that have already disconnected.

Observability