Core concept

Backpressure is a graph property

A fast source must not outrun the slowest reliable destination. Exstream carries that demand through transforms, asynchronous work, and branches.

Short version

fast source → transform → slow writer
     ↑                         │
     └────── demand ───────────┘

The writer accepts one record at a time. That capacity travels upstream. The source pulls the next input only when the connected graph can make progress.

Backpressure is not a speed boost. It is what keeps a speed difference from turning into an ever-growing memory buffer.

`mapAsync()` bounds

const enriched = exstream(records).mapAsync(loadCustomer, {
  concurrency: 16,
  ordered: true,
})

concurrency: 16 limits active calls. With ordered: true, a later result may need to wait for an earlier one, so ordering also contributes to the amount of retained work. Both choices belong in the operator contract.

InputPulled on demand

Active workBounded by concurrency

Output orderPreserved by default

AbortStops useless work

Branches vs observers

fork() is reliable: every branch participates in backpressure. If one branch is slow, the shared source must respect it.

observe() is best-effort: it does not slow the main flow. Because a non-blocking observer can fall behind, it needs an explicit buffer limit and overflow policy. Use it for metrics, sampling, and diagnostics—not for data that must arrive.

Open “Backpressure” in the playground

Source limits

A pull-based async iterator naturally waits until the next pull. A pausable Node.js stream can stop producing temporarily. An EventEmitter or EventTarget may be hot and non-pausable: once events arrive, software can only buffer, drop, or fail.

An API can preserve backpressure semantics only where the source primitive makes pressure possible. Hot sources need an explicit overflow decision.

Review checklist

Before shipping a pipeline, answer these questions:

Which destination is allowed to set the pace?
How many records can each asynchronous operator retain?
Does preserving order add head-of-line blocking?
Are all fork() branches actually reliable?
What happens when a hot event source outruns the pipeline?
Which signal cancels work after a failure or early stop?