OakfieldCore
← Research journal

Scalability

Queues have edges

A queue can absorb a burst. Its limits become visible when work arrives faster than consumers can finish it.

Queue and consumers diagram
Queue and consumers

Distinguish a burst from a trend

Imagine an import service that accepts files during a morning upload window. A queue lets the interface acknowledge receipt while workers parse the files. That is a useful separation only if the status page can distinguish receipt from completion.

Measure arrival and completion in comparable units. Counting files is misleading when one file contains a single row and another contains a day of transactions. Consider an estimate of remaining work alongside queue length and the age of the oldest waiting job.

Budget for waiting

During a sustained interval, outstanding work grows by arrivals minus completions. If workers finish less than arrives, extra storage only extends the time until the backlog becomes unacceptable. Adding workers helps until another shared resource, such as a database, becomes the limiting stage.

Choose a waiting-time budget from the user task. A preview requested interactively might expire before a long import completes. Put those workload classes under different scheduling rules when their promises differ. Make any refusal to accept more work visible to the caller.

Rehearse a slow drain

Replay a burst followed by a quiet period in an isolated environment. Track how long the oldest job remains outstanding and which shared resource saturates first. Repeat with a worker unavailable and include the cost of retries in the measurement.

The outcome is a decision about admission, concurrency and recovery. Document what operators should see while a healthy queue drains. That reference helps distinguish a long but predictable recovery from a backlog that is still accelerating.

Back to the journal