Capacidad, utilización y colas
Por qué un Nodo se vuelve lento mucho antes de llenarse, qué significan las tarjetas ámbar y roja y cómo las Colas cambian espera por seguridad.
A Node gets slow long before it is full. Every Node has a capacity, the most work it can do per second, and how full it is decides how long requests wait. That relationship is a curve, and it explains most of the slowdowns you meet in Quorum.
Utilization#
Utilization, written ρ (rho), is the work that arrives divided by the work the Node can do:
Work is measured in the Node's own unit. An App Server counts CPU time, worker slots, memory, and network. A Database counts CPU and connections. The tightest limit sets the Node's capacity. The load bar on each card shows utilization, and the card changes color as it rises:
Under 70%. Healthy. Requests barely wait.
70% to 90%. Amber. Waiting time rises fast.
90% and above. Red. The tail is several times the median.
Over 100%. Saturated. The Node processes what it can and drops the rest as errors.
The hockey stick#
Waiting time grows with ρ / (1 − ρ) times the service time. At 50% utilization, a request waits
about as long as it takes to serve. At 90%, it waits about nine times as long. The curve stays flat
for a long time and then shoots up, like a hockey stick.
So a Node at 85% is not "fine with 15% to spare". It is already slow, and a small rise in traffic makes it much slower. The same curve means a small upgrade to a hot Node can help a lot.
A Node runs at 90% utilization. You double its capacity, so it runs at 45%. About how much does its queueing wait change?
Headroom is not waste#
Demand follows a daily cycle, and the evening peak is far above the daily average, so a Design sized for the average runs red every evening. Spare capacity also absorbs a traffic spike and the loss of one Node. A healthy Design therefore runs well under the amber line most of the time.
You can buy headroom three ways:
Scale up. Resize a Node to a bigger instance. Each step doubles vCPU and RAM, but the Node restarts and comes back cold. Some software cannot use the extra cores.
Scale out. Add copies behind DNS or a Load Balancer. Each copy of a Component costs more to place than the one before.
Scale with demand. An Autoscaler adds copies when load passes its target and removes them when load falls. New copies take time to boot, so keep a margin anyway.
See vertical and horizontal scaling and the Infrastructure reference for instance sizes and prices.
Slots, not only CPU#
An App Server can be full while its CPU is idle. Runtimes such as PHP-FPM, Gunicorn, Puma, and Tomcat hold a worker or thread for each request until it finishes, including the time it waits on the Database. When every slot waits, new requests queue even though the CPU has nothing to do.
Little's law explains it: slots in use equal the arrival rate times the time each request holds a slot. Double the wait and a fixed pool serves half the requests. You can fix the slow call, add workers, or move to a runtime that waits without a slot. The Runtimes pages show which model each runtime uses, and How requests flow explains why this links your App Servers to your Database.
Queues: waiting on purpose#
A Queue is a Component that stores work so that Workers can do it later. An App Server linked to a Queue hands over the slow, asynchronous part of some requests, such as resizing an upload or building an export. The user waits only for the enqueue.
A Queue does not drop work when its Workers fall behind. It stores the excess as depth, and the Workers drain it when they can, so a burst becomes a delay instead of errors. Depth has a price:
A new task waits behind every task already in the Queue. Users score an upload or export on when its result is ready, so a deep Queue frustrates them.
With no length limit, depth can grow without end when Workers drain slower than tasks arrive.
A Queue's max length is backpressure. Past it, the Queue refuses new tasks, and the request that sent one fails at once.
A task that fails comes back for another attempt later, and without a dead-letter queue, a task that always fails keeps coming back. See dead-letter queue and the Poison message incident.
Reading a red card#
When a card turns red, ask what its tightest limit is. If it is CPU, add capacity or make the work cheaper. If it is slots, look downstream for the slow call. If it is connections, read Errors and retries. The bottleneck marker tells you which red card to fix first.