解説

レイテンシとパーセンタイル

リクエストの時間がどこへ行くか、平均が最も遅いユーザーをなぜ隠すか、ゲームが p50、p95、p99 をどう計算するか。

The Garage view shows mean latency, and it is the least useful number on the screen. Your users do not experience the mean. They experience their own request, and a few of them wait much longer than the rest. Quorum computes latency for every request kind on every path through your Design, and it pays you by the wait, so the slow tail costs real revenue.

Where the time goes#

A request's latency is the sum of every hop on its synchronous path. Each hop adds four parts:

  • Fixed overhead. Every Component adds a small base latency. The first hop, across the internet from Users to your edge, is the largest one in a small Design.

  • Service time. The CPU time the work needs, divided by the runtime's speed, plus I/O on data Nodes.

  • Queueing wait. The time a request waits for a free slot. It grows sharply as a Node fills up. See Capacity, utilization, and queues.

  • Network. A call inside one zone is cheap. A call to another zone costs more, and a call to another region pays the round trip.

On top of these, rare tail events hit a small share of requests: a garbage collection pause, an event loop stall, a cold start. They add a lot of time to a few requests.

An asynchronous hop does not add to what the user waits: a request that goes through a Queue waits only for the enqueue, so the score for that kind counts the time until the result is ready.

Why the mean lies#

Most requests are fast and a few are very slow, which is the textbook single-server queue, where response times follow an exponential curve. On that curve the 99th percentile is about 4.6 times the mean of the variable part.

So a mean of 100 ms can hide users who wait half a second. A tail event that hits 2% of requests barely moves the mean, because the other 98% are unchanged. The p99 jumps.

A percentile asks the better question: how slow is the slowest share of my users? The p50 is the typical request. The p95 and p99 are your unlucky users, and on a busy site that is a lot of people.

理解度チェック

A GC pause adds 150 ms to about 2% of the requests on one App Server. Which number moves the most?

What you see and when#

The Garage view shows mean latency. The Percentiles research adds the p99 to the HUD and p50, p95, and p99 to each request kind. The Ledger computes percentiles from the start, and slow requests cost you revenue before you can see them. Research changes only what the screen shows.

A percentile comes from the model's distribution rather than from a sample. The fixed parts add directly, the exponential part sets the curve, and tail events land on the slowest share of requests, equal to their probability. For how to act on the number, see Read p99.

Tails add up along a path#

A request collects the tail events of every Node on its path. If one Node stalls 1% of requests and the next stalls 1%, a request that crosses both has about a 2% chance to stall, because two chances a and b combine as 1 − (1 − a)(1 − b). A long chain of Nodes that each stall now and then has a worse p99 than any one of them.

Fan-out makes this worse. When one request makes many backend calls and waits for all of them, the slowest call decides its latency. A pulled feed reads many posts per request, so its tail is long. This is the tail at scale and fan-out.

Runtimes shape the tail#

Some tail events come from your Stack:

  • Garbage collection. When an App Server's heap is nearly full, its collector pauses a small share of requests. The length depends on the runtime and the collector.

  • Event loops. On Node.js, one CPU-heavy request, such as a password hash at login, stalls every request behind it on that loop.

  • Cold starts. A new or restarted App Server runs slowly until it warms up, so scaling out during a peak can make the p99 worse for a while.

These show in the p99 long before they move the mean. The Runtimes pages list each runtime's behavior, and the glossary explains GC pauses, the event loop, and JIT warmup.

サンドボックスGCポーズAPI読み取りのp99を110 ms未満に90秒間保ってください。

Timeouts cut the tail, at a price#

A Link waits for an answer up to its timeout. A request that waits longer fails as a timeout. A short timeout keeps a slow callee from holding the caller's slots for long, but it turns slow requests into failed ones. See Errors and retries for how that trade works.