キャッシュ

ホットなデータが載っている間は、読み取りクエリをメモリから返します。書き込みは素通りし、キャッシュを無効化します。

このページはまだ日本語に翻訳されていません。

A Cache answers repeated reads from memory so the Database does not have to, as Redis, an in-memory store, does.

When to place it#

  • Place it when reads dominate and Database CPU is high. The Read Mostly Sandbox shows the symptom.

  • Place it before a Read Replica when the same hot rows are read again and again: it costs the least of the three. When the hot data outgrows its memory, or reads are spread over many rows, add a replica.

  • Resize it when the hit ratio falls as users grow.

  • Switch to Memcached when a single Redis thread is the limit.

How it works#

Put it in front of the Database or the Connection Pooler. It serves a share of read queries itself, its hit ratio, and passes the misses on. Writes go through and invalidate cached data, so a write-heavy mix pulls the hit ratio down.

The hit ratio rises with memory relative to your hot data, and your hot data grows with users. A longer TTL raises the ratio too, at the price of more stale reads. A new or restarted Cache starts cold and fills as reads arrive. See cache hit rate and cache-aside.

Options#

設定項目設定値表示条件
戦略キャッシュアサイド / リードスルー / ライトスルー / ライトビハインド常時
コアレス毎回のミス / コアレススタンピード対策の後
早期失効TTL で失効 / 早めに更新スタンピード対策の後
再検証中は古い値を返す失効時にミス / 古い値を返すスタンピード対策の後
ウォームアップコールドスタート / ウォームスタートキャッシュウォーミングの後
エンジンRedis / Memcached常時
TTL10 s / 1 min / 1 h常時
ホットキーシャード 1 つ / 分割ハイブリッドファンアウトの後

Strategy#

Cache strategies adds the three strategies a Cache can run besides cache-aside:

  • Cache-aside (the default): reads come from the Cache and writes go around it. A write can race a concurrent miss, so the Cache may keep an old row until its TTL runs out.

  • Read-through: the Cache loads a miss itself, which closes most of that race. Most of the TTL's own staleness is still there.

  • Write-through: every write goes to the Cache and the Database together. Nothing behind the Cache is stale, and each write waits for one more cache write.

  • Write-behind: the write returns from the Cache and reaches the Database late, so writes are fast and the Database takes less write load. A miss reads a Database that lags, and a Cache crash loses what it has not flushed.

The Inspector shows which way a write travels. Data tier holds the table.

Stampede options#

Stampede protection adds three switches. All three answer the moment a popular key expires and every request misses at once.

  • Coalesce: one request reloads the key while the others wait. The Database sees one miss instead of the whole wave.

  • Early expiry: refreshes a key a little before it expires, so a short TTL keeps more of its hits and the Database takes a few extra reads.

  • Stale while revalidating: answers from the old value while one request refreshes it, so an expired key never misses, and those answers count as stale.

See Cache stampede protection.

Engine and TTL#

  • Engine: Redis runs on one main thread and reloads part of its data after a restart. Memcached uses every core and comes back empty.

  • TTL: how long a value stays. A longer TTL keeps more of the hits and serves staler data.

  • Warming: with Cache warming, Warm start fills the Cache before it takes traffic, so a restart does not begin cold. See Cache warming and tiered caching.

  • Hot keys: with Hybrid fan-out, Split puts a hot key on its own shard, so one key stops setting the pace. See Hot keys and hot shards.

How it fails#

  • A long TTL serves stale data, which fails reads that need fresh data.

  • When the hot data does not fit its memory, every refill pushes an older key out: the card and the Dashboard's Cache row show the evictions a second, and the hit ratio falls with them. A bigger Cache fits the data and stops them.

  • A Cache does not reduce connections: it passes its callers' connections through.

Cache stampede#

  • Signs: The Hit ratio on the card falls toward zero, and the Database behind it turns amber, then red. While the Database stays overloaded, refills time out and the Cache stays cold.

  • Do now: Take load off the Database: turn on Coalesce on the Cache card, turn on Shed low value on the App Servers, or size the Database up. Coalesce needs Stampede protection, and Shed low value needs Priority load shedding.

  • Prevent: Research Stampede protection and Cache warming, then turn on Coalesce and Warm start.

Hot key#

  • Signs: One Cache turns red while the others idle, because one key takes a large share of the reads.

  • Do now: Set Hot keys on the Cache card to Split. It needs Hybrid fan-out.

  • Prevent: Research Hybrid fan-out and leave Split on.

Stale reads#

  • Signs: stale errors appear on API reads and checkouts, and the Cache's answers are older than the writes behind them.

  • Do now: Shorten the TTL on the Cache card.

  • Prevent: Keep a short TTL on write-heavy data, switch the strategy to Write-through after Cache strategies, or turn on read-your-writes routing (Read-your-writes routing).

Instance failure#

  • Signs: The Cache card goes grey and its callout reads "The host under this VM died. It still bills." Requests sent to it are stranded.

  • Do now: Select the Node and click Replace. It costs Cash and comes back cold. A Node with a Standby fails over by itself, or at once when you press Fail over.

  • Prevent: Turn on Standby in the Zone section, which needs Multiple zones, or run a second Cache.

Zone outage#

  • Signs: Every Node in one zone goes grey at once, the Cache with them.

  • Do now: Press Fail over on the banner, or on the Node's card. It needs Multiple zones and promotes the Node's Standby at once; the Node then keeps no standby until you give it one. A Node with no standby stays down until the incident ends, at most 120 sim seconds, and one with a standby also fails over by itself after a share of that time.

  • Prevent: Research Multiple zones, turn on Standby in the Zone section, and keep a copy in a second zone.

Region outage#

  • Signs: One region's board goes grey and the Cache in it goes down.

  • Do now: Press Fail over on the banner. It needs Multiple zones and a second open region, and starts the Node again in the nearest other region, cold, for the price of a replacement. Otherwise it comes back when the incident ends: 180 sim seconds in the catalog, 60 once the error budget counts.

  • Prevent: Research Multiple regions, keep a copy in a second region, and add GeoDNS to send users there.

サンドボックススタンピードApdexを0.9以上に90秒間保ってください。

Tips#

  • Turn on coalescing as soon as you own Stampede protection.

  • Use Memcached when one Redis thread is the limit.

ルール#

  • 追加のコピーは 1 つ前より 25% 高くなります。
  • 売却すると、最後のコピーの価格の 50% が戻ります。
  • サイズは 8 段階です。1 段ごとに容量と支出が倍になります。
  • 新しいコピーはコールドスタートで、温まるまで遅いままです。

一緒に使えるもの#

アプリサーバー、データベース、コネクションプーラー、リードレプリカ、ドキュメントストア、ワーカー、イベントストリーム、サーバーレス関数、ベクトルインデックス、CDCコネクター

次に取るリサーチ#

キャッシュを手に入れると、次のリサーチがそれを土台に進みます。

キャッシュ戦略、スタンピード対策、ハイブリッドファンアウト

エンジニアリング用語#