Add a cache
Put a Cache between the App Server and the Database so repeated reads are answered from memory and the Database keeps its capacity for writes.
Place a Cache between the App Server and the Database to answer repeated reads from memory.
Move up when#
Database CPU is high and most of its work is read queries.
p99 climbs with Users while the App Server has room.
The Design#
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| App Server | Instance size | medium | Medium, so the App Server is not the limit. |
| Database | Instance size | small | Small, and every API read lands on it. |
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| App Server | Instance size | medium | Unchanged. |
| Cache | Engine | redis | One Redis thread answers far more operations than this traffic sends, and it reloads part of its data after a restart. |
| Cache | TTL (s) | 60 | A minute keeps most hot reads as hits and caps how old a stale answer can be. |
| Database | Instance size | small | Unchanged. It now takes only the reads the Cache misses, plus every write. |
Research this Design needs: Caching.
In the first picture every read query reaches the Database. In the second, the App Server calls the Cache, and the Cache calls the Database only on a miss. Writes pass through.
Why it works#
The Cache serves its hit ratio of reads itself. The Database sees a smaller share of them, so its load bar drops and p99 falls. Redis answers on one thread, which is far more than this traffic needs. The TTL trades hit ratio for freshness: a longer one holds more hits and serves older data.
What it costs#
You buy stale reads. A write invalidates data, but a cache-aside race can keep an old row until its TTL ends, so the ledger shows a small stale share. A write-heavy mix pulls the hit ratio down. A new or restarted Cache starts cold, and the Database takes the full load until it fills. Memory that is too small for the hot data lowers hits as Users grow.
In Quorum#
Research Caching to unlock the Cache. The Read Mostly Sandbox shows the symptom. Cache-aside covers the pattern, and Stampede protection handles a cold start. The next rung is to add a read replica.