Nine servers, one database: Stack Overflow
Stack Overflow kept one SQL Server and a handful of big web servers, and put Redis and Elasticsearch beside it.
Stack Overflow served about 209 million HTTP requests a month in early 2016. Nine primary IIS web servers took them, behind four HAProxy load balancers, with one SQL Server primary, a hot standby, read-only copies, two Redis servers, and three Elasticsearch servers. Nick Craver's architecture series lists every box.
The system#
The site was one .NET application, multi-tenant across the whole Stack Exchange network, behind four HAProxy load balancers. Redis held an L1/L2 cache. Elasticsearch answered searches, so the database saw only the writes and the cache misses. The database stayed one big box with a hot standby, and when the team needed more it bought RAM and a faster disk before it considered sharding.
The board#
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| Load Balancer | Policy | least-conn | Traffic goes to the copy with the most room, so a slow web server takes less before it is dropped. |
| App Server | Instance size | large | Large, because the site runs one application on a handful of big web servers and keeps them lightly loaded. |
| App Server | Copies | 9 | Nine copies stand for the nine primary IIS servers that served the network in 2016. |
| Cache | Engine | memcached | The site uses Redis, which runs one thread. Memcached is the game engine that scales with cores, so the board does not understate the cache tier. |
| Cache | Cache strategy | aside | The application fills the cache on a miss and writes straight to the database, the L1/L2 pattern the site uses. |
| Cache | TTL (seconds) | 60 | A minute bounds how stale an answer can be. |
| Cache | Instance size | medium | Medium, enough for the hot set in front of the database. |
| Database | Instance size | xlarge | Extra large, because the site buys one big SQL Server rather than many small ones. |
| Database | Standby | On | A second box in another zone takes over when the primary zone fails, the hot standby the site keeps. |
| Database | Backups | nightly | A nightly snapshot is the cheap floor for a single primary. |
| Search Engine | Product | elasticsearch | The site runs Elasticsearch beside the database, and the search tier takes the search queries off it. |
| Search Engine | Refresh interval | 1 | A new post is searchable within a second. |
| Search Engine | Instance size | small | Small, because search is a small share of the traffic. |
Research this Design needs: Load balancing, Caching, Full-text search, Caching, Full-text search, Multiple zones.
The Load Balancer stands for the four HAProxy boxes. Nine App Server copies stand for the nine primary IIS servers. The Cache is Redis, the Database is the SQL Server primary and its standby, and the Search Engine is the three Elasticsearch boxes. The links are the request path: the app reads the cache, a miss crosses to the database, and search goes to its own tier.
Why it works#
The shape spends money on three things that remove work from the database: a large buffer cache, a Redis tier, and a search tier. The database keeps the writes and the reads that miss the Redis tier. Scale up first is the same decision, and search beside the database is the third tier.
Where the game stops#
Quorum has no SQL Server, so the Database runs Postgres in the board, and the caption is Quorum's result, not the site's latency. The board cannot show a 384 GB buffer cache or the read-only SQL copies. It shows the shape: one primary database with a cache and a search tier beside it.
Sources#
Stack Overflow: The Architecture, 2016 Edition, Nick Craver: the server list, the request counts, and the cache and search tiers.
Stack Overflow: The Hardware, 2016 Edition, Nick Craver: the RAM, disk, and network of each box.
Engineering Stack Overflow with Roberta Arcoverde, Hanselminutes #847: the 2022 view of the same architecture and the team that runs it.