Several app servers behind a load balancer
Put a Load Balancer in front of identical App Servers so traffic splits across them and capacity grows by adding copies.
Place a Load Balancer in front of two or more identical App Servers and let it split the traffic.
Move up when#
The biggest instance size you can afford has a full CPU bar, or the next size costs more than a second box.
One App Server is a single point of failure you can no longer accept.
The Design#
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| App Server | Instance size | small | Small was enough a rung ago. At this many Users its CPU is over full. |
| Database | Instance size | medium | Medium keeps the Database out of the way while the App Server is the problem. |
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| Load Balancer | Algorithm | weighted | Both servers are the same size, so splitting by capacity is an even split. Pick least connections when sizes or runtimes differ. |
| App Server | Instance size | small | The same small box as before, now taking half the traffic. |
| App Server | Instance size | small | A twin of the first server, so either can answer any request. |
| Database | Instance size | medium | Unchanged. Both servers share this one Database. |
Research this Design needs: Load balancing.
The first picture is one App Server over capacity. In the second, Users reach the Load Balancer, which passes each request to one of two App Servers. Both call the same Database.
Why it works#
Each server now takes half the requests, so its load bar falls under full, drops stop, and p99 returns to normal. The default algorithm splits by capacity. With identical servers that is an even split, and with mixed sizes each server takes a share that matches its capacity. Least connections follows live load instead, which helps when servers slow unevenly, and needs research.
This works because the previous rungs left no state on the server. If a user's next request lands on the other copy, it finds the same rows and files.
What it costs#
The Load Balancer is one more hop and one more Node to size. Alone, it is the new single point of failure. Both servers hold their own connections to the Database, so the connection count roughly doubles with each copy, which is the next problem. Turning on sticky sessions pins users to a copy and unbalances the load.
In Quorum#
Research Load balancing to unlock the Load Balancer, then link it to each App Server. Load balancing algorithms adds least connections. The glossary defines the load balancer. The next rung is dropping dead nodes with health checks.