Load balancing algorithms
Pick how a load balancer chooses among servers, because the right rule depends on whether servers differ and whether requests do.
A load balancer needs a rule for choosing a server, and the simplest rule fails when servers or requests are uneven.
Use it when#
One server runs hot while its neighbours idle.
Servers differ in size, or some requests take far longer than others.
A new server falls over when it joins, because it gets full traffic while still cold.
How it works#
Round robin deals requests out in turn, so a small server gets as many as a large one. Weighted round robin gives each server a share by size. Least connections sends the next request to the server with the fewest open ones, which adapts to slow requests. Power of two choices picks two servers at random and takes the less busy, and it needs no shared state, so it scales across many balancers. Consistent hashing sends the same key to the same server, which helps caches.
Slow start ramps traffic to a new server over its first seconds. Sticky sessions pin a user to one server, which keeps local state working but unbalances the load.
What it costs#
Least connections needs the balancer to track every connection. Power of two choices compares only two servers, so it can still pick a busy one, and the game also splits a little of its traffic evenly whatever the load. Hashing ignores load, so a hot key overloads one server. Sticky sessions make a restart log users out.
In Quorum#
A Load Balancer has a policy. Weighted and round robin are open from the start, and least connections, two choices, and consistent hash need Load balancing algorithms. Sticky and slow start are switches. The Design mixes a larger and a smaller server and uses least connections.
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| Load Balancer | Policy | least-conn | New requests go to the copy with the most spare room right now, so a big box takes more than a small one and a slow one takes less. |
| App Server | Instance size | medium | A Medium copy, twice the vCPUs of its neighbour. |
| App Server | Instance size | small | A Small copy. Round robin would hand it as much as the Medium. |
| Database | Instance size | small | Small gives the Database two vCPUs, so the figures below reflect the Cache and not a starved Database. |
Research this Design needs: Load balancing, Load balancing algorithms.
The glossary defines load balancer and power of two choices, and consistent hashing is the third.