Autoscale the compute tier
Link App Servers to an Autoscaler so the number of copies follows the load, inside a floor and a cap you choose.
Let an Autoscaler add and remove App Server copies as load changes, instead of sizing for the worst hour.
Move up when#
Demand swings through the day, and you either pay for idle copies at night or run hot at peak.
You add copies by hand after every spike.
A spike hurts before a person can react.
The Design#
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| App Server | Instance size | small | Each copy is a small box, so capacity grows in small steps. |
| App Server | Copies | 2 | The Autoscaler starts here and moves the number as load changes. |
| Autoscaler | Signal | rps | Requests per copy follow traffic whatever the runtime. CPU can read low while worker slots are full. |
| Autoscaler | Min copies | 2 | Two copies keep serving while a new one boots, which takes minutes on a VM. |
| Autoscaler | Max copies | 8 | The cap is the cost guard. A loop or an attack cannot buy more than this. |
| Database | Instance size | medium | More copies mean more connections and more queries, so the Database has to take what they send. |
Research this Design needs: Load balancing, Autoscaling, Load balancing, Autoscaling.
Users go through a Load Balancer to the App Server, which calls the Database. The App Server also links into the Autoscaler. No traffic passes through the Autoscaler. It watches its Signal, compares it with the target, and changes the copy count.
Why it works#
When the Node runs above target, the Autoscaler asks for more copies, and when it idles well below, it gives one back. The Signal should fit the runtime. CPU misreads a runtime that waits on I/O, because its worker slots fill while CPU stays low. Request rate follows traffic whatever the runtime, and queue depth suits a Worker. Min copies keeps the service up while a new copy boots, and Max copies is the cost guard.
What it costs#
New copies are not instant. A VM takes minutes to boot, so a sudden spike hurts before the copies arrive, and new copies start cold. More copies send more connections and queries to the Database, so it needs room, and a pooler may follow. A scaler with no cap can turn a bug into a large bill.
In Quorum#
Place an Autoscaler after Autoscaling, and put copies behind a Load Balancer. Scheduled scaling provisions ahead of a known peak. The pattern is on Autoscaling, and Survive a traffic spike uses it. The next rung keeps copies of data in sync: Stream changes with CDC.