A WebSockets tier for realtime
Hold realtime connections open on a Realtime Hub that runs a runtime built for them, and route only realtime traffic to it, so chat and live updates stop eating App Server workers.
Give realtime traffic its own Hub, because a connection that stays open for minutes cannot share a worker pool with requests that last milliseconds.
Move up when#
Realtime messages fail as refused, or App Server workers sit waiting on open connections.
Chat or live updates arrive late while pages and reads are fine.
After a restart, clients reconnect all at once and the Hub refuses half of them.
The Design#
Drag to pan. Hold Ctrl or Cmd and scroll to zoom.
| Where | Setting | Value | Why |
|---|---|---|---|
| App Server | Instance size | medium | Medium runs the request kinds that the Hub does not take. |
| Realtime Hub | Stack | node | The default Stack holds a worker for each open connection, so it holds none. An event loop holds each connection for a few KB. |
| Realtime Hub | Clustering | On | Without it one box takes every connection, so a bigger Hub adds Burn and no room. |
| Realtime Hub | Presence | On | The Hub tracks who is online, so each message earns more and each join costs CPU. |
| Realtime Hub | Reconnect backoff | On | Clients return after a restart with jitter, so the Hub does not refuse half of one reconnect wave. |
| Database | Instance size | medium | Medium, to match the App Server. |
| Link Load Balancer → App Server | Kinds carried | page,read,write,login | Every kind this Design sends except realtime. A Hub is the only Node here that can hold a realtime connection open. |
| Link Load Balancer → Realtime Hub | Kinds carried | rt | Only realtime messages go to the Hub, so the long connections never sit in the App Server. |
Research this Design needs: Load balancing, WebSockets, Node.js.
The Load Balancer splits traffic by request kind. Realtime goes to the Hub and every other kind goes to the App Server. The Hub takes realtime and sends nothing on.
Why it works#
A runtime that holds a worker for each connection holds none, so the Hub refuses everything. An event loop or a green thread holds each connection for a few KB, so a Node.js Stack lets the Hub keep many open. Clustering lets bigger Hub sizes add room, and without it one box takes every connection and a larger size only adds Burn.
Presence earns more per message because social features pay for knowing who is online, and it spends CPU on every join. Reconnect backoff spreads a returning wave over time instead of refusing it.
What it costs#
A Hub is a stateful Node. A restart drops every connection and invites a reconnect wave, which is the Reconnect storm. Clustering needs a pub/sub backbone, and a split cluster loses the messages that cross the cut. The game has no long polling or server-sent events option, so it cannot show a cheaper fallback for light use.
In Quorum#
Place a Realtime Hub and link it from a Load Balancer. Set the Link's kinds so only realtime reaches it. Research WebSockets, and a runtime such as Node.js. The next rung, sharding the database, splits the data tier once one primary cannot take the writes.