ReferenceIncidents
Era 4 · Series BSplit brain
A network partition cut a Distributed Erlang cluster in two. Each half kept serving its own clients, and half of the realtime messages never reached the other.
What happens#
A network partition cut a Distributed Erlang cluster in two. Each half kept serving its own clients, and half of the realtime messages never reached the other.
How to prevent it#
Chaos experiments find the partition before it does. A restart rejoins the cluster.
The postmortem opens the first of these that you do not own yet: