The client was worried because one of the Asterisk servers had gone down without any notice at all. The overseer process on the other Asterisk server had noticed, and had taken over as the active server. The disruption was minimal. Agents were at work. But the client wanted to know what had happened, and what we could do to prevent the issue in the first place. After 30 minutes of poring over logs, digging around, and contacting the colo, we discovered that a tech had decided to swap out the power bar connected to that server. There was no notice, and not even a courtesy three-finger salute. Continue reading “What To Do Before Someone Lets The Smoke Out of Your Server”