Calm at 3 a.m.: a second machine that takes over while you sleep

On 15 October the power went out where our own server lives. Twice. The machine came back on its own, but the sites on it did not — and the worst part was the silence. Nobody was told. We found out by accident, almost two hours later. Everything that was supposed to warn us lived on the same machine that had just gone dark.

So we built the thing we wished we had had that morning, and it is now part of MAESTRO.

A second machine that keeps up

You add a second, small machine anywhere else — another provider, another city. MAESTRO keeps a fresh copy of your sites on it every few minutes: files, databases, even the certificates. No history piles up; it is always just the latest state. That copy waits quietly, using almost nothing.

What happens at 3 a.m.

• Your main machine goes quiet. The second one notices from the outside, and after three minutes you get a calm note on Telegram. Most power cuts end here — the machine comes back by itself and you get a second note saying so.
• If it stays quiet, the second machine takes over. It starts your sites, checks that they really work — not just that something answers — and only then moves your addresses to itself.
• When the first machine wakes up, it sees the addresses have moved and stops its own copies. Only one machine ever writes, so nothing gets split in two.
• At 7:00 a short report waits for you, with your decisions as buttons: bring everything home, or stay on the backup for today.

Bringing everything home is one button

When you are ready, one tap brings it all back. What was written on the backup during the night comes home. The last few minutes from before the outage — the ones the backup never saw — are not thrown away; they are kept aside for you to look at. Then the addresses move back and the copying starts again. It never happens by itself: going home is your call.

Honest numbers

You can lose up to a few minutes of writes — the time between two copies. Visitors may see an interruption of around ten to twenty minutes while the world learns the new address. That is the price of a calm night with two ordinary machines, and we would rather tell you than promise magic. You can rehearse a takeover, or a homecoming, at any time — nothing live is touched.

It lives in the Stay alive room. If your machine dies tonight, you will hear about it — and you will not have to get up.

← all articles