It seems like every time you just want to do some silly maintenance task you end up screwing something up royally well. A few weeks ago I accidentally fsck’d the running ext3 filesystem on this server, ignored the warning message it spit out, and killed the filesystem. That’s what I get for mistyping the /dev device and ignoring warnings. The problem with the fsck warning is that it was really easy to bypass. Given, that’s what I expect from a UNIX-ish system.
There’s still plenty of things broken here and on the server as a whole. It’ll all be back eventually. Note to self: when the backup server fails, replace it sooner rather than never. :(