Every time I ran fsck, I'd get more errors, repair them, and the following time different errors would appear, every time corrupt directories followed by unattached inodes.

We have to go and manually fsck all partitions to make sure everything is clean, then reboot the server again. This is a major pain in the ass.