We should move Hackage onto a separate set of VMs on Rackspace. For one, the current VMs are insanely underpowered and according to Datadog are at almost 100% CPU all the damn time.
We should move Hackage onto a set of two servers - one serving the site (with an nginx frontend), and another doing builds. Having both on the same machine is a security nightmare. Both can be reasonably powerful to ensure they can last for a while.
Second, we should think hard about the security enhancements for the builder. I'm personally favorable to installing grsecurity and hardening the machine. We can think about the Nix situation later (which should be easier to deploy anyway), but in the mean time just beefing it up and locking it down are the priority.
We also need to migrate the data over. I think this can be done pretty easily and efficiently without much downtime, we just have to announce it (especially now that we have http://status.haskell.org)