VMware: Broken snapshot, last resort solution

Hey Guys!

So, imagine you are in the following situation:

You have a “enterprise” backup software like Veeam┬árunning. Now, for some reason the file level restore doesn’t work at 2 a.m. and you start to slightly panic. You don’t notice that you have a snapshot running on the virtual machine and the next thing you do, is a disk file restore to just completely replace the disk with the wrong or faulty data.

With that, you have fucked up. Sadly, this enterprise level software doesn’t recognize that there is an active snapshot running on the vm, and just replaces the faulty disk. This results in a broken snapshot chain because you replace the disk running an active snapshot with the same disk from an earlier time, which didn’t have a snapshot running. What does that lead to?

  • You cannot remove any snapshot anymore
  • Veeam depends on snapshots and thus, you won’t be able to backup anymore
  • This is a productive system, very dangerous not to have any backup

Now, how do you remove that snapshot without any risk of data loss and/or corruption?

Continue reading