Unable To Delete Hyper-V Root Snapshot in Hyper-V Manager

During a build-out for a customer it became necessary to move some virtual machines between a Hyper-V 2012 cluster and a Hyper-V 2012 R2 cluster but when trying to do so, all sorts of nasty errors came cropping up:

Live Migration Error Due To Differencing Disk

Error (12700)
VMM cannot complete the host operation on the host1.contoso.com server because of the error: Virtual machine migration operation for ‘MachineToMove.contoso.com’ failed at migration destination ‘host2.contoso.com’. (Virtual machine ID 1D5042AA-1A93-4635-9F0A-F7C7B0D10BDD)

Failed to access disk ‘C:\ClusterStorage\Volume2\MachineToMove.contoso.com\Windows Server 2012 DC with SP1_disk_1_3F40B5A6-E8DC-4752-873C-D9742C9419F4.avhdx’: ‘The system cannot find the file specified.'(‘0x80070002’).
Unknown error (0x800b)

Error (23753)
The virtual machine or tier load balancer configuration requires an IP pool and there are no appropriate IP pools accessible from the host.

Recommended Action
Select a host with access to an appropriate IP pool and try the operation again.

Live Migration Error Due To Differencing Disk 2

Error (12700)
VMM cannot complete the host operation on the MachineToMove.contoso.com server because of the error: Virtual machine migration operation for ‘MachineToMove.contoso.com’ failed at migration source ‘Host1’. (Virtual machine ID 1D5042AA-1A93-4635-9F0A-F7C7B0D10BDD)

Virtual machine migration for ‘MachineToMove.contoso.com’ failed because configuration data root cannot be changed for a clustered virtual machine. (Virtual machine ID 1D5042AA-1A93-4635-9F0A-F7C7B0D10BDD)
Unknown error (0x8005)

Recommended Action
Resolve the host issue and then try the operation again.

You may notice in the top error that the disk path is pointing to an odd file name. Looking at the settings for the machine in Hyper-V Manager and inspecting the disk, we find:

Live Machine Properties

Lo and behold, it’s a differencing disk. Let’s try removing the snapshot that created it:

Hyper-V Snapshot Missing Delete

And there’s the problem – no delete option!

Let’s look at the snapshot in PowerShell. To do so, open an elevated PowerShell session on a Machine with the Hyper-V PowerShell tools installed and run:

Get-VMSnapshot -VMName MachineToMove.contoso.com -ComputerName host1.contoso.com | fl

Here’s the output for the above VM:

SnapshotType : Recovery
VMId : 1d5042aa-1a93-4635-9f0a-f7c7b0d10bdd
VMName : MachineToMove.contoso.com
State : Off
Key : Microsoft.HyperV.PowerShell.SnapshotObjectKey
IsDeleted : False
ComputerName : host1.contoso.com
Id : 4382dc53-2fdd-476f-91b8-81963c292d24
Name : MachineToMove.contoso.com - Backup - (1/16/2014 - 6:00:19 PM)
Version :
Notes : #CLUSTER-INVARIANT#:{434c76e7-5581-463a-b1b4-71027d39770f}
Generation :
Path : C:\ClusterStorage\Volume2\MachineToMove.contoso.com
CreationTime : 16/01/2014 20:22:24
IsClustered : True
SizeOfSystemFiles : 49254
ParentSnapshotId :
ParentSnapshotName :
MemoryStartup : 8589934592
DynamicMemoryEnabled : False
MemoryMinimum : 536870912
MemoryMaximum : 1099511627776
ProcessorCount : 4
RemoteFxAdapter :
NetworkAdapters : {MachineToMove.contoso.com}
FibreChannelHostBusAdapters : {}
ComPort1 : Microsoft.HyperV.PowerShell.VMComPort
ComPort2 : Microsoft.HyperV.PowerShell.VMComPort
FloppyDrive : Microsoft.HyperV.PowerShell.VMFloppyDiskDrive
DVDDrives : {DVD Drive on IDE controller number 1 at location 0}
HardDrives : {Hard Drive on IDE controller number 0 at location 0, Hard Drive on SCSI controller
number 0 at location 0}
VMIntegrationService : {Time Synchronization, Heartbeat, Key-Value Pair Exchange, Shutdown...}

Time to remove it:

Get-VMSnapshot -VMName MachineToMove.contoso.com -ComputerName host1.contoso.com | Remove-VMSnapshot

You can run this command while the machine is running and if you look in Hyper-V after running this you’ll see that the differencing disk will quickly merge into the parent and then the recovery-point snapshot will be removed. Migrating the VM in this state should go without a hitch.

What caused this?

In this instance, the environment is running HP Data Protector 8.0 which is HP’s incredibly powerful (albeit rather old-looking) backup platform. The environment had been configured to back up the machines in the Hyper-V cluster using the HP StoreVirtual P4000 VSS/VDS Providers along with Application Aware Snapshot Manager. As I understand it, this uses the differencing disks so that incremental backups can be achieved – they’re merged and renewed during each Full backup. This is why you see the word “Backup” in the snapshot name along with the data and time that Data Protector took the backup.

HP IMC Firewall Rules

I use HP’s Intelligent Management Center a reasonable amount as I often install it as the NMS of choice for HP network deployments. As a result, I’m often caught by problems with things hitting the Windows Firewall and breaking.

Here’s a list of the exceptions that you need to add to the Windows Firewall for IMC. This list is incomplete but I’ll add to it as I find more:

Base Platform:

  • HTTPS – TCP 443 – This is assuming you’ve changed from the default of 8443 which I recommend
  • SNMP Traps – UDP 162 – <ServerInstallDir>\server\bin\imcfaultdm.exe
  • TFTP – <ServerInstallDir>\server\bin\tftpserver.exe – This fixes issues such as IMC not being able to back up HP ProCurve switch configurations

Wireless Services Manager [WSM]:

  • WLAN – (<ServerInstallDir>\server\bin\imcwlandm.exe) – This fixes a MSM not being able to connect to the Mobility Manager IP set in the Web Administration. For reference, this port is TCP 7668.

If you know of any others, please do let me know!

Updates:

2014-01-17 – SNMP Traps

Hyper-V 2012 -> 2012 R2 Cluster Migration Issues

Quick post more to document an oddity than anything…

Migrating machines from a 2012 cluster to a 2012 R2 cluster using VMM 2012 R2 with mixed results. In particular, I’m seeing the machines duplicated in Failover Cluster Manager – one of the two seems to be the live machine and the second, prefixed with SCVMM (as all VMM created machines are) seems to be broken with various errors such as ID 21502 “Missing or invalid virtual machine ID resource property”. Simply removing the duplicate starting with SCVMM and all seems to be ok.

Odd though.