Tip

Troubleshooting Hyper-V storage problems

One of the nice things about Microsoft Hyper-V is its relative simplicity compared with competing server virtualization

    Requires Free Membership to View

platforms. It tends to be stable and reliable, but storage-related issues can and sometimes do occur. This can lead to failed backups, corrupt virtual machines and even host server crashes. While the platform's relative simplicity goes a long way toward making it easy to diagnose and correct storage-related issues, it's important to understand what can go wrong. 

One of the most common Hyper-V storage-related problems that you might encounter is that virtual machines become corrupted such that starting the virtual machine is impossible. In some cases, individual virtual machine components might disappear entirely. Some examples of error messages that are displayed in such situations include:

  • "The requested operation cannot be performed on a file with a user-mapped section open (0x800704CB)."
  • "VMName' Microsoft Synthetic Ethernet Port (Instance ID{7E0DA81A-A7B4-4DFD-869F-37002C36D816}): Failed to Power On with Error 'The specified network resource or device is no longer available' (0x80070037)."
  • "The I/O operation has been aborted because of either a thread exit or an application request (0x800703E3)."

All three of these problems occur as a result of improper antivirus software usage. Administrators often deploy antivirus software within the host operating system, failing to realize the damage that such software can do to the virtual servers. As a general rule, antivirus software that is running within the host operating system should be configured to exclude virtual machine configuration directories, directories containing virtual hard disks and any directory containing virtual machine snapshots. Microsoft also recommends excluding the Vmms.exe file and the Vmwp.exe file from scanning.

If the host server is running Windows Server 2008 R2 and is operating as a Hyper-V cluster node, the ClusterStorage folder on the C drive must also be excluded from scanning. This folder maps directly to the Cluster Shared Volume, which normally contains virtual machine files.

Host server Blue Screen of Death

Another Hyper-V storage problem that you may encounter is a Blue Screen of Death (BSOD) error on the host server. There are a number of events that can trigger a BSOD. The most common is memory corruption. However, some Hyper-V administrators have experienced problems with a BSOD occurring every time they attempt to back up their Hyper-V hosts.

The easiest way to tell whether or not the BSOD is related to the backup process is to check the server's logs and determine whether or not the time of the crash coincides with the backup. If you do determine that the backup process is causing the crash, there are several things that you should check.

First, check to make sure that you have the latest firmware for both your storage hardware and backup hardware. Outdated firmware can sometimes cause the backup process to fail catastrophically.

Next, if your Hyper-V deployment is making use of a Cluster Shared Volume, you should check to make sure that your backup software supports backing up Cluster Shared Volumes. Many of the backup applications that are available for Hyper-V are not cluster-aware. Backing up a Cluster Shared Volume with a non-cluster-aware backup application rarely results in a catastrophic failure, but it can happen. More often, the end result is a corrupt backup.

Finally, a BSOD error can occur as a result of a problem with the Volume Shadow Copy Service (VSS). Hyper-V backups use VSS to ensure that virtual machine contents do not change while a backup is being created, thereby ensuring a consistent backup. The actual copy process is made possible by a VSS provider. Oftentimes, the VSS provider comes from Microsoft and the file copy process is handled by the native Windows file system. However, when specialized storage hardware is used, the storage vendor may supply its own VSS provider. These custom VSS providers have been known to cause BSOD errors. If you suspect that your VSS provider may be the cause of the error, check with your storage vendor to make sure that you have the latest version of the VSS provider and that it is properly registered with the operating system.

Storage contention

One of the most common Hyper-V storage problems is that of storage contention. Regardless of whether your virtualization host is using dedicated storage or a Cluster Shared Volume, all of the virtual machines on the host must compete for disk I/O. If a Cluster Shared Volume is being used, virtual machines running on every host in the cluster must compete for disk I/O because each host is linked to the same storage volume.

Storage contention becomes a problem when the storage subsystem cannot deliver sufficient disk I/O to meet the demands of the virtual machines. This can occur as a result of either a storage array bottleneck or a storage network bandwidth bottleneck. The only way to truly resolve the problem is to use performance monitoring software to identify the bottleneck and then upgrade the hardware.

Sometimes, however, you might be able to reduce the symptoms of storage contention by restructuring scheduled workloads on your virtual machines. For example, backups are notorious for generating a lot of disk I/O. If you are performing guest-level backups of multiple virtual machines, you might consider adjusting the backup schedules so that the backups do not occur simultaneously. Doing so will reduce the I/O load that is generated by the backup process.

Brien Posey is a freelance technical writer who has received Microsoft's MVP award six times. He has served as CIO for a national chain of hospitals and health care companies and as a network administrator for the U.S. Department of Defense at Fort Knox, Ky.

This was first published in July 2012

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.