Server virtualization has often been problematic for data storage administrators, especially when it comes to tracking the relationships between virtualized servers and data storage assets. Many storage management products have been modified for this new environment, enabling users to monitor virtual servers, the applications they host and the storage they’re implementing.
Within each data center, there are typically silos pertaining to particular functional areas, each with a group of staff members dedicated to their management. There are people in charge of managing particular data center resources, including the network, servers, storage systems and virtualization. Each team focuses on managing its specific area and collaborates with other teams when it’s necessary to handle integration points between them. Should a new server need shared storage, the group dedicated to servers works with the storage group to provision storage and present it to the server.
In a traditional physical server environment, the relationships between storage and physical servers can easily be managed by the storage group; a logical unit number (LUN) created on a storage area network (SAN) is assigned to a physical server, and only that server uses the LUN. Things are very different in a server virtualization environment. Storage is perhaps the most critical component of a virtual infrastructure, so it must be implemented and managed properly for maximum performance and reliability. The relationship between server virtualization and storage is a tight one, so the management must be as well.
VMs add complexity to the storage environment
Virtualization revolves around multiple virtual machines (VMs) sharing physical resources. Virtualization file systems, such as VMware’s VMFS, let several physical servers simultaneously read and write to the same LUNs. This is enabled by a special locking mechanism that ensures that multiple hosts have exclusive access to each of their VMs on a shared LUN. Adding to the complexity of the environment are high availability and workload load balancing across virtualization clusters. In addition, VMware’s vMotion and Storage vMotion can move running VMs across hosts and across storage devices, respectively.
On top of the movement of VMs at the virtualization layer, many storage arrays also have an automated storage tiering feature constructed around tiers of devices with differing performance attributes, including solid-state drives (SSDs) and SATA drives that are pooled and presented to the host. The array transports data dynamically across tiers, depending on performance needs. This movement occurs at the storage layer, unbeknownst to the host.
While advantageous, features that allow for the movement of VMs can also cause problems for storage and virtualization managers, because of the dynamic nature of the relationships among VMs, their physical hosts and the storage device in which their virtual disk resides. These relationships have the greatest impact on troubleshooting and performance monitoring. Neither the storage administrator nor the virtualization administrator is able to see what occurs at the virtualization layer and the storage layer, respectively; as a result, neither administrator is able to see the overarching picture.
Plug-ins fill the management gap
Storage vendors recognized the importance of tight integration between storage and server virtualization and have worked to develop software integration with existing virtualization management tools like VMware’s vCenter Server. VMware offers a solid set of vSphere APIs that allows third-party vendors to integrate their products with vSphere. Also, vCenter Server has a plug-in architecture that makes it easy for third-party plug-ins to seamlessly integrate with the vCenter Server admin interface. Plug-ins appear as a tab inside the vSphere Client, and their behavior and appearance can be customized. This allows options or information that’s specific to a particular object like a VM, host or cluster, to be displayed.
STORAGE VENDORS' vCENTER SERVER PLUG-INS
Not all storage vendors were quick to develop vCenter Server plug-ins, but most of the major storage vendors today offer plug-ins that allow their storage arrays to be monitored and managed from within vCenter Server. Each vendor’s storage plug-in typically only supports specific storage array models and families, and the plug-in’s functionality and features vary from vendor to vendor. Generally, storage plug-ins may offer these capabilities:
- Simplified expansion of virtual datastores. LUNs are created and presented to hosts, which then create datastores such as VMFS volumes from them. To increase the size of a datastore, the underlying LUN on the storage array needs to be increased first. The plug-in allows both the LUN and VMFS volume to be increased from the same console.
- Storage provisioning. Storage admins can assign chunks of storage to virtual environments; this allows virtualization administrators to create and size their own LUNs and manage the configuration of the storage.
- Storage management. A plug-in can give virtualization administrators the ability to manage storage array capabilities like LUN masking and thin provisioning, set multi-pathing and tiering policies, optimize I/O settings and define access lists.
- Automated VM storage mapping. This type of plug-in lets you monitor and manage the physical/virtual relationships among the virtual machines, hosts and storage arrays. This can help the virtualization admin by mapping between the virtualization identifier and the storage array identifier for the same disk.
- View detailed storage information. This brings information from the virtualization layer and the storage layer into a unified view, and lets you see the exact details of the physical storage layer from within the virtualization console.
- Physical storage health monitoring. This capability provides information on the physical health of storage arrays so virtualization admins will know when physical hardware fails or becomes degraded.
- VM cloning. The cloning of VMs is basically just a data copy job that can be offloaded to the array, which can do it more efficiently. This is especially useful in virtual desktop environments that have larger VM density.
- Backup and recovery at the storage layer. This allows you to create point-in-time snapshots on the storage array of VM datastores. You can then mount the snapshot and restore VMs from it as needed.
Anatomy of a plug-in
The joining of the storage and virtualization layers allows the virtualization admin to stay within the context of the virtual management user interface (UI) without having to grant access to a specialized storage management UI. Most of the storage plug-ins let you define credentials for the storage arrays that will be managed inside the virtualization management console. This allows seamless integration between the two consoles, and it’s also good from a security perspective as you don’t have to grant virtualization admins direct access to the storage management console.
Hewlett-Packard (HP) Co.’s approach to the integration of storage management into vCenter Server was to leverage its Insight Control management console and integrate portions of it within vCenter Server as a plug-in. In addition to a module to manage HP storage, the company included a module to manage HP server hardware. So both server and storage hardware can be managed from a single console. When the plug-in is installed, it creates a special HP storage privilege within vCenter Server that allows access to be granted to the HP storage plug-in. The plug-in brings storage management into virtualization but not vice versa. vCenter Server has very granular permissions and roles can be defined so access to storage-specific information can be granted to storage admins. This allows storage admins to have a single console to all the storage arrays integrated with vCenter Server.
The HP Insight Control Storage Module for vCenter Server currently supports the firm’s EVA, P4000, P9000/XP and P2000/MSA storage arrays. The plug-in creates an HP Insight Software tab in vCenter Server that appears whenever a VM, host or cluster is selected; it also offers a menu option for actions such as cloning/creating virtual machines or creating datastores. The tab provides a storage overview of the object selected, such as storage provisioned to a host, and storage provisioned by a host and the arrays it’s connected to; it also provides links to directly launch the storage management console for an array. There are various views you can select to see different information such as storage disks and host bus adapters (HBAs) and paths; in addition, you can customize the columns and choose from the many detailed storage fields that are available. Sections allow you to see specific storage objects related to the object you’ve selected such as virtual machines, hosts and datastores.
In addition to information about storage arrays, there are storage tools that can perform actions such as cloning a virtual machine by utilizing array-based replication, creating batches of new virtual machines, or provisioning storage and creating VMFS volumes. While these tasks can be accomplished within vCenter Server, the HP plug-in provides automation and offloads the tasks to the storage array, which can handle it more efficiently.
The marriage of storage and virtualization in a single console allows for tighter management integration, which benefits virtualization admins, but is less beneficial to storage admins. Virtualization admins can get more involved with some of the storage-related functions, but those are traditionally handled by storage admins who may be reluctant to give up their control of provisioning and managing storage resources. Demonstrating the integration and features, and granting storage admins access to the virtualization console may help convince them to empower the virtualization admin to perform some basic storage management. Even if virtualization admins aren’t allowed to manage storage resources, being able to view detailed storage array information is advantageous by itself.
While there are plenty of management apps that integrate with VMware, there are plug-ins available for other hypervisors like EMC Corp.’s Virtual Storage Integrator for Hyper-V, which integrates with System Center Virtual Machine Manager (SCVMM). Vendors have focused on VMware because of its popularity and because VMware has a deeper and more mature set of APIs and SDKs. The storage integration plug-ins that are available for virtualization are relatively new and vendor offerings are still evolving with more features and better integration. No matter what hypervisor you have, storage plug-ins are a must-have for any virtualization environment, as they provide better visibility and integration, and enhance your ability to monitor, manage and troubleshoot your critical storage resources.
Eric Siebert is an IT industry veteran with more than 25 years of experience who now focuses on server administration and virtualization. He’s the author of VMware VI3 Implementation and Administration (Prentice Hall, 2009) and Maximum vSphere (Prentice Hall, 2010).
This was first published in July 2011