Managing Dell MD Storage with SCVMM 2012

In the new SCVMM 2012, you can now centrally manage storage devices using SMI-S providers from various vendors. Till now, only a small handful of storage devices are supported in SCVMM 2012. Among Dell storage portfolios, only the expensive Compellent series are officially supported. There is actually a SMI-S provider (MD Storage Array vCenter Plug-in) for the cheaper Dell MD storage but it is meant for VMWare vCenter. Fortunately being open standard, you can use the same provider even for SCVMM 2012. Download either the x86 or x64 program. As this provider is a proxy type, install it on a Windows Server 2003 or 2008 (WS2012 is not supported) that is reachable by the VMM server. Run the executable to install. Ignore any vCenter settings, since we are using SCVMM here.

Post Installation of Dell MD SMI-S on Proxy Server

1. Create a local user account on the proxy server. Run "cimuser -a -u username -w password"  (where username is the same local user account) using local Administrator credential (not just elevated prompt) to add it as a CIM user account. The command can be found under "C:\Program Files (x86)\Dell\pegasus\bin".
2. Create/Edit ArrayHosts.txt in the directory "C:\Program Files (x86)\Dell\pegasus\providers\array". Add the management IP addresses of the storage array. Use a new line for each IP address.
3. Restart cimserver service using "services.msc". Check TCP 5988 is running using "netstat -ano"
4. Enable host firewall rule to permit TCP 5988

Configuration of SCVMM 2012 SP1 on VMM Server

1. Create a RunAs account using the same credential that you created earlier using cimuser.
2. Go to "Fabric". Right click on "Providers" to add storage devices. Choose the one with "SMI-S" option.
3. On next page, choose "SMI-S CIMXML" protocol. Specify the proxy server on the Provider IP address and select the RunAs account.

Complete the remaining wizard step and the MD storage can now be managed as part of SCVMM 2012 Storage Fabric. You can create new LUN and assign them to the VMs directly under the same console seamlessly.

New Storage Spaces, SMB3.0 and SOFS in Windows Server 2012

One big game-changing difference in Windows Server 2012 is the new Storage Spaces feature and SMB3.0. Storage Spaces organizes a bunch of hard-disks neatly into virtual storage pools that can be easily expanded by simply adding more hard-disks. Virtual storage pool can also be added to support Clustered Storage Space for fail-over clustering (see: "How to Configure a Clustered Storage Space in Windows Server 2012"). Minimally, each node needs to have direct access to shared disk resources (at least 3 x SAS drives) i.e. SAS JBOD with no RAID sub-system, such as using PCIe non-RAID HBA connection.

SMB3.0 provides many enhancements for improved performance, resiliency and security, including:

1. SMB Scale-Out: transparently redirect SMB client connections to a different file server cluster node.  
2. SMB Direct (SMB over RDMA): enables direct memory-to-memory data transfers between servers, with minimal CPU utilization and low latency, using standard RDMA-capable network adapters (iWARP, InfiniBand, and RoCE). Any application which accesses files over SMB can transparently benefit from SMB Direct.
3. SMB Multichannel – takes advantage of multiple network interfaces to provide both high performance through bandwidth aggregation, and network fault tolerance through the use of multiple network paths to data on an SMB share.  Fast data transfers and network fault tolerance.
4. Transparent Failover and node fault tolerance – Supporting business critical server application workloads requires the connection to the storage back end to be continuously available. The new SMB server and client cooperate to make failover of file server cluster nodes transparent to applications, for all file operations, and for both planned cluster resource moves and unplanned node failures.
5. Secure data transfer with SMB encryption – protects data in-transit from eavesdropping and tampering attacks. Encrypting File System (EFS) is still required to protect data at rest though.

Reference: "SMB 2.2 is now SMB 3.0".

You can enable cluster disk as new Cluster Shared Volume (CSV). CSV enables all cluster nodes to "own" a "shared" volume at the same time i.e. Active-Active configuration. When CSV 1.0 was first introduced in W2K8 R2, it was only meant for Hyper-V storage to support Live Migration. Match the new CSV 2.0 storage with SMB3.0, it provides a real solid NAS-based alternative to SAN for performance and resiliency at a better value known as Scale-Out File Server (SOFS). Meanwhile, Microsoft is working with hardware partners to create a cluster-in-a-box (CiB) architecture if you prefer appliance-based SOFS solution. 

SOFS can be used as file-based storage spaces for Hyper-V and MS SQL clusters over SMB 3.0. Without expensive SAN storage (replaced by SOFS using shared SAS JBOD) in the picture, the new Hyper-V and SQL cluster would look like below (taken from TechEd 2012):

Subsequently, I did a quick test on SOFS using iSCSI storage. Even without fail-over clustering, "Share Nothing Live Migration" is also possible for non-clustered Hyper-V hosts using Hyper-V replica.

Having singing much praises to SOFS, do note that it is still not meant for every situation. Microsoft recommends that SOFS should not be used if your workload generates a high number of metadata operations, such as opening files, closing files, creating new files, or renaming existing files, which is typical for end-user file shares. Microsoft publishes the following chart to help you to decide when to use traditional file share and SOFS (taken from "When to use Scale-Out File Server"):

If you're already running 10Gigabit Ethernet or higher in your data center, you should further optimise your existing investment and leverage on the full performance benefits of SMB Direct (i.e. SMB over RDMA). Do note that the servers should have Network Interface Cards (NICs) that support RDMA (iWARP or RoCE). Here is a link on RDMA enabled NICs that support Windows Server 2012.

I've also come across this informative MVP blog about the new SOFS that can potentially replace SAN-based solution for server clustering. Here's the extract:

Scale Out File Server (SOFS)

Normally we want our storage to be fault tolerant. That’s because all of our VMs are probably on that single SAN (yes, some have the scale and budget for spanning SANs but that’s a whole different breed of organisation).  Normally we would need a SAN made up fault tolerant disk tray$, switche$, controller$, hot $pare disk$, and $o on. I think you get the point. Thanks to the innovations of Windows Server 2012, we’re going to get a whole new type of fault tolerant storage called a SOFS.

When I’ve talked about SOFS many have jumped immediately to think that it was only for small businesses.  Oh you fools!  Never assume!  Yes, SOFS can be for the small business (more later).  But where this really adds value is that larger business that feels like they are held hostage by their SAN vendors.  Organisations are facing a real storage challenge today.  SANs are not getting cheaper, and the storage scale requirements are rocketing.  SOFS offers a new alternative.  For a company that requires certain hardware functions of a SAN (such as replication) then SOFS offers an alternative tier of storage.  For a hosting company where every penny spent is a penny that makes them more expensive in the yes of their customers, SOFS is a fantastic way to provide economic, highly performing, scalable, fault tolerant storage for virtual machine hosting.

Failover Clustering with StarWind iSCSI

Recently, I attended a well-known iSCSI SAN vendor seminar. The product is good and it provides all kind of storage virtualization (except deduplication), including replication, thin provisioning etc. The main selling point is the frameless architecture that is scalable and you can manage the whole lots of their SAN boxes as one virtual instance. The main drawback is pricing and it can't inter-operate with other storage solutions. Hence, there is a potential vendor lock-in.

I recalled some MVP speaker in Las Vegas introduced a software iSCSI target called "StarWind iSCSI". I decided to give it a try and set it up at my home network. The setup looks like this:


To prevent single box failure, I mirrored 2 virtual volumes across both StarWind servers (which are now iSCSI SAN boxes. Joining them to domain would make administration even easier). Creating the virtual HA volumes and exporting them as iSCSI targets is easy with StarWind with this step-by-step guide from the vendor.

Next, I setup 2-node failover cluster and both nodes are able to connect to the iSCSI targets with MPIO. I added file service to the cluster with a large musical video file that I captured in Las Vegas. I mapped a drive from the client PC on the public net and start playing it. During the play, I purposely shut down StarWind1 server (which is the source target). The video paused a few seconds before the partner (StarWind2) took over. I'm impressed.

Storage Virtualization

As we are implementing Microsoft virtualization, more and more storage space are being used up rapidly. Another issue is storage availability. As we cluster up more VM hosts, major single points of failure still remain on the shared cluster storage. Even if storage can be fully replicated within a single site, any site-wide disaster (like flood, fire etc) can wipe off any data shortly. This is where storage virtualization comes into the picture. Wiki defines storage virtualization as the abstraction (separation) of logical storage from physical storage.

Let's take a look at the jargon used and how they can help solve the above issues, mainly on over-provisioning (that lead to high costs) & availability/DR related issues.

• RAID: Some said RAID is the earliest form of storage virtualization, as a logical volume can span across multiple disks to prevent single disk failure.
  
• I/O Multipathing (MPIO): In the event that one or more of these components fails, causing the path to fail, multipathing logic uses an alternate path for I/O so that the servers & applications can still access their data.
• Remote synchronization: To eliminate storage as single point of failure, data across two separately located storage are replicated over the network on a per volume basis. It presents a single logical volume to the servers, although it may span across different storage boxes. It is also essential to implement multi-site failover clustering for Windows 2008 servers.
  
• Thin provisioning: It is easier and less troublesome to extend a volume rather than shrinking it. Hence, most administrators tend to over-provision storage space for applications. To reduce wastage, thin provisioning allows administrators to provision a large volume but only a small fraction is actually allocated until the applications occupy more space over time.
  
• Thin replication: You replicate a "thinly" provisioned volume. Only delta changes will be replicated across and save network bandwidth.
  
• Point in time Snapshot: To simplify data restoration & DR recovery, periodic snapshots on the storage are taken over time. It allows you to rollback data to certain points in time.
  
• Deduplication: When you implement server virtualization or VDI, most of the bits and bytes of the VHDs are identical. Deduplication further optimizes storage space by removing duplicated bits & bytes.