How to delete Virtual Disk in Dell MD during Operation?

How to delete Virtual Disk in Dell MD during operations? Answer is you can't, you've to wait for the operation to complete.

That sounds fair but not when you've waited several hours (if not days) for the operation to complete. Power cycle the MD storage still can't abort the operation. From the DellMD manager console, you've absolutely no idea to know when it would ever complete.

The next best news is that you can run a command utility "smcli.exe" to roughly gauge the progress. This is typically found under  "C:\Program Files (x86)\Dell\MD Storage Manager\client" folder.

> smcli -n ArrayName -c "show virtualdisk [\"VirtualDiskinOperation\"] actionprogress";

Performing syntax check...

Syntax check complete.

Executing script...

Virtual Disk xxx
Action: RAID Level Change
Percent Complete: 66%
Script execution complete.

SMcli completed successfully.

How to setup Windows iSCSI with MPIO on Dual Controller Storage Target

Let's say you've acquired an iSCSI storage target with dual controllers (e.g. Dell MD3xxx-i series). And you wish to configure Windows iSCSI initiator with MPIO to enable multi-path connection for High Availability and Load Balancing. How should you go about it?

iSCSI Port Configuration

Let's begin with the iSCSI storage controller setup. Typically, each controller should have four iSCSI ports. Hence, you should configure 4 different VLANs on your iSCSI switches. Each port on each controller should connect to a VLAN (also an IP subnet). Jumbo frame (e.g. MTU 9000) is also recommended. Assign a valid IP address to each port. The port configuration should look something like this:

Create new LUN and assign the preferred path to either Controller
Typically, each LUN can only be accessed via a target controller at any one time. The connections to the preferred controller should be active and the other controller should be standby. Assign the new LUN to either preferred controller and remember its iSCSI port addresses.

Enabling MPIO on Windows host

Similarly, configure the four or more iSCSI network connections (enable Jumbo Frame) on the Windows host. Install the necessary provider software given by the storage vendor. Add new Windows MPIO feature by executing "Install-WindowsFeature Multipath-IO -IncludeManagementTools" on Powershell. Activate MPIO and restart the host by clicking on the red boxes as follows:

Windows iSCSI Initiator 

Start the Windows initiator by executing the command "iscsicpl". On Discovery tab, configure the Target Portals connecting to the iSCSI target directly. Go back to the Targets tab and click the "Connect" button.

By default, only 1 session connection is made. Disconnect the session. We can add more iSCSI sessions on clicking on the Properties button as above. Always check the "Enable multi-path" box whenever you see it.

Each session identifier represent each session to each controller. As you can see, I've already added two sessions to both controllers. You may add more by clicking on the "Add session" button. Check the "Enable multi-path" box and "Advanced" button. Assign the "Target portal IP" to the preferred controller address.

Multiple Connected Session (MCS) Policy
If you have more than 1 iSCSI NIC on the server for each session, click on the "MCS" button to add additional connections to each session. To add more connections to each session, click on the "Add" button and select the appropriate iSCSI initiator and target addresses.

You can also choose a load-balancing algorithm. By default, simple "Round Robin" is used to distribute the loads evenly among the multi-paths.

Verify Multi-Path for each connected LUN
Click on "Device" and then "MPIO" button. Verify that each connected LUN can be accessed by more than 1 session.

Conclusion
In summary, there are two levels of path redundancy defined. Firstly, the session to each controller. Secondly, within a session, you can also have multiple connections defined under the MCS Policy. You may define different load-balancing algorithms for each level. In this setup, I clicked on the Device MPIO and defined "Failover-Only" for session level (remember the LUN can only be accessed via one controller at a time). Under each session, I defined "Round Robin" for multiple connections under MCS Policy for load-balancing.

Comparision between iSCSI and FCoE on Cisco Nexus

There are much confusions and debates over the choice between iSCSI and FCoE (Fiber Channel over Ethernet) for SAN storage choice, especially when you have invested on the Cisco Nexus data center infrastructure. Both choices are well supported by Nexus and both can definitely run concurrently together with the data network traffic. On top of that, you can even run both Jumbo and non-Jumbo frames without any loss in packets and performance! This truly fulfill the promise of "Unified Fabric Data Center".

The main advantage of iSCSI is its low-cost and its ease of implementation. You don't need any additional hardware on your existing servers and you can run it off on any server-class Gigabit Ethernet NICs that support TCP offload (even though it's non-mandatory). Because iSCSI runs on top of TCP/IP, you may suffer some slight deterioration in performance due to its inherent latency, especially when you have to route the traffic over L3 hops.

FCoE, on the hand, is a replicate of existing FC technologies (except the physical cables, it simply replaces the Fibre Channel physical layer with 10GbE). If you do own existing FC infrastructure (i.e. Cisco MDS) and intend to "migrate", "converge" or "consolidate" into Ethernet infrastructure, FCoE would be the choice. You can simply map VSAN into VLAN on the Lossless Ethernet-based Nexus for inter-operability. Typically, you would need Converged Network Adapters (CNAs) on the servers, which contain both Fibre Channel Host Bus Adapter (HBA) and Ethernet Network Interface Card (NIC) functionality on the same adapter card. CNAs have one or more physical Ethernet ports, which facilitate transition and migration from FC to FCoE.

Below is a good comparision table between iSCSI and FCoE taken from this NetApp site.

Storage Virtualization

As we are implementing Microsoft virtualization, more and more storage space are being used up rapidly. Another issue is storage availability. As we cluster up more VM hosts, major single points of failure still remain on the shared cluster storage. Even if storage can be fully replicated within a single site, any site-wide disaster (like flood, fire etc) can wipe off any data shortly. This is where storage virtualization comes into the picture. Wiki defines storage virtualization as the abstraction (separation) of logical storage from physical storage.

Let's take a look at the jargon used and how they can help solve the above issues, mainly on over-provisioning (that lead to high costs) & availability/DR related issues.

• RAID: Some said RAID is the earliest form of storage virtualization, as a logical volume can span across multiple disks to prevent single disk failure.
  
• I/O Multipathing (MPIO): In the event that one or more of these components fails, causing the path to fail, multipathing logic uses an alternate path for I/O so that the servers & applications can still access their data.
• Remote synchronization: To eliminate storage as single point of failure, data across two separately located storage are replicated over the network on a per volume basis. It presents a single logical volume to the servers, although it may span across different storage boxes. It is also essential to implement multi-site failover clustering for Windows 2008 servers.
  
• Thin provisioning: It is easier and less troublesome to extend a volume rather than shrinking it. Hence, most administrators tend to over-provision storage space for applications. To reduce wastage, thin provisioning allows administrators to provision a large volume but only a small fraction is actually allocated until the applications occupy more space over time.
  
• Thin replication: You replicate a "thinly" provisioned volume. Only delta changes will be replicated across and save network bandwidth.
  
• Point in time Snapshot: To simplify data restoration & DR recovery, periodic snapshots on the storage are taken over time. It allows you to rollback data to certain points in time.
  
• Deduplication: When you implement server virtualization or VDI, most of the bits and bytes of the VHDs are identical. Deduplication further optimizes storage space by removing duplicated bits & bytes.

Multiple Path I/O (MPIO) for iSCSI storage

To ensure server to storage path continuity, we usually deploy redundant physical path components, including NICs (for iSCSI) and HBAs (for FC, SCSI etc). In the event that one or more of these components fails, causing the path to fail, multipathing logic uses an alternate path for I/O so that the servers and applications can still access their data.

Each storage vendor may introduce their own Device Specific Module(DSM) solution, such as PowerPath for EMC. If you did not cater for budget to purchase specific vendor MPIO module, Microsoft did introduce a generic DSM free. I tried it on my W2K8 R2 server on Hyper-V VM and Dell-EMC AX4-5i storage using Round Robin to load balance among the four iSCSI paths. If you are also using W2K8, install MPIO as a feature using Server Manager or "Add-WindowsFeature" cmdlet in Server Core.

Click on Administrative Tools -> MPIO and check on "Add Support for iSCSI devices" on the "Discover Multi-Paths" tab. Reboot the server. On Server Core, you may run 

>"mpclaim -r -i -d "MSFT2005iSCSIBusType_0x9""

Invoke iSCSI initiator and connect to the iSCSI targets - make sure that the option "Enable multi-path" is checked. On server core, you may invoke the same initiator using "iscsicpl.exe" command.

Click "Advanced", connect using different path each time. Repeat and rinse for the number of different paths that you have. Click on "Devices" -> "MPIO". And you should the load balancing policy and the multiple paths linking to the Disk devices.

 For a more complete Server Core configuration, see "MPIO with Windows 2008 R2 Server Core and iSCSI".

Storage options for Hyper-V

Found this excellent online resource that discuss various storage options for Hyper-V. This table is particularly useful:

Delete Volume Group in Openfiler

Openfiler is a free open-source iSCSI solution, which I mentioned earlier. I've been trying to delete a Volume Group (VG) via the Brower GUI on Openfiler. The VG still remains.

I search the Internet and found this blog on how to remove the VG using CLI instead.

Step 1: Disable VG
vgchange –a n

Step 2: Remove VG
vgremove

How I assign storage to a VM

Someone asked how I typically assign storage to a VM. This is what I usually did - separate the data from the binaries to minimize the chance of corruption in the event of outage.

C: System OS drive. Typically assign ~60GB fixed sized VHD.

D: is application drive where I would install the application binary in VHD. Size varies on application requirements.

E: is the data or log drive where I would store the system & application data & logs. If syslog or ftp is the application, expect a big storage space. Typically, I would assign direct SAN LUN with RAID (e.g. iSCSI) to this volume. I would also redirect the host Firewall/IIS logs here for audit purposes.

In summary, C & D drives are typically assigned with VHDs that are stored on the host's direct or SAN storage while I would assign direct LUN with redundant RAID to E drive. The rationale is that the system & application binaries in C & D can be easily restored by installation but not the logs/data on E drive. Always remember to keep data and binaries separate.

Quick Tutorial on DiskPart

Some quick tutorial on using DiskPart (menu-driven utility to manage/create disk, partition & volume) if you happen to work on Server Core or Hyper-V Server 2008

How to setup iSCSI Initiator on Server Core R2 and Hyper-V server

In my earlier post, I mentioned about using OpenFiler as a free iSCSI Target server and even build a pair of failover cluster VM server on Hyper-V. Setting up iSCSI initiator in full graphical W2K8 installation is pretty straightforward. What about Server Core R2 and HyperV server that provide minimal GUI and mostly CLI?

After digging through MS TechNet, I've figured out and tested it on my test HyperV server.

Step 1: Create a RAID volume, configure the iSCSI target, and map a LUN on it

Step 2: MS added a nice GUI of iSCSI initiator. Otherwise, you have to make do with the "iscsicli" cmdlet, which is difficult to manage. At the server core command prompt, just type "iscsicpl.exe"

Step 3: On "Discovery", enter the iSCSI target IP address or DNS. Make sure you add it as favorite targets, so that it will reconnect everytime the server restarts. On "Targets" tab, click "Connected".

Step 4: Type "diskpart.exe" on command prompt. Do "List Disk" to ensure that the new disk is added. Type "Select Disk", "Create Partition", "Assign Letter" to disk and finally "Format quick". All these commands are equivalent to what you normally do on a graphical "Disk Management" console.

Step 5: Check on the newly created disk and it is ready to go.