I have not had much free time the last several months, so my tinkering with my storage system has been spotty. When I have had the chance, I have been testing different configurations with napp-it and disk layouts, ironing out stability issues and benchmarks.
Here is the setup that I have decided to stick with.
Build’s Name: Artemis
Operating System/Storage Platform: OmniOS/Napp-it
CPU: Xeon E3-1230v2 CPU
Motherboard: Supermicro X9SCM-F-O
Chassis: Rosewill RSV-L4411 Chassis
Drives: 8x Hitachi 2TB Disks, 4x Toshiba 3TB Disks, 1x SanDisk SDSSDP 64gb SSD for ZIL and Read Cache, 1x Crucial M4 64GB SSD for OS
RAM: 16GB UDIMM Kingston RAM
Add-in Cards: 2x IBM M1015 crossflashed/9211 IT mode, 2x Intel Dual Port Server NIC
Power Supply: Antec EarthWatts 650 Power Supply
Usage Profile: Storage, iSCSI Target, SMB Shares, MediaTomb
Pools:
I decided to go with mirrors with both of my pools. While the performance penalty of ZFS pools versus mirrors were negligible, they were noticeable. I decided to go with speed and the redundancy of mirrors.
Bonnie Benchmarks:
Comstar - iSCSI Config
Switch Config:
I am using a 16 port ZyXEL GS1510-16 smart switch configured with 5x VLANs with Jumbo Frames enabled.
napp-it config - Comstar:
Server Config:
Build’s Name: Zeus
Operating System/Storage Platform: Windows Server 2008 R2 w/ MPIO iSCSI
CPU: Xeon E3-1230 CPU
Motherboard: Tyan S5510
Chassis: iStar 4U Chassis
Drives: 4x Western Digital RED 1TB Disks, 4x Western Digital RE3 512GB Disks, 1x SanDisk SDSSDP 64gb SSD for OS
RAM: 32GB UDIMM Crucial RAM
Add-in Cards: 3Ware 9650SE RAID Controller, 2x Intel Dual Port Server NICs
Power Supply: Raidmax 530SS 530 Watt Power Supply
Other Goodies: 2x Supermicro 5x Bay Hot-Swap Cages
Usage Profile: Hyper-V
iSCSI Initiator - MPIO Config:
The Test:
To verify that MPIO was working properly, I copied an Exported VM that was roughly 40GB to the new iSCSI disk. The average transfer rate was roughly 390MB a second once the cache was leveled out on the napp-it box.
Success!
Here is the setup that I have decided to stick with.
Build’s Name: Artemis
Operating System/Storage Platform: OmniOS/Napp-it
CPU: Xeon E3-1230v2 CPU
Motherboard: Supermicro X9SCM-F-O
Chassis: Rosewill RSV-L4411 Chassis
Drives: 8x Hitachi 2TB Disks, 4x Toshiba 3TB Disks, 1x SanDisk SDSSDP 64gb SSD for ZIL and Read Cache, 1x Crucial M4 64GB SSD for OS
RAM: 16GB UDIMM Kingston RAM
Add-in Cards: 2x IBM M1015 crossflashed/9211 IT mode, 2x Intel Dual Port Server NIC
Power Supply: Antec EarthWatts 650 Power Supply
Usage Profile: Storage, iSCSI Target, SMB Shares, MediaTomb
Pools:
I decided to go with mirrors with both of my pools. While the performance penalty of ZFS pools versus mirrors were negligible, they were noticeable. I decided to go with speed and the redundancy of mirrors.
Bonnie Benchmarks:
Comstar - iSCSI Config
Switch Config:
I am using a 16 port ZyXEL GS1510-16 smart switch configured with 5x VLANs with Jumbo Frames enabled.
napp-it config - Comstar:
- Created 4x additional VLANs and bound 4x NICs to their own VLAN with Jumbo Frames
- Created 4x Target Portals and assigned each their own IP form the NICs in the 4 new VLANs
- Created 4x iSCSI targets and added them as members to a Target Group
- Created a 2TB Volume on the iSCSI Pool and imported it as a Logical Unit in Comstar
- Created a View for the Target Group in Comstar
Server Config:
Build’s Name: Zeus
Operating System/Storage Platform: Windows Server 2008 R2 w/ MPIO iSCSI
CPU: Xeon E3-1230 CPU
Motherboard: Tyan S5510
Chassis: iStar 4U Chassis
Drives: 4x Western Digital RED 1TB Disks, 4x Western Digital RE3 512GB Disks, 1x SanDisk SDSSDP 64gb SSD for OS
RAM: 32GB UDIMM Crucial RAM
Add-in Cards: 3Ware 9650SE RAID Controller, 2x Intel Dual Port Server NICs
Power Supply: Raidmax 530SS 530 Watt Power Supply
Other Goodies: 2x Supermicro 5x Bay Hot-Swap Cages
Usage Profile: Hyper-V
iSCSI Initiator - MPIO Config:
- Configured the 4x NICs with static IP Addresses to their respected VLANs
- Added the Multipath I/O Feature
- Added the 4x Target Portal IP Addresses in the Discovery Tab of the iSCSI Initiator
- Connected the Inactive Targets while enabling Mulit-Path
- Configured the Disk for Round Robin the Devices Tab for the iSCSI Initiator
- Initialized the presented iSCSI Target in the Windows Disk Management Tool
- Checked the Add Support for iSCSI devices in the MultipathIO Configuration Tool and rebooted the server
- Verified the Sessions on the Discovered Targets from the Target Properties button in the iSCSI Initiator Tool
The Test:
To verify that MPIO was working properly, I copied an Exported VM that was roughly 40GB to the new iSCSI disk. The average transfer rate was roughly 390MB a second once the cache was leveled out on the napp-it box.
Success!
Last edited: