For the last couple of weeks I've been building and configuring my all-in-one ESXi hypervisor server. It is nothing huge or fancy, just enough for my lab needs and home network. Here are the specs:
Hardware
The hypervisor is running a couple of Windows Server 2012 R2 VMs (i.e. DCs, Plex, etc..) but also my new NAS VM which is Oracle Solaris 11.1 with Napp-It installed. My IBM M1015 card is passed through to the NAS VM and has 4 x 3TB WD Reds connected to it. The HDDs are configured as a RAIDZ pool.
My storage strategy is as follows.. All important and critical data is stored on it's own ZFS filesystem with SMB sharing turned on. That is to be able to access it from all my Windows based machines. The obvious reason for having the data stored on ZFS is to take advantage of RAIDZ parity and to be able to use snapshots. My non-critical data is my personal movie collection that I have ripped and is being shared via Plex Media Server. At first, all the movies were also stored on a ZFS filesystem with SMB sharing turned on. I then mapped the share on my PMS server. However I found this solution a bit sluggish. Transfer speeds were just OK. Since I've setup LACP and jumbo packets I figured the speeds should be higher. So i started experimenting with Comstar iSCSI. The downside of this being of course that I had to create a NTFS volume of the iSCSI block device, so the Windows-based Plex Media Server could access my movies. So no more snapshots of my movie collection. I however figured that this data is not critical. I could loose it and have to re-rip my movies, even though this is a pain in the b**t.
It is suggested to use a thin provisioned LU (file) in Napp-It as block device and pass it to a server via iSCSI. This is what I did first. Speeds got better. The speeds according to Windows when copying a 7GB BluRay-rip (.mkv) was at 102 MB/s. While I was moving all my movies to the new block device via iSCSI I was reading Oracles official documentation on creating and configuring iSCSI. In all the examples and other various articles on the subject they were using volume LUs. So I thought I'd switch from file LU to volume LU. Doing so resulted in lower transfer speeds. I was now getting around 40-50 MB/s when copying the same movie. Very often the transfers even stalled.
I would therefor like to ask the community if anyone else is using similar setup as I do and what the preference is? Any pros/cons for using volume LUs instead of thin prov. file LUs and most importantly why did I receive a speed drop when using volume LUs? Any suggestions to my storage strategy? How would you set it up for my purpose? Any feedback and/or suggestions is always greatly appreciated, cheers!
Hardware
- CPU: Intel Xeon E3-1230v3
- Motherboard: SuperMicro X10SLM-+F
- RAM: 4 x Kingston 8GB unbuffered ECC
- HBA: IBM ServeRAID M1015
- PSU: Seasonic Platinum 80 Plus Modular Fanless 4600W PSU
- HDD (SSD): 1 x Samsung 840 Pro Series 512GB (at the moment used as local VM datastore)
- HDD: 4 x Western Digital Red 3TB
- Cassi: Fractal Design DEFINE Mini
The hypervisor is running a couple of Windows Server 2012 R2 VMs (i.e. DCs, Plex, etc..) but also my new NAS VM which is Oracle Solaris 11.1 with Napp-It installed. My IBM M1015 card is passed through to the NAS VM and has 4 x 3TB WD Reds connected to it. The HDDs are configured as a RAIDZ pool.
My storage strategy is as follows.. All important and critical data is stored on it's own ZFS filesystem with SMB sharing turned on. That is to be able to access it from all my Windows based machines. The obvious reason for having the data stored on ZFS is to take advantage of RAIDZ parity and to be able to use snapshots. My non-critical data is my personal movie collection that I have ripped and is being shared via Plex Media Server. At first, all the movies were also stored on a ZFS filesystem with SMB sharing turned on. I then mapped the share on my PMS server. However I found this solution a bit sluggish. Transfer speeds were just OK. Since I've setup LACP and jumbo packets I figured the speeds should be higher. So i started experimenting with Comstar iSCSI. The downside of this being of course that I had to create a NTFS volume of the iSCSI block device, so the Windows-based Plex Media Server could access my movies. So no more snapshots of my movie collection. I however figured that this data is not critical. I could loose it and have to re-rip my movies, even though this is a pain in the b**t.
It is suggested to use a thin provisioned LU (file) in Napp-It as block device and pass it to a server via iSCSI. This is what I did first. Speeds got better. The speeds according to Windows when copying a 7GB BluRay-rip (.mkv) was at 102 MB/s. While I was moving all my movies to the new block device via iSCSI I was reading Oracles official documentation on creating and configuring iSCSI. In all the examples and other various articles on the subject they were using volume LUs. So I thought I'd switch from file LU to volume LU. Doing so resulted in lower transfer speeds. I was now getting around 40-50 MB/s when copying the same movie. Very often the transfers even stalled.
I would therefor like to ask the community if anyone else is using similar setup as I do and what the preference is? Any pros/cons for using volume LUs instead of thin prov. file LUs and most importantly why did I receive a speed drop when using volume LUs? Any suggestions to my storage strategy? How would you set it up for my purpose? Any feedback and/or suggestions is always greatly appreciated, cheers!