OmniOS build "ashift", how to test ashift for SSDs and which is better 512N or 4KN HDDs

[Jon S]

Hello,

For my new micro server, I am purchasing a SuperServer 5028D-TN4T with with X10SDV-TLN4F mobo (D-1541 CPU) in which I plan to do a lot of system consolidation and virtualization in my small office lab. Yes I know the 10G ethernet ports are not supported in OmniOS (yet?) but I can live with the two (2) 1GB ethernet ports.

In this system I am putting in 2 x 4TB HDDs I am thinking of Seagate Constellation ES.3 either ST4000NM0023 ( 512N /sector) or ST4000NM0004 (4KN / sector). I also plan to put in 2.5” Samsung PM863 1.92 TB SSDs MZ-7LM1T9E. All drives are SATA 6Gb/s since that's what the X10SDV-TLN4F mobo supports.

Question #1 For the spinning rust (Seagate HDDs) which “is best” for this system 512N or 4KN ? why (future expansion, etc.)?

Question #2 For the SSDs (PM863) which “is correct” for this system 512N, 4KN, 8KN or 16KN? How do you test to determine the correct ashift value for SSDs ? Heck I would even use CentOS or Windows just to get the right sd.conf settings for optimal alignment.

Notes:
1. Seems like SSD lie about their “true” sizes e.g. Hardware - OpenZFS
2. From zfs/zpool_vdev.c at master · zfsonlinux/zfs · GitHub it does appear that a sister product MZ7WD960 has a 8KN size.

 

[gea]

For disks, never use any 512n disks.
512e and 4 k disks are treated the same by ZFS

For SSDs, I would just use the default unless you need some optimasion for a very special workload.

btw.
I have seen discussions at Illumos dev that Intel X550 (10G) and Intel X710 (40G/ 4 x 10G or 2 x 40G/ 8 x 10G) driver are on development and on the way.

 

[Jon S]

Thanks Gea,

Your right about the 10G woprk <daleg_> (on #illumos IRC on May 12, 2016) specifically indicated to me he has a 5028D-TN4T with a D-1541 System-on-Chip as a test for getting the 10Gb interface supported under ixgbe.

Also <rmustacc> (on #illumos IRC on May 12, 2016) also hinted to me that the NVMe won't quite work, but 'we' can probably help make it work, depending on the amount of work you want to do. I am planning on buying a "cheapo" NVMe M.2 to work on this capability myself, as in the past the illumos community at large has been great in helping me with non-compatible hardware hiccups and development.

I can't wait to get my hands on my 5028D-TN4T (shipping now "plug for wiredzone.com"), yes I have a dozen+ Sun servers, IBM servers, and Supermicro at the office lab - but this 6"x6" motherboard seems awesome (and more so once the 10G and NVMe kinks for illumos distros are worked out). My goal is to get rid of everyone of them keeping only my pFsense firewall and one Sparc based system.

On another note, Gea - say I spin up "napp-it" on OmniOS can I still use the system a a general purpose server. In my case I want to have about a dozen to two dozen light weight zones (bastion, honeypot, dns, web01, web02, wordprs01, prod01, prod02, prod03, dev01, dev02, ... dev09). of course the "dev##" boxes won't all be booted at the same time. As long as I can continue to use my box as a general purpose server with napp-it installed I would be quite happy? What's your take on this am I crazy - should I dedicate a third box for "napp-it"?

I am starting to use SmartOS a lot (maybe a second 5028D-TN4T) I was curious if you have had any progress with this unique distro. I read napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana, Solaris and Linux : Downloads and from my research I also came across this Is SmartOS suitable for a file server - SmartOS Documentation - SmartOS Wiki

I am thinking this might be a "fun" project to make napp-it (Samba, NFS, iSCSI, CIFS) work on stock SmartOS USB sticks and a good chance to really learn more a lot more about the nuances of SmartOS.

 

[cperalt1]

SmartOS is great. The learning curve is to learn that absolutely nothing is persistent except for the bits that you load at boot from /opt , /opt/custom/smf . I have 3 old desktops serving iscsi this way. For managing SmartOS nodes there is SmartDataCenter as well as projectFIFO.

 

[gea]

Napp-it does only global system settings to simplify the use of console commands. You can even disable napp-it and continue with console commands. The only known incompatibility when using napp-it as a general use server (while napp-it comes with a whole amp stack, ftp, tftp, afp, rsyncd, owncloud, dlna etc) are some Perl related packages thay may interfere with my setup and some pkgin packages from Joyent if you have added add-ons from napp-it and you need a pkgsrc package from a different release.

I have tried SmartOS once as a possible option but it seems quite complicated to use it as a filer compared to OmniOS or OpenIndiana so this may be a fallback option only.

btw
You do not need SAMBA. The kernelbased and multithreaded SMB server from Sun is now zone aware on any Illumos.