what's your ZFS flavor?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
OmniOS was a up to date Illumos distribution with full system support, stables and long term stables. The problem is/was that there were not enough paying customers.
There are discussions about a continuation of the current OmniOS lts.

Beside that the main alternative is OpenIndiana, the successor of OpenSolaris. This is more or less ongoing pure Illumos with an additional GUI (Mate 1.18). It comes in a minimal, a text (similar to OmniOS) and the GUI option.

SmartOS (Joyent, owned by Samsung), another Illumos distribution comes with superiour VM capabilities as it supports Solaris zones, LX container, KVM and Docker. It runs from RAM and USB stick. Due to some limited global zones features (due the running from RAM approach) it requires some work to act as a storage appliance like save/restore users and settings. But it has the potential of beeing the next killer appliance for storage and VM.

The former main OmniTi software engineer, responsible for OmniOS is now at SmartOS
 

Davewolfs

Active Member
Aug 6, 2015
339
32
28
Would running SmartOS on ESXI make things any easier? I guess I'll stick with the LTS release for now and pray that I don't hit any bugs.
 

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
SmartOS would be a replacement of ESXi and OmniOS
but currently OmniOS is much better suited for storage
 

GaryM

New Member
Feb 27, 2013
8
1
3
Belleville, ON
For those who were on OmniOS curious to know what you plan to do :) FreeNAS can't seem to get off of ver. 9. That leaves Linux or FreeBSD.
Many of us are in the same boat.
FreeNAS 11 looks good but it is bloated with things I don't need or want. I want a secure storage system, with the ability to handle disk replacements without fault or drama.
I am trying out OI minimal with napp-it as the GUI. I know I can identify and reliably replace a disk, failed or not with napp-it.
As long as OI keeps the security updates coming, it looks like a good eventual replacement for OmniOS once this last OmniOS gets long in the tooth and if we don't see it come back to life.
 

CJRoss

Member
May 31, 2017
91
6
8
Many of us are in the same boat.
FreeNAS 11 looks good but it is bloated with things I don't need or want. I want a secure storage system, with the ability to handle disk replacements without fault or drama.
Can you elaborate on these bloated things?
 

GaryM

New Member
Feb 27, 2013
8
1
3
Belleville, ON
Can you elaborate on these bloated things?
Bloated to me might not be bloated for you. You might want jails, docker, VMs and all the apps that go with it. The more stuff that gets thrown in adds complications that could affect the operation of the NAS software, this is advertised as a NAS system after all.

What suffers when these extras are added, are the things needed to make HD handling faultless, optimize throughput etc.. For e.g. ID the drive, by menu, that you want to replace, without first having to make labels to attach to the drive tray. They have lights to ID drives. Menu driven ID and replace would be nice. It should be possible to easily and reliably replace a drive using the menu system. Freenas is promoted as a menu driven interface. Someone told me to put in a feature request but these things are basic storage things, not features. If I have to ask for or tell them what they should have in FreeNAS, they aren't doing their jobs.

When software developers focus on features to attract more users they often forget their roots and doing something as best they can, becomes, it's good enough it's not that important.
When I want a VM I open one in ESXi, nothing will interfere with my running NAS and all my NAS software run in VMs under ESXi.
I have found OI minimal + napp-it an almost ideal NAS system. Faster than FreeNAS and It doesn't have VMs, Jails, docker or extra apps.
Being able to remove menu items would be nice, you do need to be careful of bloat. ;)
 
  • Like
Reactions: Nugget

Fritz

Well-Known Member
Apr 6, 2015
3,370
1,375
113
69
I have 2 FreeNAS boxes. Both are used for storage only. I rarely touch them. Disk failures/replacements couldn't be easier. I've had 3 failures in the last 2 years and all were easy peasy to fix. I also moved one to another box and that too was easy. I backed up the config, did a fresh install of FreeNAS, restored the config file and that was it. Could not have been easier. Ease of recovering from a disaster is mucho important to me. Hard drives fail, other parts fail too.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
Bloated to me might not be bloated for you. You might want jails, docker, VMs and all the apps that go with it. The more stuff that gets thrown in adds complications that could affect the operation of the NAS software, this is advertised as a NAS system after all.

What suffers when these extras are added, are the things needed to make HD handling faultless, optimize throughput etc.. For e.g. ID the drive, by menu, that you want to replace, without first having to make labels to attach to the drive tray. They have lights to ID drives. Menu driven ID and replace would be nice. It should be possible to easily and reliably replace a drive using the menu system. Freenas is promoted as a menu driven interface. Someone told me to put in a feature request but these things are basic storage things, not features. If I have to ask for or tell them what they should have in FreeNAS, they aren't doing their jobs.

When software developers focus on features to attract more users they often forget their roots and doing something as best they can, becomes, it's good enough it's not that important.
When I want a VM I open one in ESXi, nothing will interfere with my running NAS and all my NAS software run in VMs under ESXi.
I have found OI minimal + napp-it an almost ideal NAS system. Faster than FreeNAS and It doesn't have VMs, Jails, docker or extra apps.
Being able to remove menu items would be nice, you do need to be careful of bloat. ;)
I'm curious how this would work in a data-center/business environment where you likely can't get on WiFi or cellular networks from within feet of the server, likely can't hear anyone if you could make a call... so do you set the failed drive to flash from your lab/workstation take a walk to the data center see the flashing light, remove that and replace it, and then go back to your workstation and disable the flashing light?
 

GaryM

New Member
Feb 27, 2013
8
1
3
Belleville, ON
I'm curious how this would work in a data-center/business environment where you likely can't get on WiFi or cellular networks from within feet of the server, likely can't hear anyone if you could make a call... so do you set the failed drive to flash from your lab/workstation take a walk to the data center see the flashing light, remove that and replace it, and then go back to your workstation and disable the flashing light?
You are seeing the value of the blinking light. The light is just the drive activity light. The pro version of napp-it lets you build a drive map which would be even easier to use in a data center. HPE, Dell Lenovo etc have lights on the drives to tell you which are bad, off or failed. IF the physical drives don't have lights, at least the management software shows you a graphic of the drive housing (map) and you know which drive to replace.

What I do is ID the drive via menu or if you don't have a menu choice ID via the dd command. Drive activity light on my racks turn steady red versus the normal flickering. Setting the drive offline should turn off the light. I unlatch the drive but not eject it, set it to offline, eject and replace it with a new drive, select replace via menu, done. There is no concern that I have pulled the wrong drive. If you are at a distance from the drive you could have someone else unlatch and pull the drive from the tray with the solid red light and replace it and you can do the required menu manipulations.
Others have said how easy it is for them to change a chosen drive but I have not seen evidence of that. It doesn't matter, I have found a free system that has worked for me for at least 5 years and I will stick with what is tried and true in my case.
My FreeNAS VM for now has been deleted. Enjoy your computing.
 

fasting

New Member
Jan 24, 2017
3
4
3
37
On the client side I prefer Debian + ZoL, mainly because I've been using Debian on my desktop for a long time.

However, on the server side I prefer plain FreeBSD on bare metal. I have been using OpenIndiana/OmniOS and Debian with ZFS on servers as well, e.g. for KVM hosting and iSCSI targets, but for some reason FreeBSD personally feels more correct.

At home I therefore run FreeBSD 11-RELEASE using geli to encrypt all my drives. Samba and NFS are configured on the for all NAS-oriented stuff, and bhyve for my VMs for additional applications and services. I'm comfortably enough with a terminal and not needing a GUI (web or otherwise). Although I understand why (probably) most would prefer one, e.g. FreeNAS or napp-it (keep up the good work @gea). The easy way to get an overview of the server's health, resources and services is really neat. For just this purpose I wrote a script just to present some basics in a Web GUI, but not for any configuration. All that is being done by connecting to it with SSH.

To summarize: I wouldn't say any specific OS is the perfect one for ZFS. They all have their pros and cons, most of them already discussed in this thread.
 
  • Like
Reactions: EluRex

CJRoss

Member
May 31, 2017
91
6
8
Bloated to me might not be bloated for you. You might want jails, docker, VMs and all the apps that go with it.
Agreed, hence why I asked.

The more stuff that gets thrown in adds complications that could affect the operation of the NAS software, this is advertised as a NAS system after all.

What suffers when these extras are added, are the things needed to make HD handling faultless, optimize throughput etc.. For e.g. ID the drive, by menu, that you want to replace, without first having to make labels to attach to the drive tray. They have lights to ID drives. Menu driven ID and replace would be nice. It should be possible to easily and reliably replace a drive using the menu system. Freenas is promoted as a menu driven interface. Someone told me to put in a feature request but these things are basic storage things, not features. If I have to ask for or tell them what they should have in FreeNAS, they aren't doing their jobs.

When software developers focus on features to attract more users they often forget their roots and doing something as best they can, becomes, it's good enough it's not that important.
When I want a VM I open one in ESXi, nothing will interfere with my running NAS and all my NAS software run in VMs under ESXi.
I have found OI minimal + napp-it an almost ideal NAS system. Faster than FreeNAS and It doesn't have VMs, Jails, docker or extra apps.
Being able to remove menu items would be nice, you do need to be careful of bloat. ;)
I can see where you're coming from but I don't think FreeNAS has added that much bloat between 9 and 11. And they are still focused on the core of keeping your data safe.
 

realtomatoes

Active Member
Oct 3, 2016
251
32
28
44
i recall working with openfiler and freeenas back in the days for the lab, long before nas4free was a thing.
for production, before virtualization, it was always windows file servers for remote sites and emc for core sites.
clearly, i'm no linux/unix guy, so when it was time to setup zfs, i opted to go with freenas because i didn't want to have to handle the OS. lol
as for the features they've been adding onto freenas, never used them. just zfs for my smb shares and nfs for my ESXis.
 

talsit

Member
Aug 8, 2013
112
20
18
I've played with a few distros and was a long time Unraid user.

My current fav is ubuntu with zfs installed from the repository. This is home use only so I'm not too worried about lights and drive maps.

I'm only managing about 30 drives, when I put one in a drive cage I print a label with part of the SN and attach it to the handle. I have one 8 drive raidz1 that I tried using docker on but it just became too painful. I wanted more stability out of my home NAS.

I got a good deal on another server, installed ESXi 6.5, installed a LSI 2116-4e, attached a Se2016 JBOD enclosure and passed it through to an ubuntu vm. I passed through an onboard 10Gbe and 1GBe NIC (10GBe attaches to the first server for a full mirror) built two raidz2 arrays and haven't looked back.

It is easy to maintain and has been rock solid. The bare metal server is a dual E5-2660 with 100Gb of RAM (way overkill for an 8x3TB raidz1), the ESXi is a dual E5-2660 with 64Gb of RAM, the ubuntu VM for ZFS has 2vCPU and 8Gb. The disk are limited to SATA1 speeds but it has zero impact on my use case.

Currently I am playing with using the ZFS disks as primary storage for other VMs on the ESXi server, it works well and allows me to easily keep everything backed up (ESXi ZFS rsyncs to stand alone ZFS nightly, stand alone ZFS is 100% replicated to CrashPlan, really important stuff is also synced to AWS Glacier upon change).

The excess CPU cycles on the stand alone are going to Boinc and making Gridcoin (about $30/month worth right now).
 

sas2000

New Member
Dec 16, 2017
2
0
1
58
Oracle Solaris is yet the fastest ZFS server
@gea do you think it's still the case? I need to setup a couple of server to offer iscsi luns, do you think I should consider solaris to get better performances over say omnios?
 

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
Sequential resilvering on Solaris is way faster than resilvering on Open-ZFS. On my last tests 10G/40G performance on NFS/SMB was also 10-30% faster than on OmniOS. I have not compared Comstar (FC/iSCSI) so the question is how much of the advantage is related to genuine ZFS v.37 and how much is related to driver quality.

But while Oracle has reduced Solaris development, there is much more Open-ZFS development now (not only on the free Solaris fork Illumos but also BSD and ZoL) so I am not sure how long Oracle can keep its advantages.
 
Last edited:

whitey

Moderator
Jun 30, 2014
2,766
868
113
41
I'm curious how this would work in a data-center/business environment where you likely can't get on WiFi or cellular networks from within feet of the server, likely can't hear anyone if you could make a call... so do you set the failed drive to flash from your lab/workstation take a walk to the data center see the flashing light, remove that and replace it, and then go back to your workstation and disable the flashing light?
Typically in a datacenter you have an out of band mgmt network or vlan that is either heavily firewalled or just a stub vlan (non-routed) that you could easily plug a laptop/crash cart/IP KVM access into an apopropriate switchport set for mgmt vlan access to see the stg mgmt interface if your into 'blinky lights' I suppose :-D
 

sas2000

New Member
Dec 16, 2017
2
0
1
58
On my last tests 10G/40G performance on NFS/SMB was also 10-30% faster than on OmniOS.
This is very interesting, NFS is the other way I could consume the storage. Do you use a set of common test scripts to compare performances between platforms?

Thanks!