What do you think about NexentaStor?

voodooFX

Active Member
Jan 26, 2014
243
50
28
I need an operating system for my storage server, it's a simple single node server with 8xHDD and 2xSSD.
I need iSCSI and NFS for an esxi server (SSD and HDD) and NFS+SMB for storing "NAS stuff", so video/photo/docs etc.

I have plenty of ethernet interfaces (1G) for iSCSI multipath.

At the moment I'm doing all I need "manually" with CenOS7.1 but I wondering if something like NexentaStor (Community) will work fine?
Any experience with this OS? Something I should be aware, careful, informed of and so on?

Thanks
 
  • Like
Reactions: T_Minus

EluRex

Active Member
Apr 28, 2015
209
70
28
Los Angeles, CA
NexentaStor (Community) has license limit on RAW HD space (which does not work out for me). If you want ZFS and just iscsi over ethernet, you may look into FreeNAS (freebsd based) or ZFS on Linux (ZoL), they offer zfs on centos and also ubuntu as well.
 
  • Like
Reactions: T_Minus

wildchild

Active Member
Feb 4, 2014
394
57
28
i switched from nexentastor to omnios and nappit...
nexenta is usually a rather old illumos with lots of restrictions, including a max 18 tb raw limit.
i find the omnios and nappit does the same, or even better of a job, has a much better community.
only reason for ever thinking nexenta is VAAI, which nexenta still hasnt released to the community as the have promissed now for over 3 years.
 

voodooFX

Active Member
Jan 26, 2014
243
50
28
18TB limit is not an issue for me
what I'm looking for is stability and central/easy management (and of course a platform with an active development)

how do you evaluate FreeNAS compared to NexentaStore? Better? Equal? Below? (why?)
 

wildchild

Active Member
Feb 4, 2014
394
57
28
i'd still be more inclined to use a solaris derivative.
zfs,comstar and cifs , ad intergration native. proven stability list goes on and on.
only downside would be hardware support.
if you have enterprise grade hardware (intel lsi ) i think it would be a no brainer (for me anyway)
 

neo

Well-Known Member
Mar 18, 2015
672
361
63
I'm curious why you are aiming to use something so esoteric as NexentaStor rather then a FreeNAS, omnios, or nappit. Which have a large community and the support to go along with it.

Not trying to sound cynical, just curious.
 
  • Like
Reactions: CreoleLakerFan

EluRex

Active Member
Apr 28, 2015
209
70
28
Los Angeles, CA
omnios & nappit __ freebsd & freenas

one provides foundation platform and one provide front end gui and task specific driven package. Both of them are great. it just that my mellanox connectx3 vpi card is not supported by omnios

18 tb raw limit by Nexenta is easily reached... your configuration has 8 hhd, which means each disk needs to be less than 2tb in order to avoid that limitation and this gives many people a cold feet and turn away from nexenta and thus less and less people knows about it and can support on it.

here is othere ppl compare nappit vs freenas FreeNAS vs. OmniOS / Napp-It - b3n.org
no doubt... omnios wins
 

voodooFX

Active Member
Jan 26, 2014
243
50
28
I'm curious why you are aiming to use something so esoteric as NexentaStor rather then a FreeNAS, omnios, or nappit. Which have a large community and the support to go along with it.

Not trying to sound cynical, just curious.
The answer is easy ;)
First of all, even if I know of they're existence since many years I never really used one of them (I mean nexenta, FreeNAS and OmniOS)
As second point, my needs (I think) are really the base of what these systems are made for (iSCSI/NFS/SMB over RaidZ pools) so as long as I have a good admin manual and a stable platform I don't care too much if there is a big or small community out there because I don't plan to do/need nothing more.

So why nexenta? simply because the documentation is great, it looks more serious/professional than freenas and less complicated than OmniOS+NappIt
Why you think it is "esoteric"?

Anyway I opened this thread exactly to ask if there is a reason I should not go for Nexenta :D
 

cperalt1

Active Member
Feb 23, 2015
178
51
28
39
For ESXi. If you need VAAI use nexenta since that has not made it into upstream Illumos yet.
 

voodooFX

Active Member
Jan 26, 2014
243
50
28
yes VAAI will be great since it will be used also as esxi storage..

question: is nexenta installation supported (recommended?) on USB sticks?
 

CreoleLakerFan

Active Member
Oct 29, 2013
477
176
43
So why nexenta? simply because the documentation is great, it looks more serious/professional than freenas and less complicated than OmniOS+NappIt
Why you think it is "esoteric"?
In my experience the documentation is "adequate", but the community support is lacking. If you're attempting something not covered in the documentation, say serving block storage over infiniband (as I was), then you will find the small community a problem.

I was never able to get it working under Nexenta - granted I didn't invest a lot of time into it. By contrast, it "mostly" worked right out of the box with OmniOS+Napp-It and I was able to find plenty of community assistance to complete the configuration.

VAAI is a handy feature that I test in my lab, but I didn't think it was worth the hassle and licensing limitations of Nexenta.
 
  • Like
Reactions: neo

CreoleLakerFan

Active Member
Oct 29, 2013
477
176
43
yes VAAI will be great since it will be used also as esxi storage..

question: is nexenta installation supported (recommended?) on USB sticks?
It isn't recommended, but it works. I got a mirrored installation working on USB3 sticks.
 

voodooFX

Active Member
Jan 26, 2014
243
50
28
Hi CreoleLakerFan.

Atm I'm using iSCSI (LIO targetcli) over IB (40G) on CentOS7: it's cool but since my SSDs mirror speed is about 500MB/s (max) I do not have a real advantage over a 4x1G multipath (active/active).

About the USB sticks: is 16G enough? How do you create a USB stick mirror? You install the OS on the first and then mirror it from the GUI?

Thanks
 

whitey

Moderator
Jun 30, 2014
2,770
866
113
38
i switched from nexentastor to omnios and nappit...
nexenta is usually a rather old illumos with lots of restrictions, including a max 18 tb raw limit.
i find the omnios and nappit does the same, or even better of a job, has a much better community.
only reason for ever thinking nexenta is VAAI, which nexenta still hasnt released to the community as the have promissed now for over 3 years.
BING, you hit most everything RIGHT on the head. I hear FreeNAS and Nexenta have VAAI support but have not tried either yet. Another guy who entrusts all his data to OmniOS here, played w/ napp-it but want the version that I can zfs send/recv BETWEEN ZFS arrays and i think this requires a license. I need to stop being a cheap arse and just pay it up to Gea...it's absolutely worth it from what I hear and he deserves it hand over fist w/out a doubt. I think it's just a few hundred bucks for home/lab use if memory serves me correct.

I'm just 'old-school' I guess. Banging on a shell is fun for me, must be sick.
 

whitey

Moderator
Jun 30, 2014
2,770
866
113
38
In my experience the documentation is "adequate", but the community support is lacking. If you're attempting something not covered in the documentation, say serving block storage over infiniband (as I was), then you will find the small community a problem.

I was never able to get it working under Nexenta - granted I didn't invest a lot of time into it. By contrast, it "mostly" worked right out of the box with OmniOS+Napp-It and I was able to find plenty of community assistance to complete the configuration.

VAAI is a handy feature that I test in my lab, but I didn't think it was worth the hassle and licensing limitations of Nexenta.
I hear community/support was lackluster as well, had a buddy that had HA arrays go down as well from 'NOT so well developed/implemented HA methodology/architecture'. YMMV, At least they finally delivered Nexenta 4.0 right and moved into Illumos codebase.
 

whitey

Moderator
Jun 30, 2014
2,770
866
113
38
All of my comments being said and keeping VAAI support for ZFS based distro's in mind why does OmniOS not have it and when will they? If Nexenta/FreeNAS are supporting a few of the feature sets I figure the API/code is open/provided from VMware to storage array vendors, maybe it's a time/$/development hours/priorities thing for now...anyone have any in's/contacts over at OmniTI?
 

TuxDude

Well-Known Member
Sep 17, 2011
615
336
63
All of my comments being said and keeping VAAI support for ZFS based distro's in mind why does OmniOS not have it and when will they? If Nexenta/FreeNAS are supporting a few of the feature sets I figure the API/code is open/provided from VMware to storage array vendors, maybe it's a time/$/development hours/priorities thing for now...anyone have any in's/contacts over at OmniTI?
There's nothing that VMware can do to help with this - VAAI is not really a VMware API anymore (except for VAAI for NFS protocol). Current VAAI and also MS's ODX are just fancy names for using standard T10 SCSI commands to offload work to the array, and any block storage that supports those SCSI commands will show up as supporting hardware acceleration in VMware.
 
  • Like
Reactions: whitey

cperalt1

Active Member
Feb 23, 2015
178
51
28
39
All of my comments being said and keeping VAAI support for ZFS based distro's in mind why does OmniOS not have it and when will they? If Nexenta/FreeNAS are supporting a few of the feature sets I figure the API/code is open/provided from VMware to storage array vendors, maybe it's a time/$/development hours/priorities thing for now...anyone have any in's/contacts over at OmniTI?
It is a time vs priority thing. Nexenta did publish the VAAI code to Illumos/Nexenta and work is being done to get it upstream to Illumos/gate. Once it is in Illumos/gate then Omni will have VAAI.

[OmniOS-discuss] VAAI Testing
[OmniOS-discuss] Status of VAAI