Moving from Solaris 11.3 to OmniOS (or OpenIndiana): anything important to know?

TheBloke

Active Member
Feb 23, 2017
200
40
28
40
Brighton, UK
Hi all

I am currently running Solaris 11.3 on my home server. I went with Solaris 11 because I was a Solaris 10 + OpenSolaris admin for many years, so it is what I know. Plus I have a free license for Solaris 11, so licensing/patches are not a concern for me.

But now I am thinking I made a mistake, and I would much rather be with the open source community, where new development is happening. I had already been thinking that for a little while, and then when I saw that LX zones are back in OmniOS, it made me decide I really should move. I used Linux 2.4 zones in Solaris 10 a lot and would love to play with them again.

Plus I keep forgetting that Illumos-based distros also have KVM - I am used to using VirtualBox for VMs on Solaris, but KVM looks like it will be nicer to manage.

I have one final opportunity to easily move, as I just bought a new server/chassis. So I can easily install OmniOS in parallel on the new machine and check all is OK, and then move across my pool. I know that I will need to re-create my ZFS pool as unfortunately when I re-created it last week I made it as version 37. But I have some spare drives, and the array currently only contains 5TB of data, so it will be possible to make a backup on some spare drives and then re-create the full pool on OmniOS and fairly quickly migrate the data across.

My question is: is there anything I should be aware of migrating from Solaris 11 to OmniOS? Besides the ZFS incompatibility that I know about.

I mean things like: are there major OS features that won't be available, or software that is not available? I assume OmniOS can run 99.9% of Solaris 10/11 software, but maybe there are notable exceptions? Or popular software that is in the Solaris 11 repo but not OmniOS? (Although there's always OpenCSW). Is it possible to install and use Sun Studio without problems?

It is only a home server so I am not using a very complex setup. But I like to experiment with advanced features, even if they are not vital.

I already know that OmniOS doesn't really have a desktop component, which I don't mind too much. In the past I did like having an X desktop in a zone, in case I wanted to run any graphical installer or similar. But that's not a big deal, I have an OSX install at home so I can always export DISPLAY. I did briefly think about OpenIndiana instead, because it still has a desktop, but it doesn't yet have LX zones and I get the feeling OmniOS is more regularly updated.

I don't really know what else to ask exactly, as I don't know what could be different. I have googled a fair bit, but I can't find much info about advantages/disadvantages, besides licensing/patching. Maybe that means there aren't really many differences, which would be great.

Any help/advice would be very much appreciated from anyone who knows both Solaris 11 and OmniOS well.
 
Last edited:

TheBloke

Active Member
Feb 23, 2017
200
40
28
40
Brighton, UK
OK thanks a lot gea, that's good to know.

I'm still undecided as to whether to make the switch. I do like the idea of LX zones and KMV. Then again, I do benefit from the faster resilvers you mention, and having read through all of the Oracle What's New documents from 11.1, 11.2 and 11.3, I see a few other features added in 11.x that I quite like, such as:
  1. zpool monitor and zfs diff -r (added in Solaris 11.3)
  2. share.smb and share.nfs (Solaris 11.1)
  3. Recursive zone packaging and live zone reconfig (Solaris 11.2)
  4. Not tried them yet, but SMF Stencils look cool (Solaris 11.2)
I am still in the process of rebuilding/upgrading my server. I finally got a proper server chassis/mobo etc, and am now using all LSI cards. I will probably do a test OmniOS install and try both in parallel a bit before I commit to making my final pool.

I suppose I could make my pool on v28 in Solaris 11 to keep my options open a bit longer, then I could flip back and forth between the two OS to test them both. Only about 10% of my current data is compressible, so it's not a huge deal to be stuck on an older version (I guess I would lose the faster resilvers in 11.x as well.) I can rewrite the compressible data later once I've settled on an OS and upgraded the pool to either v37 or v1000.
 

gea

Well-Known Member
Dec 31, 2010
2,502
842
113
DE
When you use Pool v28 you must care that you use ZFS v5. Oracle use a newer and incompatible v6.
Other problem is that v28 is from a 5 years ols OpenSolaris. You will not get newer Oracle features nor newer Open-ZFS features.
 
  • Like
Reactions: TheBloke

TheBloke

Active Member
Feb 23, 2017
200
40
28
40
Brighton, UK
When you use Pool v28 you must care that you use ZFS v5. Oracle use a newer and incompatible v6.
Other problem is that v28 is from a 5 years ols OpenSolaris. You will not get newer Oracle features nor newer Open-ZFS features.
Yeah, I would only be using a v28 pool temporarily so I can switch freely back and forth between Solaris 11.3 and OmniOS while doing some comparisons and tests. I don't want to run it long term - I want lz4 compression and the other newer features! :)
 

AveryFreeman

ESXi + ( ILLUMOS / ZFS ) = HAPPY
Mar 17, 2017
165
19
18
39
Near Seattle
averyfreeman.com
Yeah, I would only be using a v28 pool temporarily so I can switch freely back and forth between Solaris 11.3 and OmniOS while doing some comparisons and tests. I don't want to run it long term - I want lz4 compression and the other newer features! :)
Did you ever look into getting a support contract from Oracle? What does it cost for a regular Joe Shmoe with a server in his basement, anyway?
 

m4r1k

Member
Nov 4, 2016
52
6
8
31
Did you ever look into getting a support contract from Oracle? What does it cost for a regular Joe Shmoe with a server in his basement, anyway?
I cannot find anymore the official Oracle source.
Today it’s 1k€/$ per year for support and updates.
The thing is that Oracle has habit to change license cost at contract renewal. You know where you start but not at how much you’re gonna spent in a few years ... plus it’s oracle ...
 

AveryFreeman

ESXi + ( ILLUMOS / ZFS ) = HAPPY
Mar 17, 2017
165
19
18
39
Near Seattle
averyfreeman.com
I cannot find anymore the official Oracle source.
Today it’s 1k€/$ per year for support and updates.
The thing is that Oracle has habit to change license cost at contract renewal. You know where you start but not at how much you’re gonna spent in a few years ... plus it’s oracle ...
JTFC! That is insane. Why does Oracle have to suck so bad?

Can't you get a free license for use-only if you're a developer?
 

AveryFreeman

ESXi + ( ILLUMOS / ZFS ) = HAPPY
Mar 17, 2017
165
19
18
39
Near Seattle
averyfreeman.com
For noncommercial use (demo and development) you can download for free.
It requires only a registration, Oracle Solaris 11 Downloads | Installation from CD/DVD or USB
Oh, yeah I know, I've downloaded it a few times since the release of 11.0 - but one can't receive OS or driver updates without a support contract, right?

Isn't a support contract required for access to an SRU?

I was actually considering running Solaris 11.3 or 11.4 originally for Napp-It since benchmarks for ZFS were a bit better for ESXi datastores than OpenZFS, but the cost of a contract along with the lack of compatibility makes it a non-starter...
 

m4r1k

Member
Nov 4, 2016
52
6
8
31
JTFC! That is insane. Why does Oracle have to suck so bad?

Can't you get a free license for use-only if you're a developer?
Solaris is free for personal and development usage. Paying the subscription one gets access to updates and support.
Take this example, 11.3 had a broken SMB 2.1 implementation which has been fixed in one of the various RSU.
The version you can download today is the original 11.3 GA with all original bugs...
 

AveryFreeman

ESXi + ( ILLUMOS / ZFS ) = HAPPY
Mar 17, 2017
165
19
18
39
Near Seattle
averyfreeman.com
Solaris is free for personal and development usage. Paying the subscription one gets access to updates and support.
Take this example, 11.3 had a broken SMB 2.1 implementation which has been fixed in one of the various RSU.
The version you can download today is the original 11.3 GA with all original bugs...
Yep. Lame.
 

gea

Well-Known Member
Dec 31, 2010
2,502
842
113
DE
Solaris is the most feature rich and fastest ZFS server but it is a commercial datacenter OS. Luckily we have Illumos (and as a consequence of the fork Open-ZFS now on Free-BSD, Linux, OSX and propably Windows in future) with many of the Solaris features as OpenSource.
 

AveryFreeman

ESXi + ( ILLUMOS / ZFS ) = HAPPY
Mar 17, 2017
165
19
18
39
Near Seattle
averyfreeman.com
Solaris is the most feature rich and fastest ZFS server but it is a commercial datacenter OS. Luckily we have Illumos (and as a consequence of the fork Open-ZFS now on Free-BSD, Linux, OSX and propably Windows in future) with many of the Solaris features as OpenSource.
Yeah, I've really been enjoying ZFS on other platforms, but it seems rather hack-ish when compared to Solarish OS.

Unfortunately, the trade-off is lower compatibility, fewer OS features, practically non-existent dev community, and as the years go by the gap between, say, OmniOS and Linux (or even FreeBSD) gets more and more pronounced.

I still like it, though, because for the single use-case of being a storage server, OmniOS still seems the best. Just not sure how long that will last.

SmartOS seems like it's more actively developed, how's the prospect of running Napp-It on SmartOS for a storage server?

Is OmniOS making any sort of comeback now that there's a new community edition/developers?
 

gea

Well-Known Member
Dec 31, 2010
2,502
842
113
DE
Sun invented not only ZFS but also NFS, added their own kernelbased SMB server to Solaris and services like Comstar, an enterprise class FC/iSCSI stack and Crossbow (network virtualisation) among other groundbreaking inventions regarding security and service management. This makes Solaris a self sufficient operating system especially for storage needs. You do not need any 3rd party software. Everything is under full control of Oracle for Solaris or Illumos for the free forks. Regarding Illumos, alls firms work together to maintain these core features. SmartOS (Joyend/Samsung) or Delphix, OpenIndiana or OmniOS must not care about.

So the question may be, why are there Illumos distributions like NexentaStor, OmniOS, OpenIndiana or SmartOS to name a few? Why not just Illumos as a distribution like Solaris that is the only distribution from Oracle.

The correct answer is maybe that you may want more than these core services. OpenIndiana minimal for example is quite pure ongoing Illumos as a distribution. The main reason for OpenIndiana is that they are maintaining a GUI (mate) and a lot of programs (Desktop applications and services ex databases, webserver etc).

Nexenta for example added commercial support for software and hardware, HA, ESXi integration with certified vendors. SmartOS added cloud functionality (KVM, LX zones, Bhyve, Docker etc).

OmniOS has a strong focus on stability and support, The distribbutionis very munimalistic and covers mainly the core services, nothing else (a just enough ZFS storage OS). They additions they add are from SmartOS (LX zones, Bhyve). What makes OmniOS unique, for exampe over a quite similar OpenIndiana minimal/text are the stable and long term stable repositories, For every release they provide a freeze of the repository. No newer featurs added afterwards beside security fixes. This is why OmniOS is the premium distribution if you want storage and stability. They even continue the commercial support option from OmniTi.

OmniOS CE is not a comeback but a continuation of the Idea from OmniTi. They exactly followed theire release schedule without a break, see Release Schedule.

SmartOS as a replacement of OmniOS may be possible but as it is fully rambased with limited features in the global zone this would require some work. OpenIndiana is a 1;1 replacement of OmniOS without the dedicated stable/long term stable repositories.
 
  • Like
Reactions: TRACKER

tony&

New Member
Sep 12, 2018
5
1
3
Oh, yeah I know, I've downloaded it a few times since the release of 11.0 - but one can't receive OS or driver updates without a support contract, right?

Isn't a support contract required for access to an SRU?

I was actually considering running Solaris 11.3 or 11.4 originally for Napp-It since benchmarks for ZFS were a bit better for ESXi datastores than OpenZFS, but the cost of a contract along with the lack of compatibility makes it a non-starter...
I've asked Oracle through their email/chat functions for the possibility of a cheaper license for Solaris similar to their Oracle Linux counterpart ("Oracle Linux Network Support") which would only allow access to their IPS repository and I was told by that they would look it up and reach back to me. Not that I would expect a feedback from Oracle as an end user without a current support contract, but it was a couple of months back now... Certainly, Oracle aren't interested at all in the SMB segment since many years back and apparently they don't even attempt to sell hardware anymore to the enterprise segment - all focus is on their own SAAS/PAAS infrastructure.
During the 11.4 announcement I suggested in their social media channels that they should look into establish some kind of evaluation license but the response back was only "Interesting idea.".

I was considering Oracle Solaris for my new build in order to take advantage of the ZFS encryption and their polished networking stack, but instead I went for (at least for now) OmniOS CE bloody. What I like about OmniOS is that they retain on IPS and that they have implemented bhyve. Although, while my guests seem to be stable, I have some tuning to look into when it comes to my Linux guest encrypted zvol write performance (20-50MB/s). Happy user so far, although I would prefer ZFS built in encryption.
 

liv3010m

New Member
Jul 22, 2018
16
2
3
Buenos Aires, Argentina
I've asked Oracle through their email/chat functions for the possibility of a cheaper license for Solaris similar to their Oracle Linux counterpart ("Oracle Linux Network Support") which would only allow access to their IPS repository and I was told by that they would look it up and reach back to me.
Hi tony&, did you finally get some definitive answer from them?
 

AveryFreeman

ESXi + ( ILLUMOS / ZFS ) = HAPPY
Mar 17, 2017
165
19
18
39
Near Seattle
averyfreeman.com
If you don't have to pay for a ridiculous license then stick with Solaris would be my vote.
Hi gigatexal, exactly!, that's the reason why I'm asking. ;):)
Oracle requires support contracts to use Solaris to any real degree.

You can download and 'evaluate' Solaris for free, but once you want to update a package, or need a device driver, you have to have a support contract to access their package library.

It blows.

I use OmniOS.

SmartOS is also very nice and I think most likely the most actively developed 'Solarish' OS. Also made to integrate with pkgsrc as primary package manager, which is very, very nice. Only problem is, it's meant to run off a USB flash drive as a hypervisor appliance, so it can take some finagling to get it to act like a 'normal' OS (it's like comparing FreeNAS to FreeBSD).

OmniOS definitely the easiest to get into but older pkg system, packages not as current, etc. I like it fine. It's very, very stable.