Oracle Solaris 11.3 and Intel X552/X554 10GbE drivers

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tic226

Member
Feb 2, 2017
52
29
18
122
DE
I have a Supermicro X10SDV-2C-7TP4F [1], the onboard Pentium D1508 has integrated 10GbE (X552/X554 based) interfaces which i'm trying to use with Oracle Solaris 11.3 which didn't work so far, the interfaces don't show up in
Code:
dladm show-link
I'm new to Solaris and really could need a hand in finding a solution. So far i found out that the ixgbe drivers should work with the X552, OmniOS uses them and the interfaces show up there. I'm still a bit stumped by the lack of information i could find about the NIC, on a system with working drivers (linux, OmniOS, ESXi) they show up as 'X552/X554 based'; searching on Intel's website for X552 doesn't yield any useful results, just a plethora of different cards, Supermicro's block diagram of the mainboard in the manual only shows the 10GbE PHY. Searching the Solaris documentation from Oracle for articles about the ixgbe drivers didn't lead anywhere either. Apparently they exist and should be installed automagically, searching for related packages didn't yield any results.

How can i obtain the ixgbe drivers if those are the ones i need?
Is this related to (possbile) differences between the free version of Oracle Solaris (which i'm using) and the paid/licensed version?



[1]https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-2C-7TP4F.cfm
 

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
Driver development for Oracle Solaris is d0ne at Oracle while developments of the free Solaris forks is done at Illumos. Last year the ixgbe driver for Illumos (OmniOS, OI, SmartOS etc) was updated to support the new Intel 55x and 710 nics.

I suppose there is a new ixbge driver available at Oracle as well but Oracle offers updates for Solaris 11 only with a paid subscription.
You options are now:

With Oracle Solaris and ZFS v37
- I am not sure if Intel offers a Solaris driver but I doubt
- Ask Oracle if the driver is available after a subscription (1k$/year)
- Use an All-in-One setup with a virtualised SAN (where the nic is managed by ESXi, usefull if you want to virtualise more systems)
- wait for Solaris 11.next, due mid 2017

With Illumos and Open-ZFS v5000
- Use a OS from the Illumos platform (OmniOS 151022, OI 2017.04, SmartOS or nonfree commercial NexentaStor)
 

tic226

Member
Feb 2, 2017
52
29
18
122
DE
Driver development for Oracle Solaris is d0ne at Oracle while developments of the free Solaris forks is done at Illumos. Last year the ixgbe driver for Illumos (OmniOS, OI, SmartOS etc) was updated to support the new Intel 55x and 710 nics.

I suppose there is a new ixbge driver available at Oracle as well but Oracle offers updates for Solaris 11 only with a paid subscription.
You options are now:

With Oracle Solaris and ZFS v37
- I am not sure if Intel offers a Solaris driver but I doubt
- Ask Oracle if the driver is available after a subscription (1k$/year)
- Use an All-in-One setup with a virtualised SAN (where the nic is managed by ESXi, usefull if you want to virtualise more systems)
- wait for Solaris 11.next, due mid 2017

With Illumos and Open-ZFS v5000
- Use a OS from the Illumos platform (OmniOS 151022, OI 2017.04, SmartOS or nonfree commercial NexentaStor)
Hi!

thank you for explaining the driver issue, this clears up a lot for me!

The reason i went with Oracle Solaris was the ZFS version, i want to use the built-in encryption. I had a OmniOS (151022) with napp-it on top but zfs encryption doesn't work, zfs version is 28, that's the reason why i installed Oracle Solaris 11.3 yesterday as a test to see if encryption works there. Only to find out that the onboard 10GbE NICs don't work. Intel only offers linux drivers for their hardware, nothing for Solaris/SunOS based systems.
Frankly i don't want to virtualise with ESXi (which i tested yesterday as a last attempt), this host is used for storage only and i don't want to introduce a virtualisation layer just because the NICs don't work.
And i read about the lofi method, creating encrypted 'files' and using them as vdevs but that is more of a 'hack' than a real solution.
I'll ask Oracle support if there's a way to obtain the driver somehow. I assume OpenZFS v5000 doesn't support native encryption?
 

tic226

Member
Feb 2, 2017
52
29
18
122
DE
Not yet, but there is work at Datto to add native encryption based on Illumos
as this was nearly ready in OpenSolaris when Oracle closed the doors.

It is nearly ready for ZoL with pull requests for Illumos and OSX.
I am quite sure you will see it at BSD as well.
https://zfs.datto.com/slides/caputi.pdf
I'm thinking about staying with linux and finally try ZoL, after learning about the entrenched battlefield that is Solaris/SunOS/etc. and their obscure versioning/licensing i'm really put off of it. I'm beginning to understand why so many people seem to absolutely hate Oracle with a passion.
I would have loved to use napp-it though, i bought a license when i had it running on a (virtualised) Oracle Solaris test system. Never would i have thought that a missing driver (i apparently have to pay 1k for) would fail the whole project.

I'll have to re-evaluate my stance on a virtualised storage server or encrypted 'file-vdevs', i already have a fully functioning test setup with OmniOS + napp-it on the new hardware and it works fantastically. Would hate to scrap it...
 

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
Oracle Solaris is a commercial OS not OpenSource. The free version is only for evaluating and demoing. No production use allowed.. It is like a Windows or OSX server. You must pay to use. In my test it was always the fastest ZFS server and it is the only one that offers fast sequential resilvering and native encryption. The free version is updated every two years (next should be mid 2017), then with newer drivers and propably smb3.

Sequential resilvering and native encryption is currently not available on any Open-ZFS so you must wait there as well. And from my own experience, a Solarish very minimal install just work out of the box with perfect ZFS integration, native iSCSI/FC, NFS, SMB and network virtualisation - all maintained either by Oracle or Illumos, no third party bits involved. Update is only a pkg update and everything works, not the adventure game of Linux.
 

tic226

Member
Feb 2, 2017
52
29
18
122
DE
Still, pay-to-use is different than offering an outdated version (2 years by now, so why even bother) for a product which has been pronounced dead many moons ago. One can use a Windows or OSX server for 30/60/90 days for evaluation, with all updates/patches and all the bells and whistles advertised on the can. On the other side of the spectrum there is RHEL, SLES, etc.; Also non production, no support but more frequent releases than every 2 years. It's just a dickish move on Oracle's part.

Calling a linux solution an adventure game in light of this is a bold statement i must say; The only way to achieve a solution you described would be to fork out cash for a licensed Oracle Solaris version, fine for an enterprise approach but this is servethehome. So the only alternative to the enterprisey approach is to either use an outdated, dead OS lacking support for modern hardware to have all the advertised ZFS functionality or use an open source derivative where people play catch up with a (now) closed source OS's functionality (ZFS), with varying success and incomplete feature-set. This is a long way from 'everything works'.
The only thing even remotely adventurous in a linux approach is the ZFS part, thanks to Oracle apparently.
I read the 'iSCSI-NFS-SMB is so easy because it's integrated(tm)' argument very often but to be fair, it's not rocket science on a linux OS either.
And don't forget the mind-boggingly overcomplicated things on Solaris/SunOS, setting up DNS resolving comes to mind. Now that is an adventure, not so on linux, it really works 'out of the box' there.
 

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
You can only choose from whats on the table.

And this means Oracle Solaris with ZFS v37.
Many do not like the update/price policy of Oracle or some weird network and share settings
on current Solaris 11.3 but it is as it is.

The other option is Open-ZFS. Featurewise regarding the ZFS filesystem
this is quite identical with ZFS v5000 on BSD, Illumos, Linux or OSX,
each platform with its specific advantages.

Main advantage of Linux is the best of all hardware support due its marketshare.
 

tic226

Member
Feb 2, 2017
52
29
18
122
DE
You can only choose from whats on the table.

And this means Oracle Solaris with ZFS v37.
Many do not like the update/price policy of Oracle or some weird network and share settings
on current Solaris 11.3 but it is as it is.

The other option is Open-ZFS. Featurewise regarding the ZFS filesystem
this is quite identical with ZFS v5000 on BSD, Illumos, Linux or OSX,
each platform with its specific advantages.

Main advantage of Linux is the best of all hardware support due its marketshare.
OpenZFS's encryption feature seems to be just around the corner, i'd hesitate to be an early adopter of a filesystem though...

So i think i'll keep my virtualised (ESXi) test system with Oracle Solaris for the moment and keep fingers crossed that the impending release will add ixgbe driver support. What are your thoughts/experience with such a virtualised approach?
 

gea

Well-Known Member
Dec 31, 2010
3,140
1,182
113
DE
CPU wise, the performance degration of physical vs virtual is minimal.
As ESXi requires 1-2GB RAM you have some RAM less.

As you should use a dedicated HBA in pass-through mode,
you will get same disk performance with native Solaris drivers.

Main performance problem may be the vnic adapter as this is virtualized.
You must use the vmxnet3 one from vmware tools (and increase buffer size in vmxnet3 settings for 10G).

VMware-tools for Solaris were included in ESXi up to 5.5 or 6.
For current ESXi 6.5 its a separate download from vmware,