Oracle Solaris 11.4

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Gremlin190

New Member
Dec 14, 2018
4
0
1
Pass through of an LSI 2308 with ESXI 6.7U1 and 11.4 doesn't seem to work. System just freezes
Tried copying the mpt files from omnios 15028 and get a bunch of 'invalid kernel relocation type' messages and the driver does not load.
Any suggestions or am I out of luck with this controller?
If I am out of luck, can someone recommend one that would work?

Snapshot of the freeze by using the “-m verbose -v” after $kern on file /rpool/boot/grub/grub.cfg
 

Attachments

Last edited:

Gremlin190

New Member
Dec 14, 2018
4
0
1
This has been an issue before with possibly *some* LSI 9207 cards[URL="https://forums.servethehome.com/index.php?threads/esxi-6u1-passthrough-with-solaris-11-3.8595/#post-78498"]ESXI 6U1 passthrough with Solaris 11.3[/URL]

I bought the IBM 1015 and cross flashed it to the LSI 92111 and all is fine. The card had worked for months in OmniOS

Everything is working better than expected. One thing I'd like to know is how I can get rid of the picket fence when writing. I tried adding write cache, but it still happens. I understand that it's a flush from memory, but I have over 550MB/s bandwidth and I am only writing at gigabit speeds

upload_2018-12-22_13-12-44.png
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
This is the effect of the writecache flush in Solaris every 5s.
Solaris collects all (small) writes in RAM for 5s and writes it then as a single large sequential write to pool. This is why performance goes to upper limit (write to RAM) and zero duting a flush to pool.

Open-ZFS (OmniOS) behaves different. There the cache is size determined, ex 4GB/ 10% of RAM. A flush to disk is initiated there when the cache is full not after a given time.

The result is that on short write peaks Solaris ZFS is much faster (ok, Solaris is mostly faster) while Open-ZFS has a more even write behaviour on a steady write load without reaching the upper limit but without the drops. The effective performance is the area covered by the graph over say your 60s.
 
Last edited:
  • Like
Reactions: sth

DedoBOT

Member
Dec 24, 2018
44
13
8
Is this real, i want to believe but its suspicious, zpool iostat 2 output:IMG_20190526_161104.jpg

10k +write iops?! 16 hdds in raid10, no dedicated cache drives. There is something fishy.
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
10k write operations in 2s means around 5k per second.
You are correct that this is far above of the pool's physical capability.

A single disk can give around 100 physical io iops. A pool from 8 mirror vdevs can give around 800 write iops and 1600 read iops. Your remark about a cache drive points into the right direction. But a cache drive (L2Arc) is only for reads.

Here you see the effect of the rambased write cache. On Solaris it has a size of around 5s of writes until it is flushed as a single large write operation to disk. Your 5k write operations are mainly writes to ram.

If you enable sync without an Slog where every write commit must land on disk your write operations per second should be more in the region of the physical capability of the pool.
 

DedoBOT

Member
Dec 24, 2018
44
13
8
Thank Gea for detailed explanation.
Default pool's setup,atime=off datasets. Ram 128 gb .
 
Last edited:

Evan

Well-Known Member
Jan 6, 2016
3,346
598
113
I wound up @gea estimate of 100 write IOP’s to maybe 150 though on the newer disk would be possible.
But a safe rule of thumb as pointed out is 100 iops per spinning disk.
 

DedoBOT

Member
Dec 24, 2018
44
13
8
Not bad for predominately second hand eBay's setup , except hdds and more ram .
Filled mostly with random files, multiplied. The goal was 80 percent, but exceeded it with little bit - 18 percents free space left:
aja-d63-64tb-16GBtest.jpg
aja's disk cache disabled , zfs sync disabled ,atime off,
Wife got new camera. I'm ready :)
 

Attachments

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
A little tuning may even improve performance

On Solaris:
enable Jumbo
increase tcp buffers

On Windows
enable Jumbo
With Intel nics: use newest Intel drivers and disable int throttelling in driver settings

On Switches
enable Jumbo

But even with current values, you are able to edit 4k videos from server.
Only care about pool fillrate with disks (due increased fragmentation).
ZFS becomes slower with pool fillrate and slow > 80%,
 
Last edited:

DedoBOT

Member
Dec 24, 2018
44
13
8
Yeh Gea thanks,
Played with the NICs drivers setting on both ends. At the Windows side the major difference made interrupt moderation from default "adaptive" to "low" and disabling the flow control . Thats the cost of under 5 percent CPU load . With other types of calculations offloaded to the CPU the load go above 10 percent and I skip them . Same with the Jumbos - the wifi and the wired LAN are in bridge - I want other devices to be able to connect to the storage and to the windows host too. Iperf3 gives me 9 Gb/s in both directions , its fine for me, I'm not jealous :)
About the pool filtrate - mmy next free day I will torture it with the test in conditions of 15, 10 and finally 5 percent free space.
Just wonder about the outcome . Then I will destroy the pool and create it again.
 

gea

Well-Known Member
Dec 31, 2010
3,141
1,182
113
DE
Announcing the First Oracle Solaris 11.4 CBE


"I'm very happy to announce that today we are releasing a new version of Oracle Solaris 11.4 for free/open source developers and non-production personal use.

Today marks the first delivery of our "Common Build Environment" (CBE)
releases for the Oracle Solaris 11.4.


To enable us to make new features and fixes available quicker and to more systems Oracle Solaris now uses a continuous delivery model of SRU/micro releases rather than much larger minor releases every few years.


The GA release of a major or minor was historically the release intended for non-production use for developement of free/open source software, testing, proof of concept deployments. With the switch to a continuous delivery model many new features that have been added to Oracle Solaris 11.4 are not available in a release with a non-production use license.


The SRUs also contain updates to the free and open source software that is included with Oracle Solaris. The source code repository with build instructions and patches for the open source software is available on our solaris-userland GitHub repository. Some of the Oracle Solaris patches enable free/open source software to take advantage of functionality delivered after the 11.4.0 release.


You can upgrade an existing Oracle Solaris 11.4.0 system to the CBE release today and get access to the new features released in SRU/micros since 11.4.0.

  1. How do I get the Oracle Solaris 11.4 CBE releases?
    If you already have a system with Oracle Solaris 11.4.0 GA release installed (it has the pkg.oracle.com/solaris/release IPS publisher configured) then a simple pkg update is sufficient. Alternatively, if you have a local IPS repository, you can copy the CBE, and update from there.

    Intial installation ISO images on the Oracle Solaris 11.4 downloads page are planned to be made available soon.

    Documentation for Oracle Solaris 11.4 is available from docs.oracle.com and also on our oraclesolaris-contrib GitHub repository."
 

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
How about opensourcing solaris (again)? :D

(To be honest I think even an opensourced version of solaris doesn't make any sense for enterprises or homelabs nowadays)
 

dragonme

Active Member
Apr 12, 2016
282
25
28
back when Oracle closed off open source people started realizing its ALL ABOUT THE DATA...

ZFS was on a wining path, and with open source helping move it in both features and testing it was well on its way to becoming a possible new standard.

Now that openZFS and oracle ZFS have hard forked and are no longer compatible, in my opinion, openZFS will win due to cross platform compatibility and momentum

Oracle is dead
 
  • Like
Reactions: TRACKER