EXPIRED Hyve "S-SKUD 2" - 1U, 8x LFF SATA6, 2x LGA 3647 Skylake-SP, 2x PCIe x16, 2x OCP PCIe x8 w/2x 25Gbps Connectx-4 installed - $149 + Shipping

RolloZ170

Well-Known Member
Apr 24, 2016
2,240
555
113
55
I have a cascade lake CPU on hand, I could try swapping the microcode and see if it boots.
i doubt that. it's not only the microcode, the memory controller requires a different initialisation (Intel Memory RC)
you all have to get the 2nd gen BIOS if you want to run cascade lakes.
i tried to get one for my MS Olympus boards from the motherboard OEM (Wiwynn). but without NDA authorisation by Microsoft the BIOS is not available.
the MS Olympus is based on Project Olympus, but this project does NOT include any firmware. Firmware is copyrighted/licensed.
 

Fritz

Well-Known Member
Apr 6, 2015
2,958
979
113
68
Glad I passed on this deal. I would have been way over my head.
 

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
Glad I passed on this deal. I would have been way over my head.
This is actually why I jumped in on the deal... It's time I learn to hack BIOSes for my needs, and @RolloZ170 is helping guide several of us on the STH forums. :)

I have a mountain of projects that I could "fix" the BIOSes on, dating all the way back to 2008, once I get hold of the ropes. The community here being excited about this S-SKUD 2 has given me the push to dig into and learn these things while I am between gigs.
 
  • Like
Reactions: Fritz

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
So, I am going to order a bunch of cables and right-angle adapters for this setup of 10x HDDs.

There's 1/2" of clearance in front of the fans.

 

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
So, I've been researching OCP SAS cards and finally stumbled upon a few models I am pretty sure will fit as is.

Tyan has one of the best published "Storage Mezzanine cards" guides. Too bad their tyan.com DNS server seems to be hacked (lots of porn, giveaways, contests - all garbage URL Links) in sub-domains like "www.giveaway.tyan.com", "www.ww.tyan.com" <- tries to load a Javascript virus, and so on. If you manually fix the URL, the regular site works.

Unfortunately, I can't find many Tyan SAS controllers on ebay. Maybe someone here knows a cheap source? Listed below are the models that should work without issue.

SAS3 8-port

Unfortunately, the heatsink may interfere with the card you insert into the PCIe riser in that area. Maybe, maybe not...

SAS2/SATA6 (if you can use slower)

Gigabyte also had some OP SAS3 cards. however, their versions are much longer (out of OCP spec) and seems it would interfere with the TPM module as well as one of the SATA ports (and would block BOTH sata ports!).

I think I am done searching for SAS OCP cards as this point and will just use a normal PCIe version for now, until I get rid of the SAS drives.
 

plessley

New Member
Dec 1, 2019
28
23
3
So I have been doing 90 degree pcie adapters

and cheap garbage AOC from supermicro (Supermicro AOC-S2308L-L8e 8-channel 6Gbps SAS2/SATA3 PCIe 3.0 x8 IT/HBA Adapter | eBay ). Admittedly they are only sas2, and lsi2308 chips... And probably going to bite me in the arse....

Oh, and sas 4 port breakout cables
Mini SAS 36P SFF-8087 to 4 SFF-8482 Connectors With SATA Power Cable 1M US SHIP | eBay

Some of this stuff can be found cheaper on bitdeals but I've had zero luck with them.
 

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
So I have been doing 90 degree pcie adapters

and cheap garbage AOC from supermicro (Supermicro AOC-S2308L-L8e 8-channel 6Gbps SAS2/SATA3 PCIe 3.0 x8 IT/HBA Adapter | eBay ). Admittedly they are only sas2, and lsi2308 chips... And probably going to bite me in the arse....

Oh, and sas 4 port breakout cables
Mini SAS 36P SFF-8087 to 4 SFF-8482 Connectors With SATA Power Cable 1M US SHIP | eBay

Some of this stuff can be found cheaper on bitdeals but I've had zero luck with them.
My problem is I'd like to install both a cache drive (Intel Optane) and a dual 40GbE QSFP+ card - since I'm all 40G here. Normally that would be fine, but I only have 5x SATA 10 TB drives. The dozens of other 10 TB drives are all SAS3... So, that's 3x total cards (because I can't find an OCP 40GbE card to fit in that OCP slot!).

It's aiight, as I going to use it as a backup server at this point. So the dual SPF28 cages, for 10G on my network as the lowest common denominator, should suffice for ZFS snapshots. It's just, that initial load of 50 TB... ouch...
 

plessley

New Member
Dec 1, 2019
28
23
3
My problem is I'd like to install both a cache drive (Intel Optane) and a dual 40GbE QSFP+ card - since I'm all 40G here. Normally that would be fine, but I only have 5x SATA 10 TB drives. The dozens of other 10 TB drives are all SAS3... So, that's 3x total cards (because I can't find an OCP 40GbE card to fit in that OCP slot!).

It's aiight, as I going to use it as a backup server at this point. So the dual SPF28 cages, for 10G on my network as the lowest common denominator, should suffice for ZFS snapshots. It's just, that initial load of 50 TB... ouch...
That's where I'm going to be, these 4 are going to become a storage cluster, but 10gbe is all I have for switching. I've been debating on doing something weird with either a dual port 100gbe nic or a pair of single port 100gbe that I can break out to 25s and make a software based switch out of an old r620
 

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
That's where I'm going to be, these 4 are going to become a storage cluster, but 10gbe is all I have for switching. I've been debating on doing something weird with either a dual port 100gbe nic or a pair of single port 100gbe that I can break out to 25s and make a software based switch out of an old r620
Here ya go:


If you just want a switch, you can pickup Brocade ICX6610 (with 4x 40GbE, 8x 10GbE, and 24/48x 1 Gbps ... For around $120 or a bit more. Actually, I have like 3 or 4 I need to box up and sell at some point.

And of course, the infamous STH thread on these Brocade switches:

 
  • Like
Reactions: plessley

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
Seller ended the listing... :(

They said they are going to get rid of them. If someone wants a couple more, get them now by messaging them!

They dropped the price to $149 + shipping at the end.


Seller said no messaging them. They are gone. Sorry!
 
Last edited:

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
As a small update, I was able to source one of the Tyan SAS2 x8 OCP cards and while it works when plugged in and Linux can see it!, I don't have any SAS cables for my drives - yet. Doh! There are a few sellers on eBay. I got one of the $80/$90 ones.

I also snagged an 8124M Platinum. In case you don't know about this off-roadmap Amazon-special CPU:
  • 18C/36T
  • 3.0 Ghz base !!! <- try to find ANY cpu at that speed with over 8C
  • 3.4 Ghz turbo up to 9C active
  • ~25 MB cache (24.75)
IMO, this is the best Skylake-SP CPU of the bread... 18C/36T

I got mine for $205 "dealing" on eBay for about a month. I haven't installed it yet as I'm travelling a lot these days. But by all accounts, it should be a drop-in like any other Skylake-SP.

 

RolloZ170

Well-Known Member
Apr 24, 2016
2,240
555
113
55
I haven't installed it yet as I'm travelling a lot these days. But by all accounts, it should be a drop-in like any other Skylake-SP.
of yourse no. 8124M is a TDC=255A processor and requires a non standard VRM configuration.
this Hyve mboard uses 6+1 power stage conf. which is promising.
good luck !
 
  • Wow
Reactions: eduncan911

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
For those that missed this deal, here's a similar setup for a standard Supermicro (or similar) 1U motherboard build I've had in my watchlist for quite some time (waiting for a price drop):


Doesn't quite qualify for a "Great Deal", not until they lower the prices more as the S-SKUD2 was $280 + tax once you figured in price + shipping, and you got 2x PCIe slots and mobo to boot.

Maybe someone can shoot off a few Offers for a discount on a couple, and report back?

EDIT: There's also this one I've had in my watchlist for a while, waiting on a discount...

 
  • Like
Reactions: Fritz

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
of yourse no. 8124M is a TDC=255A processor and requires a non standard VRM configuration.
this Hyve mboard uses 6+1 power stage conf. which is promising.
good luck !
Well, the system starts up immediately with an "FF" error code when the Intel Xeon 8124M is inserted. I may pull/review the bios posted earlier to see about updating the microcode (since this is an off roadmap CPU).

Doh!
 

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
Here is some component connections via reverse engineering, the expensive way...
  • Both OCP x8 (LEFT) and x16 (RIGHT) are connected to CPU1.
  • Both PCIe x16 FHHL slots are both connected to CPU2.
  • Tyan OCP 2.0 Card M7094-2308-8I-SYN (x8 PCIe 3.0) works for SAS2 in the spare OCP x8 slot!
  • So far, I can't see anything in the M.2 slot.
  • Get 36"/3.3FT/1-meter SAS/SATA cables. There's plenty of reach for additional drives.
I found the Tyan card here:


I'll see about picking up another 3106 to activate those other two PCIe slots, because I have cards in them.

PXL_20220906_172936468.jpg
 

RolloZ170

Well-Known Member
Apr 24, 2016
2,240
555
113
55
Well, the system starts up immediately with an "FF" error code when the Intel Xeon 8124M is inserted.
as expected(unfortunaly)
I may pull/review the bios posted earlier to see about updating the microcode (since this is an off roadmap CPU).
does not help: 8124M is OEM but not special stepping, boring H0 (50654) as all other Skylake like silver 4110.
check the used VRM controller, a upgrade is easy if its MPS/TI one.
 
  • Like
Reactions: eduncan911

eduncan911

The New James Dean
Jul 27, 2015
651
509
93
eduncan911.com
This may be my last update as I'm done fracking with this machine.
  • CPU0: OCP x16 and OCP x8
  • CPU0: 8x SATA + 6x sSATA (secondary SATA)
  • CPU1: both x16 PCIe 3.0 Risers (yes, neither riser works without both CPUs!)
  • CPU1: M.2 Slot (that's right! you can't use the M.2 slot without both CPUs!)
  • You can install 4x extra DDR4 DIMMs into the 4x additional DCPMM (Optane) slots and the system initializes it. So, running 16x4GB sticks nets me 64 GB.
    • However, since these Skylake-SP CPUs are only 6-channel, I removed the ram in the DCPMM slots as I am sure that is downclocking/interleaving memory or some other mis-matched channel restriction. So, 48 GB it is for me.
  • 2x Xeon 3106 (8C @ 1.7 Ghz) works fine.
    • At $55ea, very inexpensive backup server.
I'll post more wattage usage from my KIll-a-Watt when I finish moving some servers around.