Supermicro X10DRU-i+ - use for internal PCIe x8 port

[oldpenguin]

Repurposing this as it was gathering dust, but in the current case (no idea what's the equiv CSE number, but it's a 2U unit with 12LFF having the PCIE specs/risers of 6018U-TR4T+ except in a 2U case):
- 2 x16 free slots on the WIO riser (hello 2 asus hyper cards)
- 1 x8 slot on same WIO (used by their AOC-3008 HBA)
- 1 x8 slot "internal" (looks like on the UIO riser).

The internal x8 slot looks mechanically compatible, but it's pushed more to the inside of the case, so I'm guessing it takes a certain type of card to fit in there.
Has anyone ever had any use for that port, or have some recommended cards that fit?

I'd happily switch the 3008 with any other 12G SAS HBA (IT mode) that fits there so I could squash in a dual SFP+/QSFP in the regular x8 and keep both the x16 ports for NVMEs.

Thank you in advance for any recommendations (checked their "matrixes" but maybe I was looking in the wrong spot).

 

[BlueFox]

Supermicro shipped servers with a 3008/3108 based HBA in that location. Just requires a bracket to support it. The slot itself is not proprietary.

 

[oldpenguin]

Just pulled that riser (aoc-2ur6-i4xt) out - much to my wonder it has another x16 FH on the other side. Not that I'd ever complain about it, I was fully sure the config for this was 2x16 + 1x8 + 1x8 internal.

Anyway, decided to challenge satan and simply pulled out the bracket from the HBA, then stuffed it into the internal x8 port hanging in the air, seems quite solid, kicked the ignition on the roaring turbofans and voila:


zpools all in order after boot, nothing out of order. Temps to be measured for further impact analysis.
Remaining for NVME population: 3 x16 slots plus the desired one for the extra NIC card.

No idea what a bracket/extender should look like, found zilch on SM parts list, the distance between the regular bracket slot and the beginning of the card is a [roughly] 60mm / 2.35". I'll happily take any recommendations - due to the bunch of radiators from the X540-2AT, I'm tempted to steer clear of a 3d-printed improv in this case.

 

[BlueFox]

You can email Supermicro's support and inquire. They will sell individual parts from time to time. As for what it looks like: https://www.supermicro.com/a_images/products/views/6028U-E1CNR4T__top.jpg

 

[oldpenguin]

Will give it a try - thank you!

 

[sko]

Any chance this is a SYS-629U-TR4T ( SYS-6029U-TR4T | 2U | SuperServers | Products | Super Micro Computer, Inc. ) /w expander backplane? Those are relatively common and easily available, I also run one of these with 2x E5-2260v4 as my home server.
The risers for those systems are also relatively easy to find, except the 2xSFP+ variant (most have the 4x 10G copper which are a waste of PCIe lanes IMHO...).

Why would you want to switch the SAS3008 HBA? Nothing wrong with that, the backplane (/w expander) isn't pcie-capable anyways, so tri-mode HBAs are a waste. I recently switched to 6x HPE/Sandisk 1.92 SAS SSDs as my main storage pool in that system, all the spinning rust went out (and power consumption went down by ~50W)
The HBA sits in the internal x8 slot as intended by supermicro; the external LP x8 in the middle is populated by the Supermicro AOC-STGN-i2N NIC.
In terms of NVMe I switched the left riser with 4x PCIe x8 on the leftmost side and use the AOC-SLG3-2M2 carriers (much better cooling than quad M.2 carriers!).
For x16 slots I ordered several of these a while ago: https://www.aliexpress.com/item/1005005316780981.html ; currently having 3 of them running in total (one in this system).
The slot above the PSUs might see a bit higher temps, so don't put space heaters like Samsung in there (or anywhere else if you care about cooling requirements/noise...). I currently have 2 Micron 7400 Pro in that carrier that run fine with temps at ~65°C at high loads (package builds running) and ~45-50° under normal load/idle.

 

[oldpenguin]

Ref mobo/chassis - based on BMC reported part number, motherboard looks like it used to be part of a 6018U-TR4T+, however chassis got "promoted" to a 2U counterpart labeled CSE829U on the left side of the chassis.

The sole purpose of switching the 3008 was if I'd find something with a longer bracket to keep it in place in the so-called "internal" x8 slot, otherwise couldn't give a dead rat's ass about it, it only holds 4 SAS 12g rustmills (because of capacity) and a bunch of 2.5" SSDs (because they were doing nothing around, now they're still doing nothing but mainly decreasing their SMART power on hours and eating away mostly solar power). I'm usually overprovisioning space, just in case I may need it sometime.

Since this ain't gonna do anything heavy from a workload perspective (worst I can think of is some gitlab runners rebuilding CI/CD containers and a test-stage kubernetes), if the bracket-less improvisation with SAS3008 in the internal slot works, that just means more commodity NVMEs available to shove in whenever/if-ever needed. Currently using an AOC-SLG3-2M2 for them, but hypers are on the way already. As for using commodity NVMEs, I may be against the trend but since it's configured to nicely shut itself off as soon as the 3KVA UPS hits 20% remaining on batteries, I can't say I have many worries about it (wouldn't have any even if they died out of the blue, due to backups).

 

[sko]

I'm not using the 7400 Pro because of the PLP - I also got a UPS /w external battery pack for ~40min runtime for the whole rack (and another ~5-6 days when I connect it to my BEV...), I primarily use them for their sustained performance.
Those consumer-grade drives plummet to absolute garbage throughput levels as soon as their cache is saturated. I just had the 'pleasure' to copy yet another 2 dying samsung NVMes at work that are marketed with fantastic performance, but after 32 (or 64) GB they drop to ~200MB/sec levels and after a few minutes even (much) lower because of their crappy thermal design...

Those microns can be continuously abused for hours and still have amazing performance (i.e. micron seems to advertise the *sustained* achievable throughput, not the peak/cached values) while still having enough headroom temperature wise to not fall into thermal throttling. and at 140EUR for 1.92TB and 3.5/14.4PBW they are an absolute bargain.
If sustained performance is irrelevant and only price and write endurance are important, the Transcend MTE220S are amazing at 100eur for 2TB / 4.4PBW.

RE the HBA: just remove the slot bracket and put it in the internal slot. that's how it is intended by supermicro. the support brackets can be usually found for a few bucks on ebay or just run it without a bracket - that works too and if you have vibrations at your servers that could shake out the card from the socket I highly suspect you have much bigger problems than that HBA...