The Green Hornet - All Flash Mini-ITX NAS - 100TiB usable

[itronin]

Project: Little Black NAS

Build’s Name: The Green Hornet
Operating System/ Storage Platform: TrueNAS Core (now) or TrueNAS Scale (maybe)
CPU: intel Xeon D-1520
Motherboard: Supermicro X10SDV-4C-TLN2F with active cooler
Chassis: iStarUSA S-35EX
Drives: Micron 5100 ECO 7.68TB (14), Micron 5300 Pro 7.68TB (4), samsung PM983 M.2 3.84TB (2), intel Optane 900P U.2 280GB, intel DC S3500 120GB (2)
RAM: 128GB PC4-2400T - 32GB PC4-2400T (4)
Add-in Cards: Christian Payne customized bifurcation riser x8x4x4 supports m.2 22110 double height m.2 sockets; Broadcom 9400-16i HBA
Power Supply: Sparkle SPI270LE 80+ Flex ATX
Other Bits: Icy Dock 6x9.5mm 2.5" Hotswap in 5.25" HH (3), slim ODD "shelled", illuminated momentary switch, green diffuse LEDs (2), black hex head screws, various sized nylon washers, various screws, sata and Mini-SAS HD forward breakout cables, intel m.2 to u.2 adapter, molex power splitters (3), sata power splitter, 40mm Noctua fan, 120mm Noctua fan, zip ties

Usage Profile: Itinerant storage for my digital life and media going back about 35 years.

Storage connected as follows:

Qty 14 Micron 5100 ECO 7.68TB connected to Broadcom 9400-16i HBA
Qty 2 Micron 5300 Pro 7.68TB connected to Broadcom 9400-16i HBA
Qty 2 Micron 5300 Pro 7.68TB connected to motherboard sata (4,5)
Qty 2 intel DC S3500 120GB connected to motherboard sata (2,3)
Qty 1 intel Optane 900P 280GB u.2 connected via onboard m.2 socket
Qty 2 samsung PM983 m.2 22110 3.84TB connected to c_payne bifurcation riser

Storage configured as follows:

Boot Pool: Mirror intel DC S3500 120GB
Capacity Pool: 3 x 6 RZ1 7.68TB sata Drives, 1 intel Optane 900P SLOG device, ~97TiB
VM Pool: Mirror PM983 3.84TB ~3.3TiB

Total usable storage ~100TiB

Status: This build is roughly complete and usable... a few changes coming... but it may never truly be finished.



The Green Hornet (other objects in image are there for scale)
(okay maybe I was just too lazy to clean up the table before taking the picture)



custom bifurcation riser and 9400-16i HBA



Mess 'o cables


Project Started: roughly 3/2021
Testing Started: 12/2021

Keys and LL of this build:
patience, trial and error, taking risks
basic understanding of pcie bifurcation practical implementation
do your research, leverage what others have done, read and understand the motherboard manuals before you buy anything then make a plan
start small to see what can be done then incrementally try more.
the funky mounting points at the top of the 5.25" bay area are for a slim ODD even though the bezel is not cut for it
don't use sata port 0 (superdom) if you are going to pull x4 from the onboard m.2 slot

Additional Details and pictures will be posted in subsequent reserved posts to this thread.

References:
U-NAS NSC-800 (this will be explained when I detail how I ended up starting this project.)
Tweaktown iStarUSA S35 Mini-ITX NAS Tower Chassis Review

Credits and Thank Yous for sharing:
@maes Fully loaded compact NAS (NSC-800) sending me some drive adapters for the NSC-800 and information about Christian Payne's work.
@JimPhreak Mini-ITX (or not?) storage chassis for SSDs (and especially the posts about the S-35EX)
@matt_garman Home Server Rebuild: S-35-DE5SL...
@pgh5278 Supermicro Xeon D / FCBGA 1667 Heatsink information
@IamSpartacus for this post about heatsink solutions
@craig24x for this post about heatsink solutions
@Danic for selling me his 6 bay icy dock to get me to three in this chassis (and for searching for the mounting screws)!

 

[itronin]

Bill of Materials

Item	Manufacturer	Source	Description	Qty
chassis	istarusa	bh photo	S-35EX 3 x 5.25HH mini itx case	1​
chassis	icy dock	sth/onhand	6 x 2.5x9.5mm hot swap 5.25 bay	3​
power	sparkle	ebay	270W 1U flex atx	1​
motherboard	supermicro	ebay	SM X10SDV-4C-2NLF active cooling 10gbe	1​
memory	samsung	ebay	32GB PC4-2400T LRDIMM	4​
fan	noctua	amz	120mm	1​
fan	noctua	amz	40mmx10mm for HBA	1​
hba	broadcom	ebay	9400-16i	1​
cable	various	amz	mini sas hd to sata .5m	4​
cable	various	amz	.3m sata	2​
cable	supermicro	onhand	.8m sata	2​
cable	intel	onhand	m.2 to u.2 adapter / cable	1​
card	c_payne	c_payne	custom dual sided nvme m.2 22110 bifurcation riser	1​
disk	various	onhand	shelled slim ODD mod for dual boot drive	1​
disk	micron	ebay	5100 ECO 7.68TB	14​
disk	micron	ebay	5300 PRO 7.68TB	4​
disk	samsung	ebay	PM983 NVME 3.84TB	2​
disk	intel	ebay	DC S3500 120GB	2​
disk	intel	onhand	Optane 900p 280gb	1​
pcie slot #	physical	data	speed	item
1​	x16	x16	pcie 3.0	c_payne riser
1​	x16	x8	pcie 3.0	9400-16i
1.1​	x4 m.2	x4	pcie 3.0	pm983
1.2​	x4 m.2	x4	pcie 3.0	pm983
2​	x4 m.2	x4	pcie 3.0	optane 900p

 

[itronin]

Reserved 2

 

[itronin]

Reserved 3

 

[itronin]

Customizing the iStarUSA S-35EX

Completed

• Changed the chrome On/Off momentary switch for a black illuminated switch
• Changed the red and green clear LEDS for diffuse green LEDS
• Changed the chrome hex head screws for black hex head screws
• Built a dual system disk tray from an old and salvaged slim ODD

Future

• Make a new 4mm aluminum bezel - these items will probably never happen
  • Spacing for dual vertically aligned USB 3.0 ports
  • Bottom cutout for push button LCD front panel in bezel and enlarge chassis cutout
  • Powder coat the new bezel

Overall
I really did not like the flashy chrome bits on this chassis and wanted a more subdued look. The chrome hex head screws and on/off switch had to go. I looked around and figured if I could use an illuminated switch then I would free up one of the front panel LEDS too. The installed activity red and green LEDS are super duper bright. too bright. I decided I'd use the activity LEDS for NIC1 and NIC2 on the motherboard. The on/off switch LED is green, all of the ICY Dock activity LEDS are green. Both activity LEDS needed to be green too.

Changing the On/Off momentary switch

I measured the existing switch's width and the threaded barrel. Looked to be 16mm.
I ordered the black and green illuminated Weideer 16mm Momentary pushbutton switch from AMZ.
Replacement was super easy and relatively straightforward. Since I already had an assembled system I had to disconnect the HBA cables, remove the HBA and riser to get my fingers into the motherboard area to install the connectors.

Changing the red and green clear LEDS for diffuse green LEDS.

Back in the day Red and Green LEDS were pretty much what you got with system chassis. I have a pile of salvaged LEDS in a parts box.
Simple enough to remove the existing clear LEDS clip off their leads and solder on a pair of salvaged green diffuse LEDS.
Again super simple and straightforward.

Changing the chrome hex head screws for black hex head screws.

I have a general purpose hardware store near me (ACE) and so I took one chrome hex head with me and drove over for a visit.
I test fitted the existing screw against some metric nuts. IIRC correctly they were M6 - but please verify if you try this - do not take my word for it. After that I simply went through the various bins to find something that looked nice and had sufficient thread length to pass through the bezel and hold securely to the chassis skeleton.

Build a dual system disk tray from an old slim ODD

When I received the S-35EX I stripped it down. The first thing I noticed were the mounting holes at the very top of the drive bay. Nothing in the documentation indicated a use. Nothing in the various iStarUSA videos indicated what they were for. Later through the build my brain jogged and I thought to myself the only thing that might fit there was a slim ODD since its wide at the top part, a little narrower at the bottom - but there is not pre-made cutout in the bezel for it. I contacted iStar through their support channel asking if they had additional detailed drawings for the front bezel and whether the top mount was for a slim ODD, the bottom hole for an LCD mount and would they make custom bezels. I received one reply immediately - we'll get back to you. Nothing else - that was about 3 months ago. I have a couple of salvaged slim ODD's from Compellent SC030s. I test fit one and yes the holes lined up - perfect fit. Well except for the bezel. The ODD bezel has to be removed since it interferes with the chassis front bezel.

The Red circles in the following pictures show the mounting holes for the slim ODD.


Front View
Red - Mounting holes for slim ODD


Top down view

Decisions, Decisions, do I mount the system disks using the slim ODD or do I mount the Optane 900P there? Cable length didn't really matter either position would work. Power, I can get power there in two different ways. During system build and before trying to fit the ODD I mounted the optane on top of the icy docks and used cut foam to keep it in position. Probably fine for a purely static use case and careful moving for maintenance but I really wanted the solution to be more permanent. Ultimately I decided to mount the Optane in the side 2.5" drive position and use the ODD for the system disks.

Getting to work on the mount I took apart the ODD.



First I needed to remove standoff protrusions. I simply used increasing drill bit sizes to drill those out. Then I used a hand file to get rid of the metal burrs.

I scanned the bottom of a 2.5" ssd, and a 1:1 print created a template. The sheet metal of the ODD housing is really thin so I drilled a pilot hole and then increasingly larger diameter holes till I had what I needed.

I moved the template around to find drive mounting positions that more or less lined up in the interior and the top of the shell.
I marked those with a sharpie using the template and then took some snips to the back edge of the bottom of the shell to remove bits that would block mounting the drive. A hand file removed the burrs and sharp edges.

A quick trip to the hardware store to get some semi-thick nylon washers and I mounted the drives on the shell halves. I used the washers to provide just a bit of an air gap between the top and bottom of the drives.



Next I put the halves back together and put in the screws that hold them together.



In order to mount the ODD shell and get the chassis mounting holes to line up I loosened the 4 hex head screws that hold the front bezel in place. Using the screws from the ODD mounting rails I attached the ODD shell to the chassis and then tightened the bezel screws.



Power is supplied to the two system disks utilizing a SATA power splitter (M to 2xF) from one of the two SATA connectors on the PSU's SATA power cable. I know that the SATA splitters are have a history of combusting. I think the system disks' power load is sufficiently low and infrequently used that it won't combust. I could have daisy chained a molex to 2 x SATA splitter into the cables that feed the icy docks but those are already split once and I don't like splitting splitters.

Future

I feel that iStarUSA did a nice job on this chassis. I believe they may have intended this design to be used for OEM manufacturers due to the inclusion of the slim ODD mounting position (makes it kind of like the early gen HPE microservers) and the chassis cutout (blue circle in image) which would allow a cable feed and the pins/protrusions from an LCD or LCD/keypad to be mounted to the bezel. If the cutout were slightly larger (5.25" bay or so) a simple cutout in the bezel could be fashioned and I could utilize one of the various LCD/keypad in 5.25" bay solutions.

The chassis design is clearly dated by use of the front USB 2.0 ports. However that area is modular and removable (yellow circle in image). Fashioning a new bezel and removable mounting plate would allow one of the various USB 3.0 vertical port assemblies to fit.


Front View
Red - Mounting holes for slim ODD
Yellow - Mounting for vertial dual USB connectors
Blue - Front cutout - maybe for LCD panel and cable pass through to Bezel?

I'm not really knowledge about CNC work but I suspect that getting the bezel design and positiong correct for my proposed customizations are probably cost prohibitive to have a new bezel made.

 

[itronin]

Reserved 5

 

[NoncarbonatedClack]

Holy crap. This is one of the craziest build's I've seen. very nice!

Have you done any benchmarking? Surely you saturate 10GbE.
Are you running VM's on this? or serving out to another host for compute via NFS/iSCSI?

Can you elaborate on this bit please, when you have time?

"basic understanding of pcie bifurcation practical implementation"

 

[Wasmachineman_NL]

holy SHIT I desire to know where you got that hexagon "mousepad" from!

Insane build btw! If I was Mr. Money Bags I too would like a all-flash NAS/VM machine in the meter cupboard.

 

[itronin]

sorry premature reply. edited to fix some grammar stuff and an inaccuracy.

Holy crap. This is one of the craziest build's I've seen. very nice!

Have you done any benchmarking? Surely you saturate 10GbE.
Are you running VM's on this? or serving out to another host for compute via NFS/iSCSI?

Can you elaborate on this bit please, when you have time?

"basic understanding of pcie bifurcation practical implementation"

thanks. informal benchmarking however the cpu is thermal throttling and my easy source for a few TB's of transfer was a zfs replication from spinning rust. Was in the 4-5Gb/s range which is more than adequate. nvme storage is for local vm's (bhyve or docker depending on TN version) and sharing via NFS to a couple of nodes I intend to build and run prox on. I have two plans to try and alleviate the thermal throttling. Not sure how much of an impact that will have on transfer speeds. I also have an all flash TNC box that handles my larger systems vm storage needs and I will doing some transfers from there to the new box as well to see what speeds I can get.

by basic understanding I mean you need to be able to read the block diagram in the manuals, translate that to the BIOS settings. You should understand the difference between x8x4x4 and x4x4x8. Helps if you can look at the traces on a riser too to figure out what is what.

this board has a small pcie switch on it too, if you use SATA0 (dom port) you will yank at least 1 pcie lane (probably two due to spec) and cut your m.2 slot bandwidth down.

If you are working with a larger board say mATX then it gets more interesting, and full size boards more so. Pls realize I was evaluating boards to go into this box - I didn't start with this motherboard or this chassis.

the reserved posts are so I can add more info - I posted that and intended to immediately write more and then work happened. still is. there will be more. history, some of the simple mods, and then the can opener mod to make use of the internal slim ODD mounting.

holy SHIT I desire to know where you got that hexagon "mousepad" from!

Insane build btw! If I was Mr. Money Bags I too would like a all-flash NAS/VM machine in the meter cupboard.

The Armory Probably circa '88 or '89 or maybe even '90 I think? Older than the laptops you typically work on!

That's no mousepad! Its a battlemat!

I use it to protect the table surface and its anti-static (probably not by design) and it does look pretty cool!
oh yeah it also works as a mousepad too. Its about 30"x30".

This one is similar to the one I have. Says they ship worldwide.

 

[Evan]

Wow! Love it.
About people thought I was a bit crazy with 6x3.84tb SSD for main data. Once you go full ssd everything is just horrible.

 

[itronin]

Customing the iStarUSA S-35EX added.

 

[Gene]

What is your idle power usage and max power under load? I always thought something like this would be great for being on a liveaboard sailboat where minimal power usage and tiny footprint is key

 

[itronin]

What is your idle power usage and max power under load? I always thought something like this would be great for being on a liveaboard sailboat where minimal power usage and tiny footprint is key

I had not collected that nor thought about collecting it. Its on a shelf de-energized right now as I have work projects going on. May not be able to answer this until May. PSU is 270w so obviously less than that. the spec on the 5100 ECO is 6w max for writes and the 5200 pro is 3.6w max for writes. so 100 w for the drives max give or take. The optane is 14w max. the sammys probably max of 16 combined. cpu/board/10Gbe probably about 60w max. - running flat out with 10gbe and keeping everything fed will likely be close to 200w. If the network was 1gbe then maybe half since you can;'t saturate the drives?

 

[Gene]

Yeah no rush on any of that was just wondering if you had the numbers. On a liveaboard sailboat every watt counts since you generate it yourself. Im still years away from buying a boat

 

[Judeling]

I know this is an old thread but I notice there seems to be a lot of extra space where the PSU is mounted and was wondering if the case could be modded to fit an SFX PSU instead.

 

[itronin]

I know this is an old thread but I notice there seems to be a lot of extra space where the PSU is mounted and was wondering if the case could be modded to fit an SFX PSU instead.

I don't know and I don't have one to physically check.

Istar's page says:

Supported PSU Type

1U Flex

next time I open one of them up I will see about measuring the psu bay to see if one will fit - using - 125 x 63.5 x 100 mm

 

[itronin]

I know this is an old thread but I notice there seems to be a lot of extra space where the PSU is mounted and was wondering if the case could be modded to fit an SFX PSU instead.

I'm doing a mobo swap on one of my hornets.

The tray is xyz
100mm, 48mm, 175mm
PSU standoffs are about 24mm high.

You have some overhang on both sides of x so you'd probably be good there.
if you decrease the psu standoff height ou can pick up the 16 mm you need in y. Note you may have to replace the power cable extension.

You may also be able to remove the psu tray and standoffs all together - vent the bottom of the chassis perhaps and create a mounting mechanism. The power cable extension cable will need to be replaced if you do that.

 

[T_Minus]

Swapping as-in replaecment or upgrade to something?

 

[itronin]

Swapping as-in replaecment or upgrade to something?

I had a Gigabyte 370N mitx in one of them. Was playing with TN scale. Pulled that and put an X10SDV into it and going back to TNC. The first post was the Green Hornet. Second one I built (smaller disks) I went with blue LEDs and the plastic Icy dock 6 bay with blue LED's so Blue Hornet(?). I also pulled the Optane 900P u.2's and swapped in Optane P1600X's. Other than LED color they are both pretty much identical now.

Hoping later this year or early next to play with 3d printing a new face plate to add in a usb keypad display thingie I got for cheap.

 

[Judeling]

I'm doing a mobo swap on one of my hornets.

The tray is xyz
100mm, 48mm, 175mm
PSU standoffs are about 24mm high.

You have some overhang on both sides of x so you'd probably be good there.
if you decrease the psu standoff height ou can pick up the 16 mm you need in y. Note you may have to replace the power cable extension.

You may also be able to remove the psu tray and standoffs all together - vent the bottom of the chassis perhaps and create a mounting mechanism. The power cable extension cable will need to be replaced if you do that.

Thanks for taking the time to measure, I appreciate that