Converting an HP DL380e Gen8 14xLFF Server to a Disk Shelf

audiophonicz

New Member
Jan 11, 2021
5
6
3
I see, so youre using full server chassis. I guess youre right and i should be thanking omegadraconis too lol. In all honesty I thought the pic of the two 120mm fans was yours. Thats what I was asking about mountings to. But good to know about the 80mm too since Im thinking of going with a cut down oem chassis. also thinking about a straight 12v psu to try to cut down on the size and heat.

I'd also like to move from the 9207-8i card I'm currently using in the R730 to a 9207-8e and maybe use some SFF-8087 to SFF-8088 cables, but I haven't done much research yet, so I'm not sure about that.
So thats exactly what Im doing, except with the 9206-16e. Same chip, same cables, I just have 4 ports. should work fine for ya.

Looks like you've got a nice clean setup there, and a fine looking rack. Would you mind doing a quick run through of what you've got in there, please. The box above your storage looks to be quite interesting, what model is it?
Thanks! I actually use R730xd's and MD1400s at work, so while I wanted a setup Im used to I also wanted compact, low power, and low noise.

The rack itself is a 13" deep NavePoint 12U Wall Mount Rack, but the usable rack depth is only 10.5" rail to wall.


DAS1.jpg

Top to bottom:

-24 port patch panel to cat6 running thru the house
- MikroTik 24 port 1G switch
- MikroTik 8 port 10G router. Those are all 10G SFP+ Twinax cables.

-The next 2 are my ESXi cluster, theyre both OnLogic 1.5U rack cases flipped backwards. i7-6700T, 32GB, Samsung SSD 500GB boot vols, FSP modular 1U psu, and a Intel X520-DA2 10G card. The little blue lights are HDMI emulators.

-the 2U case is my TrueNAS Core box. Its a modified plinkusa 2U front access case.
I cut about 3.5" off the back of the case so it would fit in the rack depth, used a PicoATX 130w psu and put the power jack on the bottom corner (power brick on the bottom) and above that 3d printed a riser mount and cut a 5.25 bay in the case for a 4x 2.5" hotswap bay. In there i have 4x 1TB Crucial SSDs in RAID5 that is the datastore for the virtual host cluster. i7-8500T, 16GB, dual 64GB SDCards in USB adapters for the mirrored boot vol, the LSI 9206-16e and Intel X520-DA2.

-obviously next is the homebrew DL380 DAS. 6x 4TB WD in RAID6 + 2x HS. I think the max this can run is 6TB drives. those are next. mounted on the shelf with the shelf's lip you can see it takes up more than 2U. It bothers me.:)

-and on the bottom with all the power is a Digital Logger Web Power Switch.

Both ESXi hosts and the NAS have 1x 10G connection to the main network and 1x 10G connection to the iSCSI network. All 3 have the ThermalTake Engine 27 1U LP CPU cooler and Crucial memory. The whole setup is pretty quiet, I cant hear anything from the other side of the garage unless its dead silent. The T i7 chips are the low power ones. The entire rack only uses about 200watts average.
 

amp88

New Member
Jul 9, 2020
20
13
3
I see, so youre using full server chassis. I guess youre right and i should be thanking omegadraconis too lol. In all honesty I thought the pic of the two 120mm fans was yours. Thats what I was asking about mountings to. But good to know about the 80mm too since Im thinking of going with a cut down oem chassis. also thinking about a straight 12v psu to try to cut down on the size and heat.
Cool. I'm also hoping that moving from an ATX PSU to the server PSU will improve efficiency a bit. The 450W 80+ Gold Seasonic unit I'm using isn't terrible, but it's at the low end of the load/efficiency curve, so if I can drop a few watts that'd be nice.

So thats exactly what Im doing, except with the 9206-16e. Same chip, same cables, I just have 4 ports. should work fine for ya.
Thanks for the reassurance, I'll hopefully move to that soon then.

Thanks! I actually use R730xd's and MD1400s at work, so while I wanted a setup Im used to I also wanted compact, low power, and low noise.

The rack itself is a 13" deep NavePoint 12U Wall Mount Rack, but the usable rack depth is only 10.5" rail to wall.

Top to bottom:

-24 port patch panel to cat6 running thru the house
- MikroTik 24 port 1G switch
- MikroTik 8 port 10G router. Those are all 10G SFP+ Twinax cables.

-The next 2 are my ESXi cluster, theyre both OnLogic 1.5U rack cases flipped backwards. i7-6700T, 32GB, Samsung SSD 500GB boot vols, FSP modular 1U psu, and a Intel X520-DA2 10G card. The little blue lights are HDMI emulators.

-the 2U case is my TrueNAS Core box. Its a modified plinkusa 2U front access case.
I cut about 3.5" off the back of the case so it would fit in the rack depth, used a PicoATX 130w psu and put the power jack on the bottom corner (power brick on the bottom) and above that 3d printed a riser mount and cut a 5.25 bay in the case for a 4x 2.5" hotswap bay. In there i have 4x 1TB Crucial SSDs in RAID5 that is the datastore for the virtual host cluster. i7-8500T, 16GB, dual 64GB SDCards in USB adapters for the mirrored boot vol, the LSI 9206-16e and Intel X520-DA2.

-obviously next is the homebrew DL380 DAS. 6x 4TB WD in RAID6 + 2x HS. I think the max this can run is 6TB drives. those are next. mounted on the shelf with the shelf's lip you can see it takes up more than 2U. It bothers me.:)

-and on the bottom with all the power is a Digital Logger Web Power Switch.

Both ESXi hosts and the NAS have 1x 10G connection to the main network and 1x 10G connection to the iSCSI network. All 3 have the ThermalTake Engine 27 1U LP CPU cooler and Crucial memory. The whole setup is pretty quiet, I cant hear anything from the other side of the garage unless its dead silent. The T i7 chips are the low power ones. The entire rack only uses about 200watts average.
Interesting, thanks for the run through. That plinkusa front access case was the one which caught my attention, but the 1.5U OnLogic cases are pretty neat too. Very impressive to see what you've managed to fit within the constraints of the rack.

FWIW, I'm running 8TB drives in my setup (currently got 9 drives in two separate pools), so you should be OK with that too. I don't have any higher capacity drives to test with at the moment though, so I don't know about anything bigger.

Good luck with your future experimentation! :)
 
  • Like
Reactions: audiophonicz

audiophonicz

New Member
Jan 11, 2021
5
6
3
Cool. I'm also hoping that moving from an ATX PSU to the server PSU will improve efficiency a bit. The 450W 80+ Gold Seasonic unit I'm using isn't terrible, but it's at the low end of the load/efficiency curve, so if I can drop a few watts that'd be nice.
Honestly, while Im not a fan of seasonic, I doubt a HP server psu is going to be any better, and probably a bit louder. But id be interested to see if im wrong.

FWIW, I'm running 8TB drives in my setup (currently got 9 drives in two separate pools), so you should be OK with that too. I don't have any higher capacity drives to test with at the moment though, so I don't know about anything bigger.
8TB drives in the HP cage? nice, I swore i read the cards only do 8TB, and the DL380 G8 maxes out at 6TB, but that could have been referencing the stock HP raid card or just out of date. Good to know. Incidentally, yesterday I just got my first pair of 16TB drives and put them into my backup NAS with 0 issues. That one has a 9202-16e (direct connect breakout cable) and since its the previous 2008 chip family, I would expect the 2308 chip family 9206 and 9207 series to play nice as well.
 
  • Like
Reactions: amp88

amp88

New Member
Jul 9, 2020
20
13
3
Honestly, while Im not a fan of seasonic, I doubt a HP server psu is going to be any better, and probably a bit louder. But id be interested to see if im wrong.
My random guesstimation is about one hard drive worth of power saving, so <10 watts going from the 80+ Gold Seasonic to an HP Platinum Plus 460W unit, but I'll see if there's any noticeable difference.

8TB drives in the HP cage? nice, I swore i read the cards only do 8TB, and the DL380 G8 maxes out at 6TB, but that could have been referencing the stock HP raid card or just out of date. Good to know. Incidentally, yesterday I just got my first pair of 16TB drives and put them into my backup NAS with 0 issues. That one has a 9202-16e (direct connect breakout cable) and since its the previous 2008 chip family, I would expect the 2308 chip family 9206 and 9207 series to play nice as well.
Yeah, I've got a mix of 8TB drives in two pools in the HP cage (with the SAS expander backplane, 647407-001). One pool has shucked 5400rpm WD white label drives and the other has a mix of 7200rpm drives (Seagate SATA and HGST SAS), and I haven't had any issues so far. Those spec sheets and upper limits for capacity are often outdated/misleading, unless it's a known hard cap (e.g. the 2TB limit on some older hardware), but I'm happy to report 8TB works here. I don't know when I'll get my hands on any larger drives; I'm gonna take some time to let my wallet recover! ;)
 
  • Like
Reactions: audiophonicz

audiophonicz

New Member
Jan 11, 2021
5
6
3
So... this thread finally got me the motivation to refresh my DAS. I hope this helps someone else like it did me.

The bad news first: I have confirmed that the HP DL380 G8 Expander cage does NOT recognize 16TB drives. :(
Besides that, upon further inspection of the HP DL380 chassis, the fan mounts are pushed too far back for me to fit everything I need, so I went ahead with the alternate plan. Progress is as follows:

got me some 80mm fans like you have, and a little PCI bracket fan controller thingie. Its basically just a pot with the right connections. I made a bracket to mirror the stock type, spacing them out so it pulls thru the little slits and also covers some of the expander chip heatsink. Its held down by 3x standard M3 case screws threaded into the shelf plate. The first fan has a grill to keep the miniSAS connectors away from it.

DAS1.jpg
DAS2.jpg


I cut and soldered the fan wires to the connector for the fan adjustment, which was kinda pointless cuz theyre inaudible at full blast, but turns out it has a nice little blue led so I know when the unit is powered on. The old PSU had a green one but the new PSU has no lights. Mounted the adjustment on the rear side of the case for easy access thru the side panel.

DAS3.jpg


Of course I decided to use a straight 12v PSU so for the 3.3v line I got some of these little guys and integrated one into the wire harness. Just wrapped some electric tape to hold it to the connector. There is another one pictured on the side so you can see how tiny they are. Theres also a little red LED on these so you can verify theyre working.

DAS4.jpg
DAS5.jpg

All finished, so far so good. Thanks for yous guys contributions and getting me motivated to fix my DAS. Good luck to you on yours!
(I tried to get a pic of them all lit up but they just move too fast...)

DAS6.jpg
DAS7.jpg
DAS8.jpg
 

Attachments

  • Like
Reactions: amp88

amp88

New Member
Jul 9, 2020
20
13
3
So... this thread finally got me the motivation to refresh my DAS. I hope this helps someone else like it did me.
Very nice job, looks great. Well done. :)

The bad news first: I have confirmed that the HP DL380 G8 Expander cage does NOT recognize 16TB drives. :(
Hmm, I'm just looking through your post and I see you've used a voltage regulator to provide the 3.3 volt input that's in the original connector. I don't know if you've already considered this, but there's a possibility the 16TB drives you have won't function when they're supplied with a 3.3 volt input. If you've already had them running in your backup NAS with a 3.3 volt input in the power connector then just ignore this because it won't be the issue.
Some drives, especially shucked (e.g. the WD white label drives that come in external enclosures) or newer enterprise drives, simply won't spin up when connected to a power source which has a 3.3v input. I don't know if there's a comprehensive list anywhere, but if you search for the model number of the drive along with "3.3v", "3.3v fix", "PWDIS" or "Power Disable" you may get some information. If you can't find a definite answer one way or the other you can test for it quite easily by covering the 3.3v pins on the SATA power connector of the drive with some insulating tape (ideally something that's not too thick, like Kapton, just so it's not too tight a fit inside the SATA power connector). Here's a video from Art of Server on YouTube with some more explanation, and a short guide on Instructables.
 

audiophonicz

New Member
Jan 11, 2021
5
6
3
Hmm, I'm just looking through your post and I see you've used a voltage regulator to provide the 3.3 volt input that's in the original connector. I don't know if you've already considered this, but there's a possibility the 16TB drives you have won't function when they're supplied with a 3.3 volt input. If you've already had them running in your backup NAS with a 3.3 volt input in the power connector then just ignore this because it won't be the issue.
Some drives, especially shucked (e.g. the WD white label drives that come in external enclosures) or newer enterprise drives, simply won't spin up when connected to a power source which has a 3.3v input. I don't know if there's a comprehensive list anywhere, but if you search for the model number of the drive along with "3.3v", "3.3v fix", "PWDIS" or "Power Disable" you may get some information. If you can't find a definite answer one way or the other you can test for it quite easily by covering the 3.3v pins on the SATA power connector of the drive with some insulating tape (ideally something that's not too thick, like Kapton, just so it's not too tight a fit inside the SATA power connector). Here's a video from Art of Server on YouTube with some more explanation, and a short guide on Instructables.
This kills me... why you ask?

Because, yes, the 16TBs are specifically shucked WD Whites.

Because way back when, I couldnt find a pinout for those connectors so I actually bought an OEM one that specifically had colored wires instead of all black, and re-pinned my connectors based off that. That wire was red, not orange, so for the last year ive been running 5v to that pin. I only went with the 3.3v because your pinout pic had that pin at 3.3v. All my other WD Blues spin up fine, but since you mentioned specifically WD Whites...

and Because I did put the WD Whites into my backup NAS, and formated them... in a 3in2 hotswap bay that can be powered by 15pin SATA or 4pin Molex connectors, and of course i went with the 4pins that dont actually provide the 3.3v, so I cant say for sure whether its converting down to 3.3v for the internal SATA/SAS or not.

Now the only way i can confirm/deny this for sure is to pull it all out again and swap the 3.3v buck down for a 5v one (which i already purchased with the 3.3v in case I needed the 5v.) or cut it off completely. ...then again i do have tons of kapton.
 
  • Like
Reactions: amp88

amp88

New Member
Jul 9, 2020
20
13
3
This kills me... why you ask?

Because, yes, the 16TBs are specifically shucked WD Whites.
I'm about 99.9% sure that's the issue then.

Now the only way i can confirm/deny this for sure is to pull it all out again and swap the 3.3v buck down for a 5v one (which i already purchased with the 3.3v in case I needed the 5v.) or cut it off completely. ...then again i do have tons of kapton.
I'd definitely try Kapton on the pin from either of the 2 links above as a first resort.