[WTB] CSE-PTJBOD-CB3

[lastb0isct]

yeah, not all mainboards, but most of the supermicro boards can do it.
Problem with this solution is that the mainboards take up a lot of space compared to the cb3.
I did several conversions of intertech mining cases and with the cb3, i am able to fit 50 disks easily with more space between the drives to keep temps and noise down, even with sth small like a micro atx i would loose at least the last two rows, which would bring that down to more like 35 disks with similar spacing and a lot of wasted space.
But yeah around 190€ each is not cheap.

Do you have any pictures of the intertech cases customized? How did you do it? Did it require custom machining?

 

[UnknownPommes]

Do you have any pictures of the intertech cases customized? How did you do it? Did it require custom machining?

I am probably going to do a dedicated post soon.

Upfront, i dont want this to get too long so not gona explain everything here. Also yes i considered options with backplanes and i know that wiring can get messy sometimes, but with the right cable mangement it is fine.

Case,
So as base any high depth intertech case should be fine (all that are 68 or 695mm deep), they are all nearly identical and only in the front different, all parts that are easily removable via a couple screws, in therms of model numbers those are,
4129L, 4129N, 4W2, 4W40, 4F28,
with my favorite the 4w2, bc it is the cheapest and it also has two additional 80mm fan mounts in the back of the normal pcie slots.
Many miners used them and on local marketplaces i found several under 50€, with the cheapest one i ever bought at 25€.

Fans,
from my experience a 3, 3, 1, 2 configuration is the best for temp and noise (living room level quite) but 9 fans can get expensive, so if you dont care about noise or very low temp you can go with a 3, 0, 1, 0 configuration. Front, Mid, in front of sas expanders and cb3, rear exhaust in this order
Most of the intertech cases ship with molex fans, which are actually kinda usable but quite loud. If you want a little upgrade (to add pmw control) but still be on the cheap side, go with sth like Arctic P12 PMW, they are 30€ for a pack of 5 or if you want low noise and low temps you can go with noctua, generally go for pressure optimized fans.

Layout,
For the inside, i am basically removing everything from the case besides the walls and designed / 3d printed my own mounting system for the inside, everything is in a topload config with several rows of drives, for full rows that have no obstructions you can go with up to 15 disks for density or 12 for better temps. How many disks you will be able to fit in depends on what psu you are using and what other components like sas expanders take up space.

Basic Layout Plan


SAS Expanders,
this really depends on if you want to go with sas2 or sas3 6/12gb
i am usually going with sas3 bc it allows me to only use 4 uplink cables to the server and still dont bottleneck my hdds even when all are wrote to at the same time.
The absolute best option in my opinion are the adaptec 82885T, they are full 12gb sas offer 2 external ports at the back (that can be used as uplink) as well as 6 internal ports for a total of 24 disks connected per expander (yes there is a physical 7th port on the inside but cant be used)
Additionally they have a molex connector for power so you can directly power them of the psu without any pcie power adapters.
They are also cheap, i ususally get them from china for 60€ each. But basically any expander should work with some adaptions

Cable - Data
If we stay at 12gb we have to use sff8643 to 4x sff8482 break out cables or sff8643 to 4x sata.
This is one of the most expensive parts bc you need a lot of them and they are ~25€ each

Cable - Power
depending on the psu you use you might need to make your own cables, very short version, dont create a fire hazard, dont use cheap injection molded molex to sata connectors, do the math on how much you can put on one cable and how much one psu rail can handle, specially sas disks can draw quite a lot of power dont underestimate that!!!!!!!!

Also as you probably noticed by now we are using no backplane which means we have to wire each disk, also we are wiring on the top for better accessibility and cooling of the cables. Another thing for the power cables is, since they chain four to six disks you would be blocking other disks if you go in straight lines. This is why i connect them in a zig zag s, this allows to just unplug a single disks move the cables to the side and pull it out without interfering with other disks. More on that with plan in separate post, coming soon.

PSU
In my option two options
1. consumer psus, problem hdds need mostly 5v but basically all consumer psus can only provide a small part on the 5v rail (ususally 25A) bc all the high draw stuff like gpus is mostly 12v. This means depending on how many disks you want to use you will probably need 3 psus like the corsair rmx series ones, which not only blocks nearly the entire back but also is expensive. There are ways to reduce this like using staggered spinup, pls do your research there are tons of posts on this.
2. My prefered option, get used salvaged pdbs and psus from enterprise systems. If you are lucky you can buy a pdb from sth like a supermicro 847 for under 50€ add two supermicro superquite 1000w+ psus and you are good to go, some of them are rated over 100A at the 5V rail, so pleanty and you get redundant power and plat or gold efficiency.
Basically all of them are molex tho, so you will need to make or get adapter cables from molex to sata, again watch out, preferably make you own cables, it is not that hard, you can get packages of 100+ molex and sata clip on connectos for a couple bucks and good quailty cables for maybe 40. You dont even need to solder for that, there are ones that just snap on the cable you press a cap on and done, this also allows you to get them in the perfect spacing.

Control
The last point, how to control everything,
i am nearly always using the supermicro jbod board bc it is just super convenient, but sadly expensive, lets hope for a project that used pis or Arduinos for that
JBOD Board 2, if you just want to use the physical power button and sth to power you pmw fans (~70€)
JBOD Board 3, if you want full ipmi with remote power on / off and fan control (160-200€)
Nothing, if you are using the stock intertech molex fans you only need a power switch, you can get those simple 24pin to power switches for 10€ on amazon

So yeah thats it, i originally started this several years ago and have bin constantly improving this, so far i built 14 JBODs like this, some for my own datahoarding problem, some for friends

Price can vary drastically on what parts you are using, the cheapest one i did was around 350€, but most are around 600€ for the entire jbod without disks, which is not to bad in my option bc you can built a jbod that

- fits ~55 3.5" disks
- can be 12g with future easy upgrades to even more
- can be living room level quite
- fits in normal racks (70cm length)
- incredibly flexible
- much lower vibration and temp than most enterprise jbods if you dont go for max density

(btw you will need a 3d printer, preferably with experience in abs and tpu printing)
TPU (elastic filament) is used on some parts of the interior for vibration dampening and helps a lot but is pain to print in dual extrusion with ABS.


Anyway thanks for reading

 

[mattventura]

It might be neat to use just a single server PSU and a homegrown PDB (since the only thing it would need to do in this case is provide the 12v -> 5v conversion and tell the PSU to turn on). The PDBs are often more expensive than the PSU in terms of secondhand prices.

 

[tinfoil3d]

When my 28-bay 3U JBOD is fully populated the top temps are ~37C, with ambient at a controlled constant 22C. Fans are attached to 12V directly, just the yellow(rpm) wire is connected to SAS backplane(as they were before) to indicate a fault.
I think there's just way too much tinkering you guys do with these platforms. If you fill all the slots, just run it at full speed. If you don't, maybe use a different platform(a server chassis) with more granular fan control. I initially wanted to lower the fan speeds but when i filled the whole thing some drives were above 40C at idle even with default control. TBH i don't understand the purpose of CB2 either. It doesn't seem like it's doing anything at all. Maybe that's what automatically powers up the PSUs, I didn't try removing it! In my case when I power up either of the PSUS it just starts up. I've never touched the front panel buttons.

 

[mattventura]

If you don't, maybe use a different platform(a server chassis) with more granular fan control

But the purpose of CBs is so that you can use a server chassis as a disk shelf.

Maybe that's what automatically powers up the PSUs, I didn't try removing it! In my case when I power up either of the PSUS it just starts up. I've never touched the front panel buttons.

That's the main point of CB1/2. The front header stuff is neat too, but you can also power on any PSU by shorting the green pin on the ATX 24-pin to ground. You just won't be able to power cycle it without pulling the plug(s), but that's a pretty minor inconvenience.

It's mostly the CB3 that's interesting, since it can monitor PSU/fan/etc over the network and remotely power cycle. But it's a little overpriced for that job compared to other options.

 

[tinfoil3d]

I'd say CB3 costs more than JBOD itself in my case... And I use PDUs to do all the resetting stuff but wonder if that can even happen that SAS backplane dies(and reset helps in that case).
I'd love to try netapp/dell but I already have supermicro. If that fails for whatever reason I may switch to dual-controller JBODs