48 Drive 4U Chassis: IBM Slicestor 3448 aka Chenbro RM43348

llowrey

Active Member
Feb 26, 2018
111
75
28
TL;DR - Upgraded version of NR40700 with SAS3 (12Gbps) backplanes. Heavy and loud.

I recently picked up an IBM Slicestor 3448 (Chenbro RM43348) on eBay. They're currently listed at $595 but the seller accepted my Best Offer which was considerably less. It's definitely more expensive than a Chenbro NR40700 but the bandwidth upgrade is worth the delta, imo.

profile.png

box.jpg box inside.jpg box inside open.jpg box inside open ducts.jpg box inside open bare.jpg

Included:
  • 48 drive 4U chassis
  • (2) 1100W 1U 80+ Gold power supplies
  • (1) Dual LGA2011 motherboard (unknown, likely custom) with heatsinks and ducts
  • (2) LSI SAS9300-8i SAS3 12Gbps HBAs
  • (4) Mini-SAS HD cables
  • (4) Dual axis (counter rotating) 80mm fan modules
  • (2) 24 drive backplanes with SAS3x36 12Gbps SAS expanders
NOT Included:
  • No drive caddies (top load or rear 2.5")
  • No CPUs
  • No RAM
  • No power cables
  • No rails
As mentioned, this is a Chenbro RM43348 that is sold by IBM. Here's the product page on the Chenbro website:
Chenbro - Products

They even have a marketing video on Youtube complete with dramatic music:


Backplanes

My main interest in this unit, as opposed to the Chenbro NR40700 , is in the backplane expanders. I'm new to SAS expanders so their capabilities may be entirely conventional but I was impressed none the less. They are 36 port SAS3 12Gbps and all 36 ports can be wide (dual link) and either initiator (HBA) or target (drive). They are wired up for 24 drives each with 3 mini-SAS HD ports for 12 initiator ports. This allows for 13.2GB/s (1,100MB/s x 12) per backplane or 26.4GB/s for the chassis!

Another attribute of interest is that the expanders buffer slower devices (3Gbps and 6Gbps) and transfer to the initiator at full 12Gbps. I confirmed that under linux my 6Gbps SATA drives reported a negotiated link speed of 12Gbps. This means that the initiator links can run at full speed at all times instead of dropping down to match the target's speed.

The expanders do not support SATA-1 or SAS-1 devices (1.5Gbps). The datasheet for the expander does not list 1.5Gbps capability and my old SATA-1 drives are not detected at all.

The drive connectors appear to be wired for wide links so fast SSDs should run at full speed.

The backplanes include (1) red LED per drive adjacent to the drive connector and the LED is controlled via SGPIO. I was able to control the LEDs using the linux ledctl utility. The front panel has 24 dual LEDs (red/green) per backplane. The green LED indicates activity and the red LED mirrors the backplane LED.

The backplanes are mounted on a tray which is (relatively) easy to remove.

backplanes.jpg backplanes closeup.jpg backplane expanders.jpg backplane cabling R.jpg backplane cabling L.jpg racked 2.jpg backplanes tray.jpg

Drive Trays

No drive rails/trays/caddies are included and, as far as I can tell, are simply not available anywhere. I ended up designing and 3D printing rails. There are 3D models for the NR40700 but I'm not sure they would have worked since they appear symmetrical. This chassis's drive slots are asymmetrical with the width being 18mm for the side closest to the connector and 15mm for the other side. Plus, with the LED on the backplane I wanted a light pipe. The light pipe is just a 2mm acrylic rod.

rails 1.jpg rails 2.jpg drive.jpg

This first pass at rails isn't ideal but works in a pinch. There really isn't clearance for screws so I'm stuck with pins but the printed pins aren't exactly strong. I might go to stainless pins at some point.

Motherboard

The included motherboard is dual LGA2011 with 16 DIMM sockets and 3 x16 slots (not sure if they're also wired x16). I didn't have any CPUs to install so I could only verify that the IPMI was operational. The board MAC addresses indicate the origin as https://www.cas-well.com/. I've never heard of them but my guess is that they made a custom board for IBM. The BIOS is dated 2016 and it's unlikely any updates will be available.

box inside open ducts.jpg box inside open bare.jpg

Cables
  • (1) 24-pin ATX power
  • (2) 8-pin 12V (CPU power)
  • (1) cable with (2) SATA power
  • (1) cable with (1) 4-pin Molex
  • (1) USB 8-pin header cable for (2) front panel ports
  • (4) 5-pin fan connectors with (2) tach wires (fits standard 4-pin header but only signals 1 fan in that case)
  • (1) PM SMB
  • (1) Front panel (Power LED, ID LED, System Fault LED, Power Button, Reset Button, ID Button, IPMI SMB)
The front panel connector didn't match my motherboard so I had to construct my own cable. Fortunately, Chenbro has good documentation on the pinout and it was a simple task.

front panel cable 1.jpg front panel cable 2.jpg

Power Supplies

There are (2) 1,100W 1U power supplies that are 80+ Gold rated. According to the Chenbro documentation the 1,100 W rating is when supplied with 220VAC and it's only 800W when supplied with 110VAC. These guys are LOUD, as you would expect. They power up at full tilt (or nearly) then slowly ramp down to a near tolerable level. Interestingly, the fans do run when the system is powered down, but at a low speed.

power supply.jpg

The power supplies connect to a module via edge connectors which in turn connects to the two backplanes via edge connectors. The module provides breakout cabling. It is very well documented by Chenbro.

power distribution cabling.jpg

Fans

There are (4) dual axis, counter rotating, 80mm fan modules. Apparently, I didn't fully seat one of them because when I powered up the system the fans came on at 100% wide-open-throttle. Holly smokes they were LOUD and moved more air than I thought possible. If this thing was just a few pounds lighter I suspect it would be capable of powered flight. I have zero concerns about keeping 48 drives cool. Fortunately, once I ensured that all 4 modules were properly seated, the fan speed was much more tolerable. Even at the lowest fan speeds, you won't want to be in the same room for prolonged periods. The rest of the gear in my rack is loud so I really don't notice but I definitely did when I was running it in my office.

fan module.jpg

Racking


The chassis top panel is split front to back, allowing the top front to be opened without the chassis fully outside of the cabinet. The chassis is DEEP but this isn't all bad since I can push the two chassis above this one a bit forward to access the back half of the top panel to access the motherboard. I find that very convenient.

racked 1.jpg racked top.jpg racked rear.jpg
(don't hate on my cable mismanagement... I'm working on it!)

Miscellaneous

Also included are (2) LSI 3008 8 SAS3 12Gbps port cards. The firmware was old but was IT. It was trivial to update to the latest firmware. The inclusion of these cards almost pays for the price delta.

I bought a set of rails for a Chenbro NR40700 and they fit just fine. They were easy to source when I bought them but appear to be out of stock everywhere now so they may be hard to acquire.

The chassis has a removable motherboard tray with a complete set of standoff positions. I came up short by one standoff for my motherboard. It takes a rather peculiar standoff but it's the standard Chenbro standoff so, again, easy to source. The tray will accept boards from Micro-ITX all the way up to E-ATX. The tray also includes (2) 2.5" hot-swap bays. The tray supports low-profile cards only.

motherboard tray.jpg

Updated 2020-08-17: My 3D printer models for drive rails are available at the link below. Both STL and f3d files are included.


The LED light pipe is a 2mm acrylic rod.
 
Last edited:

azev

Active Member
Jan 18, 2013
757
226
43
nice write up, I really like this setup. Now, if only you can get the drive tray or any kind of aftermarket tray that will work I might just get one.
 

llowrey

Active Member
Feb 26, 2018
111
75
28
Very interesting! Any idea on how easy this would be to convert to JBOD?
No conversion necessary, it's JBOD already. The included HBAs came with IT firmware loaded so it's clear that the chassis is meant for JBOD use cases.
 

pchangover

Member
Dec 12, 2016
49
24
8
MD
Damn you may have sold me but my only hang-up right now is the hard drive caddy issue. Did you try contacting Chenbro?
 

wallabi

New Member
Jan 3, 2020
11
10
3
lol i bought 1 of these the other day on a whim.. (like a instant buy click)... did not think caddies would be such a issue for something, after some google searches i find this post.. did not even try to give them a lower offer than the 595 lol :p DARN.. but yea the sas controllers alone.. let alone a 48bay case is well worth the money :) i am currently using a hp msa60 storage works with 12x2tb drives and a hpz800 with a hp raid controller for my plex server setup.. this will make a nice all in one upgrade solution supporting way more space, more cores, better performance and just about the same power bill :)
 

pchangover

Member
Dec 12, 2016
49
24
8
MD
lol i bought 1 of these the other day on a whim.. (like a instant buy click)... did not think caddies would be such a issue for something, after some google searches i find this post.. did not even try to give them a lower offer than the 595 lol :p DARN.. but yea the sas controllers alone.. let alone a 48bay case is well worth the money :) i am currently using a hp msa60 storage works with 12x2tb drives and a hpz800 with a hp raid controller for my plex server setup.. this will make a nice all in one upgrade solution supporting way more space, more cores, better performance and just about the same power bill :)
Are you planning on using the motherboard? If so, what CPUs are you going to use?
 

wallabi

New Member
Jan 3, 2020
11
10
3
Are you planning on using the motherboard? If so, what CPUs are you going to use?
going to try e5-2628L v4 low power 12 core cpu's to run gentoo linux as a zfs and the plex server.. the goal i had was to upgrade my previous setup without doubling the powerbill... i was looking at a lot of options, cheap 24bay netapps, another msa60(12bay+400w psu), i was tempted to buy this HP MDS600 on ebay for about 200 bucks... but that thing idles at 300w from the wall with no drives lol. quad 1600w power supplys for a jbod is a little overkill...

--edit ill post back if it accepts those cpu's.. the only ibm doc i could find says it came with E5-2698v4, so im just gonna guess anything e5-26xx v4/3 should work...
 
Last edited:

pchangover

Member
Dec 12, 2016
49
24
8
MD
going to try e5-2628L v4 low power 12 core cpu's to run gentoo linux as a zfs and the plex server.. the goal i had was to upgrade my previous setup without doubling the powerbill... i was looking at a lot of options, cheap 24bay netapps, another msa60(12bay+400w psu), i was tempted to buy this HP MDS600 on ebay for about 200 bucks... but that thing idles at 300w from the wall with no drives lol. quad 1600w power supplys for a jbod is a little overkill...

--edit ill post back if it accepts those cpu's.. the only ibm doc i could find says it came with E5-2698v4, so im just gonna guess anything e5-26xx v4/3 should work...
Any luck with those CPUs?
 

wallabi

New Member
Jan 3, 2020
11
10
3
Any luck with those CPUs?
did not get them yet....

but i got a bunch of caddy's that i tried...

if u cut the bottom off a dell optiplex caddy (blue plastic things) it fits perfectly snug. nothing else i tried even remotely fits. a hp 774026-001 is to small.. u could probably put 2 dabs of hot glue on both sides so it comes in contact with the slots and it would work.

 

pchangover

Member
Dec 12, 2016
49
24
8
MD
did not get them yet....

but i got a bunch of caddy's that i tried...
I like the attempt there! I actually have one of these sitting around so I can verify that as well.

Right now I'm working through some performance issues with freenas.

What are you going to be running on this Wallabi?
 

wallabi

New Member
Jan 3, 2020
11
10
3
I like the attempt there! I actually have one of these sitting around so I can verify that as well.

Right now I'm working through some performance issues with freenas.

What are you going to be running on this Wallabi?
--edit i found these also, but do not have any.. they look like they might work !
Supermicro MCP-220-94601-0N 2.5" or 3.5” Drive Tray For CSE-946 Chassis

they look like they might fit, i just orderd 1 so ill let u guys know!!


a gentoo build using zfs and a plex server running on the box aswell, it will also use nfs,afs,ftp and samba..
the only linux i use is gentoo, takes along time to set up workstations as adding use flags can make circular dependency hell.. but for servers its not really to much trouble... i have heard good things about freenas and unraid, but i have been using gentoo long before those things came to life, so i do not mind the couple hours of cfg file writing.. u should try gentoo.. the performance u get from compiling everything native is pretty nice!! :) the gentoo handbook is very well written, easy to follow and can have you up and running in no time.. custom kernels can take awhile (going through hundreds of lines enabling stuff only your server has to cut the size down)... not really needed unless ur doing somthing super specific where size or security matters, using genkernel and just chaning the cpu and core kernel options for ur machine is all u need to do really..
 
Last edited:
  • Like
Reactions: pchangover

Bert

Active Member
Mar 31, 2018
429
165
43
43
This is such an awesome system and thanks for the write up. May I learn the best offer price accepted for this server?
 

wallabi

New Member
Jan 3, 2020
11
10
3
ok i got the Supermicro MCP-220-94601-0N caddy today, it does fit.. perfectly.. except the latch lock does not have anything to grab on, i cant test it as i do not have the cpu's for the mobo yet.. but the plastic light rods seem to line up aswell.. cant tell fore sure.. but it looks like it does ( on the bottom its in the same exact spot, but on front they are spaced) they fit decently snug enough without the latch bar having somthing to latch to. the lid does close with this. they are kinda pricy at 12 dollars each though.. when i get the cpu early next month ( should be around the 8th) ill post back if that model cpu works and of course the light strip on the caddy. (im like 90% sure its a exact line up)..