Any issues or gotchas with using the 12-bay 6Gb/12Gb SAS cages (Inspur?) that have been on AliExpress for the last year or so? SATA support? Series?

[Dave Corder]

I've got my two Inspur 12-bay cages set up with a rear fan wall/enclosure, an OpenJBOD board, an Adaptec AEC-82885T expander, a pair of cables from the expander to the top two SAS ports on each cage, an HP AC-063-3 A power backplane and 460W PSU, and one hell of a hack job of power cables...

It is a tad finicky at the moment (loose pin somewhere on the 12v power line to the fan controller in the rear fan wall, I think), and it's a hell of a rat's nets behind everything at the moment. But it's all working, my TrueNAS ZFS pools are happy, Plex is back up, so the kids and wife are happy, which means I'm happy.

Side note: locate LEDs work fine on TrueNAS Scale with encled

http://imgur.com/a/eN7TVZo

 

[Dave Corder]

A couple of technical notes, in case anyone wants to know:

Expanders are Microchip/PMC-Sierra SXP 12G SAS series (PM8043, probably?)

➜  ~ lsscsi | grep enc
[1:0:0:0]    enclosu ADAPTEC  AEC-82885T       B059  -
[1:0:13:0]   enclosu HP       Gen8 ServBP 12+2 3.30  -
[1:0:25:0]   enclosu HP       Gen8 ServBP 12+2 3.30  -
[1:0:38:0]   enclosu HP       Gen8 ServBP 12+2 3.30  -
[1:0:39:0]   enclosu ADAPTEC  AEC-82885T       B057  -
[1:0:52:0]   enclosu PMCSIERA SXP 24Sx12G      RevB  -
[1:0:65:0]   enclosu PMCSIERA SXP 24Sx12G      RevB  -

They appear to be configured as 12 drive bays, two of the SFF-8643 ports as upstream to a controller/other expander, and one port downstream to four more drives (12+(2x4)+4 = 24 ports, which matches the model number).

➜  ~ sudo src/sdled/encled | grep "1:0:52:0" | sort -n --key=1.10
1:0:52:0/0    sdbf    fault_off    locate_off
1:0:52:0/1    sdal    fault_off    locate_off
1:0:52:0/2    sdau    fault_off    locate_off
1:0:52:0/3    sdap    fault_off    locate_off
1:0:52:0/4    sdam    fault_off    locate_off
1:0:52:0/5    sdan    fault_off    locate_off
1:0:52:0/6    sdaq    fault_off    locate_off
1:0:52:0/7    sdar    fault_off    locate_off
1:0:52:0/8    sdao    fault_off    locate_off
1:0:52:0/9    sdav    fault_off    locate_off
1:0:52:0/10    sdaw    fault_off    locate_off
1:0:52:0/11    sdax    fault_off    locate_off
1:0:52:0/12    ----    fault_off    locate_off
1:0:52:0/13    ----    fault_off    locate_off
1:0:52:0/14    ----    fault_off    locate_off
1:0:52:0/15    ----    fault_off    locate_off

The HP DL380e/p Gen 8 cages I have are 12 bays + two SATA ports on the back, and report 14 devices in encled - those two ports are intended to go an optical drive and a single internal SATA device, if I correctly understand the docs that I've seen.

I surmise that the Inspur cage is similar - 12 bays + one downstream port intended to go to something like a 4-bay internal drive cage or a 4-bay rear-mounted drive cage. Whether or not it is restricted to that usage, or if that's indeed the intention in the first place, is hard to know since documentation for these seems difficult to come by.

 

[seadog2441]

A couple of technical notes, in case anyone wants to know:

Expanders are Microchip/PMC-Sierra SXP 12G SAS series (PM8043, probably?)

➜  ~ lsscsi | grep enc
[1:0:0:0]    enclosu ADAPTEC  AEC-82885T       B059  -
[1:0:13:0]   enclosu HP       Gen8 ServBP 12+2 3.30  -
[1:0:25:0]   enclosu HP       Gen8 ServBP 12+2 3.30  -
[1:0:38:0]   enclosu HP       Gen8 ServBP 12+2 3.30  -
[1:0:39:0]   enclosu ADAPTEC  AEC-82885T       B057  -
[1:0:52:0]   enclosu PMCSIERA SXP 24Sx12G      RevB  -
[1:0:65:0]   enclosu PMCSIERA SXP 24Sx12G      RevB  -

They appear to be configured as 12 drive bays, two of the SFF-8643 ports as upstream to a controller/other expander, and one port downstream to four more drives (12+(2x4)+4 = 24 ports, which matches the model number).

➜  ~ sudo src/sdled/encled | grep "1:0:52:0" | sort -n --key=1.10
1:0:52:0/0    sdbf    fault_off    locate_off
1:0:52:0/1    sdal    fault_off    locate_off
1:0:52:0/2    sdau    fault_off    locate_off
1:0:52:0/3    sdap    fault_off    locate_off
1:0:52:0/4    sdam    fault_off    locate_off
1:0:52:0/5    sdan    fault_off    locate_off
1:0:52:0/6    sdaq    fault_off    locate_off
1:0:52:0/7    sdar    fault_off    locate_off
1:0:52:0/8    sdao    fault_off    locate_off
1:0:52:0/9    sdav    fault_off    locate_off
1:0:52:0/10    sdaw    fault_off    locate_off
1:0:52:0/11    sdax    fault_off    locate_off
1:0:52:0/12    ----    fault_off    locate_off
1:0:52:0/13    ----    fault_off    locate_off
1:0:52:0/14    ----    fault_off    locate_off
1:0:52:0/15    ----    fault_off    locate_off

The HP DL380e/p Gen 8 cages I have are 12 bays + two SATA ports on the back, and report 14 devices in encled - those two ports are intended to go an optical drive and a single internal SATA device, if I correctly understand the docs that I've seen.

I surmise that the Inspur cage is similar - 12 bays + one downstream port intended to go to something like a 4-bay internal drive cage or a 4-bay rear-mounted drive cage. Whether or not it is restricted to that usage, or if that's indeed the intention in the first place, is hard to know since documentation for these seems difficult to come by.

Great find.
Regarding the downstream port, they're probably for 2 bay 2.5 rear mount boot drives, think I have them last time I bought random thing from Taobao.

 

[KingKaido]

I've got my two Inspur 12-bay cages set up with a rear fan wall/enclosure, an OpenJBOD board, an Adaptec AEC-82885T expander, a pair of cables from the expander to the top two SAS ports on each cage, an HP AC-063-3 A power backplane and 460W PSU, and one hell of a hack job of power cables...

It is a tad finicky at the moment (loose pin somewhere on the 12v power line to the fan controller in the rear fan wall, I think), and it's a hell of a rat's nets behind everything at the moment. But it's all working, my TrueNAS ZFS pools are happy, Plex is back up, so the kids and wife are happy, which means I'm happy.

Side note: locate LEDs work fine on TrueNAS Scale with encled

http://imgur.com/a/eN7TVZo

Hey this is amazing! i see that you made the fan shroud for 2 of them stacked so it works for 120/140mm fans, did you upload the model onto makerworld? Also the psu box with the Adaptec AEC-82885T expander, is a cool idea

also where is the cheapest plalce to purchase the 12 bay cages? aliexpress seems cheap at face vaule but the shipping fees (atleast to the UK) is more expensive than the cage itself,

would it be cheaper on alibaba or going straight to taobao?

[Edit] i found this listing on taobao via superbuy, but without knowing the weight & dimensions, its hard to know the total price shipped

 

[Dave Corder]

Hey this is amazing! i see that you made the fan shroud for 2 of them stacked so it works for 120/140mm fans, did you upload the model onto makerworld? Also the psu box with the Adaptec AEC-82885T expander, is a cool idea

also where is the cheapest plalce to purchase the 12 bay cages? aliexpress seems cheap at face vaule but the shipping fees (atleast to the UK) is more expensive than the cage itself,

would it be cheaper on alibaba or going straight to taobao?

[Edit] i found this listing on taobao via superbuy, but without knowing the weight & dimensions, its hard to know the total price shipped

Yeah, the shipping to the US wasn't cheap either (and took a while on a boat from China), but overall it's not a bad price compared to any other options I've come across (at least in the US).

I haven't uploaded any models yet - I actually need to make the shroud a little bit deeper and re-print it, but I definitely plan to upload the files when I'm done.

The box is based around the AEC82885T expander and the OpenJBOD board from TheGuyDanish - the model for that is here: Stand-alone enclosure for OpenJBOD controller and Adaptec AEC-82885T SAS expander by Dave Corder | Download free STL model | Printables.com

 

[Gio]

Hey @Dave Corder I have just purchased this same enclosure and was setting it up last night. How many of the SFF-8643 connectors in the back are you using in your setup?

My HBA is 9500-8i which is SlimSAS - so I purchased‌ SlimSAS 8i S‌FF-8654 to 2 X Mini SAS SFF-8643 Cable

My question is, should I be using MiniSAS_0 + MiniSAS_1 or MiniSAS_1 + MiniSAS_2 in the backplane? any idea if this provides additional combined bandwidth without additional configuration? AI says I should be using MiniSAS_0 + MiniSAS_1 but unable to get StorCLI64 to confirm if the bandwidth is doubled, I only see a single SAS address for the enclosure and no way to tell if there is "multipath" going on.... any tips?

 

[Dave Corder]

I ended up using two of the SFF-8643 connectors. I don't recall their numbers off the top of my head, but they are the two that were facing the same direction.

I'm not sure if both are required, or if you can get away with just one...but at first I tried just one and upon initial boot my TrueNAS VM didn't see all the drives. I don't know if it was something wonky with TrueNAS or what, but I just went ahead and plugged in the second connector and rebooted and everything was fine and I didn't troubleshoot further.

I don't use StorCLI myself, so I can't say for certain, but I would expect there to be something somewhere that tells you how many lanes you have from your HBA to the expander chip on the backplane, and I would expect that to say 8 lanes. I could be wrong, though.

If anyone knows a nice way to view SAS topology on Linux, I'd love to hear about it (I'm too lazy and/or busy at the moment to google that myself).

 

[Quadrapole]

I ended up using two of the SFF-8643 connectors. I don't recall their numbers off the top of my head, but they are the two that were facing the same direction.

I'm not sure if both are required, or if you can get away with just one...but at first I tried just one and upon initial boot my TrueNAS VM didn't see all the drives. I don't know if it was something wonky with TrueNAS or what, but I just went ahead and plugged in the second connector and rebooted and everything was fine and I didn't troubleshoot further.

I don't use StorCLI myself, so I can't say for certain, but I would expect there to be something somewhere that tells you how many lanes you have from your HBA to the expander chip on the backplane, and I would expect that to say 8 lanes. I could be wrong, though.

If anyone knows a nice way to view SAS topology on Linux, I'd love to hear about it (I'm too lazy and/or busy at the moment to google that myself).

I've got my two Inspur 12-bay cages set up with a rear fan wall/enclosure, an OpenJBOD board, an Adaptec AEC-82885T expander, a pair of cables from the expander to the top two SAS ports on each cage, an HP AC-063-3 A power backplane and 460W PSU, and one hell of a hack job of power cables...

It is a tad finicky at the moment (loose pin somewhere on the 12v power line to the fan controller in the rear fan wall, I think), and it's a hell of a rat's nets behind everything at the moment. But it's all working, my TrueNAS ZFS pools are happy, Plex is back up, so the kids and wife are happy, which means I'm happy.

Side note: locate LEDs work fine on TrueNAS Scale with encled

http://imgur.com/a/eN7TVZo

Any chance you can upload the files for your 3d printed design?

How hot are your hot drives in that cage with that fan setup?

Mine are running pretty hot with 11 drives at 50c.

Thanks

 

[Dave Corder]

Any chance you can upload the files for your 3d printed design?

How hot are your hot drives in that cage with that fan setup?

Mine are running pretty hot with 11 drives at 50c.

Thanks

Yes, I'd be happy to share! I want to make a couple tiny tweaks before I upload them, so give me a day or two and I'll get that done.

 

[Quadrapole]

Thanks for helping a brother out. My wife thanks you too for saving me time. It's one less thing on my list of things to do and with 2 kids and a preggo wife, it's a long list.

 

[rc13502]

Sorry to revive a slightly old thread but I got one of the 12-bay ones (exactly as @seadog2441 took pictures of) this week but i’m a little confused on power. My unit came with a single 8pin mini fit to 3x Molex. On the unit there are two of the 8 pin mini fit plugs. Are both 8 plugs required to work?

I was thinking one cable from my psu to three molex would be too much for one line so i ran two lines from the psu with one feeding 2 molex and the other feeding one to balance it out a bit.

I appear to get power to the backplane and most of the drives spin up but not all of them.

I’m using a Cooler Master 750W SFX PSU so i have enough juice and extra ports for runs if needed.

If i do need a second 8pin mini fit to molex adapter anyone know where i can get one? All the ones on amazon etc are PCIe keyed.

 

[ewr_usa]

Sorry to revive a slightly old thread but I got one of the 12-bay ones (exactly as @seadog2441 took pictures of) this week but i’m a little confused on power. My unit came with a single 8pin mini fit to 3x Molex. On the unit there are two of the 8 pin mini fit plugs. Are both 8 plugs required to work?

I was thinking one cable from my psu to three molex would be too much for one line so i ran two lines from the psu with one feeding 2 molex and the other feeding one to balance it out a bit.

I appear to get power to the backplane and most of the drives spin up but not all of them.

I’m using a Cooler Master 750W SFX PSU so i have enough juice and extra ports for runs if needed.

If i do need a second 8pin mini fit to molex adapter anyone know where i can get one? All the ones on amazon etc are PCIe keyed.

In a similar situation, I can have 10 drives idling but if I put it under heavy I/O load then I start having power failures with only one 8 pin plug put in. Contacting the seller and hoping I can get a second cable with a proper fit otherwise I might have to figure out soldering and making my own with the first cable as a base for what I'm making. Hopefully it'll provide enough power for what I need. If anyone has a spare or advice on mangling together this kind of solution it would be greatly appreciated as I'm flying a bit blind.

 

[ewr_usa]

Sorry to revive a slightly old thread but I got one of the 12-bay ones (exactly as @seadog2441 took pictures of) this week but i’m a little confused on power. My unit came with a single 8pin mini fit to 3x Molex. On the unit there are two of the 8 pin mini fit plugs. Are both 8 plugs required to work?

I was thinking one cable from my psu to three molex would be too much for one line so i ran two lines from the psu with one feeding 2 molex and the other feeding one to balance it out a bit.

I appear to get power to the backplane and most of the drives spin up but not all of them.

I’m using a Cooler Master 750W SFX PSU so i have enough juice and extra ports for runs if needed.

If i do need a second 8pin mini fit to molex adapter anyone know where i can get one? All the ones on amazon etc are PCIe keyed.

I found it, if you still need it.

https://www.ebay.com/itm/377106761731