Intel 36 port expander usage issue Pls Help! ( Intel RES3tv360 expander)

[james23]

arrgg, what an annoying prior 48 hours.

I have a Adaptec 8805 that has been hooked to a Intel RES3tv360 (36 port , SAS 3 expander) for a little over a year with good results. This weekend i received 8x " new" 3tb Hitachi Drives i wanted to add to my array/the expander.

So i have 2 x questions / issues:

1) is this true that 24 Hard drives is the max that can be connected to the expander? ( i was shocked to find this tiny line buried in the manual for the RES3TV360. Why call it a 36 port expander and never mention that 24 drives are the max anywhere (but for this single, small line buried deep in the manual) :



2) Am i using the right port on the expander to connect to my Adaptec 8805? The manual says this CN4 port is only 1x1 not 1x4 so am I now limiting the bandwidth to my raid card from the expander by using this 1x1 port vs using a 1x4 port? (btw, i dont recall exactly why i choose this specific port to connect to my raid card, i do recall that when first setting this server up a year ago, i had to keep moving the connections around on the expander card until i found a setup where all 20 x of my inital drives shows up, since then its worked fine):



Also if anyone else is using a Intel RES3tv360 connected to a raid card, please if you could post your cable connection setup and any other relevant info you can (ie number of HDDs , performance , any issues or qwerks).

Thank you all for your time and help with this.

EDIT: btw, those screen shot snips are both from the manual PDF for the INTC expander, link is at:

http://download.intel.com/support/motherboards/server/res3tv360/sb/res3tv360_ug.pdf

(screen shots have my notes added in RED and iBLACK)

 

[sfbayzfs]

So how many drives have you had attached to it for the past year? (and are they spinners like the new 8 drives, or SSDs?)

I actually do have one of these, still new in the box, but I haven't used it yet, and may not ending up doing so, since SAS2 is fine for my needs, and seems to be a lot better tested than SAS3. I have used the RES2SV240 extensively though in multiple chassis', as well as HP SAS expanders and Supermicro backplanes with integrated expanders.

For the max 24 drives note in the manual, these Intel expanders, while more available directly as parts than other manufacturers' SAS expanders, are geared towards particular Intel chassis', and it is very likely that the 24 drives max is just the max of the chassis it is supposed to go into. As far as I understand it, SAS expanders of all generations are basically just a switch, so using a particular in or out port doesn't really matter, it is just a cabling recommendation. I believe the intended use of this expander is in an Intel 24-bay chassis, using 2 ports to dual link to a single controller or multipath to 2 controllers, 6 ports to connect to 24 drive bays, and the remaining port for downstream daisy-chaining to more expanders / chassis'. The manual for the SAS2 RES2SV240 clearly states which 2 ports are supposed to be linked to the controller and the other 4 are for the drives, but I have had one working perfectly single linked to a controller on one of the drive-only ports, and linked out to 20 drives via the other 5 ports with no issues whatsoever for several years now.

That takes us to your second point about CN4 - I believe that is a type-o in the manual, especially since it lists that connector as SAS ports 16-19, but if it was only x1, it would only have 1 SAS port, not 4 of them.

One point of note about LSI HBAs which probably doesn't apply to your Adaptec RAID card, I recently discovered that the BIOS only lists drives which are on expanders directly connected to the card during bootup, but not daisy-chained expanders, so it can make it look like a Supermicro SC847 36-bay chassis only has one backplane working depending on how it is cabled. If you go into the BIOS, it will list all of the drives on both backplanes though. I suspect this is a precaution against trying to list hundreds of drives in daisy-chained JBOD units at startup.

 

[sfbayzfs]

your quote above may be true 99% of the time, but i can assure you that its not true for ALL expanders (i so wish it were true though, and totally should be)). see my thread from last week below, where i just recently bumped (more like rammed or crashed) into this exact issue, with an unfortunate conclusion:

Intel 36 port expander usage issue Pls Help! ( Intel RES3tv360 expander)

TLDR: the Intel 36 por RES3TV360 SAS3 expander is called a 36 port expander, it has 9 x 4 lane SFF HD ports... however the ONLY mention of the fact that it will only support 24 HDDs (and i mean ONLY, i cant even find this on the net anywhere) is a single , very short sentence with no explanation or reasoning (see screen shot in my thread above). 2 weeks ago i tried everything to bring up 28 (or even 25 ) drives through the expander, NO LUCK at all... (im not very happy with intel right now). i had to end up tossing an extra 8405 card into the server to run 28 drives , outside the expander (so now i have 25 drive on my adaptec 8805 going to the expander, and 4 drives on a adaptec 8405 card) ... its a waste of a good 8405 card i constantly used for testing and such.

Im actually going to open a support case on this with Intel and see what their official reasoning is.

(Quoting from another thread you mentioned this issue in to keep the info together for future searchers)

Have you tried updating the firmware on the expander? I know Intel is supposedly good about providing updates.

I also heard that Intel went with a PMC chip for their SAS3 expanders, not an LSI chip like their SAS2 expanders. I have one of these new in the box, plus one of the RES3FV288 version, (as well as the HP SAS3 expander, which I believe uses the same PMC chip as these Intel ones) but in all cases I was planning to daisy-chain them to SAS2 expander backplanes, and not connect drives directly to the SAS3 expanders, I'll let you know when I figure out if that config works, but it may take me a while to get to that project.

 

[jupiturn]

Hi,
long time ago, but the well known working RES2CV360 is going to be EOL.
I agree it will be a marketing text 24 drives, as the server chassis supports only 24 drives.
I found this hardware user guide - https://www.intel.com/content/dam/support/us/en/documents/server-products/raid-products/res3tv360_ug_002.pdf - page 10 major components - "Any PHY can be SAS or SATA attached".
But has anybody tried to expand the RES3TV360 beyond the 24 drives to max 32 drives?

 

[james23]

hi, yes, that is what my OP is mostly about- i have a RES3TV360 , i was trying to attach more than 24 drives to it, but no matter what i did, it would only pass/show 24 drives (even with 28 or 32 drives attached to it, in any "plug configuration" I tried)
Im not sure if maybe you had a typo in your message, but exactly your question (as i read it) was answered in my 1st post.

lmk if any other ?
tks
btw: i still have my same setup running 24x7 (as my OP), and its been running flawlessly (ie adaptec 8 series RAID + RES3TV360, connected to 24x 3tb hitachi HUA drives... not one issue nor one hiccup, but it would be AWESOME if there were a way to add 8 more drives, so i could get all 32 ports..."some how"

But has anybody tried to expand the RES3TV360 beyond the 24 drives to max 32 drives?

 

[jupiturn]

Hi,
thanks for your reply. I'm sorry, I've forgot a question. Did it help to upgrade the firmware to a newer version?