EU HGST 4U SAS3 12GB/s Storage Enclosure 60 bays £299

Rahvin9999

Member
Jan 14, 2016
79
29
18
Rotterdam, The Netherlands
From the https://www.hgst.com/sites/default/files/resources/4U60-Storage-Enclosure-DS.pdf

Some models refer 512e sector size or 4Kn
Do you think it's only about which integrated drives that comes with it ? or if it's an hardware requirement of the unit (depending of it's model number) ?
Just the drives that come with it. The SAS Expander in the box could care less.


Mine arrived today.
Its pretty big, Without the cable management arm attached the enclosure itself measures 87cm.
Dont have QSFP+ to SAS cables in yet. So will have to wait to do tests until they do.
Did turn it on, Started out really loud but then the fans started to go down in speed.
The noise is quite acceptable for me.
Some more pictures for those who are interested,

IMG_4975.JPG IMG_4985.JPG IMG_4981.JPG IMG_4983.JPG IMG_4976.JPG IMG_4979.JPG
 

nephri

Active Member
Sep 23, 2015
535
104
43
42
Paris, France
it's seem to have 2 distincts "rear" module with each 2 QSFP connectors.
That mean you can link 16 SAS3 ports (4 x 4) ?
If it's the case and you want to connect all ports, you need 2 9300-8e :-/ ouch

Someone know if some of theses ports are only used for HA or they are also used for performance ?
 

Mirabis

Member
Mar 18, 2016
122
6
18
26
it's seem to have 2 distincts "rear" module with each 2 QSFP connectors.
That mean you can link 16 SAS3 ports (4 x 4) ?
If it's the case and you want to connect all ports, you need 2 9300-8e :-/ ouch

Someone know if some of theses ports are only used for HA or they are also used for performance ?
Mine arrives Monday, but I already received 2 QSFP connectors xD... And 1 9300 on the way.. Could always fill left side (30) and next year fill the other once the 9300 drops in price haha


Sent from my iPhone using Tapatalk
 

Rahvin9999

Member
Jan 14, 2016
79
29
18
Rotterdam, The Netherlands
it's seem to have 2 distincts "rear" module with each 2 QSFP connectors.
That mean you can link 16 SAS3 ports (4 x 4) ?
If it's the case and you want to connect all ports, you need 2 9300-8e :-/ ouch

Someone know if some of theses ports are only used for HA or they are also used for performance ?
Its dual ported sas, thats why the 2 module.
each module has an uplink and downlink port.
Depending on how they configured the SAS Expander. It might be able to dual link it on one module. Some are hardcoded tho.
My supermicro 847 backplanes dont care which port I use for uplink.
Ill try it when my cables arrive.

Mine arrives Monday, but I already received 2 QSFP connectors xD... And 1 9300 on the way.. Could always fill left side (30) and next year fill the other once the 9300 drops in price haha

Sent from my iPhone using Tapatalk
Ha, I'm starting out with 26x8TB and will grow from there.
 
  • Like
Reactions: Chuntzu

Sleyk

Well-Known Member
Mar 25, 2016
1,033
417
83
Stamford, CT
Yeah Fritz, was following this thread to see if anyone from the states snagged one of them too. I just wish it was here in the US. I still think this thing is gorgeous. The price is nice too!
 

Mirabis

Member
Mar 18, 2016
122
6
18
26
Its dual ported sas, thats why the 2 module.
each module has an uplink and downlink port.
Depending on how they configured the SAS Expander. It might be able to dual link it on one module. Some are hardcoded tho.
My supermicro 847 backplanes dont care which port I use for uplink.
Ill try it when my cables arrive.



Ha, I'm starting out with 26x8TB and will grow from there.
haha nice drives : ) I'll start with 5 or so
 

nephri

Active Member
Sep 23, 2015
535
104
43
42
Paris, France
i would like to know if it's be able to do a dual link instead of dual ported...
it's not the same things to have 8 lines for 30 disks instead of 4 lines for 30 disks....
 
Last edited:

Mirabis

Member
Mar 18, 2016
122
6
18
26
i would like to know if it's be able to do a dual link instead of dual ported...
it's not the same things to have 8 lines for 30 disks instead of 4 lines for 30 disks....
Page 43 of manual:

Enclosure Storage Module (OUT) is used to connect to another bay's Enclosure Storage Module (In) -> so I guess 2 (out) of them are for daisy chaining and the other 2 (in) are for connecting to the Host?

Section 8.6 & 8.7 also seem to confirm this, but it's not written explicitly xD

https://www.hgst.com/sites/default/files/resources/4U60-Storage-Enclosure-User-Guide.pdf
 

nephri

Active Member
Sep 23, 2015
535
104
43
42
Paris, France
That mean this enclosure is for spinning disks.
30 spinning hdds will not saturate 4x12Gb/s links. With SSD it's not the same story...
 

Rahvin9999

Member
Jan 14, 2016
79
29
18
Rotterdam, The Netherlands
Did some quick initial testing.

Power usage:
Idle no disks, fans spun down to low - 72W ( europe so 230/240v)
Idle no disks, fans at max - 235W
( thats with both psu's connected and both expanders linked up)

Idle no disks, both PSU's instaled. but only one powered - 52W
The fans are on a shared powerbus. So you only need power to one PSU to have the fans in the other spin.
If you physically remove either a sas expander module or a PSU the remaining fans go to 100%

Noise:
I would call it quiet.
Makes a tad less noise then my supermicro SC826 and SC847 cases with PWM fans set to quiet mode.
Goes up to blow dryer level if the fans run at full speed.

In/Out ports:
As I expected the Enclosure doesnt care in which port you stick your SAS cable. I can use the In or Out port. It links up and the Enclosure and disks are visible and usable.

SATA disks:
They work, ofcourse you can only see it on one of the expanders (left one viewed from the rear) since SATA disks are single ported.
Ran badblocks against a few disks to generate some load.

Dual Linking:
Seems to work. Borked my lsiutil on the test server. so cant list the output till I figure out why its borked.
The redundancy part works. Can say that much
Have 2 links going from a 9200-8e controller to the enclosure, both on the left expander, So the IN and the OUT port have a link.
Running badblocks on a few drives.
Badblocks keeps running without I/O errors as long as I only unplug one cable at a time ;)
Did that a few times now. Unplug, check, plug back in, unplug the other, check, plug back in, and so on..
If I unplug both cables the server throws up a load of I/O errors, as expected. :p

If anyone wants me to run a particular test post it here and I will see what I can do.
Running it on a linux server ( Ubuntu ) with one LSI 9200-8E controller.
Only have a few disks to test with some sata and some dual ported sas.

That mean this enclosure is for spinning disks.
30 spinning hdds will not saturate 4x12Gb/s links. With SSD it's not the same story...
Its even worse: 60 disks in this enclosure. With Streaming reads and writes you will be able to hit the limits.
Do not think it would be an issue. Its not ment for pure speed.
Nice thing is that the disk trays do have a provision for 2.5" disks so if you want to go all SSD you could. But there are better solutions for that.
 
Last edited:
Mar 7, 2016
121
9
18
Boston
Can you clarify how many SAS connectors this has?

If it is a single one, are there adapters from 2x8088 or 4x8088 to 1xQSFP? If so how would thingie figure out which drives are on what lane?
 

Rahvin9999

Member
Jan 14, 2016
79
29
18
Rotterdam, The Netherlands
Can you clarify how many SAS connectors this has?

If it is a single one, are there adapters from 2x8088 or 4x8088 to 1xQSFP? If so how would thingie figure out which drives are on what lane?
It has 2 expander/controllers in it, which have 2 ports each
You can get a cable SFF-8088 at one end and QSFP on the other.

Because it uses an expander, which is basically a switch. You can attach a lot of devices to one single HBA.
In my case im using a LSI-9207 HBA card wich has 2x4 sas lanes.
So I can direct attach 8 disks using SFF-8087 fan out cables.
Or I can attach up to 1024 disks ( max the controller supports) by using SAS Expanders and Switches

The enclosure uses expanders to make the 60 disks available through, at minimum, a single 4 lane SAS connector. Each disk gets an adddress.

Code:
[   13.093592] mpt2sas_cm0: detecting: handle(0x002a), sas_address(0x5000ccab02000a32), phy(50)
[   13.093594] mpt2sas_cm0: REPORT_LUNS: handle(0x002a), retries(0)
[   13.093625] mpt2sas_cm0: TEST_UNIT_READY: handle(0x002a), lun(0)
[   13.095069] mpt2sas_cm0: detecting: handle(0x002b), sas_address(0x5000ccab02000a33), phy(51)
[   13.095071] mpt2sas_cm0: REPORT_LUNS: handle(0x002b), retries(0)
[   13.095101] mpt2sas_cm0: TEST_UNIT_READY: handle(0x002b), lun(0)
[   13.110301] mpt2sas_cm0: detecting: handle(0x002c), sas_address(0x5000ccab02000a35), phy(53)
[   13.110304] mpt2sas_cm0: REPORT_LUNS: handle(0x002c), retries(0)
[   13.110336] mpt2sas_cm0: TEST_UNIT_READY: handle(0x002c), lun(0)
[   13.111985] mpt2sas_cm0: detecting: handle(0x002d), sas_address(0x5000ccab02000a36), phy(54)
[   13.111988] mpt2sas_cm0: REPORT_LUNS: handle(0x002d), retries(0)
[   13.112022] mpt2sas_cm0: TEST_UNIT_READY: handle(0x002d), lun(0)
[   13.113588] mpt2sas_cm0: detecting: handle(0x002e), sas_address(0x5000ccab02000a37), phy(55)
[   13.113591] mpt2sas_cm0: REPORT_LUNS: handle(0x002e), retries(0)
[   13.113625] mpt2sas_cm0: TEST_UNIT_READY: handle(0x002e), lun(0)
[   13.115121] mpt2sas_cm0: detecting: handle(0x002f), sas_address(0x5000ccab02000a39), phy(57)
[   13.115123] mpt2sas_cm0: REPORT_LUNS: handle(0x002f), retries(0)
[   13.115154] mpt2sas_cm0: TEST_UNIT_READY: handle(0x002f), lun(0)
[   13.116626] mpt2sas_cm0: detecting: handle(0x0030), sas_address(0x5000ccab02000a3a), phy(58)
[   13.116628] mpt2sas_cm0: REPORT_LUNS: handle(0x0030), retries(0)
[   13.116659] mpt2sas_cm0: TEST_UNIT_READY: handle(0x0030), lun(0)
[   13.118417] mpt2sas_cm0: detecting: handle(0x0031), sas_address(0x5000ccab02000a3b), phy(59)
[   13.118419] mpt2sas_cm0: REPORT_LUNS: handle(0x0031), retries(0)
[   13.118449] mpt2sas_cm0: TEST_UNIT_READY: handle(0x0031), lun(0)
[   13.120939] mpt2sas_cm0: detecting: handle(0x0032), sas_address(0x5000ccab02000a7d), phy(68)
[   13.120942] mpt2sas_cm0: REPORT_LUNS: handle(0x0032), retries(0)
[   13.121733] mpt2sas_cm0: TEST_UNIT_READY: handle(0x0032), lun(0)
[   13.133637] mpt2sas_cm0: port enable: SUCCESS

Phy(59) is basically disk 60 (numbering starts at 0 )
Phy(68) is the expander itself
 

sthsep

Member
Mar 7, 2016
72
10
8
Hm mine runs now also. Just got some problems to get it working. The solution was to take the other controller. Maybe one controller is broken on mine.... Or I did something wrong. It found always the enclosure but no drives. After changing the drives where found. I'm running with a 9300-8E. Funniest thing is that the controller was about 3 times cheaper (Got it cheap from an ebay bidding) then one cable.
 

Mirabis

Member
Mar 18, 2016
122
6
18
26
Hm mine runs now also. Just got some problems to get it working. The solution was to take the other controller. Maybe one controller is broken on mine.... Or I did something wrong. It found always the enclosure but no drives. After changing the drives where found. I'm running with a 9300-8E. Funniest thing is that the controller was about 3 times cheaper (Got it cheap from an ebay bidding) then one cable.
hmm that's a bummer.

I've been debating to sell mine xD ... bought a 36 bay shortly after and I don't see myself filling 96 bays with 8TB disks anytime soon hehe...then again, will I ever get a great deal like this again when I need it xD
 

sthsep

Member
Mar 7, 2016
72
10
8
The biggest problem I got now is, that I run out of windows drive letters. :rolleyes: Speeds are great on my SATA drives. Get about 700MB/s peak with 4 seagate 5900 rpm drives when I'm reading files parallel.