EU WTB: 60+ bay JBOD

[Puppetfreek]

Hey all,

Looking for a 60+ bay (4U) storage JBOD to purchase. I've been looking at the HGST 4u60, but most existing deals seem to be in the US and adding VAT + shipping usually makes these too expensive. The JBOD would be used for backup, so don't require alot of performance, so SAS2 would work - as long as it supports a bunch of random SATA 8+TB drives.

I'm limited to 4U in my current rack (Netshelter CX 18U) as well as around 34" for max length - guessing this limits my options for anything larger than 60 drives.

 

[ari2asem]

i have been in conversation with a company from UK. their price incl VAT-tax for hgst 4u60 gen.2 unit is 480 pound (around 560 euro, excl shipping and incl VAT). this is unit's SKU-number: 1ES0156 (incl rack rails), with software lock to use only hgst/wd hdd-drives.

if you are interessted, pm me. then i can give the name of the company

 

[Puppetfreek]

Hi,

That sounds interesting - i'll send you a PM. Do you know if the software lock is something that can be removed (easily)? While I do have only HGST drives currently, I would hope to be able to populate this in the future with any random brand.

 

[ari2asem]

Hi,

That sounds interesting - i'll send you a PM. Do you know if the software lock is something that can be removed (easily)? While I do have only HGST drives currently, I would hope to be able to populate this in the future with any random brand.

before you buy the unit, read very carefully the manual. specially sections anout hdd-brands and how to populate the hdd-slots, about ssd-hdd combination. population of slots is very important to have working unit.

read the manual carefully, plz

 

[Puppetfreek]

So, a small update. Was quickly going through the manuals of the different revisions of the unit and it seems you are right.

On the first revision, it does actually state:

The SEP examines the Inquiry data of all attached drives to determine if they are a supported type. If an unsupported device is attached to the 4U60 Storage Enclosure, it will not be available to access.

https://documents.westerndigital.co...latform/user-guide-4u60-storage-enclosure.pdf, Page 22

While neither the G2 version (https://documents.westerndigital.co...form/user-guide-4u60-g2-storage-enclosure.pdf), or the G3 (?) (https://documents.westerndigital.co...brid-platform/user-guide-ultrastar-data60.pdf) mentions the same limitation, I got confirmation from eBay seller that this is still true.

So i've started looking for alternative, and found the Newisys NDS-4600-JD, which seems to be a very similar product. However, I managed to get in contact with a seller on eBay that ran some tests that indicated that these are not able to detect any consumer drives (for example WD red).

 

[ari2asem]

So, a small update. Was quickly going through the manuals of the different revisions of the unit and it seems you are right.

On the first revision, it does actually state: https://documents.westerndigital.co...latform/user-guide-4u60-storage-enclosure.pdf, Page 22

While neither the G2 version (https://documents.westerndigital.co...form/user-guide-4u60-g2-storage-enclosure.pdf), or the G3 (?) (https://documents.westerndigital.co...brid-platform/user-guide-ultrastar-data60.pdf) mentions the same limitation, I got confirmation from eBay seller that this is still true.

true? do you mean true also for gen2 and gen3? that those gen's are having same "limitation" about hdd as 1st gen has?

So i've started looking for alternative, and found the Newisys NDS-4600-JD, which seems to be a very similar product. However, I managed to get in contact with a seller on eBay that ran some tests that indicated that these are not able to detect any consumer drives (for example WD red).

do you mean newisys can't detect consumer hdd's?
if yes, then i am curious about the reason why not?

back to hgst 4u60 units.

gen1 and gen2 can be used when even 1hdd is putted in the unit, as far as i could understand.
gen3 needs at least 24 hdd's to start working. reason for this they call performance. meaning putting at least 24 hdd in the unit garantee you good performance. but what happens when you put 1 hdd? is the unit still usable and hdd recognized? i don't know.

any other places to look at for jbod case
servercase uk

xcase uk

zstor de

and read this topic (20 pages)

https://forums.servethehome.com/ind...3-12gb-s-storage-enclosure-60-bays-£299.9980/

 

[Puppetfreek]

true? do you mean true also for gen2 and gen3? that those gen's are having same "limitation" about hdd as 1st gen has?

Yes, gen2 and gen3 does have the same "limitation" as the gen1 regarding disk brand and type (enterprise).

do you mean newisys can't detect consumer hdd's?
if yes, then i am curious about the reason why not?

Yeah, that is the information I received from the eBay seller. I'm also unsure why, but I guess a similar configuration has been added as to the 4u60 units. Unsure why, but might be how they were sold as units?

back to hgst 4u60 units.

gen1 and gen2 can be used when even 1hdd is putted in the unit, as far as i could understand.
gen3 needs at least 24 hdd's to start working. reason for this they call performance. meaning putting at least 24 hdd in the unit garantee you good performance. but what happens when you put 1 hdd? is the unit still usable and hdd recognized? i don't know.

I don't have any information on this - the seller of the unit on eBay didn't mention this in our conversations. So, i'm unsure if this is a enforced - like for the drives - or if this is more of a "recommendation".

any other places to look at for jbod case
servercase uk

xcase uk

zstor de

and read this topic (20 pages)

https://forums.servethehome.com/index.php?threads/hgst-4u-sas3-12gb-s-storage-enclosure-60-bays-£299.9980/

Thanks for the links! I'll take a look and see if I can find something interesting. I've also looked at the Nexsan E60X on eBay, but according to the seller:

You have to use nexsan drives with the correct drive signature

 

[ari2asem]

I've also looked at the Nexsan E60X on eBay, but according to the seller:

i should say: stay away from nexsan as the hell. they are the same as netapp s*it, with vendor lock-in as hell.

look at this. maybe for you?

InWin JBOD

 

[ari2asem]

a little update for you.

if noise is consideration point, then choose gen1 hgst 4u60. because gen1 after start of the ubits spins the fans down.
gen2 doenst spin down the fans.
so, gen2 makes much more noise.

gen1 has 6gbps io-controller/module.
gen1 has 12gbps io-controller/module.

gen1 has qsfp+ port on the backside, so you need qsfp+ to sff-8088 or to sff-8644 cable.

gen2 has sff-8644 port on the backside.

gen1 can also support seagate hdd, but to be sure it must be tested.

gen2 you REALLY have to use hgst or wd hdd.

all this informortion i have from someone who has owned both gen1 and gen2 units.
he used on gen1 also seagate hdd of 3tb and 10tb hdd.

officially there is no gen3 of 4u60.
probably data60 is so called gen3. but that is not officialy the name of data60-unit

 

[Puppetfreek]

Thanks for the update! I've read similar reports here, but I'm still doubting the G1's capability to support other vendors, such as Seagate. And starting to struggle with getting drives to be identified (and work) is the last thing I want to spend my time on, so then I rather invest abit more to get something that allows for more generic hardware. But it vaven't been easy to find alternatives, so currently looking at options from Supermicro.

 

[ari2asem]

Thanks for the update! I've read similar reports here, but I'm still doubting the G1's capability to support other vendors, such as Seagate. And starting to struggle with getting drives to be identified (and work) is the last thing I want to spend my time on, so then I rather invest abit more to get something that allows for more generic hardware. But it vaven't been easy to find alternatives, so currently looking at options from Supermicro.

just ask the seller to test with any seagate hdd.

i would go with gen1, with 1 disadvantage = 6gbps connection

 

[nephri]

Hi,
I have a GEN1 and it's not what i understood.

From :
https://documents.westerndigital.co...platform/data-sheet-4u60-storage-platform.pdf

and

https://documents.westerndigital.co...atform/data-sheet-4u60g2-storage-platform.pdf

Both use a 12Gbs connection but not the same amount of sas3 line per modules

HGST 4U60G2 : 8x 12Gb/s lines (4 per IOM)
HGST 4U60G1 : 4x 12Gb/s lines (2 per IOM)

In the GEN1, i'm not sure if the 2x 12Gb/s lines per IOM are used for multipath (redudancies) or if they aggregates them in term of bandwidth.
But whatever the truth, i think if you connect to a SAS2 card (6Gb/s connection) you will shrink the max bandwidth you can have (24G/s to 12Gb/s or 48 Gbs/s to 24Gbs/s)

If it's 24Gb/s the total bandwitdh of the Gen1, it's sure it can be a bottleneck for 60 drives (even worst if it's 60 SSD).
But this unit should not be used for building a "performance array" but instead a "capacity array".
You can hybrid it with some SSD for fullfill the bandwith that will give you descent performances.

In the GEN1 datasheet, they say "It offers 2x2x4-lane SAS 12Gb/s performance". I not understand what they means by the 2x2x4" ? It's probably the interconnection inside the unit ? 60 HDD connected inside to 16 lanes and output the unit on 4 lanes ?

I use it at home for 2 arrays:
- 1 backup array
- 1 huge array in read intensive that did'nt require a lot of bandwith

i fill in it 60x 2Tb HGST HDD.

 

[Puppetfreek]

So, some more update:

HGST 4u60:
I got feedback from a seller on eBay that already tested Seagate Ironwolf drives in the 4u60 G1 and it did not seem to work.

Newisys NDS-4600:
Recieved feedback from one seller, as stated above, that out of the box it does not work with WD Reds (for example). But after he changed the interposer (to a Dell one) - it was able to identify the WD Red drive. If I understood this, you would have to purchase 60 interposer to get all slots working. Unsure however, if this would result in the unit being unable to close, due the additional interposer.

For me, the bandwidth is not really an issue, as i'll be using this for backup.

I've went ahead with the Supermicro SC847 45-bay version now, so hopefully i'll have it within a couple of weeks. Will see if I can do a fan replacement, and perhaps install some SSDs between the front and end backplane - if I have something laying around, and there is a notable speed increase.

@nephri : Have you tried any brand except HGST, for example, Seagate?

 

[Rand__]

The officially supported device strings are HGST, WDC and SanDisk so I'd be surprised if it did support Seagate drives...
I mean if they even remove support of old (as in out of support) HGST drives I can't imagine otherwise...

 

[nephri]

No, i used only HGST drives (SAS2 drives). Theses drives are older than expected drives but they are detected like a charm.
I have only HGST drives (as HDD)

 

[Rand__]

I can also add that it didnt see neither Micron nor Intel SSDs unfortunately

 

[ari2asem]

since today i have gen1 unit, 2 seagate sata3 hdd are recognized, 3tb and 10tb.

gen1 has 6gbps io-module with qsfp+ connector. datasheet talks about sas3 as hdd interface, not as connection from io-module to host machine/server.

i thought maximum ssd in the unit allowed is 12 or 24

 

[Rand__]

Hm I have not run performance tests tbh, but that would be a bummer

But...

 

[Rand__]

Regarding speed - I have annoyed the WD support a bit and got the following results after a bit back and forth


Q:
I know the JBOD supports SAS3 harddrives, and it has QSFP connectors.
Can you please confirm that the QSFP port outputs SAS3? I assume 4 lanes per EOM?
Assuming that is the case, can i connect both EOMs to a single host to increase the bandwith? Or can this only be used for multipathing.

A: Regarding SAS3/QSFP query:
1) The frontend ports of each ESM are support QSFP+ - Refer user guide for supported QSFP+ cables.
2) Yes, Each ports internally has 4 lanes.
3) If a single Host is connected to the frontend port of both of the ESMs, then this configuration will give you both - Increase bandwidth and Multipathing capabilities.

Q:
Regarding the QSFP+ port - just to be sure, the 4 lanes carry SAS3, right?
So the maximum bandwith I can get when connecting the box to two servers is 4x12Gb/s (6 GB/s) limited by the use of a single port on a HBA/Raid Controller (4 lanes of PCIe3 = 4x985MB/s=3940MB/s) - this is just to confirm the maximum theoretical bandwith

A: QSFP+ port:
1) Yes, 4 lanes carry SAS3.
2) Yes, the maximum bandwidth that you can get with 4 lanes of PCIe3 = 4x985MB/s=3940MB/s theoretically.

Which tells me I need to get a PCIe4 HBA for these

 

[nephri]

At this time, i connected it to a "LSI SAS 9300-8e 12gbit/s Dell J91FN" on a PCIe3 x8

If i'm remember correctly, each ESM have 2 QSFP ports (an IN and an OUT).
In your plan, you want to connect 2 servers. Each to both ESM using the 4 QSFP ports ?