Choosing drives for 3 drive RAID-Z ZFS, please help

[mixer]

Hello!

I'm about to pull the trigger on some new drives for my Napp-IT All-in-one / ESXi storage system. I run very lightly loaded VMs (home file server, internet file transfer server, mail server, one-camera Zoneminder), so performance is not paramount. I'm upgrading from 3 WD RE-3 750 GB drives that have been great for three years now. The only thing I want is more space, but also quieter drives would be better. Also I could add an SSD cache drive to the RAID-Z setup which might help offset going to slower drives (5400 RPM).

I'm a bit shocked at drive prices these days, but in England right now it seems the price (incl. VAT) for three of the following 3 TB drives are:

Hitachi 5k3000 £456
Hitachi 7k3000 £465
Hitachi 7k4000 £678****
WD Red £432
Barracuda £330

--wildcards--
WD AV-GP £390
Seagate SV35.5 £414
WD RE-4 £489**
Constellations and Ultrastars are going to be around £750

****= 4TB, **= 2TB.

My thoughts on the 'DVR' optimised drives is, in case of WD, same as the RED really but better availability and price. In case of Seagate, same as Barracuda (specs seem identical) but 2 year warranty and rated for 24/7 operation and stated 1M hr MTBF. Also, though the DVR drives CAN support this ATA streaming command, best guess it that unless the OS / controller specifies that the drive will operate in a normal mode.

I'm exasperated that normal Barracuda now has 1 yr warranty and 2400 power-on hour rating, but I like the fact that it is a 3-platter drive. I've heard good things about Hitachi, and I guess I shouldn't worry about the fact that it is a 5 platter drive.

Finally, should I spend more or get less storage to go for a drive with BER of 2 ^15 instead of all the consumer drives that are 2 ^14. I just want three years of trouble-free ZFS storage!

Thanks for your input.

 

[Patrick]

I might go with the RED in that use case. Still not enough reliability data on them, but I think this is an intended application.

 

[Mike]

http://skinflint.co.uk/?cat=hde7s&xf=958_3000&sort=p might help you out with selecting, I use the mainland's version for some of the stuff I'm looking for.

I've had good luck with the WD greens although some see it as a taboo for ZFS. I've seen a few reviews on Seagate's .14 series I believe with impressive speeds closing in on 200mb/s per spindle. Newer firmwares should resolve some of the clicking these drives create(d).

 

[MiniKnight]

Reds +2 supposed to be designed for this application

 

[mixer]

Thanks for the replies so far... Mike - I like that site, better than Google Shopping.

If I feel like splurging a bit, are the Hitachi 7k4000s (or 5k4000s) recommended?

Will ZFS scrubbing make up for the lack of 10 ^15 BER on all these non-enterprise drives?

 

[cactus]

The Hitachis are not produced anymore, got bought out by WD, and for that I would stay away from them. The WD Reds are new to the market, but support is there is they do die.

 

[mixer]

Hmm. Interesting point, Cactus. I assume that if one bought a Hitachi now WD would handle the warranty and swap out drives for the term of the warranty (sadly only 2 years). But even if they do, I see your point it could be considered an EOL product even though it's one of the few 4 TB options.

 

[odditory]

I'd still go Hitachi if they're available, as they tend to outlive their warranties anyway. Aside from my dislike of WDC's business practices and gaming the market, I dont think WD Reds are worth going rates since they're essentially Greens with TLER and no APM.

If you can hold out then wait for Toshiba since they're ramping up production now and are set to release 3TB's any time now, which are essentially Hitachi drives with a Toshiba sticker. http://www.toshiba.co.jp/about/press/2012_08/pr0801.htm

Also, to the question of enterprise versus consumer class -- if its for home use meaning you're spending your own money and not a company's, no reason to go enterprise if you consider that for the cost of 1 enterprise drive, you could buy 2 desktop class drives and mirror the files for a 1:1 backup.

 

[mixer]

Thanks for replying Odditory -- about 30 mins after I rashly ordered the REDs! Haha.. Well, they ended up only being £10 each more than the WD Greens and I have a good feeling about using the drives essentially as directed, though I know that's a bit silly. I'll report back on my experience with them. Thanks all.

 

[odditory]

Don't sweat it, the Reds will be fine for your purposes. Sorry that you gents in the UK are taxed to the high hills on items like this.

 

[mixer]

Yes, 20% VAT is painful, but aside from that the price is actually lower than US Newegg right now (not that they have any for sale at the moment). Today's exchange rate puts the drive at about $183 before VAT.

 

[MiniKnight]

mixer mind posting pics and benchmarks. maybe before and after. Wonder how last gen RE3 compare to new red. Only one digit (3 to D) difference

 

[mixer]

The WD RED 3TB drives arrived today, from Lambda-Tek in the UK. Two out of the three work! The third one spins up, does 9 fast high beeps then a longer low beep then a medium beep... and repeats that set for a while. At some point it spins back down and rests peacefully. Generally it does not show up in BIOS, but once it did, but showed a size of 0.

Anyone know what that means? Besides "it's DOA"? I have an email in to Lambda-Tek for replacement and I called the UK WD RED special support line, which clearly forwarded to the US (maybe since it's after normal business hours today in the UK). The front line support guy was not going to be helpful at all, and I pretty much had to tell him that I was going to put the drive in a NAS enclosure / Drobo before he would stop telling me how it was not supposed to be used in a normal computer.

The drives seemed to be packaged/padded well enough and the other two seem happy. These are probably not the benchmarks you are looking for MiniKnight (what would you like?) but I did some quick DD write tests after a short and conveyance SMART test:


root@PartedMagic:~# fdisk -l

Disk /dev/sda: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

root@PartedMagic:~# dd bs=1M count=128 if=/dev/zero of=/dev/sda conv=fdatasync
128+0 records in
128+0 records out
134217728 bytes (134 MB) copied, 0.978857 s, 137 MB/s

root@PartedMagic:~# dd bs=1M count=1280 if=/dev/zero of=/dev/sda conv=fdatasync
1280+0 records in
1280+0 records out
1024000000 bytes (1.0 GB) copied, 7.34892 s, 139 MB/s

root@PartedMagic:~# dd bs=8k count=125000 if=/dev/zero of=/dev/sda conv=fdatasync
125000+0 records in
125000+0 records out
1024000000 bytes (1.0 GB) copied, 7.27671 s, 141 MB/s

root@PartedMagic:~# dd bs=4k count=250000 if=/dev/zero of=/dev/sda conv=fdatasync
250000+0 records in
250000+0 records out
1024000000 bytes (1.0 GB) copied, 7.34892 s, 139 MB/s

root@PartedMagic:~# dd bs=4k count=25000 if=/dev/zero of=/dev/sda conv=fdatasync
25000+0 records in
25000+0 records out
102400000 bytes (102 MB) copied, 0.792944 s, 129 MB/s

root@PartedMagic:~# dd bs=3k count=31000 if=/dev/zero of=/dev/sda conv=fdatasync
31000+0 records in
31000+0 records out
95232000 bytes (95 MB) copied, 2.12503 s, 44.8 MB/s

 

[Patrick]

Bummer you received a DOA drive. My guess is that is the case if the other two work fine.

Nice results on that drive.

 

[mixer]

Question about the 512e / 4k / ashift=12 thing: This drive reports the following:

Disk /dev/sda: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Does that mean it is one of those guilty of 'lying' to the OS about sector size and needs to have the ashift=12 override put in, or since it reports a Physical Sector size of 4096 bytes it will be detected correctly (in my case, concerned with how it is detected by OpenIndiana on Intel C204 SATA ports). Thanks!

 

[mixer]

Where art thou, RED?

Since my RMA'd drive went back no 3 TB RED drives have come into stock anywhere in the UK. Does anyone know what is going on with Western Digital?

My two drives are just sitting around waiting for a third! Should I just order some other 3TB drive and be done with it?

 

[Patrick]

What is interesting is that the stock of 3TB RED drives in the channel even in the US is not amazing. I will send Western Digital a ping today on this and see what I get back.

 

[mixer]

Thanks Patrick!

 

[Patrick]

Just sent the e-mail off. Also checked, Newegg, Provantage, TigerDirect and Amazon and didn't have any luck stateside.

 

[mixer]

I suppose you haven't heard anything, Patrick? Maybe they wanted the capacity for their new Enterprise drives, (the new REs are just out).

Still, it has been almost 2 months of no supply here since I got my three (of which two worked) RED drives and I've just about had it with the waiting.

Those new Toshiba drives are not around either.

I think to avoid a disk set that is any slower than three REDs I should choose a replacement third drive that is faster, a 7200 RPM drive, maybe a Hitachi. Thoughts?