Cringe

[Patrick]

Thread is getting a bit punchy.

Here is my reasoning on the recommendation.

Price and Write Endurance

• In terms of cost, 2x S3710 would be $135x 2 (here Intel® SSD DC S3710 Series (400GB, 2.5in SATA 6Gb/s, 20nm, MLC) SSDSC2BA400G4 | eBay ) and from those who have purchased with <200GB written means that there is still over 8PB worth of writes left.
• A single 960 Pro costs around $350 (e.g. Amazon.com: Samsung 960 PRO Series - 512GB PCIe NVMe - M.2 Internal SSD (MZ-V6P512BW): Computers & Accessories ) with 400TBW write endurance

So based on this the 960 Pro 512GB has 1/20th of the write endurance of a used S3710 400GB.

Data Protection

• The Samsung 960 Pro does not have power loss protection. The Intel S3710 400GB has full power loss and power on protection. A difference between a consumer drive and enterprise drives.
• Buying two S3710's would allow for RAID 1 operation. That means if a drive fails, the system can continue working. Unlikely given sub 0.3% AFR on the drives.
• S3710 allows for hot swap while the 960 Pro does not.

I was weighing data protection heavily as it causes application downtime.

Performance

• Given the current size of the data set, and the application, my sense is that the ingested data will hit the DRAM (capacitor protected on the S3710) on either drive then write out as sequential to NAND.
• Latency and throughput will be better on the 960 Pro assuming the drive does get a rest for GC/ TRIM. Unsure if performance wise this is a bottleneck.
• I do use Samsung XS1715 NVMe drives which I paid less per GB for than the 960 Pro that offer PLP, hot swap capability and good speeds. I have been happy with their performance.

Summary

Overall, at $270 (or less with best offer) I was suggesting a more reliable solution that is less expensive than the single 960 Pro. The ordered 850 EVOs are about $270 (Amazon.com: Samsung 960 PRO Series - 512GB PCIe NVMe - M.2 Internal SSD (MZ-V6P512BW): Computers & Accessories ) but perhaps $70 or so is worth not having to deal with this again. The application itself aggregates a ton of data and history has shown that consumer drives are dying under the load.

 

[nthu9280]

All I can say @Patriot and @T_Minus have lot of patience. I would have last it long time ago on this thread.

Sent from my Nexus 6 using Tapatalk

 

[whitey]

All I can say @Patriot and @T_Minus have lot of patience. I would have last it long time ago on this thread.

Sent from my Nexus 6 using Tapatalk

Seconded, I stayed out of this one and just SMH :-D

 

[vBuild2BAU]

They don't handle lots of writes, they have bad latency, they have poor signal integrity (in a backplane they will sometimes drop to 3gbps). They have shoddy firmware and a company that doesn't stand behind them.

NVME drives do not necessarily have the best write endurance.... Depending on the size of the database... he should probably try and get in ram.

Apart from Backblaze. Is there any information collated somewhere that end users could refer to to get statistically significant data on the life and performance of SSDs and HDDs?

I went with Micron M500DCs because the spec sheet and reviews seemed pretty good. But I guess I'll get more of a perspective for sure when I pop them into my vSAN. The drives are specced 2Mill MTBF, Power Loss Protection and a large amount of over provisioning.

Review : Micron M500DC Enterprise SSD Review | StorageReview.com - Storage Reviews

I was "Hoping" these would be on par with Intel DC SSDs. I had a good run with Crucial M500's the consumer version but that was only 4 drives and only in my lab.

 

[MiniKnight]

Intel publishes afr numbers on enterprise drives not just MTBF.

I just Binged used enterprise SSDs and look what came up Used enterprise SSDs: Dissecting our production SSD population lol top result @Patrick

That's an OK number of drives and hours but remember that Backblaze seems like a lot, but they aren't even big enough to be a direct customer. Companies like Google, Facebook, HP, Dell, Lenovo have useful numbers.

Here's the thing. Everything you read has SSDs in 0.25-0.5% afr ranges for good manufacturers. Spinners people accept 5% afr as the number.

 

[Patriot]

Apart from
Backblaze. Is there any information collated somewhere that end users could refer to to get statistically significant data on the life and performance of SSDs and HDDs?

I went with Micron M500DCs because the spec sheet and reviews seemed pretty good. But I guess I'll get more of a perspective for sure when I pop them into my vSAN. The drives are specced 2Mill MTBF, Power Loss Protection and a large amount of over provisioning.

Review : Micron M500DC Enterprise SSD Review | StorageReview.com - Storage Reviews

I was "Hoping" these would be on par with Intel DC SSDs. I had a good run with Crucial M500's the consumer version but that was only 4 drives and only in my lab.

When looking at SSDs you want to look at WPD ratings writes per day... for instance the S3710 drive is rated at 10wpd. iirc those micron m500dc drives are rated at 40gb per day for the 480gb drive or... 0.083 WPD. And when thrashed its no wonder that they curl up in a corner and rock back and forth. They can make excellent read drives, but they are not for write workloads that don't have periodic lengthy time gaps for the garbage collection to shuffle the partition around the nand and drop your iops down to 10.

 

[vBuild2BAU]

When looking at SSDs you want to look at WPD ratings writes per day... for instance the S3710 drive is rated at 10wpd. iirc those micron m500dc drives are rated at 40gb per day for the 480gb drive or... 0.083 WPD. And when thrashed its no wonder that they curl up in a corner and rock back and forth. They can make excellent read drives, but they are not for write workloads that don't have periodic lengthy time gaps for the garbage collection to shuffle the partition around the nand and drop your iops down to 10.

Not as good as the Intel but still rated at 2 DWPD.

"Random write performance is on par with drives that are twice the price. This benefited most of our mixed workload tests, especially our database profile, where the M500DC was incredibly quick. It’s not just raw speed that impressed us, it was also consistency. While not the most consistent drive we have ever tested, it far surpassed SSDs in a similar price range. At 2 drive writes per day (DWPD), the M500DC is also suited for more write intensive applications. Throw in Micron’s impressive list of enterprise features, 2M hour MTBF and 5 year warranty, and you can feel confident that your data is safe."

Source: http://www.thessdreview.com/our-rev...-480gb-great-write-performance-great-price/4/

With large amounts of overprovisiong:

"To get away with cMLC in the enterprise space, Micron sets aside an enormous portion of the NAND for over-provisioning. The 480GB model features a total of six NAND packages, each consisting of eight 128Gbit dies for a total NAND capacity of 768GiB. In other words, only 58% of the NAND ends up being user-accessible. Of course not all of that is over-provisioning as Micron's NAND redundancy technology, RAIN, dedicates a portion of the NAND for parity data, but the M500DC still has more over-provisioning than a standard enterprise drive. The only exception is the 800GB model which has 1024GiB of NAND onboard with 73% of that being accessible by the user."

Source: Micron M500DC (480GB & 800GB) Review

I bought 12 of these without much research. Jumped first with research later. Was hoping someone could give me sort of buyer's confirmation to counter-act my buyer's remorse. Sympathy shopping I guess. No one wants to give me validation here *sniff sniff*. Everyone's wants Intel.

Some guy in Australia selling a whole heap on eBay for of 480GB M500DC $140 AUD. Which is $105USD.
50 hours powered on. So pretty much new.

I guess I better go and make a forum post in good deals ........

 

[gigatexal]

The forums were bound to get a troll or two. Kudos to everyone for keeping their cool and for Patrick dropping a white paper quality post on the subject.

 

[T_Minus]

Not as good as the Intel but still rated at 2 DWPD.

"Random write performance is on par with drives that are twice the price. This benefited most of our mixed workload tests, especially our database profile, where the M500DC was incredibly quick. It’s not just raw speed that impressed us, it was also consistency. While not the most consistent drive we have ever tested, it far surpassed SSDs in a similar price range. At 2 drive writes per day (DWPD), the M500DC is also suited for more write intensive applications. Throw in Micron’s impressive list of enterprise features, 2M hour MTBF and 5 year warranty, and you can feel confident that your data is safe."

Source: http://www.thessdreview.com/our-rev...-480gb-great-write-performance-great-price/4/

With large amounts of overprovisiong:

"To get away with cMLC in the enterprise space, Micron sets aside an enormous portion of the NAND for over-provisioning. The 480GB model features a total of six NAND packages, each consisting of eight 128Gbit dies for a total NAND capacity of 768GiB. In other words, only 58% of the NAND ends up being user-accessible. Of course not all of that is over-provisioning as Micron's NAND redundancy technology, RAIN, dedicates a portion of the NAND for parity data, but the M500DC still has more over-provisioning than a standard enterprise drive. The only exception is the 800GB model which has 1024GiB of NAND onboard with 73% of that being accessible by the user."

Source: Micron M500DC (480GB & 800GB) Review

I bought 12 of these without much research. Jumped first with research later. Was hoping someone could give me sort of buyer's confirmation to counter-act my buyer's remorse. Sympathy shopping I guess. No one wants to give me validation here *sniff sniff*. Everyone's wants Intel.

Some guy in Australia selling a whole heap on eBay for of 480GB M500DC $140 AUD. Which is $105USD.
50 hours powered on. So pretty much new.

I guess I better go and make a forum post in good deals ........

I got 4x 1 died, others got many more and haven't updated the post:
M500DC Tests/Discussion

Maybe ask them how their drives are doing?

 

[vBuild2BAU]

I got 4x 1 died, others got many more and haven't updated the post:
M500DC Tests/Discussion

Maybe ask them how their drives are doing?

Thanks for the link and suggestion. I asked in that thread. I think I see the drive problem. The drive was tainted by IBM Firmware .

 

[Rand__]

German c't magazine also ran a life-time test on (consumer) SSDs (Flash-Speicher im Langzeittest) [not free unfortunately].
Result was that all SSDs survived longer than advertised, but only the 'Pro' SSDs for (several) PTWs.
But this is still little compared to the published endurance of the enterprise write oriented SSDs (let alone the expected durability).

 

[Terry Kennedy]

That's an OK number of drives and hours but remember that Backblaze seems like a lot, but they aren't even big enough to be a direct customer. Companies like Google, Facebook, HP, Dell, Lenovo have useful numbers.

Among other issues with the Backblaze data is the fact that they get (got?) their drives from all over, whoever had the best price + delivery. And that's not even counting their "please buy as many drives as you can at Circuit City and mail them to us" period. Given the way we've seen some mail order retailers pack drives, I don't know that the Backblaze data accurately reflects drives as they left the factory.

I'm certainly not up there in volume with any of the companies mentioned above, but I've dealt directly with most of the remaining spinning rust manufacturers*. While most don't have any agreements with me restricting what I can talk about, you normally won't see me mentioning my experiences with issues on specific brands / models. I assume that the companies you mentioned feel the same way about airing "dirty laundry". Not to mention that if company X says "the foo drives from bar are crap", all of their customers with that drive are going to ask "so, why did you sell those drives to us?"

* I worked on hardware / firmware design for a number of 5.25" drives from different companies (I actually started on "washing machine" drives) and I knew a lot of the "old guard" from those days. While those people have mostly retired, many of the current people recognize expertise. The trick is getting in the front door - once that's done, I've had good relationships with the remaining manufacturers.

 

[Patriot]

Among other issues with the Backblaze data is the fact that they get (got?) their drives from all over, whoever had the best price + delivery. And that's not even counting their "please buy as many drives as you can at Circuit City and mail them to us" period. Given the way we've seen some mail order retailers pack drives, I don't know that the Backblaze data accurately reflects drives as they left the factory.

They buy drives by the pallet now... the drives that were shucked came from the scramble after the flooding...and are long dead and not part of the stats.

 

[Terry Kennedy]

They buy drives by the pallet now...

The question remains - where do those pallets come from? Are they in original, unopened manufacturer multipacks, or were they re-packed to save on shipping costs? Was the entire chain-of-custody maintained with the manufacturer's environmental tolerances during shipping? BTW, this is what I'm talking about.

the drives that were shucked came from the scramble after the flooding...and are long dead and not part of the stats.

They talk about buying 3TB drives for shucking. Their 2016 stats still list 4 different 3TB models from 3 different manufacturers, as well as some 1.5TB and 2TB drives, so if they've aged out of the "fleet", they've done it relatively recently.

 

[Patriot]

The question remains - where do those pallets come from? Are they in original, unopened manufacturer multipacks, or were they re-packed to save on shipping costs? Was the entire chain-of-custody maintained with the manufacturer's environmental tolerances during shipping? BTW, this is what I'm talking about.

They talk about buying 3TB drives for shucking. Their 2016 stats still list 4 different 3TB models from 3 different manufacturers, as well as some 1.5TB and 2TB drives, so if they've aged out of the "fleet", they've done it relatively recently.

I believe it was done in the last 2 quarterly updates. I also doubt anyone follows more than half of the best practices shown in the video.

 

[Terry Kennedy]

I also doubt anyone follows more than half of the best practices shown in the video.

I can't say for sure, but I expect their large direct customers (who are the target of that video) do. For one thing, recent HGST drives have offline as well as online shock detection - if you exceed either the operating or shipping G-force / shock specs by a bit, their customer and internal diagnostic tools will immediately reject the drive. From an older version of the manual found here:

HiTest is developed to support Hitachi HDD distributors or customers of distributors focused on customer Line Integration Reject or Field Reject Failure Diagnostics. This software can be given to distributors or their customers.

*Note
“Do not return to Hitachi GST” of disposition has two means.
    NFF: No Failure Found.
    There are no error records in a drive and Head Amp and Servo Measurement are Good.
CID: Customer Induced Damage Failure (Handling Damage).
    RRO Measurement is criteria over and Seems that Disk Shift was caused by handling damage.

 

[jgreco]

He has backups... he needs drives with endurance that won't die all the time... he has burned through 12 ssds in 4 years. Going to lower endurance nvme won't help the problem.

The basic issue here for the originally described problem is an apparent lack of understanding of the need to do the basic engineering. You need to understand your workload, and you need to understand your expected lifetime, and other factors such as the availability of RAID.

The 850 Evo's have a fairly attractive combination of performance, endurance, and price point. It helps somewhat if your expected service life isn't the full five years they're warrantied for. Back around the end of 2015, we were pairing up Intel 535 480GB's with 850 Evo 500GB's in RAID1 for ESXi DAS SSD datastores, because we were calculating that we'd probably hit wearout on these in about two to three years, and the price trends on SSD suggested that we might be replacing them with larger units maybe even within a year. 2016 didn't turn out to be another year of further SSD size/price carnage, alas. However, none of the member SSD's in the deployed datastores has failed, which is actually kind of remarkable because statistically I'd have expected a few failures for non-endurance reasons.

We've also had great success with NAS using the DS416slim, where we've been using two 2TB laptop grade HDD's in RAID1, and a 960GB PNY CS1311, and a 960GB Sandisk Ultra II in RAID1. We've got four of these units, one of which is blowing through writes at 115GB/day, well above the estimated endurance, the others are running more like 20-30GB/day, which is close enough to the estimated endurance. Using cheap SSD allowed us to double the available SSD space over what it would have been with more expensive SSD's.

We still have a bunch of stuff where we need the DC S3500's and DC S3710's, but focusing on identifying what the actual workload requires of endurance can really save you a boatload of cash, while also making your environment more pleasant to use.

Or, if you choose to ignore what the workload is telling you when it's killed SSD's repeatedly and frequently, as described above, well, that's a nice way to enrich flash manufacturers.