Did some write benchmarks of a few SSD's

Discussion in 'Hard Drives and Solid State Drives' started by BackupProphet, Jun 8, 2017.

  1. BackupProphet

    BackupProphet Well-Known Member

    Joined:
    Jul 2, 2014
    Messages:
    784
    Likes Received:
    278
    pg_test_fsync and fio on Intel 6700K @ 4.6Ghz Ubuntu 17.04

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]

    Conclusion, consumer level SSD's are not worth a dime.
     
    #1
    SGN, K D, DouglasteR and 1 other person like this.
  2. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,845
    Likes Received:
    424
    Old intel 320's :)
    Have a few of them, and 520/530 doing good duty ever since installation.
     
    #2
  3. Son of Homer

    Son of Homer Member

    Joined:
    May 9, 2016
    Messages:
    173
    Likes Received:
    15
    Thanks. Very informative!
     
    #3
  4. DouglasteR

    DouglasteR Member

    Joined:
    Dec 19, 2015
    Messages:
    95
    Likes Received:
    17
    Is the newer samsungs NVME any different ?
     
    #4
  5. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,784
    Likes Received:
    1,459
    960 and 961 are the new NVME or do you mean enterprise?
     
    #5
  6. DouglasteR

    DouglasteR Member

    Joined:
    Dec 19, 2015
    Messages:
    95
    Likes Received:
    17
    Both if possible.

    I just want a good ~200GB OS boot drive.
     
    #6
  7. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,784
    Likes Received:
    1,459
    Those products are on the above graphs is what I was referencing. Sorry for lack of clarity.
     
    #7
  8. DouglasteR

    DouglasteR Member

    Joined:
    Dec 19, 2015
    Messages:
    95
    Likes Received:
    17
    Uh oh, i misunderstood too lol.
     
    #8
  9. gigatexal

    gigatexal I'm here to learn

    Joined:
    Nov 25, 2012
    Messages:
    2,683
    Likes Received:
    498
    consumer drives really are useless. Sheesh.
     
    #9
  10. DouglasteR

    DouglasteR Member

    Joined:
    Dec 19, 2015
    Messages:
    95
    Likes Received:
    17
    So...

    A S3500/3700 will feel much more faster than a 960 Pro for example ?

    AFAIK, QD1 is the most important depth for perceptive quickness, correct ?
     
    #10
    Last edited: Jun 13, 2017
  11. BackupProphet

    BackupProphet Well-Known Member

    Joined:
    Jul 2, 2014
    Messages:
    784
    Likes Received:
    278
    Absolutely yes, but it still depends on workloads. Most desktop users rarely need more than 10-100 iops.
     
    #11
    DouglasteR likes this.
  12. CookiesLikeWhoa

    Joined:
    Sep 7, 2016
    Messages:
    111
    Likes Received:
    24
    What usually makes a desktop "feel" faster, in my experience at least, is latency.

    As pointed out above most desktops don't need a ton of iops, but latency makes a noticeable difference.
     
    #12
  13. matt_garman

    matt_garman Active Member

    Joined:
    Feb 7, 2011
    Messages:
    205
    Likes Received:
    36
    So do we know why the enterprise drives are so much better? Two things I believe to differentiate consumer versus enterprise: controller and over-provisioning. How much of an effect does each have? And are there other things that make it better?

    I'd be curious to see the results these same tests run on consumer SSDs with significant amounts of over-provisioning (say 25-50%).

    Edit: Here's a related link, Write Saturation And Over-Provisioning Tests.
     
    #13
    Last edited: Jun 21, 2017
  14. i386

    i386 Well-Known Member

    Joined:
    Mar 18, 2016
    Messages:
    1,665
    Likes Received:
    400
    Firmware can add a lot of magic, just look at the intel ssds with sandforce controllers that were more reliable than kingston ssds with the same controller.
     
    #14
  15. Stephan

    Stephan IT Professional

    Joined:
    Apr 21, 2017
    Messages:
    98
    Likes Received:
    33
    Enterprise SSDs have:

    - Better cooling, so no or little need to throttle the controller and with that read/write speeds to stay within thermal envelope
    - Faster interface speeds e.g. SAS 12GBps, U.2, PCIe, instead of SATA
    - More advanced electronics like power loss protection capacitors, ECC-RAM for the controller
    - Firmware engineering is better, e.g. LDPC instead of BCH error correction, latency guarantees, reliable health warnings
    - End-to-end data protection, data is protected through additional checksums from its way to/from flash chip to/from CPU
    - MLC (2-bit per cell) instead of TLC (3-bit per cell) flash, although advances in 3D-NAND is changing that
    - Select A-die flash silicon from premium bins
    - More flash channels from controller to flash are used, primarily to increase write speeds
    - More space dedicated to over-provisioning to be able to replace bad cells for larger TBW (TeraByte Written) over drive's lifetime
    - Self-encrypting drive junkware features (you should not trust this)
    - Higher capacities available than with consumer SSDs

    Did I forget something...
     
    #15
  16. Evan

    Evan Well-Known Member

    Joined:
    Jan 6, 2016
    Messages:
    2,845
    Likes Received:
    424
    @Stephan there is plenty of enterprise sata and tlc drives out there.
    Samsung and Intel make a bunch of sata drives. Samsung especially has a heap of tlc drives, mixed use models as well not just read optimised ones.

    You touched in error rates and it's sometimes clear that enterprise have lower error rates 1 x 10 ^17 where some consumer drives it's not stated or less.
     
    #16
  17. matt_garman

    matt_garman Active Member

    Joined:
    Feb 7, 2011
    Messages:
    205
    Likes Received:
    36
    Agreed, but not all those things necessarily contribute to improved write performance.

    I'm trying to look at write performance on enterprise versus consumer in a "all things being equal" perspective (to the extent possible of course).

    In particular:
    • Cooling - is it possible to construct an environment such that the consumer drive's "lesser" cooling becomes a non-issue?
    • Interface - of course there are things better than SATA, but what if we look exclusively at SATA drives to help level the playing field
    • Power loss protection - great feature, but I don't think it helps with write performance
    • MLC vs TLC - there are MLC-based consumer SSDs
    • Better binning - does this make for faster NAND, or just more reliable/long-term stable NAND (honest question)
    • More flash channels - I'd file this under controller, but yes, I can definitely see how this would improve write performance
    • Self-encrypting - I don't think this would affect performance
    • Higher capacities - I'm interested in comparing equivalent capacities
    • More over-provisioning - definitely helps, but you can change the amount of OP on consumer drives
    I think, except for the controller, most differences that can affect write performance can be normalized. So then we could see how much difference the controller actually makes.

    Let's say, just for sake of argument, you don't care about the non-performance impacting perks of enterprise drives. Now, can you game the over-provisioning to get similar performance and a lower cost per GB? Or maybe you don't need to perfectly match performance; the graphs above show basically a 20x improvement. But what if your application only needs a 10x improvement?
     
    #17
    Last edited: Jun 22, 2017
  18. T_Minus

    T_Minus Moderator

    Joined:
    Feb 15, 2015
    Messages:
    6,784
    Likes Received:
    1,459
    Hmm, maybe I don't get it? Why?

    You can easily already see how much a controller makes, it's a huge difference.

    Everything you've talked about "normalizing" is pretty much irrelevant other than OP, which just extends the time before the consumer drive does slow down, and it will.

    You also can't compare like capacity to like capacity 'on that point' because the NAND may be 16GB 32GB 64GB etc, and having 24x 16 vs 12x32 most def. affects performance as you can see how the newer gen Intel SSD performance dropped in certain work loads due to this. So a 500GB consumer with 16GB on each chip != enterprise with 32gB on chip, etc...


    I think the problem is you're going by the "Rated Spec" and a consumer drive don't ever go for 100% drive it's like 8GB span only, so you'll NEVER get the write out of a consumer as an enterprise does. Look at Steady State performance in reviews of SSD it's clear as day to see this, and OP does help here also clear, but it's just a delay to the slow down not a remedy for it, because it's in the firmware of the enterprise SSD.


    So in the end, I'm not sure what you're trying to accomplish.
    You can't make a consumer SSD write like an enterprise one in terms of performance, consistency and ultimately reliability... sure a consumer COULD burst to the write perf. of an enterprise ssd under certain work loads, and assuming it's not constantly being written/read in the background too.
     
    #18
  19. matt_garman

    matt_garman Active Member

    Joined:
    Feb 7, 2011
    Messages:
    205
    Likes Received:
    36
    Thanks, I see it's a little more nuanced than I originally assumed.

    Just to give a little more color to where I'm coming from: originally I was curious about enterprise vs consumer strictly for home use (see this thread). In my case I know my writes are minimal. And am I willing to pay the enterprise premium for the other benefits? In my mind, since I am certain I don't need the write performance, the other benefits are like insurance. Everybody has a choice for how much insurance they want to buy, right? Do you have disability insurance, life insurance, home owner's insurance (not required if you don't have a mortgage), what level of auto insurance, etc. Except for legal requirements, there's not a right or wrong, and everybody makes their own cost vs risk analysis.

    So while I started out thinking about this in terms of my home situation, it also applies to what we're doing at work. We have a home directory server for about 40 employees using good old 7200 rpm spinners in a software raid config. It works and is stable, but the speed is lacking. If it's being lightly used, it's not bad, but if (for example) someone kicks off a big compile, then everyone suffers.

    Now... at work we also have a few 24-disk consumer (Samsung 850 Pro) SSD arrays, see here. Credit to @dba for the Dirt Cheap Data Warehouse inspiration. The "catch" though, is that these arrays are literally 99.999% reads. It's effectively a WORM workload. We add a tiny amount of incremental data every day, and the rest of the time it's just constant reading. Using this allowed the company to save a ton of money by avoiding going with a big iron storage vendor. Even now, we max out the dual 40gb network interfaces (with jumbo frames!) before the drives run out of steam.

    I made the thought mistake that, since the read performance is so wonderful with these drives, how bad could the write performance really be? Seeing the graphs above, and the related discussions, I see the answer is: pretty bad!

    So, coming back to the home directory server. While we recognize the current server is slow, the higher-ups are very "financially conservative" (I'm being generous here)... So before I understood the huge consumer vs enterprise write performance discrepancy, I was thinking: we get a 2U chassis with 24 2.5 hotswap bays. Fill it with eight 1tb consumer SSDs. Raid-6 them together for nearly 6tb of storage, and 16-bays left for future requirement increases. I haven't revisited the numbers in a while, but last I checked, the base server would be about $4k and the drives another $4k, so $8k all-in. Now the higher-ups are anchored to that number. I haven't done any cost estimation for enterprise drives yet, but I'm sure the cost is going to be much higher.

    Also, to give a little insight to the "ignoring non-performance impacting benefits of enterprise SSDs" mindset: the insurance analogy I used above is apt, because one of the company owners in fact does not have insurance on his house. He can afford to "self insure". Paradoxically, the cost of home owner's insurance is therefore trivial to him, but he believes the chance of a disaster so low that he's willing to make the trade off. So hopefully that gives you a little insight into the thought-process under which I'm working. I've worked here for over a decade; I can say with very high confidence, excepting for performance issues, the owner is definitely willing to trade all the other risks to save some money. The performance issue could be a deal-breaker, but if I can close the performance gap (even partially) with over-provisioning, then that's almost certainly what I'd be instructed to do.
     
    #19
  20. ATS

    ATS Member

    Joined:
    Mar 9, 2015
    Messages:
    96
    Likes Received:
    32
    Power Loss Protection. For things like Sync Writes that is a killer feature. Consumer drives that don't have PLP, can't legitimately Sync until the data is actually on the flash, and enterprise drive with PLP can signal Sync the second it hits the write cache.
     
    #20
    Stephan likes this.
Similar Threads: write benchmarks
Forum Title Date
Hard Drives and Solid State Drives Slow write sync performance on Intel S3700 Jan 25, 2019
Hard Drives and Solid State Drives SATA DOM with Write Protection or RAID Sep 20, 2018
Hard Drives and Solid State Drives 16-32gb ssd, good write speed $20 - 30 ? May 20, 2018
Hard Drives and Solid State Drives SSD endurance - continuous write Dec 25, 2017
Hard Drives and Solid State Drives SSD - value over lifetime 'total writes'? (ie scratch drive/extending lifespan of consumer ssd?) Nov 4, 2017

Share This Page