Firefox is chewing through your NAND

Discussion in 'Software Stuff' started by keybored, Sep 18, 2016.

  1. G-R-M

    G-R-M New Member

    Joined:
    Sep 23, 2016
    Messages:
    2
    Likes Received:
    0
    I also would like to know that, because even with these settings in my user.js, I can restore the last session.

    // Default is 10
    user_pref("browser.sessionstore.max_tabs_undo", 0);
    // Default is 3
    user_pref("browser.sessionstore.max_windows_undo", 0);
    // Default is true
    user_pref("browser.sessionstore.resume_from_crash", false);
    // Default is 1
    user_pref("browser.sessionstore.max_resumed_crashes", 0);
    // Default is true
    user_pref("browser.sessionstore.restore_on_demand", false);

    30 minutes (180000 milliseconds) Minutes for browser.sessionstore.interval does not make much sense in my opinion, because then you can directly turn off.
     
    #41
  2. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Opened 25 tabs. Not idle.
     
    #42
  3. wildpig1234

    wildpig1234 Well-Known Member

    Joined:
    Aug 22, 2016
    Messages:
    1,617
    Likes Received:
    191
    seems to me like IE and edge doesn't access disks that much on idle. at least based on what I see in resource monitor . ... am I right?

    I think the conclusion is pretty much if you are not using browser, they should be off . Only sure way to not have them keep interacting with SSD.
     
    #43
    Last edited: Sep 23, 2016
  4. sparesim

    sparesim New Member

    Joined:
    Oct 12, 2015
    Messages:
    10
    Likes Received:
    0
    Tried adblock or noscript?
     
    #44
  5. wildpig1234

    wildpig1234 Well-Known Member

    Joined:
    Aug 22, 2016
    Messages:
    1,617
    Likes Received:
    191
    This appears to be a case of what the browser was engineered to do from the beginning and not just a case of malware or adware attack. doubt any sort of adblock will help.

    There is only one sure way to stop it right now: don't have any idle browser idle open if you are using ff or chrome. My resource monitor seems to show that idle IE or edge doesn't seem to do this but I want to get this confirmed also.
     
    #45
  6. pricklypunter

    pricklypunter Well-Known Member

    Joined:
    Nov 10, 2015
    Messages:
    1,520
    Likes Received:
    431
    I'm using Maxthon 3 tonight, I have never tried any of their products before. Prompted by this thread, I decided to break from my comfort zone and look at some alternatives to the big guns. For as many years as I can remember, FB/ FF has always been my go to, but maybe this is a sign that it's time for something new...

    This isn't a plug for Maxthon by any means, but I'm finding MX3 a breath of fresh air compared to the usual suspects. I have found a couple of sites it's not happy with though, so full compatibility is not quite there, but I'm guessing that their latest and greatest browser is better in that regard. Still, it seems much snappier for most things than FF or IE and can use the IE engine with a click of the mouse if all else fails. It also loads really quickly.

    I haven't tested it with regard to runaway background writing so have no idea if it also suffers from this problem, I'm hoping it doesn't :)
     
    #46
  7. Mike-EEE

    Mike-EEE New Member

    Joined:
    Sep 24, 2016
    Messages:
    2
    Likes Received:
    0
    Wow... super interested in this thread, as I just now learned of it from the linked article. I do use Chrome, however, and it seems as if this is a bigger problem (~24GB/day) than FireFox (~12-16GB/day)? Yikes.

    FWIW, I bought a Samsung 850 PRO 512GB a year ago for my Windows 10 install, and just now checked on it. In addition to 22.83TB of data already written to it, it also has 2 fails on my SMART test:
    [​IMG]

    Not getting the warm n' fuzzies here. :p
     
    #47
  8. wildpig1234

    wildpig1234 Well-Known Member

    Joined:
    Aug 22, 2016
    Messages:
    1,617
    Likes Received:
    191
    wow,, 22tb in 1 yr? that's really bad. you must have your computer on 24/7 with several browsers open at all time. that's 60gb/day. might have to check to make sure you don't also have any malware going too...

    on my other computer which I also have running 24/7 it has 37tb over 3.75 yrs. I do have several IE windows open all the time but it seems to be ie and edge don't have this idle constant writing. still I thought 37 tb over 3.75 yrs was real bad bc I never save any download to the boot ssd drive. I think ie just saves a lot of my download to the download folder first then copy over to where I select to save? either way, I will have to look at this more...
     
    #48
  9. Mike-EEE

    Mike-EEE New Member

    Joined:
    Sep 24, 2016
    Messages:
    2
    Likes Received:
    0
    Blargh, heh. I don't think it's malware. I have Windows Defender and it has never reported anything, but who knows. That's a good point, however, it is *is* ~60gb/day, so where is it coming from? I can definitely confirm that I always have (or had :p ) Chrome open with a ton of tabs open at any given point. I also have a bunch of extensions installed as well.

    Not how I wanted to spend my Saturday. :p

    Edit: Looks like my drive is good for up to 3 Petabytes of writes on the 850 Pro 512GB:
    Testing Samsung 850 Pro Endurance & Measuring V-NAND Die Size

    Additionally, SSDLife Pro (thanks for the reco!) is saying that my drive's health is at 99%, and at this rate should be good until January 2025. sooooooo... not feeling TOO bad about where I stand currently. Those SMART fails are not good, however.
     
    #49
    Last edited: Sep 24, 2016
  10. tukoz

    tukoz New Member

    Joined:
    Sep 24, 2016
    Messages:
    5
    Likes Received:
    2
    @Mike-EEE I think you can check it here [2]

    Waaah I wrote under Sergei's paper before I received the forum email confirmation. Should have waited.

    So I launched disk IO monitoring tool `iotop` in cumulative mode on my desktop right after reading this article today morning. Below are the results after tracking any single disk READ or WRITE today. Please note that my desktop SSD has a SandForce controller and I have just no idea whether iotop takes its on-the-fly compression in account or not.

    – Firefox Aurora v50 has been running for six hours (half of it pretty active) with my current session of 190 tabs.
    – Slimjet (Chrome fork) was launched and running for one hour long (5′ of which where active) with 3-5 tabs.
    – Thunderbird was also launched and running for one hour (with automatic fetching of 5 IMAP accounts).
    – Last, my podcast client (with automatic fetching of new episodes) was launched and running for two hours.

    $ iotop -oPa
    Code:
    PID DISK READ DISK WRITE> SWAPIN IO COMMAND
    6343 46.44 M 309.09 M 0.00 % 0.00 % firefox-aurora
    15342 38.04 M 90.58 M 0.00 % 0.01 % python2 /usr/bin/gpodder
    17062 76.27 M 59.71 M 0.00 % 0.01 % slimjet
    28931 97.32 M 8.75 M 0.00 % 0.00 % thunderbird
    So taken back to one (and then twenty four) hour each, stats are as follow
    • Firefox (190 tabs): 7.74 and 51.51 MB reads/writes per hour → 0.186 and 1.236 GB reads/writes per 24h
      ‘browser.sessionstore.interval’ has been set on ten minutes even since I put an SSD in that box. ‘browser.bookmarks.max_backups’ set to three for even longer.
      Also when idling, FF writes to its cache and truly only in there.
    • Slimjet (3-5 tabs): 73.27 and 59.71 MB reads/writes per hour→ 1.758 and 1.433 GB reads/writes per 24h
      when idling writes almost nothing to its cache (which is badly located in the user's config directory)
    • Thunderbird (5 accounts): 97.32 and 8.75 reads/writes per hour→2.336 and 0.21 GB reads/writes per 24h
    • gPodder (Podcast client w/a hundred shows): 19.02 and 45.29 reads/writes per hour→0.456 and 1087 GB reads/writes per 24h.
    S.M.A.R.T. says my SanDisk Extreme SSD [1] has written 23 TB in 27632 hours:
    Code:
    9 Power_On_Hours_and_Msec 0x0032 069 069 000 Old_age Always – 27632h+35m+20.000s
    177 Wear_Range_Delta 0x0000 000 000 000 Old_age Offline – 7
    241 Lifetime_Writes_GiB 0x0032 000 000 000 Old_age Always – 23459

    So?
    Given the measured endurance for even more budget and smaller SSD is of dozen of TB written [2], that the real amount of data physically written *may* be lower than what iotop reports by an order of a magnitude (both sessionstore and else files are heavily compressible text files) thanks to the SF “magic”, and last but not the least that all my data is backed up twice a day to an external destination, I’d say what’s the big deal of all this? Focusing on a single process is OK but not cool unless taking the others in account.

    For those of us using the most hackable main stream browser available, do our job and reduce its IO activity as we wish. Gain a 50 years SSD life expectancy if that’s what *you* want ;), a higher battery life on our laptop and a (much) better reactivity by making full use of our available RAM. And think of these using another browser that can’t be hacked half of what Firefox (or Seamonkey, Pale Moon) can. E.g. using a couple of small utilities available for Linux I swear I could have FF write 10 MB or less per hour.

    [1]: SanDisk Extreme SSD (with then ubiquitous SF2281VB1-SDC) on Arch Linux.
    [2]: SSD Endurance Test - Live Testing Samsung EVO, SanDisk, Intel and Kingston
     
    #50
    keybored and polar101 like this.
  11. keybored

    keybored Active Member

    Joined:
    May 28, 2016
    Messages:
    286
    Likes Received:
    65
    Not seeing it with my tests. Fired up Edge and opened 3 tabs -- cnn.com, foxnews.com, and msnbc.com. Let the browser sit for a couple of minutes and then enabled capture with the following filters:
    edge-disk-write-procmon-config.jpg

    Shut down capture after 5 minutes and looked at the stats. Result: 54MB written to disk without me touching the browser at all. If left running, it would write 15.5GB in a day just sitting there in the background.

    edge-disk-write-5min-totals.jpg
     
    #51
    tukoz likes this.
  12. wildpig1234

    wildpig1234 Well-Known Member

    Joined:
    Aug 22, 2016
    Messages:
    1,617
    Likes Received:
    191
    I would not necessarily blame the browser in this case. you should not have any pages that have a lot of active contents refreshing itself. cnn fox msnbc are all known for a lot of content pushing. try something like google.com or en.wikipedia.com home page.

    Come to think of it, some of this blame does lay on us as well as sites having a lot of these browsers open idling on sites that keep pushing stuffs through...

    I think a true test is really having the browser open to no page or page with no active contents. we can then do a baseline comparison between browsers that way. Then you can also test out browsers on all same content pushing pages like cnn and see which streams the most
     
    #52
    Last edited: Sep 24, 2016
  13. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Try Yahoo! Finance or Sports and you will likely see a big spike in usage.

    Also - the ssdendurance project is not really relevant to these drives. In an OS drive with Firefox you have say 100GB of programs/ data. If you have a 160GB SSD that means a huge portion of the NAND is occupied 24x7 with data. On many controllers, that data will not be touched and you are basically hammering these writes on a much smaller pool of NAND. I do really like what TechReport did and what have you but a half or 60% full drive is going to be different than a drive where you are hammering the entire drive.
     
    #53
    T_Minus likes this.
  14. tukoz

    tukoz New Member

    Joined:
    Sep 24, 2016
    Messages:
    5
    Likes Received:
    2
    So I used a tiny tool that preloads most <application>'s files into memory and mounted its cache in RAM. With these, IO activity has been diminished to a fraction of what some of the colleagues reported in this thread. Launching[1] Firefox-Aurora with my 190 tabs session then going for dinner and coming back on the computer show these IO stats for a 20' idling period of time (Slimjet was running playing some youtube HD) :

    Code:
      PID       DISK READ DISK WRITE>  SWAPIN      IO    COMMAND
    21335        43.78 M      8.09 M  0.00 %  0.01 % firefox-aurora
    17062         2.96 M      7.38 M  0.00 %  0.00 % slimjet
    That's 24 MB writes per hour including launch for a loaded Firefox, and 22 MB for the Blink-based browser. Do you think I should be worrying about my SanDisk SSD wearing out because of the browser?;)

    According to SSD Write Endurance 25nm Vs 34nm , which let only 12 GB free on the drives during testing, all but two of these tiny-to-small old SSD supported from 110 to 900 TB writes 24H a day without idle time before wearing out. If writing a creazy 100 GB a day, means 65 months for the worst case (OCZ Vertex Turbo Indilinx 64 GB).

    @Patrick reading your last post was a bit of a surprise. As it looks as if you said (re)writing data on a ssd occurs only on the NAND cells free of existing data. Which is quite not the case unless you'd take wear leveling out of the equation [2]

    [1] After clearing the page cach
    [2]:http://www.thessdreview.com/daily-n...ion-and-trim-in-ssds-explained-an-ssd-primer/
     
    #54
  15. keybored

    keybored Active Member

    Joined:
    May 28, 2016
    Messages:
    286
    Likes Received:
    65
    As I recall, some drives report host writes and NAND writes separately. If someone has a drive that does this, it would be interesting to see what sort of an impact this internal compression may have. IIRC, Intel 5 series SSDs show NAND writes.

    Looking at [2], if I owned a 60GB Kingston V300 then I might be worried. Apparently those drives have something called "DLP", which stands for Drive Life Protection, and this feature will throttle them to the point of being useless if the drive "thinks" that you will exceed the TBW number specified in the warranty, which is 32TB over 3 years for this drive. That translates to about 28GB/day. And in one of my tests I saw 35GB/day in writes from Firefox alone.

    Other than the Kingston, do I think FF will actually kill your SSD? No. It'll just make it worth less, but not worthless :), which may be important if you're a person who likes to upgrade by selling old stuff.

    Edit: forgot to mention -- I don't know if there is a precedent for this or not, but some manufacturers may not be willing to exchange your drive under warranty if you exceed the listed TBW number. The NAND may last well past that number on average, but you might be sol if the drive bricks itself after 2 years for an unrelated reason and you've already exceeded the stated TBW number.
     
    #55
    Last edited: Sep 24, 2016
  16. keybored

    keybored Active Member

    Joined:
    May 28, 2016
    Messages:
    286
    Likes Received:
    65
    What we're finding out is that it's not that black&white. A lot depends on your specific usage patterns and the sites you frequent. Patrick's 24GB/day number came from an actively used Chrome session; i.e., he was actively using the browser during his data capture. I had two separate data capture sessions both of which were idle; i.e., I left the browser sitting in the background, and I captured two different numbers -- 10GB/day going to recovery.js in one case and 35GB/day going into the cookie sqlite database in the other case. If I were actively using those browsers, those numbers would have been considerably higher. I've done a few random captures with FireFox and observed that 1 click costs me about 1MB in writes on average and that's just data going into recovery.js and cookie* files.

    What tool are you using to get this SMART data? Samsung will probably exchange this drive for you since it's failing SMART.
     
    #56
  17. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    @tukoz - I have known Kent for some time (he was at LSI and now Seagate.) GC algorithms and wear leveling algorithms work best when there is more available NAND. If you look at page 2 of that article you can see why. That is a fairly foundational element of understanding how SSDs work. I did not know it would be surprising.

    It is true that write amplification can mean you have more data written to SSD NAND than the host thinks it is writing. That is workload/ controller dependent. SandForce tackled this by compressing data which meant that you could have lower write amplification. For its time it was a nice solution.

    With that said, most enterprise drives are rated using TBW using 4K random writes. Sequential writes are not as hard on drives. That XS was not using 4K random writes so we would expect higher figures. That methodology is not wrong, nor invalid. In fact, it is probably closer to real world usage. They are not testing against the standard specs most vendors quote so it is hard to compare those figures versus manufacturer specs.

    As we move to smaller processes, the TBW ratings are plummeting to the point there are sub 100TBW rated drives.
     
    #57
  18. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,939
    Likes Received:
    857
    You guys are missing the point. If you have an 850 Pro you don't have what most people have. Most people run HPs, Dells, Lenovos, or whatever brands. Those don't use 850 Pros they'll use something like the evos. Most laptops aren't 512GB+ SSD size, there's lots of people with 120-240GB SSDs and trash write endurance because most people don't have a clue or care.

    More constructive, active browsing over an hour I can get over 1MB/s.
     
    #58
    snadge likes this.
  19. tukoz

    tukoz New Member

    Joined:
    Sep 24, 2016
    Messages:
    5
    Likes Received:
    2
    Good news below but first, honour to these taking part in the discussion.

    @Patrick Thanks for clarifying this. I was referring to this declaration in #53:
    Obviously the more data present, the more blocks Garbage Collection has to move. Note that I use Linux and my system + applications and $HOME take a total of 15 GiB on the 114 GiB SSD. Then there are the VMs which tackle a lot of the available space, but I care to make any partition's free space 20% or more (even on HDDs as it helps keeping fragmentation equal or close to zero on ext4 filesystems).


    @keybored how true for sustained TBW workflows! In the above extremesystems.org batch of tests, a guy called Ao1 had a small OCZ Vertex 2 hit by Life-Time-Throttling at a ridiculous 24 TB (with TRIM time growing annoyingly before that) and throttling resisted all attempts he made to reset the drive.
    It turns out Corsair, SanDisk (maybe all SF based SSDs?) keep track of both host and NAND write. So here are the values for my 3.5 years old SanDisk Extreme SSD (SF2281VB1-SDC)
    Code:
    ID# ATTRIBUTE_NAME          FLAG    VALUE WORST  THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
    233 SandForce_Internal      0x0000   000   000   000    Old_age   Offline      -       15289
    234 SandForce_Internal      0x0032   000   000   000    Old_age   Always       -       23462
    241 Lifetime_Writes_GiB     0x0032   000   000   000    Old_age   Always       -       23462
    242 Lifetime_Reads_GiB      0x0032   000   000   000    Old_age   Always       -       8290
    231 SSD_Life_Left           0x0013   099   099   010    Pre-fail  Always       -       0
    That is a nice write amplification factor of 1.54. Also according to the MWI it's still in a good fit. Thank you for making me search on this point.


    @MiniKnight link above has older Vertex 2 40GB, Samsung F3 / 470, Intel 320, Kingston X25-V or Crucial C300.


    Now back to the topic.
    Today I checked again how much an idle fully loaded Firefox writes. Checking which of the application or our lazyness is the real culprit behind these writes :cool:
    Code:
    $ ps aux|grep aurora           # i'm looking for the Firefox's PID
    kozaki   28688 ...       Sl   17:57   0:04 /opt/firefox-aurora/plugin-c...
    kozaki   29206  ...      Sl   15:59   3:36 firefox-aurora
    kozaki   29339  ...       Sl   15:59   1:33 /opt/firefox-aurora/plugin-c...
                                                                                 
    └─> $ ps -p "29206" -o etimes=                                                
       7151              # how long it's been running in seconds
    
    └─> $ iotop -oPa
      PID       DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
    29206         4.00 K     19.60 M  0.00 %  0.00 % firefox-aurora
        312           4.00 K      7.89 M  0.00 %  0.00 % systemd-journald
    That's 9.76 MB written per hour launch included.

    Wondering how your browser's writes exceed those of FF here by an order of 3 magnitudes? And how they can be higher for a single browser than for my whole box? I mean this box: writes on a daily basis the typical amount of Arch updates, plus those for a half a dozen ssd-virtualised hosts, runs all sort of compilation, and does bilateral sync'ing with my cloud accounts and laptops any time a day. Resulting in an average 13 GB written per day in its 3.5 years life (4.08/3.12 GiB host/NAND writes a day for the last year when it was used more than ever). Anyway as power users, it's up to us to figure out how to make our browser and system follow our priority.
     
    #59
  20. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,939
    Likes Received:
    857
    @tukoz Here is the strange thing - on the main site and on the bug that has been created on Mozilla's bug tracker the owner of the feature admits it is an issue they know about. I don't know why you don't see the same writes but if other users are reporting it, and one of the people who owns the software admits it is an issue I don't really see the point in debating whether or not it is an issue. Maybe you've got a different setup which is great for you.

    TBH - I got to meet our admin in person and I can tell you he's top notch in terms of understanding modern architectures. That's why STH is one of the few independent server testing spots. I now assume he could write about almost any of the hardware tech pieces we read elsewhere.

    @Patrick yahoo finance is holy $!$@#!$ on page load. You're right. I was over 450kbps writes.
     
    #60
Similar Threads: Firefox chewing
Forum Title Date
Software Stuff Question about Firefox browser.sessionstore setting Oct 29, 2016

Share This Page