Firefox is chewing through your NAND

Discussion in 'Software Stuff' started by keybored, Sep 18, 2016.

  1. keybored

    keybored Active Member

    Joined:
    May 28, 2016
    Messages:
    286
    Likes Received:
    65
    See what sort of writes you get by following steps in post #21 of this thread. Specifically, open the same type of content in three separate FF windows. At the very least, you should see FF churning through lots of data in its cookie files just sitting idle.
    The key to replicating the issue with recovery.js is to increase the size of this file. Try to get the size up to 1.5-2MB by opening sites that may use lots of session data. I have a mailbox over at outlook.com and opening it up seems to give a good jolt. Something like facebook, google+, or linkedin may have lots of session data as well.
     
    #61
    tukoz likes this.
  2. tukoz

    tukoz New Member

    Joined:
    Sep 24, 2016
    Messages:
    5
    Likes Received:
    2
    @keybored Test done. Will post the results tomorrow as I have some urgent stuff to do before sleepin. Thanks for proposing :)

    @MiniKnight Maybe to relieve our drives of GBs writes a day?
    No one is contesting the fact than Firefox and other main stream browsers/applications have flaws in their code resulting in heavier IO than we'd like. But rather, than no application *defaults* is perfect. And that you as me are free to share the ways we found to have it follow our goals. Which is where the fun starts since Firefox is the most hackable between those. Isn't Sergei's paper titled « here is how to fix it » in the first place?
    Pointing out a flaw potentially affecting millions in a widely used open source application implies a couple next steps IMHO: search/share a) how to make the flaw a souvenir and b) to contribute improving the software (eg here).

    @keybored test results

    I reproduced the whole steps, only keeping my settings (blocking the most infamous trackers and saving the session every 10 minutes): I'd rather measure its impacts than "confirm" an issue that is acknowledged for years (eg here or there [1]), and that I got interested into around 2012.

    1. At 21:53:13 I launched Firefox-aurora (cold start)
    - Opened 10 random reviews for Samsung 850 pro and 10 more tabs for Samsung Galaxy S7 in two separate tabs groups.
    - Opened Innoreader, then opened the ten latest news sites articles that I marked for read (in 10 tabs).

    firefox disk activity 1:
    Code:
        PID  DISK READ DISK WRITE>  SWAPIN      IO    COMMAND
        30850 520.00 K     59.96 M  0.00 %  0.00 % firefox-aurora
    I noted an unusual burst of IO activity during this launch (2-3 × the average here). recovery.js:
    Code:
        ... 1,4K 25 sept. 21:48 /home/kozaki/.mozilla/firefox/XXXX/sessionstore-backups/recovery.js
    2. Left the 31 FF tabs sitting idle for 55 minutes while doing something else.
    At 22:53:44 I checked the FF total IO activity to get overall stats for the last hour, which was 17.86 MB.

    firefox-disk-activity-2:
    Code:
        PID  DISK READ DISK WRITE>  SWAPIN      IO    COMMAND                          
        30850 2.06 M     77.82 M  0.00 %  0.00 % firefox-aurora   
    recovery.{js,bak} where 52 KB each.
    SIZE TIME NAME
    1024 2016-09-25 20:57:24 bookmarkbackups/
    1024 2016-09-25 22:03:11 extension-data/
    1024 2016-09-25 22:09:09 minidumps/
    1024 2016-09-25 22:09:09 crashes/
    1024 2016-09-25 22:48:16 sessionstore-backups/
    1024 2016-09-25 22:53:31 datareporting/
    67584 2016-09-25 22:53:32 saved-telemetry-pings/
    53846 2016-09-25 22:38:16 recovery.bak
    53846 2016-09-25 22:48:16 recovery.js
    262368 2016-09-25 22:09:10 webappsstore.sqlite-wal
    32768 2016-09-25 22:09:10 webappsstore.sqlite-shm
    10485760 2016-09-25 22:10:11 places.sqlite
    524288 2016-09-25 22:10:14 cookies.sqlite
    6915 2016-09-25 22:10:41 xulstore.json
    12301 2016-09-25 22:14:15 SiteSecurityServiceState.txt
    1066109 2016-09-25 22:16:15 xmarks-baseline-bb36233e3e21208b.json
    58788 2016-09-25 22:16:15 prefs.js
    1004610 2016-09-25 22:16:16 xmarks.log
    65616 2016-09-25 22:44:59 places.sqlite-wal
    32768 2016-09-25 22:46:29 places.sqlite-shm
    590288 2016-09-25 22:46:29 cookies.sqlite-wal
    32768 2016-09-25 22:46:29 cookies.sqlite-shm

    3. Went back to Firefox and opened my (Zimbra) mailbox.
    Checked a couple emails and filters, then again, left Firefox sitting idle, but only for ~10 minutes.
    At 23:04:00 I checked the FF total IO activity for this last part of the test: 10.60 MB.

    firefox-disk-activity-3:
    Code:
        PID       DISK READ DISK WRITE>  SWAPIN      IO    COMMAND
      30850         3.21 M     88.42 M  0.00 %  0.00 % firefox-aurora
    recovery.{js,bak} where still 52 KB each.

    67584 2016-09-25 22:57:37.000000000 +0200 saved-telemetry-pings/
    1024 2016-09-25 22:58:16.000000000 +0200 sessionstore-backups/
    53846 2016-09-25 22:48:16.000000000 +0200 recovery.bak
    55787 2016-09-25 22:58:16.000000000 +0200 recovery.js
    1024 2016-09-25 23:03:31.000000000 +0200 datareporting/
    6915 2016-09-25 23:00:35.000000000 +0200 xulstore.json
    131200 2016-09-25 23:03:04.000000000 +0200 places.sqlite-wal
    32768 2016-09-25 23:03:04.000000000 +0200 places.sqlite-shm
    10485760 2016-09-25 23:03:04.000000000 +0200 places.sqlite
    @keybored hopefully you'll understand the reason behind reproducing your testing with my current settings. At under 100 MB writes an hour cold launch I have no fear for the not so fragile NAND cells, nor the battery (i.e. on laptops where I think I did a more IO efficient setup).

    PS as you may have noted I have telemetry and all the Beta reporting stats activated (as well as profile and bookmarks sync'ing) for this profile.

    [1]: See also Firefox fork called Pale Moon which devs set the default timer to write recovery.js each 10 minutes (makes pretty much sense for a browser that shines on low pro boxes IMH)
     
    #62
    Last edited: Sep 26, 2016
  3. Mydst

    Mydst New Member

    Joined:
    Sep 26, 2016
    Messages:
    5
    Likes Received:
    1
    I've been following this thread with interest and I've found that Chrome is doing the same thing. Leaving Chrome idle it will still write many GBs to the drive.

    However, I've found a new super offender- Spotify. As I was tracking Chrome's usage on my desktop, Spotify would quickly outpace it with writes- even when it wasn't streaming or playing music. I left Spotify open, but not streaming and it wrote 50GBs to my disk overnight! I've attached a screenshot as it is almost unbelievable. There are posts about this on the Spotify forums but there has never been any sort of reply or solution that I can see.
     

    Attached Files:

    #63
  4. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    @Mydst that is really interesting. What is Spotify doing? Are you trying to download content locally? I wonder if it is caching content like album art. I also wonder if all of that is going over the WAN.
     
    #64
  5. tukoz

    tukoz New Member

    Joined:
    Sep 24, 2016
    Messages:
    5
    Likes Received:
    2
    @Mydst Holy cow! what sort of caching is Spotify doing? As Patrick I'd like to see its network activity...

    Even Bird is not a contender: From Heavy SSD Writes from Firefox | Hacker News:
     
    #65
  6. Mydst

    Mydst New Member

    Joined:
    Sep 26, 2016
    Messages:
    5
    Likes Received:
    1
    Spotify is doing nothing at all (that I can see) which is the mystery. It doesn't matter if I set it to cache music (play offline) or if I just stream. It's even worse when I'm actually streaming music. My network shows only about 300MB of usage for Spotify and yet it's writing tens of gigabytes to the drive. I've tried completely blocking the cache folder (which was only a few hundred MB) but it did nothing. I don't know how Spotify actually handles its files, perhaps it's downloading it to a database that is constantly being refreshed? But again, why is it doing this when nothing is being streamed?

    I've attached a screenshot of Spotify during the same period it wrote 50GB to disk. Only 316MB in downloads during that time. This was playing music for maybe an hour and then just letting it sit overnight doing nothing.

    I've also used Spotify for about 2 hours this morning and it's already burned through 12GB of writes.
     

    Attached Files:

    #66
    Patrick likes this.
  7. Mydst

    Mydst New Member

    Joined:
    Sep 26, 2016
    Messages:
    5
    Likes Received:
    1
    So after some more research it looks like Spotify is misbehaving in a way similar to Chrome.

    Spotify is constantly reading "mercury.db" which appears to be a SQlite table. So there's constant read/writes to this table even when nothing is streaming. The data then appears to be written inside the Window user's appdata temp folder.

    So basically Spotify is constantly reading an SQLite table then writing to a temp file. Process monitor is showing this happening many times per second even when music is not playing and no network is present.

    This is really bad behavior. I've used 30GB of writes in the last few hours just letting Spotify sit open.
     

    Attached Files:

    #67
  8. capn_pineapple

    capn_pineapple Active Member

    Joined:
    Aug 28, 2013
    Messages:
    356
    Likes Received:
    80
    Out of curiosity, has anybody approached any of these companies with a "why is this a thing?"??
     
    #68
  9. Mikhoul

    Mikhoul New Member

    Joined:
    Sep 26, 2016
    Messages:
    3
    Likes Received:
    1
    On Firefox there is a good way to mitigate this problem right now without sacrificing anythings. :D

    You just have to use the Addon Auto Unload Tabs: https://addons.mozilla.org/en-US/firefox/addon/auto-unload-tab/

    It can unload tab on various condition like: Unload tab ffter X minutes, Unload tab if size is over X megabytes etc etc.

    The Addon is fully compatible with E10. :cool:

    N.B.: Personally it unload my tabs after 15 minutes if I have not used them.

     
    #69
  10. Filz

    Filz New Member

    Joined:
    Sep 27, 2016
    Messages:
    2
    Likes Received:
    1
    I think you're reading the incorrect data in Process Explorer. When choosing the columns to view, you want to pick "Write Bytes" under the tab for "Process Disk," not the one for "Process I/O" which doesn't report exactly what you want. Basically you want the column to read "Disk Write Bytes," and not "I/O Write Bytes."

    This ghacks article shows how to set it up for Process Explorer:
    Display the disk activity of any process in Windows - gHacks Tech News

    I/O Write Bytes is different, at least that's what I'm reading as of late.
    What does "I/O Reads or Writes" and "I/O Read Bytes or Write Bytes" mean?
    Another post about "I/O" vs "disk" but this time concerning Process Hacker, another tool kind of like Process Explorer: Inconsistent info: Disk/io vs. process view i/o. - Process Hacker Forums
    I found this thread from the main article (Firefox is eating your SSD - here is how to fix it) that got posted everywhere lately and I noticed that the Process Explorer picture is setup incorrectly and showing the inflated number 34.2GB I/O Write Bytes. Again, the column should read as "Disk Write Bytes."
     
    #70
    Last edited: Sep 27, 2016
    Patrick likes this.
  11. wildpig1234

    wildpig1234 Well-Known Member

    Joined:
    Aug 22, 2016
    Messages:
    1,617
    Likes Received:
    191
    have a quick question, I am still using ie8 on one of my win7 computer. When you right click and save a file on a webpage, ie doesn't start saving that file to the directory that you choose but it first download to a temp file on the system drive somewhere. Does anyone know where and how I can change the location of this? I think this is what also causing a fair amount of write to my boot ssd. thanks
     
    #71
  12. Mydst

    Mydst New Member

    Joined:
    Sep 26, 2016
    Messages:
    5
    Likes Received:
    1
    I really hope this is the case but I tried to setup Process Explorer like that guide lists but those options don't exist in my version (the newest one). I've attached a screenshot. The disk I/O is the only option. Taskmanager and SSDready were also reporting the I/O bytes written the same way so it seems consistent with Process Explorer. Anyone else seeing this with different monitoring software?
     

    Attached Files:

    #72
  13. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,939
    Likes Received:
    857
    I tested Firefox in Windows VM I had around. The hypervisor disk writes seemed to match what I was seeing on the main site post in terms of disk activity. I took from that that it was working. Trivial to monitor this type of activity when in a VM.
     
    #73
  14. Mikhoul

    Mikhoul New Member

    Joined:
    Sep 26, 2016
    Messages:
    3
    Likes Received:
    1

    You can just use Perfmon, it's more ez and already in Windows:

    Click on the Start Button and key in perfmon.exe /res and hit Enter. This should launch the Resource Monitor

    [​IMG]
     
    #74
    Marsh likes this.
  15. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    The Samsung 750 EVO drives for endurance testing arrive tomorrow
     
    #75
  16. Filz

    Filz New Member

    Joined:
    Sep 27, 2016
    Messages:
    2
    Likes Received:
    1
    You need to run Process Explorer as admin and the "Process Disk" tab will appear. You can elevate privileges by going to the file menu and clicking on "Show Details for All Processes."
     
    #76
  17. Mydst

    Mydst New Member

    Joined:
    Sep 26, 2016
    Messages:
    5
    Likes Received:
    1
    Hmmm...I did what you said and got the correct disk tab to show up but it's constantly empty for everything. Maybe it's a limitation of my drive? It's a PNY SSD. If that's the case I guess there's no way to see what is actually happening and if this I/O data is actually writing to my disk. I tried ProcessHacker too and the same result even run with admin permissions- the write column is empty for every program which of course can't be correct either.

    This is definitely a case where I really hope I'm wrong and this is all nothing! spotifyprocesshacker.jpg
     
    #77
  18. Mikhoul

    Mikhoul New Member

    Joined:
    Sep 26, 2016
    Messages:
    3
    Likes Received:
    1

    Why don't you use my method ? It's more simple and work on any computer with any HD. ;)

    http://tinyurl.com/zxgf3tq
     
    #78
  19. Larson

    Larson Member

    Joined:
    Nov 10, 2015
    Messages:
    96
    Likes Received:
    7
    Does anybody know how Internet Explorer 11 compares?
     
    #79
  20. enigmaxg2

    enigmaxg2 New Member

    Joined:
    Oct 31, 2016
    Messages:
    2
    Likes Received:
    0
    Spotify uses Libcef.dll (which is an embedded Chrome engine), so it acts like it. I'm pretty sure that other web-wrapper desktop apps like Reditr, WhatsApp Desktop, Steam and uPlay (which also use Libcef) do exactly the same.
     
    #80
    Last edited: Oct 31, 2016
Similar Threads: Firefox chewing
Forum Title Date
Software Stuff Question about Firefox browser.sessionstore setting Oct 29, 2016

Share This Page