Firefox is chewing through your NAND

keybored

Active Member
May 28, 2016
286
65
28
...

Wondering how your browser's writes exceed those of FF here by an order of 3 magnitudes? And how they can be higher for a single browser than for my whole box? ...
See what sort of writes you get by following steps in post #21 of this thread. Specifically, open the same type of content in three separate FF windows. At the very least, you should see FF churning through lots of data in its cookie files just sitting idle.
The key to replicating the issue with recovery.js is to increase the size of this file. Try to get the size up to 1.5-2MB by opening sites that may use lots of session data. I have a mailbox over at outlook.com and opening it up seems to give a good jolt. Something like facebook, google+, or linkedin may have lots of session data as well.
 
  • Like
Reactions: tukoz

tukoz

New Member
Sep 24, 2016
5
2
3
49
London ».«Paris
@keybored Test done. Will post the results tomorrow as I have some urgent stuff to do before sleepin. Thanks for proposing :)

@MiniKnight Maybe to relieve our drives of GBs writes a day?
No one is contesting the fact than Firefox and other main stream browsers/applications have flaws in their code resulting in heavier IO than we'd like. But rather, than no application *defaults* is perfect. And that you as me are free to share the ways we found to have it follow our goals. Which is where the fun starts since Firefox is the most hackable between those. Isn't Sergei's paper titled « here is how to fix it » in the first place?
Pointing out a flaw potentially affecting millions in a widely used open source application implies a couple next steps IMHO: search/share a) how to make the flaw a souvenir and b) to contribute improving the software (eg here).

@keybored test results

I reproduced the whole steps, only keeping my settings (blocking the most infamous trackers and saving the session every 10 minutes): I'd rather measure its impacts than "confirm" an issue that is acknowledged for years (eg here or there [1]), and that I got interested into around 2012.

1. At 21:53:13 I launched Firefox-aurora (cold start)
- Opened 10 random reviews for Samsung 850 pro and 10 more tabs for Samsung Galaxy S7 in two separate tabs groups.
- Opened Innoreader, then opened the ten latest news sites articles that I marked for read (in 10 tabs).

firefox disk activity 1:
Code:
    PID  DISK READ DISK WRITE>  SWAPIN      IO    COMMAND
    30850 520.00 K     59.96 M  0.00 %  0.00 % firefox-aurora
I noted an unusual burst of IO activity during this launch (2-3 × the average here). recovery.js:
Code:
    ... 1,4K 25 sept. 21:48 /home/kozaki/.mozilla/firefox/XXXX/sessionstore-backups/recovery.js
2. Left the 31 FF tabs sitting idle for 55 minutes while doing something else.
At 22:53:44 I checked the FF total IO activity to get overall stats for the last hour, which was 17.86 MB.

firefox-disk-activity-2:
Code:
    PID  DISK READ DISK WRITE>  SWAPIN      IO    COMMAND                          
    30850 2.06 M     77.82 M  0.00 %  0.00 % firefox-aurora
recovery.{js,bak} where 52 KB each.
SIZE TIME NAME
1024 2016-09-25 20:57:24 bookmarkbackups/
1024 2016-09-25 22:03:11 extension-data/
1024 2016-09-25 22:09:09 minidumps/
1024 2016-09-25 22:09:09 crashes/
1024 2016-09-25 22:48:16 sessionstore-backups/
1024 2016-09-25 22:53:31 datareporting/
67584 2016-09-25 22:53:32 saved-telemetry-pings/
53846 2016-09-25 22:38:16 recovery.bak
53846 2016-09-25 22:48:16 recovery.js
262368 2016-09-25 22:09:10 webappsstore.sqlite-wal
32768 2016-09-25 22:09:10 webappsstore.sqlite-shm
10485760 2016-09-25 22:10:11 places.sqlite
524288 2016-09-25 22:10:14 cookies.sqlite
6915 2016-09-25 22:10:41 xulstore.json
12301 2016-09-25 22:14:15 SiteSecurityServiceState.txt
1066109 2016-09-25 22:16:15 xmarks-baseline-bb36233e3e21208b.json
58788 2016-09-25 22:16:15 prefs.js
1004610 2016-09-25 22:16:16 xmarks.log
65616 2016-09-25 22:44:59 places.sqlite-wal
32768 2016-09-25 22:46:29 places.sqlite-shm
590288 2016-09-25 22:46:29 cookies.sqlite-wal
32768 2016-09-25 22:46:29 cookies.sqlite-shm

3. Went back to Firefox and opened my (Zimbra) mailbox.
Checked a couple emails and filters, then again, left Firefox sitting idle, but only for ~10 minutes.
At 23:04:00 I checked the FF total IO activity for this last part of the test: 10.60 MB.

firefox-disk-activity-3:
Code:
    PID       DISK READ DISK WRITE>  SWAPIN      IO    COMMAND
  30850         3.21 M     88.42 M  0.00 %  0.00 % firefox-aurora
recovery.{js,bak} where still 52 KB each.

67584 2016-09-25 22:57:37.000000000 +0200 saved-telemetry-pings/
1024 2016-09-25 22:58:16.000000000 +0200 sessionstore-backups/
53846 2016-09-25 22:48:16.000000000 +0200 recovery.bak
55787 2016-09-25 22:58:16.000000000 +0200 recovery.js
1024 2016-09-25 23:03:31.000000000 +0200 datareporting/
6915 2016-09-25 23:00:35.000000000 +0200 xulstore.json
131200 2016-09-25 23:03:04.000000000 +0200 places.sqlite-wal
32768 2016-09-25 23:03:04.000000000 +0200 places.sqlite-shm
10485760 2016-09-25 23:03:04.000000000 +0200 places.sqlite
@keybored hopefully you'll understand the reason behind reproducing your testing with my current settings. At under 100 MB writes an hour cold launch I have no fear for the not so fragile NAND cells, nor the battery (i.e. on laptops where I think I did a more IO efficient setup).

PS as you may have noted I have telemetry and all the Beta reporting stats activated (as well as profile and bookmarks sync'ing) for this profile.

[1]: See also Firefox fork called Pale Moon which devs set the default timer to write recovery.js each 10 minutes (makes pretty much sense for a browser that shines on low pro boxes IMH)
 
Last edited:

Mydst

New Member
Sep 26, 2016
5
1
3
40
I've been following this thread with interest and I've found that Chrome is doing the same thing. Leaving Chrome idle it will still write many GBs to the drive.

However, I've found a new super offender- Spotify. As I was tracking Chrome's usage on my desktop, Spotify would quickly outpace it with writes- even when it wasn't streaming or playing music. I left Spotify open, but not streaming and it wrote 50GBs to my disk overnight! I've attached a screenshot as it is almost unbelievable. There are posts about this on the Spotify forums but there has never been any sort of reply or solution that I can see.
 

Attachments

Patrick

Administrator
Staff member
Dec 21, 2010
11,805
4,760
113
@Mydst that is really interesting. What is Spotify doing? Are you trying to download content locally? I wonder if it is caching content like album art. I also wonder if all of that is going over the WAN.
 

Mydst

New Member
Sep 26, 2016
5
1
3
40
@Mydst that is really interesting. What is Spotify doing? Are you trying to download content locally? I wonder if it is caching content like album art. I also wonder if all of that is going over the WAN.
Spotify is doing nothing at all (that I can see) which is the mystery. It doesn't matter if I set it to cache music (play offline) or if I just stream. It's even worse when I'm actually streaming music. My network shows only about 300MB of usage for Spotify and yet it's writing tens of gigabytes to the drive. I've tried completely blocking the cache folder (which was only a few hundred MB) but it did nothing. I don't know how Spotify actually handles its files, perhaps it's downloading it to a database that is constantly being refreshed? But again, why is it doing this when nothing is being streamed?

I've attached a screenshot of Spotify during the same period it wrote 50GB to disk. Only 316MB in downloads during that time. This was playing music for maybe an hour and then just letting it sit overnight doing nothing.

I've also used Spotify for about 2 hours this morning and it's already burned through 12GB of writes.
 

Attachments

  • Like
Reactions: Patrick

Mydst

New Member
Sep 26, 2016
5
1
3
40
So after some more research it looks like Spotify is misbehaving in a way similar to Chrome.

Spotify is constantly reading "mercury.db" which appears to be a SQlite table. So there's constant read/writes to this table even when nothing is streaming. The data then appears to be written inside the Window user's appdata temp folder.

So basically Spotify is constantly reading an SQLite table then writing to a temp file. Process monitor is showing this happening many times per second even when music is not playing and no network is present.

This is really bad behavior. I've used 30GB of writes in the last few hours just letting Spotify sit open.
 

Attachments

Mikhoul

New Member
Sep 26, 2016
3
1
3
44
On Firefox there is a good way to mitigate this problem right now without sacrificing anythings. :D

You just have to use the Addon Auto Unload Tabs: https://addons.mozilla.org/en-US/firefox/addon/auto-unload-tab/

It can unload tab on various condition like: Unload tab ffter X minutes, Unload tab if size is over X megabytes etc etc.

The Addon is fully compatible with E10. :cool:

N.B.: Personally it unload my tabs after 15 minutes if I have not used them.

 

Filz

New Member
Sep 27, 2016
2
1
3
120
I've been following this thread with interest and I've found that Chrome is doing the same thing. Leaving Chrome idle it will still write many GBs to the drive.

However, I've found a new super offender- Spotify. As I was tracking Chrome's usage on my desktop, Spotify would quickly outpace it with writes- even when it wasn't streaming or playing music. I left Spotify open, but not streaming and it wrote 50GBs to my disk overnight! I've attached a screenshot as it is almost unbelievable. There are posts about this on the Spotify forums but there has never been any sort of reply or solution that I can see.
I think you're reading the incorrect data in Process Explorer. When choosing the columns to view, you want to pick "Write Bytes" under the tab for "Process Disk," not the one for "Process I/O" which doesn't report exactly what you want. Basically you want the column to read "Disk Write Bytes," and not "I/O Write Bytes."

This ghacks article shows how to set it up for Process Explorer:
Display the disk activity of any process in Windows - gHacks Tech News

I/O Write Bytes is different, at least that's what I'm reading as of late.
What does "I/O Reads or Writes" and "I/O Read Bytes or Write Bytes" mean?
I/O Write Bytes - The number of bytes written in input/output operations generated by a process, including file, network, and device I/Os. I/O Write Bytes directed to CONSOLE (console input object) handles are not counted.
Another post about "I/O" vs "disk" but this time concerning Process Hacker, another tool kind of like Process Explorer: Inconsistent info: Disk/io vs. process view i/o. - Process Hacker Forums
It's not that easy for Windows to report disk activity. Say you have a file that's mapped into multiple processes, and they all write to it. In reality the mapped page writer flushes the data to disk (so System is reported as having disk I/O), but under your system which process should be associated with this?

Now, with "I/O" vs "disk". They're different concepts. If a process writes to a named pipe, that's I/O, but it has nothing to do with the disk(s) on your computer. Similarly if a process does I/O on a file, the contents don't have to be written out to disk immediately.
I found this thread from the main article (Firefox is eating your SSD - here is how to fix it) that got posted everywhere lately and I noticed that the Process Explorer picture is setup incorrectly and showing the inflated number 34.2GB I/O Write Bytes. Again, the column should read as "Disk Write Bytes."
 
Last edited:
  • Like
Reactions: Patrick

wildpig1234

Well-Known Member
Aug 22, 2016
1,749
259
83
45
have a quick question, I am still using ie8 on one of my win7 computer. When you right click and save a file on a webpage, ie doesn't start saving that file to the directory that you choose but it first download to a temp file on the system drive somewhere. Does anyone know where and how I can change the location of this? I think this is what also causing a fair amount of write to my boot ssd. thanks
 

Mydst

New Member
Sep 26, 2016
5
1
3
40
I think you're reading the incorrect data in Process Explorer. When choosing the columns to view, you want to pick "Write Bytes" under the tab for "Process Disk," not the one for "Process I/O" which doesn't report exactly what you want. Basically you want the column to read "Disk Write Bytes," and not "I/O Write Bytes."

This ghacks article shows how to set it up for Process Explorer:
Display the disk activity of any process in Windows - gHacks Tech News

I/O Write Bytes is different, at least that's what I'm reading as of late.
What does "I/O Reads or Writes" and "I/O Read Bytes or Write Bytes" mean?


Another post about "I/O" vs "disk" but this time concerning Process Hacker, another tool kind of like Process Explorer: Inconsistent info: Disk/io vs. process view i/o. - Process Hacker Forums

I found this thread from the main article (Firefox is eating your SSD - here is how to fix it) that got posted everywhere lately and I noticed that the Process Explorer picture is setup incorrectly and showing the inflated number 34.2GB I/O Write Bytes. Again, the column should read as "Disk Write Bytes."
I really hope this is the case but I tried to setup Process Explorer like that guide lists but those options don't exist in my version (the newest one). I've attached a screenshot. The disk I/O is the only option. Taskmanager and SSDready were also reporting the I/O bytes written the same way so it seems consistent with Process Explorer. Anyone else seeing this with different monitoring software?
 

Attachments

MiniKnight

Well-Known Member
Mar 30, 2012
2,981
884
113
NYC
I tested Firefox in Windows VM I had around. The hypervisor disk writes seemed to match what I was seeing on the main site post in terms of disk activity. I took from that that it was working. Trivial to monitor this type of activity when in a VM.
 

Mikhoul

New Member
Sep 26, 2016
3
1
3
44
I really hope this is the case but I tried to setup Process Explorer like that guide lists but those options don't exist in my version (the newest one). I've attached a screenshot. The disk I/O is the only option. Taskmanager and SSDready were also reporting the I/O bytes written the same way so it seems consistent with Process Explorer. Anyone else seeing this with different monitoring software?

You can just use Perfmon, it's more ez and already in Windows:

Click on the Start Button and key in perfmon.exe /res and hit Enter. This should launch the Resource Monitor

 
  • Like
Reactions: Marsh

Patrick

Administrator
Staff member
Dec 21, 2010
11,805
4,760
113
The Samsung 750 EVO drives for endurance testing arrive tomorrow
 

Filz

New Member
Sep 27, 2016
2
1
3
120
I really hope this is the case but I tried to setup Process Explorer like that guide lists but those options don't exist in my version (the newest one). I've attached a screenshot. The disk I/O is the only option. Taskmanager and SSDready were also reporting the I/O bytes written the same way so it seems consistent with Process Explorer. Anyone else seeing this with different monitoring software?
You need to run Process Explorer as admin and the "Process Disk" tab will appear. You can elevate privileges by going to the file menu and clicking on "Show Details for All Processes."
 

Mydst

New Member
Sep 26, 2016
5
1
3
40
You need to run Process Explorer as admin and the "Process Disk" tab will appear. You can elevate privileges by going to the file menu and clicking on "Show Details for All Processes."
Hmmm...I did what you said and got the correct disk tab to show up but it's constantly empty for everything. Maybe it's a limitation of my drive? It's a PNY SSD. If that's the case I guess there's no way to see what is actually happening and if this I/O data is actually writing to my disk. I tried ProcessHacker too and the same result even run with admin permissions- the write column is empty for every program which of course can't be correct either.

This is definitely a case where I really hope I'm wrong and this is all nothing!spotifyprocesshacker.jpg
 

Mikhoul

New Member
Sep 26, 2016
3
1
3
44
Hmmm...I did what you said and got the correct disk tab to show up but it's constantly empty for everything. Maybe it's a limitation of my drive? It's a PNY SSD. If that's the case I guess there's no way to see what is actually happening and if this I/O data is actually writing to my disk. I tried ProcessHacker too and the same result even run with admin permissions- the write column is empty for every program which of course can't be correct either.

This is definitely a case where I really hope I'm wrong and this is all nothing!View attachment 3498

Why don't you use my method ? It's more simple and work on any computer with any HD. ;)

http://tinyurl.com/zxgf3tq
 

enigmaxg2

New Member
Oct 31, 2016
2
0
1
29
Spotify uses Libcef.dll (which is an embedded Chrome engine), so it acts like it. I'm pretty sure that other web-wrapper desktop apps like Reditr, WhatsApp Desktop, Steam and uPlay (which also use Libcef) do exactly the same.
 
Last edited: