New Linux-Bench request: Firefox Compile time
- Thread starter Patrick
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
Update for those not following the git repo. Made something for Patrick to review.
General thoughts on compile tests... I can't really see how they are useful. So much depends on the compiler, storage medium, parallelability of the code for compiling, libraries linked in, etc. Hard to decide on a single factor or two that you are really stressing. But I've seen the tests so it's definitely interesting at the minimum. I never realized how long FF takes...
General thoughts on compile tests... I can't really see how they are useful. So much depends on the compiler, storage medium, parallelability of the code for compiling, libraries linked in, etc. Hard to decide on a single factor or two that you are really stressing. But I've seen the tests so it's definitely interesting at the minimum. I never realized how long FF takes...
While I second all of Chuckleb's concerns, my job involves trying to make compiles faster. At some point you've gotten to diminishing returns for the build system and are stuck trying to throw hardware at it. I don't know how useful it is as a benchmark but it would certainly be nice to be able to have some numbers for how much the new thing helps.
I'm getting several emails a month to include compilation tests (or, compilation in a VM) when I review a processor, hence my request to Patrick.
Yeah, there are a lot of factors involved. I/O could perhaps be less of a bottleneck by using a RAMDisk?
Or retests, making sure we get a fresh build if the test is run on the same system.
There are options to use different compilers as well. gcc tends to be the request I get given. But if you can remove the IO from the issue, it becomes a system test rather than a pure synthetic of CPU/DRAM and cache management.
Not a problem, any kind of repeatable test is good.
This test will need to be bare metal if we are using a ramdisk as Docker only passes through 64MB to the ramdisk. That can be fixed but makes things messy... Not the end of the world though.
It also is making me push to compartmentalize the tests, something on the wish list anyway. So that you can run specific tests.
It's written... Are there other compiles sources that are interesting?
This test will need to be bare metal if we are using a ramdisk as Docker only passes through 64MB to the ramdisk. That can be fixed but makes things messy... Not the end of the world though.
It also is making me push to compartmentalize the tests, something on the wish list anyway. So that you can run specific tests.
It's written... Are there other compiles sources that are interesting?
Anand used to run a Chromium compile benchmark (in windows) which was well received.
Obviously with all of this there needs to be a constant build version for repeatability, so it might be worth hosting elsewhere...
>VLC? VideoLAN - VideoLAN developers - VLC media player
>OpenOffice?
>Second Life viewer? Open Source Portal - Second Life Wiki
Obviously with all of this there needs to be a constant build version for repeatability, so it might be worth hosting elsewhere...
>VLC? VideoLAN - VideoLAN developers - VLC media player
>OpenOffice?
>Second Life viewer? Open Source Portal - Second Life Wiki
I've started a github issue on it we can track there as well.
Firefox Compile Benchmark · Issue #69 · STH-Dev/linux-bench · GitHub
Additional compile benchmarks · Issue #70 · STH-Dev/linux-bench · GitHub
Firefox Compile Benchmark · Issue #69 · STH-Dev/linux-bench · GitHub
Additional compile benchmarks · Issue #70 · STH-Dev/linux-bench · GitHub
@IanCutress and @Chuckleb I was also thinking we do compile UnixBench 5.1.3. Not a well-known application but we are testing gcc speed.
SecondLife - not a huge fan of using this.
VLC maybe.
Cromium maybe.
If we do need to host a copy can do that.
SecondLife - not a huge fan of using this.
VLC maybe.
Cromium maybe.
If we do need to host a copy can do that.
You probably also need a static copy of your compiler and toolchain; there can be substantial performance changes between GCC versions.
This begs the question of how repeatable and exact it has to be. For my work, we build down to the compiler and libraries level, rarely pulling from any repos, hosting our own copy of source. Docker containers (or any image) is better since we lock down the components.
I think I will fork a test script for compiler times so we can see how long it takes. It'll be fun.
I think I will fork a test script for compiler times so we can see how long it takes. It'll be fun.
This would actually be a perfect application for Docker. Create a container with a specific version of your compiler and source code, and have it run the benchmark when it's started. Outputting timing to the logs. Simple to reproduce, and almost no moving parts.
@Patrick yep, that's what I was thinking. This is how we should dev it in the future since you may not be ready to take results, etc.
@Scott Laird. Yep, I built up the Firefox compile as a docker already. It's the only way I can keep clean dev environments and ensure that I get all the deps tracked. Then that gets transferred into main script.
@Scott Laird. Yep, I built up the Firefox compile as a docker already. It's the only way I can keep clean dev environments and ensure that I get all the deps tracked. Then that gets transferred into main script.
@IanCutress - I just wrapped up a simple test script, can you test it for me?
You should be able to run it after pulling it:
I didn't get to test it yet, but should be good to run. It will prompt for how many threads you want to run it with, I didn't add in the autodetect cores code. I have it running out of /dev/shm and the code consumes about 6GB RAM so you'll need 2xRAM due to the fact that /dev/shm is usually 50% RAM. So you'll need 12GB minimum.
Tell me if it fails and I'll fix. I don't have my dev station working yet so can't fully test., but built it out of my docker dev env.
Code:
wget https://raw.githubusercontent.com/chuckleb/linuxbench-dev/master/firefox-compile.sh
Code:
bash firefox-compile.shTell me if it fails and I'll fix. I don't have my dev station working yet so can't fully test., but built it out of my docker dev env.
Fails on same place.
So this is the script determining if gcc works. On a base LiveCD install, it seems it does not...?
Code:
configure: error: installation or configuration problem: c++ compiler cannot create executables
From the live cd I did
and got
Code:
wget https://raw.githubusercontent.com/chuckleb/linuxbench-dev/master/firefox-compile.sh
chmod +x firefox-compile.sh
./firefox-compile.sh
Number of cores: 8
Code:
configure: error: Library requirements (gtk+-2.0 >= 2.18.0 gtk+-unix-print-2.0 glib-2.0 >= 2.22 gobject-2.0 gdk-x11-2.0) not met; consider adjusting the PKG_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix so pkg-config can find them.
make: *** No targets specified and no makefile found. Stop.