Hello. I'm new to the forums here, and I like the Linux-Bench, especially as it's becoming a standard across more and more review sites. I was running the bench on an ARM SBC (Banana Pi) and noticed that the start-up portion of the script checks for and installs GCC-4.6 as the default compiler. That version of GCC is pretty poorly optimized for ARM, with some programs showing a 30-50% speed improvement just changing to GCC-4.7. Are there plans to update things for Linux-Bench 2015, or are you keeping things as-is for comparison sake?
ARM benchmarks
- Thread starter mojoxer
- Start date
Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
We do have the ability to version Linux-Bench so we can do this. The next version of the web results viewer (@Chuckleb saw a few iterations ago) will be MUCH better and we can filter based on script version.
Just wondering, did it complete the run?
Actually, it finally did, although it took more than 10 hours!
Results are at ref link 19231422052939.
Fun... Well I haven't been contributing to Linux-Bench in a while... but now I have a Banana pi incoming... So there is that.
They are ~$40 on amazon btw... @Chuckleb I can punch a hole through my router once I get it setup so we can share it if you don't want to grab your own.
I think @Chuckleb has an arm based mini box already Happy to sponsor.
BTW - I think two of those at least are issues with some of the binaries. ARM is going to be an issue for those tests. We may have to pre-compile and have an alternate download logic. I am worried we will have to pre-compile a bunch of different versions on the ARM side.
@mojoxer Thank you for the result and feedback. When I did the benchmarking on the Raspberry Pi it took an even longer time.
Happy to sponsor.BTW - I think two of those at least are issues with some of the binaries. ARM is going to be an issue for those tests. We may have to pre-compile and have an alternate download logic. I am worried we will have to pre-compile a bunch of different versions on the ARM side.
@mojoxer Thank you for the result and feedback. When I did the benchmarking on the Raspberry Pi it took an even longer time.
Ok, starting to play with things to see if I can troubleshoot. I've got a TBS2910 Matrix that I'm testing on, results are at 34241422115834. I've made new issues in the github page to track (Test out GCC 4.7 for ARM processors · Issue #56 · STH-Dev/linux-bench · GitHub
There aren't terribly many ARM64 dev boards available that don't cost an arm and a leg... no pun intended.
@mojoxer
I've grabbed the first ARM chip I had handy (my Raspberry Pi) and ran some tests. I definitely see speed improvements in the C-Ray and the Redis tests, the other ones do not change much. Plus it allowed me to find the programs that are failing (Stream, NAMD, NPB) and I've made jobs for them.
I'll grab faster hardware and be able to develop faster. The 30-36 hour RPi runs are rather slow.
GCC 4.6 = 43261422251323
GCC 4.8 = 00281422416340
I'll have to do some detection and whatnot to properly load the correct libs, this all depends on the upstream providers as well.
On a related note, it made me check my other installs. From my Docker build test environment, it looks like Ubuntu 14.04 defaults to GCC 4.8 so that's good. CentOS is 4.4.7... sigh.
Thanks for bringing it up. We'll get changes into another version at some point in the future here, I have to fix bugs first.
I've grabbed the first ARM chip I had handy (my Raspberry Pi) and ran some tests. I definitely see speed improvements in the C-Ray and the Redis tests, the other ones do not change much. Plus it allowed me to find the programs that are failing (Stream, NAMD, NPB) and I've made jobs for them.
I'll grab faster hardware and be able to develop faster. The 30-36 hour RPi runs are rather slow.
GCC 4.6 = 43261422251323
GCC 4.8 = 00281422416340
I'll have to do some detection and whatnot to properly load the correct libs, this all depends on the upstream providers as well.
On a related note, it made me check my other installs. From my Docker build test environment, it looks like Ubuntu 14.04 defaults to GCC 4.8 so that's good. CentOS is 4.4.7... sigh.
Thanks for bringing it up. We'll get changes into another version at some point in the future here, I have to fix bugs first.
Thanks for looking into the issue(s)! Another quick observation: something is keeping UnixBench from recognizing the multiple cores in multi-core ARM chips. I've run the bench on a Banana Pi (2 Cores) and an ODROID-C1 (4 cores) but both times UnixBench only ran the single thread benchmarks. Looking at the logs:
SysInfo (lscpu) reports the correct # of cpus (cores/threads/sockets) in its test.
HardInfo doesn't report the number of cores/cpus.
UnixBench self-reports 0 (zero) CPUs and runs only the single-thread benchmark
C-Ray greps /proc/cpuinfo to find out how many threads to use
Stream greps /proc/cpuinfo
OpenSSL calls nproc
SysBench calls nproc - even through nproc returns the correct number of CPUs/cores, sysbench runs the multi-thread benchmark on one core
REDIS doesn't appear to care about threads/cores
NPB is non-functional, but greps /proc/cpuinfo for cpus/cores
NAMD is non-functional, but greps /proc/cpuinfo for cpus/cores
p7zip does not seem to care
Maybe there needs to be a standardized way of determining the core/cpu count and passing that to the benchmarks, to ensure that all cores are used on the multi-core benchmarks.
SysInfo (lscpu) reports the correct # of cpus (cores/threads/sockets) in its test.
HardInfo doesn't report the number of cores/cpus.
UnixBench self-reports 0 (zero) CPUs and runs only the single-thread benchmark
C-Ray greps /proc/cpuinfo to find out how many threads to use
Stream greps /proc/cpuinfo
OpenSSL calls nproc
SysBench calls nproc - even through nproc returns the correct number of CPUs/cores, sysbench runs the multi-thread benchmark on one core
REDIS doesn't appear to care about threads/cores
NPB is non-functional, but greps /proc/cpuinfo for cpus/cores
NAMD is non-functional, but greps /proc/cpuinfo for cpus/cores
p7zip does not seem to care
Maybe there needs to be a standardized way of determining the core/cpu count and passing that to the benchmarks, to ensure that all cores are used on the multi-core benchmarks.
@Chuckleb I think we need ARM binaries :-/ I wonder if they will work cross platform if we pre-compile them.
You should log this at the GitHub page if you can, then you can get updated when I did them.
This was originally developed for x86 so done of the core detection routines are Intel and AMD focused (they both report cores and threads differently). I'll have to spend some time to look at how cores are counted in ARM, especially with the octo-core chips that are really two dual core of different types... I can imagine that being harder to understand.
Great feedback, it adds to the next round of test script... Maybe need to fork an ARM-specific test script as some of the tests are not as relevant. At least detect ARM and adjust accordingly. For example, NPB is using a really large case that would never run on an ARM...
Lastly, there are quite a few more distros on ARM then I was expecting and the ports collection for each. Will have to think how to do this.
This was originally developed for x86 so done of the core detection routines are Intel and AMD focused (they both report cores and threads differently). I'll have to spend some time to look at how cores are counted in ARM, especially with the octo-core chips that are really two dual core of different types... I can imagine that being harder to understand.
Great feedback, it adds to the next round of test script... Maybe need to fork an ARM-specific test script as some of the tests are not as relevant. At least detect ARM and adjust accordingly. For example, NPB is using a really large case that would never run on an ARM...
Lastly, there are quite a few more distros on ARM then I was expecting and the ports collection for each. Will have to think how to do this.
I think I am subscribed. About 1000 non-junk e-mails a day at this point so stuff does go unread.
I am 100% with you. It is actually an awesome experience though in the idea that Intel to ARM in Linux is not an issue at all. I have heard several people tell me "just compile and everything will work the same on ARM"
@mojoxer New version posted, give it a shot. I tested against oDroid and RPi (crashes Rpi at times, need to fix). I've standardized the proc detection into one area and aliased all the calls into it. Can be cleaned up later. I added rudimentary processor detection so that ARM and x86 are different. They each report cores differently. This will work for now and can be made more robust later.
@Patrick - you have version numbers now to add to parser. Also ARM or x86 is listed as well.
Feel free to post bugs in the github.
@Patrick - you have version numbers now to add to parser. Also ARM or x86 is listed as well.
Feel free to post bugs in the github.
I have quite a few ARM devices: ARM Cortex A7, A9, quad core, dual core, from different manufacturers. I began using Linux-Bench and got some results. I see the beta linux-bench site has a way to compare results, is there a way to tag the results correctly and upload them?