Introducing the STHbench.sh benchmark script

Patrick

Administrator
Staff member
Dec 21, 2010
11,938
4,892
113
As many readers on this site know, this site has always focused on creating benchmarks that could be run on any system without the need for proprietary software. This year we took the major step of moving all server benchmarks over to Linux from Windows after three years of using Windows as our benchmark reference point.

The other main goal was to make this as simple as possible to run for users. For example, following 30 different pages worth of instructions to collect different pieces of information seems not so user friendly. With the current benchmark, one can do a clean Ubuntu Server installation (12.04 or higher) and the script will update the system, install dependencies, download benchmarks, compile files as needed and then run the benchmarks.

Current Release
Currently the tests being run are:


  • Hardinfo
  • UnixBench 5.1.3 Single and Multi-threaded
  • c-ray 1.1 low and higher detail renders
  • crafty chess benchmark
  • Phoronix Test Suite
    • STREAM
    • 7-Zip Compression
    • OpenSSL
    • PyBench

To run the current release:
Code:
wget http://forums.servethehome.com/pjk/STHbench.sh
chmod +x STHbench.sh
./STHbench.sh
After running this benchmark there will be a file named "STHbench.sh.log" which will have detailed information about the tests run.

Current Development Version
The current benchmark is what we have been using this year. Instead of letting that be, we did want to add additional tests to the suite. As a result, we do have a development instance.

Dev002

  • Adds sysbench to the suite
  • Runs sysbench CPU test
  • Runs two redis-benchmark tests
  • Removes apt-get spam from logs (this generates well over 1,000 lines of text)
  • Make a version that can work on CentOS also

Dev003

  • Fixes OS detection for Ubuntu (even development/ daily builds)

Dev004

  • Fixes redis-benchmark issue under some OSes and adds a 6379.conf file for the benchmark

Dev005
  • Debian install and redis fixes

Dev006
  • Installer fixes
  • Shutdown redis-server after benchmarking is complete

Dev007

  • Added check to ensure script is run as root/ SU
  • Added removal code for benchmarks
  • Updated Debian installation fixes

Dev008

  • Added STREAM benchmark (non-PTS)
  • Added OpenSSL benchmark (non-PTS)
  • Added crafty benchmark back (from source)
  • Added lscpu for CPU information
  • Lowered the prime problem size for sysbench
  • Added unzip

Dev009

  • Updated STREAM benchmark (non-PTS)
  • Added NPB benchmark

To run (note this may not work!!!!):
Code:
wget http://forums.servethehome.com/pjk/STHbench-Dev009.sh
chmod +x STHbench-Dev009.sh
sudo ./STHbench-Dev009.sh
Thanks to nitrobass24, mir and Chuckleb for making MAJOR updates to the script starting with Dev002, including heavy lifting getting it to work under both Ubuntu or CentOS!

Current Enhancement List
  • Add Apache / nginx benchmarks
  • Add memcached benchmarks
  • Build script parser to extract relevant information from the log file
  • Make into LiveCD for and easy to run benchmark
  • Make tests switched using CLI so you can run a single test at a time
  • Make an easy way to run the script multiple times
  • Add x264 video encoding (see here)
  • Upload script data to web application for online result viewer functionality

How you can help
First, please test this out on your own systems. Feel free to review the script's code prior to running. We intentionally made the code easy to read so that you can make an informed decision to proceed.

Great points of feedback can be:
1. "This works/ does not work" (and here is the log file)
2. "I would like to see benchmark XYZ and here is what should be added to the script for that to happen."
3. "The code is ugly, here is a fixed version."
4. "Can I help with building the parsing or web viewer piece?"
5. "Hey here's an idea..."

I am certainly not a good coder by any means so help and feedback is always appreciated.
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,487
223
63
I will run this on a server tomorrow. Here's another idea - in memory Hadoop benchmark. I don't know how to set this type of thing up though.
 

nitrobass24

Moderator
Dec 26, 2010
1,083
127
63
TX
I went ahead and modified the current script to run on CentOS as well.
use link in first post

1) only the update/install process needed to be changed for CentOS.
2) Crafty does not exists for RHEL/CentOS....could try and compile it from source, but would make the script take a lot longer. Maybe we should consider something else for a CPU bench that is more universal.

Later this week I will post an update that allows us to maintain one script/document and have the script auto-detect OS.
 
Last edited:

nitrobass24

Moderator
Dec 26, 2010
1,083
127
63
TX
Couldn't Sleep...
use link in first post

Added code to auto-detect Distro
Created functions for Update Processes
Fixed some dependency issues on Centos (I am using min server install)
 
Last edited:

nitrobass24

Moderator
Dec 26, 2010
1,083
127
63
TX
Just realized I missed the Redis and sysbench stuff in my version...I have posted a new copy to the same link above.

Will someone else run on Ubuntu/Debian and/or Centos/RHEL and report back if there are any errors in my OS detection and Install/update logic?
 
Last edited:

sboesch

Active Member
Aug 3, 2012
405
37
28
Columbus, OH
I ran the current release script on my Linux Mint 15 install, worked great.
I will test nitrobass24's here in a bit.
 

MiniKnight

Well-Known Member
Mar 30, 2012
2,987
892
113
NYC
Gosh I was testing the first NB update script then saw the update. Patrick please update?
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,938
4,892
113
Updated the first post. Nitrobass24 - the current Dev002 is your latest revision.

Super cool! Running on a few systems overnight.
 

MiniKnight

Well-Known Member
Mar 30, 2012
2,987
892
113
NYC
When I re-run the Dev002 benchmark I'm getting:

Code:
Hint: To run 'make test' is a good idea ;)

make[1]: Leaving directory `/home/miniknight/redis-stable/src'
[sudo] password for miniknight:
mkdir: cannot create directory â/etc/redisâ: File exists
mkdir: cannot create directory â/var/redisâ: File exists
./STHbench-Dev002.sh: line 127: /redis-benchmark: No such file or directory
./STHbench-Dev002.sh: line 140: unexpected EOF while looking for matching `)'
./STHbench-Dev002.sh: line 151: syntax error: unexpected end of file
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,938
4,892
113
I got your e-mail on this. Fixed 2 things but did not rev the script.

  • Added the -p switch to the mkdir function
  • Added a "." that was missing and that caused your line 127 and beyond errors

Re-running on used machines now. Tried with only a rm -R * to see if it runs properly on a "non clean" install.
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,938
4,892
113
Made two more fixes in Dev002. The big one being that we were missing a ")" on the redis benchmark.

One of the two machines was not running the hardinfo benchmark. Need to troubleshoot why soon.

Re-running the script on two more machines.
 

nitrobass24

Moderator
Dec 26, 2010
1,083
127
63
TX
Hey nitrobass24

I just tried this on the Atom S1260 clean 13.10 installation. It seems like it is not detecting even though there is /etc/debian_version file.

http://forums.servethehome.com/pjk/STHbench-Dev002.sh.log

Seeing the same thing on a few other VMs
Hmmm....I see that it detected it, but obviously did not run the install/update process.

Does it work some of the time?
How are you installing Ubuntu?
Link to ISO you downloaded?
Package selection: Min Server, Min Desktop, etc.?

I would like to try and recreate the same OS install in a VM so i can test it.
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,938
4,892
113
Hmmm....I see that it detected it, but obviously did not run the install/update process.

Does it work some of the time?
How are you installing Ubuntu?
Link to ISO you downloaded?
Package selection: Min Server, Min Desktop, etc.?

I would like to try and recreate the same OS install in a VM so i can test it.
It was from: Ubuntu Server 13.10 (Saucy Salamander) Daily Build

The server amd64 ISO
 

Patrick

Administrator
Staff member
Dec 21, 2010
11,938
4,892
113
O Come on! - You cant roll Alpha/Beta OS on me! :)

JK - Let me see what I can do
Give it a few days for 13.10 to come out :) Seriously though, since I have been getting a lot of "rarer" hardware, it helps to have a single-step OS installation process.

Also - please do not underestimate how long it takes to install Ubuntu with an Atom S1260. As one may have seen in the main site post yesterday, the C2750 feels like a normal processor while the S1260 is rather slow.