Introducing the STHbench.sh benchmark script

Discussion in 'Processors and Motherboards' started by Patrick, Oct 6, 2013.

  1. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    As many readers on this site know, this site has always focused on creating benchmarks that could be run on any system without the need for proprietary software. This year we took the major step of moving all server benchmarks over to Linux from Windows after three years of using Windows as our benchmark reference point.

    The other main goal was to make this as simple as possible to run for users. For example, following 30 different pages worth of instructions to collect different pieces of information seems not so user friendly. With the current benchmark, one can do a clean Ubuntu Server installation (12.04 or higher) and the script will update the system, install dependencies, download benchmarks, compile files as needed and then run the benchmarks.

    Current Release
    Currently the tests being run are:


    • Hardinfo
    • UnixBench 5.1.3 Single and Multi-threaded
    • c-ray 1.1 low and higher detail renders
    • crafty chess benchmark
    • Phoronix Test Suite
      • STREAM
      • 7-Zip Compression
      • OpenSSL
      • PyBench

    To run the current release:
    Code:
    wget http://forums.servethehome.com/pjk/STHbench.sh
    chmod +x STHbench.sh
    ./STHbench.sh
    After running this benchmark there will be a file named "STHbench.sh.log" which will have detailed information about the tests run.

    Current Development Version
    The current benchmark is what we have been using this year. Instead of letting that be, we did want to add additional tests to the suite. As a result, we do have a development instance.

    Dev002

    • Adds sysbench to the suite
    • Runs sysbench CPU test
    • Runs two redis-benchmark tests
    • Removes apt-get spam from logs (this generates well over 1,000 lines of text)
    • Make a version that can work on CentOS also

    Dev003

    • Fixes OS detection for Ubuntu (even development/ daily builds)

    Dev004

    • Fixes redis-benchmark issue under some OSes and adds a 6379.conf file for the benchmark

    Dev005
    • Debian install and redis fixes

    Dev006
    • Installer fixes
    • Shutdown redis-server after benchmarking is complete

    Dev007

    • Added check to ensure script is run as root/ SU
    • Added removal code for benchmarks
    • Updated Debian installation fixes

    Dev008

    • Added STREAM benchmark (non-PTS)
    • Added OpenSSL benchmark (non-PTS)
    • Added crafty benchmark back (from source)
    • Added lscpu for CPU information
    • Lowered the prime problem size for sysbench
    • Added unzip

    Dev009

    • Updated STREAM benchmark (non-PTS)
    • Added NPB benchmark

    To run (note this may not work!!!!):
    Code:
    wget http://forums.servethehome.com/pjk/STHbench-Dev009.sh
    chmod +x STHbench-Dev009.sh
    sudo ./STHbench-Dev009.sh
    Thanks to nitrobass24, mir and Chuckleb for making MAJOR updates to the script starting with Dev002, including heavy lifting getting it to work under both Ubuntu or CentOS!

    Current Enhancement List
    • Add Apache / nginx benchmarks
    • Add memcached benchmarks
    • Build script parser to extract relevant information from the log file
    • Make into LiveCD for and easy to run benchmark
    • Make tests switched using CLI so you can run a single test at a time
    • Make an easy way to run the script multiple times
    • Add x264 video encoding (see here)
    • Upload script data to web application for online result viewer functionality

    How you can help
    First, please test this out on your own systems. Feel free to review the script's code prior to running. We intentionally made the code easy to read so that you can make an informed decision to proceed.

    Great points of feedback can be:
    1. "This works/ does not work" (and here is the log file)
    2. "I would like to see benchmark XYZ and here is what should be added to the script for that to happen."
    3. "The code is ugly, here is a fixed version."
    4. "Can I help with building the parsing or web viewer piece?"
    5. "Hey here's an idea..."

    I am certainly not a good coder by any means so help and feedback is always appreciated.
     
    #1
  2. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,463
    Likes Received:
    214
    I will run this on a server tomorrow. Here's another idea - in memory Hadoop benchmark. I don't know how to set this type of thing up though.
     
    #2
  3. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    I went ahead and modified the current script to run on CentOS as well.
    use link in first post

    1) only the update/install process needed to be changed for CentOS.
    2) Crafty does not exists for RHEL/CentOS....could try and compile it from source, but would make the script take a lot longer. Maybe we should consider something else for a CPU bench that is more universal.

    Later this week I will post an update that allows us to maintain one script/document and have the script auto-detect OS.
     
    #3
    Last edited: Oct 7, 2013
  4. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    Couldn't Sleep...
    use link in first post

    Added code to auto-detect Distro
    Created functions for Update Processes
    Fixed some dependency issues on Centos (I am using min server install)
     
    #4
    Last edited: Oct 7, 2013
  5. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,927
    Likes Received:
    854
    Here's to insomnia. Which version should I use? Is this the new official or dev version? I'm guessing dev version right?
     
    #5
  6. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    Yea mine is the new dev...i guess i could have labeled it as such.
     
    #6
  7. 33_viper_33

    33_viper_33 Member

    Joined:
    Aug 3, 2013
    Messages:
    195
    Likes Received:
    2
    your loss is our gain;)
     
    #7
  8. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    Just realized I missed the Redis and sysbench stuff in my version...I have posted a new copy to the same link above.

    Will someone else run on Ubuntu/Debian and/or Centos/RHEL and report back if there are any errors in my OS detection and Install/update logic?
     
    #8
    Last edited: Oct 7, 2013
  9. sboesch

    sboesch Member

    Joined:
    Aug 3, 2012
    Messages:
    358
    Likes Received:
    20
    I ran the current release script on my Linux Mint 15 install, worked great.
    I will test nitrobass24's here in a bit.
     
    #9
  10. sboesch

    sboesch Member

    Joined:
    Aug 3, 2012
    Messages:
    358
    Likes Received:
    20
    nitrobass24's script worked on my Mint install as well.
     
    #10
  11. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,927
    Likes Received:
    854
    Gosh I was testing the first NB update script then saw the update. Patrick please update?
     
    #11
  12. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    Updated the first post. Nitrobass24 - the current Dev002 is your latest revision.

    Super cool! Running on a few systems overnight.
     
    #12
  13. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,927
    Likes Received:
    854
    When I re-run the Dev002 benchmark I'm getting:

    Code:
    Hint: To run 'make test' is a good idea ;)
    
    make[1]: Leaving directory `/home/miniknight/redis-stable/src'
    [sudo] password for miniknight:
    mkdir: cannot create directory â/etc/redisâ: File exists
    mkdir: cannot create directory â/var/redisâ: File exists
    ./STHbench-Dev002.sh: line 127: /redis-benchmark: No such file or directory
    ./STHbench-Dev002.sh: line 140: unexpected EOF while looking for matching `)'
    ./STHbench-Dev002.sh: line 151: syntax error: unexpected end of file
    
     
    #13
  14. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    I got your e-mail on this. Fixed 2 things but did not rev the script.

    • Added the -p switch to the mkdir function
    • Added a "." that was missing and that caused your line 127 and beyond errors

    Re-running on used machines now. Tried with only a rm -R * to see if it runs properly on a "non clean" install.
     
    #14
  15. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    Made two more fixes in Dev002. The big one being that we were missing a ")" on the redis benchmark.

    One of the two machines was not running the hardinfo benchmark. Need to troubleshoot why soon.

    Re-running the script on two more machines.
     
    #15
  16. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    #16
  17. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    Hmmm....I see that it detected it, but obviously did not run the install/update process.

    Does it work some of the time?
    How are you installing Ubuntu?
    Link to ISO you downloaded?
    Package selection: Min Server, Min Desktop, etc.?

    I would like to try and recreate the same OS install in a VM so i can test it.
     
    #17
  18. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    It was from: Ubuntu Server 13.10 (Saucy Salamander) Daily Build

    The server amd64 ISO
     
    #18
  19. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    O Come on! - You cant roll Alpha/Beta OS on me! :)

    JK - Let me see what I can do
     
    #19
  20. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,497
    Likes Received:
    4,440
    Give it a few days for 13.10 to come out :) Seriously though, since I have been getting a lot of "rarer" hardware, it helps to have a single-step OS installation process.

    Also - please do not underestimate how long it takes to install Ubuntu with an Atom S1260. As one may have seen in the main site post yesterday, the C2750 feels like a normal processor while the S1260 is rather slow.
     
    #20

Share This Page