Introducing the STHbench.sh benchmark script

Discussion in 'Processors and Motherboards' started by Patrick, Oct 6, 2013.

  1. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,466
    Likes Received:
    216
    Here is my redis output where I truncated a bunch of the errors:
    Code:
    Starting Redis server...
    Writing to socket: Connection refused
    Writing to socket: Connection refused
    .....
    Writing to socket: Connection refused
    Writing to socket: Connection refused
    SET: 290782.22 requests per second
    GET: 327118.09 requests per second
    
    1 18793
    5 23121
    10 24143
    20 24266
    30 24963
    40 24894
    50 25006
    60 25069
    70 25000
    80 25157
    90 25063
    100 25272
     
    #61
  2. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    Jegss - A few questions so i can try to replicate this.

    What OS/Version?
    User Account Type - Standard or Root?
    Are you running the script as Sudo?
     
    #62
  3. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    So this code is in the Dev05 build, but its after the apt-get cmd, so its basically useless. I am moving it up in 006.

    Might be, but im unsure what other options are better suited.
     
    #63
  4. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Ok not going to mess with this if you are nitrobass. Maybe the answer is to compile and run stream, pybench, openssl bench and 7-Zip benchmarks instead of using PTS. Will look into it.
     
    #64
  5. nitrobass24

    nitrobass24 Moderator

    Joined:
    Dec 26, 2010
    Messages:
    1,081
    Likes Received:
    125
    Pat - Looking at your memcached comments in the file. What do you think about memtier_bench instead? memtier_benchmark: A High-Throughput Benchmarking Tool for Redis & Memcached | Redis & Memcached

    Also Dev006 is in your inbox. I made the following changes:
    • Removed all "sudo" commands - Running the whole script as sudo/root prevents the script from requiring re-auth/stalling out on slower systems as the whole script will run under a single parent process as root
    • Added check for root privileges and user alerting
    • Updated Debian Install process based on feedback from Mir
    • Added 'apt-get install -f' based on errors i saw on Ubuntu 12.04.03
    • Added Multi-threading to sysbench
    • Removed verbose flag from Tar commands
    • Fixed Crafty not found bug
    • Cleaned up some previously commented out code
     
    #65
  6. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Ha! Wow thanks nitrobass24! I saw your e-mail and was going to ask what is in this version. Will post it in two hours or so when I am at the main workstation.

    Memtier_bench could be really cool! I have a feeling that is going to require a lot to get setup though.
     
    #66
  7. mir

    mir New Member

    Joined:
    Dec 21, 2013
    Messages:
    9
    Likes Received:
    0
    Since some of the software is installed outside the package repository using make install would it not be an idea to remove that software before the script terminates?
     
    #67
  8. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    It would be awesome to be able to do that. Right now the focus is more on getting a set of benchmarks completed, then really uninstalling them. Certainly see the wisdom in removal though.

    Will post Dev006 as soon as I can if you wanted to take a look.
     
    #68
  9. mir

    mir New Member

    Joined:
    Dec 21, 2013
    Messages:
    9
    Likes Received:
    0
    Since Dev06 is running as user root through sudo the software will get installed on the host and stay there. Running a benchmark should not leave executable files unknown to the user running the script.
    I will take a look. Shouldn't be to difficult to make uninstall - use ./configure --prefix=show_odd_folder && make install && study what gets installed && add rm -f files_found_discovered_in_odd_folder
     
    #69
  10. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Thanks nitrobass24 and mir:

    Dev006 posted: http://forums.servethehome.com/pjk/STHbench-Dev006.sh
     
    #70
  11. mir

    mir New Member

    Joined:
    Dec 21, 2013
    Messages:
    9
    Likes Received:
    0
    Is this the crafty the scripts wants to install: crafty - state-of-the-art chess engine, compatible with xboard ?
    If so what is the purpose of installing this?
     
    #71
  12. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    mir - great question. Crafty is the chess engine. I think it is also compatible with xboard (Crafty - XBoard). We started this as Ubuntu only so we were using the repo version rather than download/ make.
     
    #72
  13. MiniKnight

    MiniKnight Well-Known Member

    Joined:
    Mar 30, 2012
    Messages:
    2,939
    Likes Received:
    857
    IMO here is what you guys should focus on:

    Benchmark updates
    - Get rid of crafty and PTS installs - just compile these manually
    - Do the memtier that nitrobass was talking about

    I still think STHbench needs some sort of normal web server nginx or apache benchmark with php. Maybe a MySQL/ MariaDB one also. I know these are hard to size if you are doing Atom and ARM up to multi processor

    Script updates
    - Uninstall lines
    - Option to run individual tests or sets of tests
    - Option to run script multiple times

    One that is done, you need to start parsing and getting to an online results viewer.
     
    #73
  14. mir

    mir New Member

    Joined:
    Dec 21, 2013
    Messages:
    9
    Likes Received:
    0
    Here is a diff containing changes which includes removing what gets installed as well as some fixes for minor bugs.
    Code:
    --- STHbench-Dev006.sh	2013-12-24 14:23:29.982863453 +0100
    +++ STHbench-Dev006.sh.new	2013-12-25 13:25:55.917188997 +0100
    @@ -11,13 +11,19 @@
     
     #############Set Functions################
     
    +NEED_PTS=1
    +
     # Update and install required packages (Debian)
     Update_Install_Debian()
     {
     apt-get -y update && apt-get -y upgrade && apt-get install -f
    -apt-get -y install build-essential libx11-dev libglu-dev hardinfo crafty sysbench expect make gcc php5-curl php5-common php5-cli php5-gd libfpdi-php
    +apt-get -y install build-essential libx11-dev libglu-dev hardinfo sysbench expect php5-curl php5-common php5-cli php5-gd libfpdi-php
     apt-get install -f
    -wget -N http://phoronix-test-suite.com/releases/repo/pts.debian/files/phoronix-test-suite_4.8.6_all.deb && dpkg -i phoronix-test-suite_4.8.6_all.deb
    +dpkg -s phoronix-test-suite 2>&1 > /dev/null 2>&1
    +NEED_PTS=$(echo $?)
    +if [ $NEED_PTS > 0 ]; then
    +    wget -N http://phoronix-test-suite.com/releases/repo/pts.debian/files/phoronix-test-suite_4.8.6_all.deb && dpkg -i phoronix-test-suite_4.8.6_all.deb
    +fi
     }
     # Update and install required packages (CentOS/RHEL)
     Update_Install_RHEL()
    @@ -96,11 +102,6 @@
     # C-Ray 1.1
     wget -N http://www.futuretech.blinkenlights.nl/depot/c-ray-1.1.tar.gz && tar -zxf c-ray-1.1.tar.gz && cd c-ray-1.1 && make && cat scene | ./c-ray-mt -t 32 -s 7500x3500 > foo.ppm | tee c-ray1.txt && cat sphfract | ./c-ray-mt -t 32 -s 1920x1200 -r 8 > foo.ppm && cd ..
     
    -# Crafty
    -if [[ $DIST = "Debian" ]]; then
    -/usr/games/crafty 'log off' bench end
    -fi
    -
     # Phoroncrix Test Suite
     expect <<EOD
     set timeout -1
    @@ -133,9 +134,6 @@
     cp utils/redis_init_script /etc/init.d/redis_6379
     [[ -d /var/redis ]] || ( mkdir /var/redis && mkdir /var/redis/6379 )
     
    -cd src
    -cp redis-server redis-cli redis-benchmark /usr/local/bin/
    -
     service redis_6379 start
     
     # Original redis benchmark set/ get test
    @@ -165,4 +163,30 @@
     
     redis-cli shutdown
     
    +cd ..
    +
    +# Remove Redis
    +rm -rf /etc/redis
    +rm -f /etc/init.d/redis_6379
    +rm -rf /var/redis
    +rm -f /usr/local/bin/redis-benchmark
    +rm -f /usr/local/bin/redis-check-aof
    +rm -f /usr/local/bin/redis-check-dump
    +rm -f /usr/local/bin/redis-cli
    +rm -f /usr/local/bin/redis-server
    +rm -f redis-stable.tar.gz
    +rm -rf redis-stable
    +
    +# Remove c-ray
    +rm -f c-ray-1.1.tar.gz
    +rm -rf c-ray-1.1
    +
    +# Remove UnixBench5.1.3.tgz
    +rm -f UnixBench5.1.3.tgz
    +rm -rf UnixBench
    +
    +# Remove Phoronix test suite
    +[ "$NEED_PTS" > "0" -a "$DIST" = "Debian" ] && apt-get -y --purge remove phoronix-test-suite && rm -f phoronix-test-suite_4.8.6_all.deb
    +
     cd ~/
    +
    
     
    #74
    Last edited: Dec 25, 2013
  15. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Thanks Mir! May be a day or two before I can update due to the holidays but thank you so much for your help on this.
     
    #75
  16. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,466
    Likes Received:
    216
    mir can you post your new version? I have some time this weekend so I can give it a go.
     
    #76
  17. mir

    mir New Member

    Joined:
    Dec 21, 2013
    Messages:
    9
    Likes Received:
    0
    Here it is
    Code:
    #!/bin/bash
    
    # (c) 2013 ServeTheHome.com and ServeThe.biz
    
    # Verify if the script is executed with Root Privileges #
    ROOT=`/usr/bin/id -u`
    if [[ $ROOT -ne "0" ]]; then
    echo "Please execute the script with root privileges"
    exit 1
    fi
    
    #############Set Functions################
    
    NEED_PTS=1
    
    # Update and install required packages (Debian)
    Update_Install_Debian()
    {
    apt-get -y update && apt-get -y upgrade && apt-get install -f
    apt-get -y install build-essential libx11-dev libglu-dev hardinfo sysbench expect php5-curl php5-common php5-cli php5-gd libfpdi-php
    apt-get install -f
    dpkg -s phoronix-test-suite 2>&1 > /dev/null 2>&1
    NEED_PTS=$(echo $?)
    if [ $NEED_PTS > 0 ]; then
        wget -N http://phoronix-test-suite.com/releases/repo/pts.debian/files/phoronix-test-suite_4.8.6_all.deb && dpkg -i phoronix-test-suite_4.8.6_all.deb
    fi
    }
    # Update and install required packages (CentOS/RHEL)
    Update_Install_RHEL()
    {
    rpm -Uhv http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
    rpm -Uhv http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
    yum -y update && yum -y upgrade
    yum -y groupinstall "Development Tools" && yum -y install wget sysbench libX11 perl-Time-HiRes mesa-libGLU hardinfo phoronix-test-suite expect php-common glibc.i686
    }
    
    #Determine OS
    # Detects which OS and if it is Linux then it will detect which Linux Distribution.
    OS=`uname -s`
    REV=`uname -r`
    MACH=`uname -m`
    
    if [ "${OS}" = "SunOS" ] ; then
    	OS=Solaris
    	DIST=Solaris
    	ARCH=`uname -p`	
    	OSSTR="${OS} ${REV}(${ARCH} `uname -v`)"
    elif [ "${OS}" = "AIX" ] ; then
    	DIST=AIX
    	OSSTR="${OS} `oslevel` (`oslevel -r`)"
    elif [ "${OS}" = "Linux" ] ; then
    	KERNEL=`uname -r`
    	if [ -f /etc/redhat-release ] ; then
    		DIST='RedHat'
    		PSUEDONAME=`cat /etc/redhat-release | sed s/.*\(// | sed s/\)//`
    		REV=`cat /etc/redhat-release | sed s/.*release\ // | sed s/\ .*//`
    	elif [ -f /etc/centos-release ] ; then
    		DIST='CentOS'
    		PSUEDONAME=`cat /etc/centos-release | sed s/.*\(// | sed s/\)//`
    		REV=`cat /etc/centos-release | sed s/.*release\ // | sed s/\ .*//`
    	elif [ -f /etc/SUSE-release ] ; then
    		DIST=`cat /etc/SUSE-release | tr "\n" ' '| sed s/VERSION.*//`
    		REV=`cat /etc/SUSE-release | tr "\n" ' ' | sed s/.*=\ //`
    	elif [ -f /etc/mandrake-release ] ; then
    		DIST='Mandrake'
    		PSUEDONAME=`cat /etc/mandrake-release | sed s/.*\(// | sed s/\)//`
    		REV=`cat /etc/mandrake-release | sed s/.*release\ // | sed s/\ .*//`
    	elif [ -f /etc/debian_version ] ; then
    		DIST="Debian"
    		PSUEDONAME=`cat /etc/debian_version`
    		REV=""
    	fi
    	if [ -f /etc/UnitedLinux-release ] ; then
    		DIST="${DIST}[`cat /etc/UnitedLinux-release | tr "\n" ' ' | sed s/VERSION.*//`]"
    	fi
    	OSSTR="${OS} ${DIST} ${REV}(${PSUEDONAME} ${KERNEL} ${MACH})"
    fi
    
    ##########Update and install required packages###########
    
    if [ "${DIST}" = "CentOS" ] ; then
    Update_Install_RHEL
    elif [ "${DIST}" = "RedHat" ] ; then
    Update_Install_RHEL
    elif [ "${DIST}" = "Debian" ] ; then
    Update_Install_Debian
    fi
    
    # Display script output and append to log
    exec > >(tee --append "$(basename $0)".log)
    exec 2>&1
    echo ${OSSTR}
    
    #############Run Benchmarks###############
    
    # HardInfo
    hardinfo --generate-report --report-format text
    
    # UnixBench 5.1.3
    wget -N https://byte-unixbench.googlecode.com/files/UnixBench5.1.3.tgz && tar -zxf UnixBench5.1.3.tgz && cd UnixBench && time make && wget -N http://forums.servethehome.com/pjk/fix-limitation.patch && patch Run fix-limitation.patch && ./Run dhry2reg whetstone-double syscall pipe context1 spawn execl shell1 shell8 shell16 && cd ..
    
    # C-Ray 1.1
    wget -N http://www.futuretech.blinkenlights.nl/depot/c-ray-1.1.tar.gz && tar -zxf c-ray-1.1.tar.gz && cd c-ray-1.1 && make && cat scene | ./c-ray-mt -t 32 -s 7500x3500 > foo.ppm | tee c-ray1.txt && cat sphfract | ./c-ray-mt -t 32 -s 1920x1200 -r 8 > foo.ppm && cd ..
    
    # Phoroncrix Test Suite
    expect <<EOD
    set timeout -1
    spawn -noecho phoronix-test-suite batch-setup
    expect {
       "Do you agree to these terms and wish to proceed (Y/n):" { send "y\n"; exp_continue }
       "Enable anonymous usage / statistics reporting (Y/n):" { send "n\n"; exp_continue }
       "Enable anonymous statistical reporting of installed software / hardware (Y/n):" { send "n\n"; exp_continue }
    
       "Run all test options (Y/n):" { send "y\n"; exp_continue }
       "Save test results when in batch mode (Y/n):" { send "n\n"; exp_continue }
    }
    EOD
    phoronix-test-suite batch-benchmark pts/stream pts/compress-7zip pts/openssl pts/pybench
    
    # sysbench CPU test prime
    echo "Running sysbench CPU Single Thread"
    sysbench --test=cpu --cpu-max-prime=1000000 run
    echo "Running sysbench CPU Multi-Threaded"
    nproc=`nproc`
    sysbench --num-threads=${nproc} --test=cpu --cpu-max-prime=1000000 run
    
    # redis Benchmark based on feedback. Next step is to add memchached as seen here: http://oldblog.antirez.com/post/redis-memcached-benchmark.html
    wget http://download.redis.io/redis-stable.tar.gz
    tar xzf redis-stable.tar.gz && cd redis-stable && make install
    
    wget http://forums.servethehome.com/pjk/6379.conf
    [[ -d /etc/redis ]] || ( mkdir /etc/redis && cp 6379.conf /etc/redis/6379.conf )
    
    cp utils/redis_init_script /etc/init.d/redis_6379
    [[ -d /var/redis ]] || ( mkdir /var/redis && mkdir /var/redis/6379 )
    
    service redis_6379 start
    
    # Original redis benchmark set/ get test
    
    redis-benchmark -n 1000000 -t set,get -P 32 -q -c 200
    
    BIN=redis-benchmark
    
    payload=32
    iterations=100000
    keyspace=100000
    
    for clients in 1 5 10 20 30 40 50 60 70 80 90 100
    do
        SPEED=0
        for dummy in 0 1 2
        do
            S=$($BIN -n $iterations -r $keyspace -d $payload -c $clients | grep 'per second' | tail -1 | awk '{print $1}')
    	VALUE=$(echo $S | awk '{printf "%.0f",$1}')
            if [ $(($VALUE > $SPEED)) != 0 ]
            then
                SPEED=$VALUE
            fi
        done
        echo "$clients $SPEED"
    done
    
    redis-cli shutdown
    
    cd ..
    
    # Remove Redis
    rm -rf /etc/redis
    rm -f /etc/init.d/redis_6379
    rm -rf /var/redis
    rm -f /usr/local/bin/redis-benchmark
    rm -f /usr/local/bin/redis-check-aof
    rm -f /usr/local/bin/redis-check-dump
    rm -f /usr/local/bin/redis-cli
    rm -f /usr/local/bin/redis-server
    rm -f redis-stable.tar.gz
    rm -rf redis-stable
    
    # Remove c-ray
    rm -f c-ray-1.1.tar.gz
    rm -rf c-ray-1.1
    
    # Remove UnixBench5.1.3.tgz
    rm -f UnixBench5.1.3.tgz
    rm -rf UnixBench
    
    # Remove Phoronix test suite
    [ "$NEED_PTS" > "0" -a "$DIST" = "Debian" ] && apt-get -y --purge remove phoronix-test-suite && rm -f phoronix-test-suite_4.8.6_all.deb
    
    cd ~/
    
     
    #77
  18. Patrick

    Patrick Administrator
    Staff Member

    Joined:
    Dec 21, 2010
    Messages:
    11,541
    Likes Received:
    4,464
    Uploaded and added Dev007 based on mir's updates above. Testing now.

    Super cool!
     
    #78
  19. TangoWhiskey9

    TangoWhiskey9 Active Member

    Joined:
    Jun 28, 2013
    Messages:
    402
    Likes Received:
    59
    Dev007 runs really slowly specifically on the prime single-threaded calculation. Took almost 2 hours one one of my lower clocked machines.
     
    #79
  20. Jeggs101

    Jeggs101 Well-Known Member

    Joined:
    Dec 29, 2010
    Messages:
    1,466
    Likes Received:
    216
    Patrick sent 13.10 server output.

    Bugs noted:
    - Crafty seems to be gone in that version.
    - pts is not working
     
    #80

Share This Page