Help Requested: Testing the STH Benchmark Script v1

[Patrick]

As many in the community know, I have wanted to make a super easy benchmark for STH readers to run on their machines so they can compare their results to the numbers we publish. Today we have a simple script with three easy commands to get

Prerequisite
A fresh 64-bit Ubuntu 64-bit server installation - preferably selecting the option to install OpenSSH server during the installation

The Script

The script assumes a perfectly new Ubuntu Server x64 installation. It runs updates for you so there is no need to do that manually. It also will install all benchmarks/ patches required. The net result will be an output log file.

How to run the script after installing Ubuntu
1. Login to the server
2. Copy/paste or type the following command:

wget http://forums.servethehome.com/pjk/STHbench-v1.sh
chmod +x STHbench-v1.sh
./STHbench-v1.sh

It will then ask for your sudo password. Feel free to look at what the script is doing to decide for yourself but the goal is that folks are installing this on a non-production machine and only using the image to benchmark "just in case".

The Ask

If you have a chance to give the script a spin and can post/ e-mail me the resulting log file that would be SUPER awesome!

 

[MiniKnight]

Nice1 much easier than your old method.

It worked fine for all but I got this output:

spawn phoronix-test-suite batch-setup

The following PHP extensions are REQUIRED by the Phoronix Test Suite:

JSON PHP JSON support is required for OpenBenchmarking.org communication.

The following PHP extensions are OPTIONAL but recommended:

PCNTL PHP PCNTL is highly recommended as it is required by some tests.
CURL PHP CURL is recommended for an enhanced download experience.
PHP FPDF PHP FPDF is recommended if wishing to generate PDF reports.


The following PHP extensions are REQUIRED by the Phoronix Test Suite:

JSON PHP JSON support is required for OpenBenchmarking.org communication.

The following PHP extensions are OPTIONAL but recommended:

PCNTL PHP PCNTL is highly recommended as it is required by some tests.
CURL PHP CURL is recommended for an enhanced download experience.
PHP FPDF PHP FPDF is recommended if wishing to generate PDF reports.

 

[Patrick]

Is that all the output you received or was there other information in the log?

 

[MiniKnight]

Oh no it ran hardinfo, UnixBench, crafty, c-ray without any issue and updated everything. Really took only three commands.

 

[Mike]

Would be fun to see this evolve into something that would do cross-distro and kernel checks too. Perhaps with more real-life benchmarks like encoding and webserver requests/s. I believe Phoronix does this from time to time?

 

[wrigleyvillain]

Is this meant for bare metal installs of Ubuntu or does it not matter?

 

[Patrick]

Just did in a VM.

Almost have the pts issue fixed.

Mike - I do want to add web server benchmarks. That is next in v2. Instead of encoding, I also really want a SQL benchmark. Suggestions appreciated.

 

[nitrobass24]

Pat - looks awesome, will try this out this week and report back.

Do you have a future roadmap for this? Some cool things i can think of off the top of my head.

LiveCD/USB version
Add hwinfo and pull Hardware specs and present as part of the report
Option to upload to STH? Think a web API to push a CSV version of the report to a STH database. Could be a really cool way to get realworld results outside of the lab.

 

[Patrick]

Ok script updated. The pts tests work now!

Potential enhancement log:

• LiveCD/USB version - good idea but this is going to take much more bandwidth
• hwinfo -> very interesting idea - you get some of this in the tests (you can see in the log file that you get CPU, memory, network adapter, OS, host bridge, video adapter, disk and controller information already with hardinfo)
• I like the option to upload to STH. That was on the future roadmap
• redis-benchmark I think that may be a good one to add
• I would LOVE to have a SQL benchmark. If someone can find me suggestions/ commands happy to add to the list
• I also want to have some sort of wordpress + nginx or apache benchmark
• Maybe memchached?
• Request - remove all of the apt-get information from the log file

Since I do have someone standing by if folks can let me know commands/ guides I can work on getting added for v2.

 

[britinpdx]

I like the option to upload to STH. That was on the future roadmap
.

When Adobe released Premiere Pro CS5 there were a couple of active members on the Adobe Forum that created a benchmark called "PPBM5" with an associated website. The benchmark can be downloaded from the website and results can be posted back. The site has a results page (with filters) in a tabular format which makes it nice to see other folks configurations and their benchmark scores.

Take a look at the PPBM5 results page

 

[Patrick]

When Adobe released Premiere Pro CS5 there were a couple of active members on the Adobe Forum that created a benchmark called "PPBM5" with an associated website. The benchmark can be downloaded from the website and results can be posted back. The site has a results page (with filters) in a tabular format which makes it nice to see other folks configurations and their benchmark scores.

Take a look at the PPBM5 results page

Looks like the benchmark is meant for Windows and requires PP to be installed? The goal is everything in the suite needs to be open source/ freely available so that anyone can run it. Will add to a list for the Windows suite 2.0 though.

 

[britinpdx]

Looks like the benchmark is meant for Windows and requires PP to be installed?

Correct, the PP benchmark is for a Windows based machine. I wasn't suggesting a Windows benchmark suite, I was simply suggesting the idea of having results (with system configuration data) posted back into a readily accessible "results table".

 

[Patrick]

Correct, the PP benchmark is for a Windows based machine. I wasn't suggesting a Windows benchmark suite, I was simply suggesting the idea of having results (with system configuration data) posted back into a readily accessible "results table".

Oh I get it now! That is certainly something I have wanted to do for years. That was an idea/ vision behind the F@H benchmark years ago but now the site is much more mature.

This time around - v1 is a log file. v2 will likely add tests. v3 I think is the online submission version. I think (might be wrong) that it would make sense to have the benchmark suite finalized before having the web application built.

 

[MikeC]

I've got a log file to upload, but I can't post more than 5000 characters in a reply and I'm not allowed to post attachments Would you like a cut down log file and if so which bits can be removed or can you allow me to make attachments? Or another suggestion?

 

[Patrick]

Sent you a PM. Currently all of the apt-get output is in the file too. Likely will have the next revision remove this.

 

[OBasel]

Please add web benchmarks.

 

[RimBlock]

Like the idea of uploading the results but .....

Linux tends to be quite fluid and if the benchmark updates to current state of the distros repositories rather than a benchmark level then changes in various app / kernel / drivers levels may skew results and so should be recorder which would seem to be a mass of information. I would suggest just updating to a set level for everything, if possible.

Oh and support for RHEL / CentOS or a bootable ISOto save any requirement for an actual install (could lock the software versions at the same time).

RB

 

[Patrick]

Like the idea of uploading the results but .....

Linux tends to be quite fluid and if the benchmark updates to current state of the distros repositories rather than a benchmark level then changes in various app / kernel / drivers levels may skew results and so should be recorder which would seem to be a mass of information. I would suggest just updating to a set level for everything, if possible.

Oh and support for RHEL / CentOS or a bootable ISOto save any requirement for an actual install (could lock the software versions at the same time).

RB

Great points RB. We are certainly looking to do something similar regarding locking benchmarks down beyond the ones we already do. I was actually thinking of just building a suite people could wget directly with the script.

 

[RimBlock]

Great points RB. We are certainly looking to do something similar regarding locking benchmarks down beyond the ones we already do. I was actually thinking of just building a suite people could wget directly with the script.

Wget may be worth the effort as a stepping stone but you are still going to be at risk of being influenced by external OS / other App / Driver factors.

I think it depends on how tightly controlled you want the environment the benchmarks are collected from. If you want it pretty tight then an ISO with a user login with no rights to update or install but only to run the benchmarks would be good. If you want it a bit looser then the Wget direction would work well but the results would not be so easy to compare.

Of course locking it fully down means you are, in essence, benchmarking the hardware only and that may not always be what you may want to do.

I would also think about having any submissions signed and encrypted allowing them only to be accepted if the keys match to try and prevent, at least casual, tampering. It would be easy for a disgruntled member / rival to seed the benchmark database with rubbish otherwise.

RB

 

[Patrick]

Wget may be worth the effort as a stepping stone but you are still going to be at risk of being influenced by external OS / other App / Driver factors.

I think it depends on how tightly controlled you want the environment the benchmarks are collected from. If you want it pretty tight then an ISO with a user login with no rights to update or install but only to run the benchmarks would be good. If you want it a bit looser then the Wget direction would work well but the results would not be so easy to compare.

Of course locking it fully down means you are, in essence, benchmarking the hardware only and that may not always be what you may want to do.

I would also think about having any submissions signed and encrypted allowing them only to be accepted if the keys match to try and prevent, at least casual, tampering. It would be easy for a disgruntled member / rival to seed the benchmark database with rubbish otherwise.

RB

Was thinking that we could use some algorithms to deal with rubbish. I do like the idea of being able to take scores from different sources.