Building the STH 2013 Linux Server Test Suite

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
As many know, I have wanted to build a Linux test suite for quite a long time. The old Windows version works well, but I want to have a version that is easy to run and install. Decided to start documenting some of the tests. After a lot of debate, will be using Ubuntu 12.04 LTS for this. The goal is to make something easy for others to replicate quickly so will be scripting a lot of this as we go.

UnixBench 5.1.2

This is the WebHostingTalk base benchmark. Cool thing is that they have a long thread of results for comparison. Only issue: the simple instructions on WHT do not work with the base Ubuntu server installation.

How to get it to work:

wget http://byte-unixbench.googlecode.com/files/unixbench-5.1.2.tar.gz
sudo apt-get install build-essential

sudo apt-get install libx11-dev
sudo apt-get install libglu-dev
tar -zxvf unixbench-5.1.2.tar.gz
cd unixbench-5.1.2
nano Makefile


In the config section look for:
GRAPHIC_TESTS = defined
and change to:
# GRAPHIC_TESTS = defined
Control x (exit) and click enter to write to Makefile then

make
./Run
May decide not to run all tests but that at least makes the benchmark work. Also thinking of moving to 5.1.3 but then it is not comparable to the WHT archive.

Next Steps
More to come. Please feel free to suggest as necessary. I will be having someone write scripts to run everything once we have a full suite and make that available for download so no more having to hunt down dependencies and such. If you do want to post a mini-guide - please use a base Ubuntu 12.04 LTS installation so that we can be sure to catch any dependencies.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
Have used pts on Linux quite a bit actually. Some of the tests you get some strange results. You also have a lot of folks generating numbers that I can't replicate in the lab. Goal is to have a STH centric benchmark. One of the other things I want to do is use a "real world" (of course cleansed) data set for things like a database test and web server test. Used pts/stream for this: Dual Xeon E5 Memory Bandwidth Comparison - DDR3 1066MHz to Hyper-Speed
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
So here's something interesting I installed crafty on the Supermicro X9SBAA-F:
patrick@ubuntu1:~$ sudo apt-get install crafty -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
linux-headers-3.5.0-17 linux-headers-3.5.0-17-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
crafty-books-medtosmall fairymax libfontenc1 libice6 libsm6 libxaw7
libxfont1 libxmu6 libxpm4 libxt6 x11-common xboard xfonts-100dpi
xfonts-encodings xfonts-utils
Suggested packages:
x-terminal-emulator xfs xserver
The following NEW packages will be installed:
crafty crafty-books-medtosmall fairymax libfontenc1 libice6 libsm6 libxaw7
libxfont1 libxmu6 libxpm4 libxt6 x11-common xboard xfonts-100dpi
xfonts-encodings xfonts-utils
........
It installed easily.

I then tried it on an Amazon EC2 instance with Ubuntu 12.04
ubuntu@domU-12-31-39-0F-62-8F:~$ sudo apt-get install crafty -y
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package crafty is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source


E: Package 'crafty' has no installation candidate
Wonder if Amazon did that on purpose?
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
Working on making the setup even better but starting to look promising.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
Monday update - spun up six servers to test. Need to get a dual L5520 box and a dual E5-2690 box in the mix clearly but interesting mix so far:

  1. Raspberry Pi (not all tests have run on this but still working on that)
  2. Amazon AWS EC2 US-EAST t1.micro instance (Intel Xeon E5430 architecture)
  3. Intel Atom S1260
  4. Intel Xeon E3-1230
  5. AMD Opteron 3380
  6. Dual Intel Xeon E5-2430L

That broad of a range is giving a really nice set of benchmark profiles. Also keeping everything in a Google Doc so that we can script these.

Still working on what the web/ database benchmarks will be. If anyone has scripted ngnix/ Apache installations/ setups + Siege for example let me know. Real goal is to get something that looks real-world. A lot of apache bench results I have seen look a lot like a simple php file possibly with a math command.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
Here's a great example using pts/openssl

Processor: Intel Xeon E5430 @ 2.66GHz (1 Core), Memory: 590MB, Disk: 8GB

Software:
OS: Ubuntu 12.04, Kernel: 3.2.0-36-virtual (x86_64), Compiler: GCC 4.6, File-System: ext4

Would you like to save these test results (Y/n): n

Estimated Run-Time: 3 Minutes



OpenSSL 1.0.1c:
pts/openssl-1.7.0
Test 1 of 1
Estimated Trial Run Count: 3
Estimated Time To Completion: 3 Minutes
Started Run 1 @ 17:49:55
Started Run 2 @ 17:50:17
Started Run 3 @ 17:50:39 [Std. Dev: 129.90%]
Started Run 4 @ 17:51:02 [Std. Dev: 138.46%]
Started Run 5 @ 17:51:24 [Std. Dev: 143.50%]
Started Run 6 @ 17:51:46 [Std. Dev: 146.74%]

Test Results:
57
5.7
5.7
5.7
5.8
5.7

Notice the first result on the t1.micro instance is 10x as fast as the others. This happened over and over again.

Another one I tried was to run Crafty 23.4 on the Raspberry Pi. Saw times way out of line when I was using pts/crafty) Was getting sub 1/2 second times which seemed way too fast. I ran the benchmark manually to see what was going on:
White(1): bench
Running benchmark. . .
.illegal position, no black king
bad string = "3r1k2/4npp1/1ppr3p/p6P/P2PPPP1/1NR5/5K2/2R5"
Illegal position, using normal initial chess position
.illegal position, no black king
bad string = "rnbqkb1r/p3pppp/1p6/2ppP3/3N4/2P5/PPP1QPPP/R1B1KB1R"
Illegal position, using normal initial chess position
.illegal position, no black king
bad string = "4b3/p3kp2/6p1/3pP2p/2pP1P2/4K1P1/P3N2P/8"
Illegal position, using normal initial chess position
.illegal position, no black king
bad string = "r3r1k1/ppqb1ppp/8/4p1NQ/8/2P5/PP3PPP/R3R1K1"
Illegal position, using normal initial chess position
.illegal position, no black king
bad string = "2r2rk1/1bqnbpp1/1p1ppn1p/pP6/N1P1P3/P2B1N1P/1B2QPP1/R2R2K1"
Illegal position, using normal initial chess position
.illegal position, no black king
bad string = "r1bqk2r/pp2bppp/2p5/3pP3/P2Q1P2/2N1B3/1PP3PP/R4RK1"
Illegal position, using normal initial chess position

Total nodes: 14864526836704026544
Raw nodes per second: 171274
Total elapsed time: 0.43
Interesting to note that these errors caused the benchmark to stop. pts/crafty did not report this just the Total elapsed time.

Good to see these variations showing up in the data.

Also now working on expanding to an Amazon EC2 m1.small and m1.medium instance.
 

OBasel

Active Member
Dec 28, 2010
494
62
28
preferring to see ngnix benchmark for web server loads here. they have momentum in the arena. are you going to share the scripts? anantech does benchmarks that I can't recreate vapus ii.

another idea ---- make usable on only one machine and have scale from atoms to 4p systems
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,506
233
63
Can you benchmark different CPU options for the C6100? I'd like to see if it's worth it to get faster Xeons. Or what about only 1 L5520?
 

Biren78

Active Member
Jan 16, 2013
550
94
28
hi - saw many reviews on the main site w/ this. watched your video on the L5520 and L5530 thread and looks like you changed stuff? what's updated?
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,168
5,234
113
Oops sorry I missed this:

Step 1 - Amazon AWS enable multiverse - it is disabled in their Ubuntu 12.04 LTS

Step 2 - Updates!
sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get install build-essential libx11-dev libglu-dev hardinfo crafty phoronix-test-suite -y


Step 3 - Benchmarks
hardinfo | less && wget https://byte-unixbench.googlecode.com/files/UnixBench5.1.3.tgz && tar -zxvf UnixBench5.1.3.tgz && cd UnixBench && time make && wget http://forums.servethehome.com/pjk/fix-limitation.patch && patch Run fix-limitation.patch &&./Run dhry2reg whetstone-double syscall pipe context1 spawn execl shell1 shell8 shell16 && cd .. && wget http://www.futuretech.blinkenlights.nl/depot/c-ray-1.1.tar.gz && tar -zxvf c-ray-1.1.tar.gz && cd c-ray-1.1 && make && cat scene | ./c-ray-mt -t 32 -s 7500x3500 > foo.ppm | tee c-ray1.txt && cat sphfract | ./c-ray-mt -t 32 -s 1920x1200 -r 8 > foo.ppm | tee c-ray2.txt && cd ..
Note I generally | tee all of this stuff

crafty bench
Having trouble with crafty. I can't get it to escape after a run or a series of runs.

phoronix-test-suite benchmark pts/stream pts/compress-7zip pts/openssl pts/pybench
Likewise, PTS has a series of menu items that I need to get scripted.