perf.py - anyone gotten this to work

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,988
113
I am totally failing at getting this to work even with the quick start (see version. benchmarks: 9923b81a1d34 README.txt)

My hg clone got the files down, but running:
Code:
python perf.py -r -b default /control/python /test/python
it fails.

Has anyone gotten this working?
 

Chuckleb

Moderator
Mar 5, 2013
1,017
331
83
Minnesota
This tool is meant to test the differences between different installations of python. So what it does is run two copies of the test and the /control/python and /test/python are paths to two instances on python that you want to compare. This example works but not ideal since it runs two copies of the test. Though you could use it as a way to verify the results. I chose a quick test and had it print out the results.

[Charles@basement benchmarks-9923b81a1d34]$ python perf.py -r -b nqueens -v /usr/bin/python /usr/bin/python
INFO:root:Automatically selected timer: time
[1/1] nqueens...
INFO:root:Running `/usr/bin/python performance/bm_nqueens.py -n 100 --timer time`
INFO:root:Running `/usr/bin/python performance/bm_nqueens.py -n 100 --timer time`

Report on Linux basement 3.10.0-327.3.1.el7.x86_64 #1 SMP Wed Dec 9 14:09:15 UTC 2015 x86_64 x86_64
Total CPU cores: 24

### nqueens ###
Min: 0.288829 -> 0.290140: 1.00x slower
Avg: 0.296371 -> 0.295594: 1.00x faster
Not significant
Stddev: 0.01070 -> 0.00763: 1.4031x smaller
[Charles@basement benchmarks-9923b81a1d34]$
 
  • Like
Reactions: Patrick

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,988
113
How about this one?
Code:
x10sdv-7tp8f-os:~/pybench/benchmarks$ python perf.py -r -v -b nqueens /usr/bin/python /usr/bin/python
INFO:root:Automatically selected timer: time
[1/1] nqueens...
INFO:root:Running `/usr/bin/python performance/bm_nqueens.py -n 100 --timer time`
INFO:root:Running `/usr/bin/python performance/bm_nqueens.py -n 100 --timer time`

Report on Linux x10sdv-7tp8f-os 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64
Total CPU cores: 32

### nqueens ###
Min: 0.174940 -> 0.175455: 1.00x slower
Avg: 0.176207 -> 0.181227: 1.03x slower
Significant (t=-2.25)
Stddev: 0.00148 -> 0.02230: 15.1180x larger
 

Chuckleb

Moderator
Mar 5, 2013
1,017
331
83
Minnesota
About twice as fast as my dual X5675s. Interesting. Some of the other tests may be better than nqueens as well. The tool is pretty good, has the ability to bind to chips, etc. So you could use against 1 core of a chip to look at single thread performance.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,988
113
About twice as fast as my dual X5675s. Interesting. Some of the other tests may be better than nqueens as well. The tool is pretty good, has the ability to bind to chips, etc. So you could use against 1 core of a chip to look at single thread performance.
Yea python is so widely used and I saw this even has a django benchmark. Just looking what this has, I liked it better than the old pybench script.

I think it is time to start looking into the next rev of Linux-Bench so I am playing with some of these. It helps now with the Sunnyvale lab that I can test across a few more varied platforms quickly.
 

PigLover

Moderator
Jan 26, 2011
2,975
1,283
113
Interesting. X10SDV-7Tp8F? Xeon D-1587? 16 core/32 thread? 65W TDP? Board listed on the early-stock drop-ship bandit sites well north of $2K.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,003
4,988
113
Interesting. X10SDV-7Tp8F? Xeon D-1587? 16 core/32 thread? 65W TDP? Board listed on the early-stock drop-ship bandit sites well north of $2K.
Or that could just be the machine name. Who knows.... :) Hardware pron thread

You would be right in thinking, is it not weird that there are D-1541 benchmarks up today. That was not the originally scheduled programming.
 

Ramos

Member
Mar 2, 2016
69
12
8
41
Python exists in a lot of different variants. Most has 2 and 3 versions because of the version splits but most professional things seems to use 2.x still, like SAP, Hadoop management, cluster python stuff etc.

- The standard one is okay but slow
- Anaconda python from Continuum is the industry standard package cause it comes with everything neatly packed in already and all essential libraries just working. Install this one in 2.7.x if in doubt. It also has the core essential numpy libraries implemented and compiled with MKL (Math Kernel Libraries) so its optimized for CPUs to run superfast.

To go beyond,
- Pypi is faster but has less features. I dunno alot about it, but mainly I hear performance people develop in Anaconda and then maybe port to pypi for some applications.
- Intel python. Its (iirc) pretty new and has some superfast optimizations for Intel CPUs built-in but Continuum works with Intel so maybe they get the same as it isnt a core product for Intel, but mainly to vendor lock python user if I read the sales material right.

I could be wrong in some of the above but its the truth that I currently know. Apols if there are facts wrong.

Could be interesting to run the perf.py python script on ALL major python interpreters on the same undisturbed iron to get an interpreter rating of different ones.