STH xmrig Monero Mining Docker Image

[Scampicfx]

Okay, so i just switched to ubuntu for testing purpose.
I ran lscpu and received this result...



Wow... This is completely different from Windows... So... I was happy and thought problem is solved!

I created docker startup scripts for two containers. I wanted to test two containers first. One container running threads of numa node 1 and one container running threads of numa node 2.

- container 1: 0-3,32-35
- container 2: 4-7,36-39

This is how my start script looks like:

#!/bin/bash

wallet=here-is-my-wallet-address
thrperinst=8
pass=x
pool=here-is-my-pool-url
port=1234

docker run -itd -e xmrpool=$pool -e startport=$port -e username=$wallet.numa-0  -e password=pass -e numthreads=$thrperinst --cpuset-cpus="0-3,32-35" servethehome/monero_xmrig
docker run -itd -e xmrpool=$pool -e startport=$port -e username=$wallet.numa-1  -e password=pass -e numthreads=$thrperinst --cpuset-cpus="4-7,36-39" servethehome/monero_xmrig

Result: There is only very little hasing in progress! Have a look:

[2017-11-07 14:26:42] speed 2.5s/60s/15m n/a 41.1 n/a H/s max: n/a H/s

So this means, the hashpower is about 40 hashes per docker-container, although system monitoring of ubuntu shows following:




I'm used to Intel CPUs... Epyc is my first AMD cpu. So I'm wondering if I am doing any severe mistake? Since I'm helpless and do not know any other solutions, I would appreciate any tipps, any feedback or any posting ... I don't have any glue why this system has such a poor performance?

Thanks so much for your time!

Chris

EDIT: Well... there is another interesting point I don't understand.

running the first container with 4 threads and cores 0-3 results in a higher hashrate (214 hashes in total) than running this container with 8 threads using cores 0-3,32-35 (180 hashes in total)

Going to take a break, now.

EDIT2: I made additional tests:

Scenario

docker run -itd -e xmrpool=$pool -e startport=$port -e username=$wallet-numa-0 -e password=pass -e numthreads=$thrperinst --cpuset-cpus="0-3" servethehome/monero_xmrig
docker run -itd -e xmrpool=$pool -e startport=$port -e username=$wallet-numa-1 -e password=pass -e numthreads=$thrperinst --cpuset-cpus="4-7" servethehome/monero_xmrig
docker run -itd -e xmrpool=$pool -e startport=$port -e username=$wallet-numa-2 -e password=pass -e numthreads=$thrperinst --cpuset-cpus="8-11" servethehome/monero_xmrig
docker run -itd -e xmrpool=$pool -e startport=$port -e username=$wallet-numa-3 -e password=pass -e numthreads=$thrperinst --cpuset-cpus="12-15" servethehome/monero_xmrig

So, basically every docker container gets 4 threads assigned and uses only real cores of only independent numa nodes.

I get 0 hashes... N/A. When I disabled all containers and then run only the first docker container, I get about ~ 230 Hashes of this first container. Enabling any other containers result in drop of performance. When there are about 4 containers "online", I don't receive any hashes anymore...

Are there any certain bios settings to consider?

 

[Scampicfx]

Guys, I think I found the mistake...

Running numactl --hardware:

numactl --hardware
available: 8 nodes (0-7)
node 0 cpus: 0 1 2 3 32 33 34 35
node 0 size: 0 MB
node 0 free: 0 MB
node 1 cpus: 4 5 6 7 36 37 38 39
node 1 size: 7915 MB
node 1 free: 6249 MB
node 2 cpus: 8 9 10 11 40 41 42 43
node 2 size: 0 MB
node 2 free: 0 MB
node 3 cpus: 12 13 14 15 44 45 46 47
node 3 size: 0 MB
node 3 free: 0 MB
node 4 cpus: 16 17 18 19 48 49 50 51
node 4 size: 0 MB
node 4 free: 0 MB
node 5 cpus: 20 21 22 23 52 53 54 55
node 5 size: 8016 MB
node 5 free: 7209 MB
node 6 cpus: 24 25 26 27 56 57 58 59
node 6 size: 0 MB
node 6 free: 0 MB
node 7 cpus: 28 29 30 31 60 61 62 63
node 7 size: 0 MB
node 7 free: 0 MB
node distances:
node   0   1   2   3   4   5   6   7
  0:  10  16  16  16  32  32  32  32
  1:  16  10  16  16  32  32  32  32
  2:  16  16  10  16  32  32  32  32
  3:  16  16  16  10  32  32  32  32
  4:  32  32  32  32  10  16  16  16
  5:  32  32  32  32  16  10  16  16
  6:  32  32  32  32  16  16  10  16
  7:  32  32  32  32  16  16  16  10

Guys, do I see it correctly that this setup lacks of memory and each numa node requires it's own memory? If so: shame on me

 

[hifijames]

I get about the same H/s, xmr-strak on bare metal, vs docker container.
On a pair of E5-2680v2, cpu 0-19, 20,28,30,38, xmr-strak on bare metal 1256H/s
each NUMA docker gets 620H/s, slightly less.

 

[azev]

I get about the same H/s, xmr-strak on bare metal, vs docker container.
On a pair of E5-2680v2, cpu 0-19, 20,28,30,38, xmr-strak on bare metal 1256H/s
each NUMA docker gets 620H/s, slightly less.

do you mind sharing your cpu config file for the baremetal ?

 

[hifijames]

do you mind sharing your cpu config file for the baremetal ?

I just use CPU ID 0-19, 20,28,30,38. Everything else is default.
Other than CPU ID 0-19, which the other 4 to use makes difference of about 50~80H/s. Using 26 threads makes it worse as well.
Sorry I am not able to post the CPU config file, because I reinstalled the whole system to switch to Ubuntu/Docker container approach for flexibility. Now I run 2 instances on each server, one use CPU 0-9, 20, 28, and the other use CPU 10-19, 30, 38. I get 620~630H/s on each.

 

[hifijames]

@Patrick Any chance servethehome/monero_xmrig can be updated to xmrig 2.50? I know there is a universal image now, but really like the flexibility of dev fee of this original image. Thanks!

 

[leerees]

Can the dev fee be set to 0 with this?

Has xmrig been updated to work with the upcoming Monero fork?

 

[Patrick]

2.50 update will be on Friday