Monero Mining Performance

[Patrick]

@SavageWS6 667-671H/s for one or two CPUs? I was at 829H/s using the docker image.

Had to pull a few nodes today. Will be down to 37 by the end of the day.

Shares: Good - 4,053,261, Bad - 375
Total Mined: 31.17 XMR
Blocks: 4

@Marsh are those V1's?

 

[SavageWS6]

@SavageWS6 667-671H/s for one or two CPUs? I was at 829H/s using the docker image.

Had to pull a few nodes today. Will be down to 37 by the end of the day.

Shares: Good - 4,053,261, Bad - 375
Total Mined: 31.17 XMR
Blocks: 4

@Marsh are those V1's?

For 2 CPU's on Server 2016, which really sucks honestly. It's also running the github project I posted xmr-stak-cpu

Later on, before I put in the 2nd CPU, I was pushing 389H/s with 1 CPU. Technically I should be getting between 770-780H/s if I just double 389 with variable hashing due to the pool, but that isn't the case right now. I've checked and no thermal throttling going on right now.

 

[Marsh]

Yes , those are E5-2650 v1 cpu.

I get the same hash rate using 1 node out 4 nodes ASUS 4nodes RS720Q-E8-RS8-P,
the dual E5-2650 v3 QSFB ASUS node only consumes about 155w instead of 255w ( E5 v1 node ).

Since we are on the subject.
I have 1 ASUS node with only single E5-2650 v3 QFSB , it produced 400H/s , but the dual E5-2650 v3 QFSB node only produced 720-740H/s using the STH docker image.

My be this weekend, I'll spend time to find out why the dual node produced less hashrate.

 

[cafcwest]

edit: think i worked past my Linux noob issues. More to follow...

 

[Patrick]

@Marsh my guess is NUMA.

The AMD GPU dockerized is giving me fits.

 

[Marsh]

With the ASUS 4 nodes chassis, I have 1 node with a single E5-2650 v3 cpu , the other nodes have dual E5-2650 v3 CPU.

single cpu node => 400H/s , minerd -t 12 threads
dual cpu node => 740H/s , minerd -t 25 threads

Reading the minerd man page:
On multiprocessor systems, minerd
automatically sets the CPU affinity of miner threads
if the number of threads is a multiple of the number of processors.

My guess is best to use -t 24 threads instead -t 25.
Is there a command line parameters to make the docker image to use -t 24 ?

Tomorrow , I like to compile the minerd , run it without docker, just to test my theory.
@Patrick , if you already have the binary for Ubuntu , then I could try it sooner .

 

[nthu9280]

May be I'm not doing something correct Or overhead/limits of free ESXi (8vcpu). Tested the STH docker cpuminer on a Ubuntu 1604LTS VM. Base is HPZ600 2xE5620. Getting 290-300H/s on 12 t. Left it running overnight. Delta of current draw of Idle vs mining on the on the PDU 0.5A (~50w).
At this rate it barely pays for the power bill

Sent from my Nexus 6 using Tapatalk

 

[cafcwest]

Okie dokie, a few updates:

- Finishing building my 4P Opteron 6200 build. Installed Wolf's CPU Miner and mucked around for a few hours in Windows (because hey, I'm a Windows guy) but could never crack more than 650H/s. Took a taste of the penguin world and installed Ubuntu Server 16.10 and Wolf CPU miner again. Started at 32 cores, ended up at 50 cores being the sweet spot (though I don't know why). Mining at 1850H/s for over 12 hours. Better performance than Patrick's 2x E5-2699 V4 testing for probably something like 1/10 the price!!

- For 3-4 weeks now, I have been using former lab C6100 nodes as miners. 8 nodes, all 2x E5620 processors. In Windows mining with the MinerGate GUI miner, the sweet spot was 12 cores for about 220H/s. Since I've already taken one taste of the Ubuntu kool-aid this weekend, I started up a Ubuntu 16.04 LiveCD, installed the miner and am currently mining at 330H/s. I read all 34 pages of the Wolf CPU thread over on bitcointalk and saw him mention multiple times that performance was greater in Linux - now I see it!

 

[cafcwest]

Ok, I am in a situation where I don't even know what I don' t know, so let me state what I want to accomplish, what I am working with, and seek the advice of those far smarter than myself.

Gear
8 Dell C6100 nodes
Server 2016 Datacenter installed on each, configured as Hyper-V hosts
(Likely not relevant, but I have free disk space on the local SSD OS drive, as well as pooled storage for the cluster via ScaleIO)

Desire
I'd like to be able to run Wolf CPU miner in a -nix environment during idle. Ideally this system would have some form of centralized administration of the individual nodes/container, so that I can check status/start & stop/etc. If not, I would program each node/container to automatically start the miner on boot/launch, then I would simply boot/start when I wanted to run.

How do I do this simply?

 

[Marsh]

@cafcwest

Where did you get your Ubuntu Wolf CPU binary?
I like to run some test today on my ASUS 4 nodes chassis.

 

[cafcwest]

@cafcwest

Where did you get your Ubuntu Wolf CPU binary?
I like to run some test today on my ASUS 4 nodes chassis.

From GitHub. Here is how I installed:

sudo sysctl -w vm.nr_hugepages=X    ((x= used CPU cores times 3 for lower core counts.  I am finding through testing that higher core count systems work better with the same number here as the core count))
sudo apt-get install libcurl4-gnutls-dev automake build-essential git
git clone https://github.com/wolf9466/cpuminer-multi.git
cd cpuminer-multi
./autogen.sh
CFLAGS="-march=native" ./configure
make

 

[Marsh]

Thanks,
I found many forks and version of the cpuminer. I want to compare apple to apple using the same cpuminer-multi.

 

[cafcwest]

Tinkering with the hugepages (matching the core count, versus the core count x 3 recommended in many places) has allowed me to bump up core count to 52 (of 64 cores total) and has the 4P system up to 1880H/s.

Being a Windows guy, I don't actually what I am adjusting, but I do know that bigger number > smaller number.

I had read on some forums 1800-2000H/s for such systems (with scant details regarding exact miner and configuration, memory configurations, processor clocks, etc.) so all in all, I am quite satisfied with my results. Now time to build 10 more!!! (maybe)

 

[Marsh]

How much power are you using for the 4P system?

I finished compile minerd ,
running -t 24 in the ASUS dual cpu node is faster than -t 25 ( docker version ).
780-790Hs (-t24 minerd ) vs 720-740H/s ( -t 25 docker version )

My Asus 4 nodes with 7 cpu ( 3 dual cpu nodes + 1 single cpu nodes ) consumes 610watt.
hash rate is 780H/s x 3 + 400H/s = 2,740H/s

 

[Patrick]

For Xenial:

sudo apt-get update
sudo apt-get install -y libcurl3 build-essential libjansson-dev automake autotools-dev autoconf libcurl4-gnutls-dev git

git clone https://github.com/wolf9466/cpuminer-multi

cd cpuminer-multi/
./autogen.sh
CFLAGS="-march=native" ./configure
make

./minerd -a cryptonight -o stratum+tcp://xmr.pool.minergate.com:45560 -u example@example.com -p x -t $cpusnum

You can change the $cpusnum to the number of threads you want to use. Did a lot of testing to get the Docker image to work well on most systems. The idea is that that is something you can use to get going quickly on a dormant cluster and that you are unlikely to tune everything for mining on the base system.

Tuning 1-5 systems is not bad. Tuning a cluster of 100 individually is a PITA.

@cafcwest what is the power consumption?

 

[Patrick]

@Marsh I see we are posting at the same time.

So is the logic that you need an even number of threads? So it is more like ((MB L3 Cache /2 )) - [((MB L3 cache) / 2 ) % 2]

 

[cafcwest]

610W on the 4P system according to the onboard systems monitor. And this is loaded with 16 x 8GB DIMMs, as I wanted to populate all the channels and then remove to see if this would have any impact on mining performance.

 

[Patrick]

That is not too bad, just a bit over 2x what the dual E5-2699 V4's are pulling.

 

[cafcwest]

That is not too bad, just a bit over 2x what the dual E5-2699 V4's are pulling.

According to the specs, 460W of that is just the processors. Pulling 12 DIMMs should bring it down a bit.

 

[Patrick]

@Patrick
32 Mb does not seem very small
-t=16 should be optimal for 7210

Just tried this. -t=16 gave about 100H/s.

I tried again using all of the AVX512 gcc flags and still got 1122H/s using 255 threads, 1127 with 256 threads.