Dedicated Crypto Mining Open Compute server build.

Discussion in 'Cryptocurrency Mining and Markets' started by Klee, May 23, 2017.

  1. Marsh

    Marsh Moderator

    Joined:
    May 12, 2013
    Messages:
    2,106
    Likes Received:
    960
    @arron

    For E5-2670 v1, I also see 975H/s

    For E5-2660 v1 = 866H/s

    You may want to double check which CPU in the machine?
     
    #181
  2. Titan

    Titan New Member

    Joined:
    Jan 14, 2018
    Messages:
    12
    Likes Received:
    2
    I see. Holding XMR is good. is there any tool that switch pools easily?
    This is my current hash on E5630 x2
    upload_2018-1-15_9-10-18.png
     
    #182
  3. Klee

    Klee Well-Known Member

    Joined:
    Jun 2, 2016
    Messages:
    1,239
    Likes Received:
    374

    I have not mined Monero in about 2 1/2 months.

    Mined Aeon until about a month ago.

    There are more profitable cryptonight coins out there.
     
    #183
    Joel likes this.
  4. Indecided

    Indecided Member

    Joined:
    Sep 5, 2015
    Messages:
    68
    Likes Received:
    16

    @polyfractal - Interesting. When I run with your config I do get approximately 930H/s, still a little short of 945H/s but not very close to 975H/s that Marsh is getting, but certainly better then before.

    The difference with your config is that i had my CPU affinity on even numbers instead of sequential.

    I.E. 0,2,4,6,8,10 vs yours as in 0,1,2,3,4,5 so on and so forth. Not sure what the logic is behind that, but I had set up even numbers after reading one of the sample config files way back several months ago. Something to do with picking the physical cores vs the hyperthreads or so. Also, not sure why your config chooses to use threads 24 and 25 specifically with a gap in between.

    I noticed with your config it pushes CPU #1 all the way and just a little of CPU #2. Wonder if that's intended behavior....

    Now I guess it's time to tweak the whole lot.... Thanks!
     
    #184
  5. polyfractal

    polyfractal Member

    Joined:
    Apr 6, 2016
    Messages:
    33
    Likes Received:
    27
    Yeah, not sure either. I also tried fiddling with the config to do every-other and similar arrangements... could never beat the auto-configuration that does it (mostly) sequential. Really not sure why it likes this, or why it seems to prefer loading up one socket more than the other. Perhaps to minimize inter-socket communication?

    It's been on my todo list to write a quick simulated annealing solver to see if the config can be optimized some more, but I haven't found time yet. I want to dig into the xmr-stak code a bit more first to see how it's using hwlock to determine these configs.

    Glad it helped, goodluck!
     
    #185
  6. Joel

    Joel Active Member

    Joined:
    Jan 30, 2015
    Messages:
    807
    Likes Received:
    155
    Note that Windows assigns cores differently than linux on HT machines so this may be what's causing the confusion.

    Windows: Even numbered cores are physical cores (0,2,4...) and odd are HT (1,3,5...).

    Linux: Physical cores are first, then HT cores. So for a 2x 2670 it would be: 0-7 (CPU0), 8-15 (CPU1), 16-23 (CPU0 HT cores), 24-31 (CPU1 HT cores)
     
    #186
    Cool Runnings and Indecided like this.
  7. Joel

    Joel Active Member

    Joined:
    Jan 30, 2015
    Messages:
    807
    Likes Received:
    155
    The other thing that's helpful is to run two separate instances of your miner, affined to the cores of each NUMA node.
     
    #187
  8. polyfractal

    polyfractal Member

    Joined:
    Apr 6, 2016
    Messages:
    33
    Likes Received:
    27
    Ah, well that'd explain what's going on. It's using 8 physical cores + 2 HT for each socket.

    Oh, interesting, even with xmr-stak? I knew you had to do that with xmrig because it didn't handle NUMA well, but I thought xmr-stak did NUMA correctly with it's affine settings.? Should I instead be running two xmr-stak instances each assigned to a socket's worth of threads (in my case, the relevant 8 physical + 2 HT threads)
     
    #188
  9. Indecided

    Indecided Member

    Joined:
    Sep 5, 2015
    Messages:
    68
    Likes Received:
    16
    Wow, I feel pretty stupid now. Thanks Joel for clearing that up!

    I think I've been able to increase hashrate another 15% or so at least, especially on the larger multicore procs like the E5-2651v2. from 750 to 885. Pretty significant.
     
    #189
  10. Titan

    Titan New Member

    Joined:
    Jan 14, 2018
    Messages:
    12
    Likes Received:
    2
    How is temp on your E5-2651 v2?
     
    #190
    Last edited: Jan 15, 2018
  11. yuzmemak

    yuzmemak New Member

    Joined:
    Aug 31, 2017
    Messages:
    27
    Likes Received:
    1
    Hi

    Which model did you order?

    Sent from my SM-G900F using Tapatalk
     
    #191
  12. yuzmemak

    yuzmemak New Member

    Joined:
    Aug 31, 2017
    Messages:
    27
    Likes Received:
    1
    Hi

    Which model did you order?

    Sent from my SM-G900F using Tapatalk
     
    #192
  13. Titan

    Titan New Member

    Joined:
    Jan 14, 2018
    Messages:
    12
    Likes Received:
    2
    upload_2018-1-18_10-7-40.png

    they told me out of stock need more 10 days.
     
    #193
  14. polyfractal

    polyfractal Member

    Joined:
    Apr 6, 2016
    Messages:
    33
    Likes Received:
    27
    A note for anyone thinking about mining Sumo: the only real exchange that I'm aware of for sumo is Cryptopia... which appears to be flaky at best. I deposited a few SUMO there as a test, and it still hasn't showed up after a few days. Seems to be a reoccurring issue with them, according to reddit posts.

    So... eh. It may look more profitable on paper, but I'm leery of wasting much time on a coin which is only usable at one flaky and illiquid exchange.
     
    #194
  15. Titan

    Titan New Member

    Joined:
    Jan 14, 2018
    Messages:
    12
    Likes Received:
    2
    Pools are gone too
     
    #195
  16. Joel

    Joel Active Member

    Joined:
    Jan 30, 2015
    Messages:
    807
    Likes Received:
    155
    I just recently found out about LiveCoin after having similar issues with Cryptopia (for some reason I'm not getting 2FA codes any more).

    One exchange with Sumo, and while a bit slow, it worked.
     
    #196
  17. Klee

    Klee Well-Known Member

    Joined:
    Jun 2, 2016
    Messages:
    1,239
    Likes Received:
    374
    I have not been running my open compute servers for about a week, so today I replaced the four E5-2660 cpu's in the first server with 4 E5-2603 four core cpu's to see what they do.

    OS is Ubuntu 17.10 server upgraded from 17.04 server.

    Powered on both servers and just idling: 104 watts.

    Fired up xmrig 2.5.0 on node 1: 130 watts with all 8 cores running.

    Fired up xmrig 2.5.0 on node 2: Both nodes mining 8 threads each is 556 H/s and using 163 watts.

    First cpu is 47c and second (the rear one) is 48c. The air coming out of the rear fans seems cool on my hand vs warm with the 2660's and hot with the 2665's.

    @node1:~/monero-xmrig-2.5.0$ ./xmrig
    * VERSIONS: XMRig/2.5.0 libuv/1.19.2 gcc/5.4.0
    * HUGE PAGES: available, enabled
    * CPU: Intel(R) Xeon(R) CPU E5-2603 0 @ 1.80GHz (2) x64 AES-NI
    * CPU L2/L3: 4.0 MB/20.0 MB
    * THREADS: 8, cryptonight, av=1, donate=1%
    * POOL #1: xmr-usa.dwarfpool.com:8005
    * COMMANDS: hashrate, pause, resume
    [2018-03-24 15:53:18] use pool xmr-usa.dwarfpool.com:8005 144.217.117.111
    [2018-03-24 15:53:18] new job from xmr-usa.dwarfpool.com:8005 diff 20000
    [2018-03-24 15:53:36] speed 2.5s/60s/15m 278.2 n/a n/a H/s max: 278.3 H/s
    [2018-03-24 15:53:50] accepted (1/0) diff 20000 (112 ms)
    [2018-03-24 15:53:51] speed 2.5s/60s/15m 278.3 n/a n/a H/s max: 278.3 H/s
    [2018-03-24 15:54:06] speed 2.5s/60s/15m 278.2 n/a n/a H/s max: 278.3 H/s
    [2018-03-24 15:54:21] speed 2.5s/60s/15m 278.3 278.2 n/a H/s max: 278.3 H/s
    [2018-03-24 15:54:35] accepted (2/0) diff 20000 (113 ms)
    [2018-03-24 15:54:36] speed 2.5s/60s/15m 278.1 278.2 n/a H/s max: 278.3 H/s
    [2018-03-24 15:54:51] speed 2.5s/60s/15m 278.3 278.3 n/a H/s max: 278.3 H/s
    [2018-03-24 15:55:06] speed 2.5s/60s/15m 278.3 278.3 n/a H/s max: 278.3 H/s
    [2018-03-24 15:55:21] speed 2.5s/60s/15m 278.3 278.3 n/a H/s max: 278.3 H/s

    EDIT: The results from earlier in the tread of the same server running E5-2660's just to compare.

    "No cores turned off.
    One server plugged in and both nodes powered off: 15w
    One node powered on, Ubuntu 17.04 server booted up and idling : 56.5w
    One node after 10 minutes mining Monero with 20 threads : 208w @ 868 H/s
    Two nodes powered on, Ubuntu 17.04 server booted up and idling : 117w
    Two nodes after 10 minutes mining Monero with 40 threads : 401w @ 1736 H/s"

    I might run the 2603's during the summer just because the power use AND heat is alot less.
     
    #197
    Last edited: Mar 24, 2018
  18. Klee

    Klee Well-Known Member

    Joined:
    Jun 2, 2016
    Messages:
    1,239
    Likes Received:
    374
    Downloading the daily build of Ubuntu Server 18.04 to give that a spin, been needing to replace the two WD black 1 TB hard drives with two of the 12 WD 250 gig drives that bought to put in all the oc server nodes. I already installed them in the other servers so now all the drives will be identical.
     
    #198
  19. Klee

    Klee Well-Known Member

    Joined:
    Jun 2, 2016
    Messages:
    1,239
    Likes Received:
    374
    Well, rebooted and ran htop since it was already installed and it shows memory use at 655 mb.


    Edit: Still does the hang on reboot just like Ubuntu 16.04, 17.04, 17.10.

    So have to still do this: echo "options mei-me disable_msi=1" | sudo tee /etc/modprobe.d/mei-me.conf
     
    #199
    Last edited: Mar 24, 2018
  20. Klee

    Klee Well-Known Member

    Joined:
    Jun 2, 2016
    Messages:
    1,239
    Likes Received:
    374
    Compiled xmrig 2.5.0 and running it now.

    The fans are so quiet, I can only hear them when I put my head about 6 inches away from them. LOL

    The power supply fan is about two or three times as loud.

    The cpu's are running so cool, 45c and 47c, that the fans are barely spinning.

    So three open compute servers running E5-2603 cpu's would be about 1675 H/s at about 490 watts, about the same as one oc server running 2660's but without the heat and noise.

    Not real impressive in hash rate but real nice as far as being quiet and more importantly the lack of heat being put out compared to the 2660 and especially the hot 2665's.

    Defiantly thinking about running them in the summer to cut down on the a/c usage which would lower my power bill a lot.
     
    #200
    pgh5278 likes this.
Similar Threads: Dedicated Crypto
Forum Title Date
Cryptocurrency Mining and Markets Dedicated Open Compute miner build- Part 2.The RandomX Edition. Aug 17, 2019
Cryptocurrency Mining and Markets My ASUS 4nodes dedicated Monero miner May 28, 2017
Cryptocurrency Mining and Markets Cryptocurrency based on Hyperledger Dec 4, 2018
Cryptocurrency Mining and Markets Cryptonight V7 and cpu mining Jun 9, 2018
Cryptocurrency Mining and Markets Zen+ at Cryptonight mining Apr 23, 2018

Share This Page