Dedicated Crypto Mining Open Compute server build.

[zreon]

Is this with the 2665's?

The second node install is going much faster.

I quickly tested Ubuntu 17.10 server mining Aeon on bare metal on the one node and its about ~150 H/s faster.

 

[Klee]

Is this with the 2665's?

Yep, now i'm installing on my 2660 nodes as I type this.

 

[Klee]

I developed an install procedure to speed things up.

1. Install a video card , keyboard and usb dvd drive.
2. Boot up and go into bios and select "UEFI Slimdvd drive" as the boot device (your device might be named differently).
3. Save and then reboot and start the install with the Ubuntu 17.10 server dvd.
4. When I got to the partition editor screen I selected "Manual"
5. I then selected the partitions I had created on the install from the other partitions now.
350 Mb UEFI dedicated partition
246 GB ext4 "/" partition
4.0 GB swap partition.
6. Continued the install and only selected "openssh server"
7. Finished the install and rebooted, and made sure it actually booted and was at the login prompt on the monitor.
8. Logged in via ssh from my main pc.
9.To fix reboot hang in newer versions of Ubuntu type this command :

echo "options mei-me disable_msi=1" | sudo tee /etc/modprobe.d/mei-me.conf

Next enter sudo password when prompted.

10. To keep from getting this error after every reboot:" Error: MEMORY ALLOC FAILED: mmap failed" with stak edit:

sudo nano /etc/sysctl.conf

Add this to the end:

vm.nr_hugepages = 256

11.Next edit:

sudo nano /etc/security/limits.conf

Add these two lines at the end:

* soft memlock unlimited
* hard memlock unlimited

NOTE: Probably not a good idea to use "memlock unlimited" on a normal install but this server is mining only and will not have any other user except for me and I will only run one miner program and so far it seems to work fine.

12. Reboot

 

[Klee]

12. If it fails to reboot, power off then power back on.

I've had to enter this command again on about half the Ubuntu installs as it does not always "stick" the first time for some reason.

echo "options mei-me disable_msi=1" | sudo tee /etc/modprobe.d/mei-me.conf

13. Reboot again if you did the above.
14. Log in via ssh from your desktop.
15. Update Ubuntu then reboot again.
16. Download and compile what ever miner you are using.
17. Configure the miner config files and then start mining.

If your running Docker you can install htop for a easy to use program you can use logged in via ssh to see that the miner is working vs using multiple docker commands.

Also you can install lm-sensors to remotely check the cpu temps.

 

[Klee]

Tips for the second node and additional nodes if you have more than one open compute server.

For the second node its a little faster.

1. Follow steps 1-15 above.
2. Install ALL required dependencies that were needed in the first install on the first node for the mining programs on this node. I'm using xmr-stak-cpu and xmrig as of right now.

xmr-stak-cpu:

sudo apt-get install libmicrohttpd-dev libssl-dev cmake build-essential libhwloc-dev

xmrig:

sudo apt-get install git build-essential cmake libuv1-dev libmicrohttpd-dev

3. Next install Filezilla on your main pc. (Makes coping files via sftp easy.)
4. Log in to the node with Filezillz and copy all the miner programs folders from the first node over to your main pc.
5. Go to the folder where you copied the miner folders from the first node.
6. Edit each config file as needed if you need to add a worker name.
7. Copy the miner folders over to the second node from your main pc.
8. Go into each miner folder and run the miner programs to make sure they work, they should and if any error out you need to make sure that you did install all the required dependencies.

Should take about 35-45 minutes per node, which is faster than trying to clone then edit each drive.

 

[Klee]

I did this because I just bought 12 identical used 250 gig WD hard drives to get rid of the range of sizes that I was using, from 80 gig to 1 tb, and I upgraded from Ubuntu 17.04 to 17.10.

 

[onsit]

Have you considered adding a 1050ti or an RX 550 into the equation, both of them are very efficient in terms of hash / watt? Assuming that PCIe slot is delivering full 50w.

 

[Klee]

Have you considered adding a 1050ti or an RX 550 into the equation, both of them are very efficient in terms of hash / watt? Assuming that PCIe slot is delivering full 50w.

These servers seem to have issues with booting with newer video cards.

 

[onsit]

Based on the $199 Quanta thread seems that was a result of Video output, I ended up ordering two of these Wiwynn i'll give it a go once I have them setup as headless, maybe try using PCI-e risers I have laying around.

I ended up going the transformer route, but even then I fear I will be maxing out my dinky 20A 120v outlet that is already powering my Home Server rack that has a pfSense box, an R710, and a GPU mining rig.

That magical outlet you found, is it a simple 12-2 single phase? Or is that a fancier 12-3 setup that was used for a Washer / Furnace?

 

[Klee]

Based on the $199 Quanta thread seems that was a result of Video output, I ended up ordering two of these Wiwynn i'll give it a go once I have them setup as headless, maybe try using PCI-e risers I have laying around.

I ended up going the transformer route, but even then I fear I will be maxing out my dinky 20A 120v outlet that is already powering my Home Server rack that has a pfSense box, an R710, and a GPU mining rig.

That magical outlet you found, is it a simple 12-2 single phase? Or is that a fancier 12-3 setup that was used for a Washer / Furnace?

It's a NEMA 6-20 20A 250v outlet for a window a/c unit.

 

[onsit]

Darn, so that means the opencompute PSU can actually run on 240v single phase. Nema 6-20 is single phase usually. 2 hots, and a ground no neutral.

Not sure why Potential Deal: 2 x Dual 2011 nodes @$199, Quanta Openrack made it sound like you need a transformer or 208 3 phase. Considering that a transformer essentially gives you 240v single phase.

 

[Titan]

@Klee Why dont you change to mining other coin instead of XMR? because as I have checked, sumokoin has better profit than XMR coin atm.
And I decide to buy a Wiwynn system in CN because it's cheaper and easier to ship to me.
Sorry for bad English

 

[onsit]

@Klee Why dont you change to mining other coin instead of XMR? because as I have checked, sumokoin has better profit than XMR coin atm.
And I decide to buy a Wiwynn system in CN because it's cheaper and easier to ship to me.
Sorry for bad English

XMR seems like a better currency to hold, in his OP he did state he does this to hold. Long term vs weekly payment scenario.

 

[Joel]

XMR seems like a better currency to hold, in his OP he did state he does this to hold. Long term vs weekly payment scenario.

Sure, but if you want to hold xmr just exchange. Even with the transfer-conversion-transfer costs you'd probably come out ahead of mining XMR.

 

[Marsh]

I want to hold XMR long term . This is the exact reason , why I am "holding my noise", still mining XMR.

Pool hashrate jump from 40MH/s to 90Mh/s with SupportXMR.com for the last few days, I just switched from minexmr.com to supportxmr.com last week.

 

[madLyfe]

I want to hold XMR long term . This is the exact reason , why I am "holding my noise", still mining XMR.

Pool hashrate jump from 40MH/s to 90Mh/s with SupportXMR.com for the last few days, I just switched from minexmr.com to supportxmr.com last week.

good decision. solid operation there.

 

[polyfractal]

High-five OCP mining buddy I setup a 4-node cluster a few years ago for work-related activity, and have them mining when idle. Each node has x2 E5-2670 and puts out ~945 h/s on cryptonight using auto-config xmr-stak. For cryptonight-lite they do around ~3200 with xmrig.

I haven't looked at power stats recently, but a 100% synthetically-loaded node eats ~290 watts...and cryptonight doesn't seem to load the cores as fully so I'd guess it's slightly less. Which probably puts these somewhere around ~3.5 H/watt.

Haven't really had time to do much tweaking since I don't have enough compute to really make a big profit, but it's a fun diversion on the side

 

[hifijames]

I want to hold XMR long term . This is the exact reason , why I am "holding my noise", still mining XMR.

Pool hashrate jump from 40MH/s to 90Mh/s with SupportXMR.com for the last few days, I just switched from minexmr.com to supportxmr.com last week.

Is there a docker image that can easily switch pools?

 

[Indecided]

High-five OCP mining buddy I setup a 4-node cluster a few years ago for work-related activity, and have them mining when idle. Each node has x2 E5-2670 and puts out ~945 h/s on cryptonight using auto-config xmr-stak. For cryptonight-lite they do around ~3200 with xmrig.

I haven't looked at power stats recently, but a 100% synthetically-loaded node eats ~290 watts...and cryptonight doesn't seem to load the cores as fully so I'd guess it's slightly less. Which probably puts these somewhere around ~3.5 H/watt.

Haven't really had time to do much tweaking since I don't have enough compute to really make a big profit, but it's a fun diversion on the side

Same here. Curiously, I'm running the same setup (2x E5-2670 C2) and on xmr-stak i'm only getting approx 868 h/s with 18 threads running vs your ~945 h/s. Any further increase of threads by even 1 brings the hashrate down to ~700 h/s. Do you mind sharing what your config looks like and/or any other tweaks you've done with them?

 

[polyfractal]

Are you running your nodes headless? Three of my nodes run headless and get the 945 number, the fourth node doubles as my desktop (I VNC into it, so it has an X server running, etc) and it trends closer to 850-900 h/s. It's a weird node though, since it's also a proxmox node and the deskto/miner are running in one of the LXC containers, which probably doesn't help.

For the headless nodes:

• Ubuntu 16.04.3 LTS
• Compiled xmr-stak with gcc/g++ 7, no Cuda/OpenCL support
• Huge pages enabled
• Post-spectre/meltdown patch :/
• Auto-configuration from xmr-stak, looks like it's using 20 of 32 threads:

"cpu_threads_conf" :
[
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 0 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 1 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 2 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 3 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 4 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 5 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 6 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 7 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 16 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 17 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 8 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 9 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 10 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 11 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 12 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 13 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 14 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 15 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 24 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 25 },

],