Newb to both building a server rig and crypto mining: I have lots of questions...

[MLG-443]

Hi. Recently I got an idea stuck in my head that I kinda grew fond of. It is to try and build a new "PC" (in quotations because it'll likely be a server) that will pay for itself (hopefully entirely) with crypto mining (CPU-favored mining algos like RandomX). With the existence of dedicated crypto miners on the market, this has been demonstrated to be a difficult task but I believe I just struck gold with rather great deals on AMD EPYC cpus (AMD EPYC 7K62 CPU Unlocked 2.6-3.3 GHz 48 Cores (OEM version of AMD EPYC 7642 ) | eBay) and a dual SP3 socket motherboard that doesn't cost an arm and a leg (the H11DSi). The whole rig might cost up to 2000 CAD before taxes but I don't think it'll go much over that.

Motivations:
Based on the validated benchmarks of XMRig (RandomX Benchmark - XMRig), a pair of my CPU of predilection can easily achieve a hash rate in excess of 70kH/s on RandomX and according to crypto profitability calculators with the (rather cheap) energy cost in my area, I should break even in 4 to 5 years. This is without accounting for the heating cost I will "save" (replace heating with crypto mining) during the winter. I don't want this rig just so it can turn a profit but it is a very big plus and even if it doesn't break even, the crypto mined will still count as a HUGE deferred rebate on a rig that is not that crazy expensive nowadays. The other reasons are first a concern for my 10+ years old AMD bulldozer rig still acting healthy (almost) all the time that I worry might kick the bucket at any moment but performance is not really an issue (yet). I use this old rig mostly to work from home, watch YT vids and do other web browser tasks (it's the one I am using right now). I have no time nowadays to play games on it but NGL.... ....just the idea of having a 96-cores 192-threads rig at my disposal when the most powerful CPU I've ever truly owned is a fake hexa-core sounds SO FレつケING LIT !

The memory:
It looks like the RAM will cost a major part of the whole budget since these 96 cores will need an equally breathtaking amount of memory bandwidth to mine efficiently so I'd like to cut cost wherever I can but I'm having doubts. Apparently, AMD EPYC cpus do support non-ECC RAM but I am having serious doubts that the motherboard H11DSi will allow it. Also, Idk what the people who say ECC RAM is cheaper than non-ECC one are smoking. This has NEVER been true in my entire online-shopping life and today is no different. I would really like to use plain and cheap desktop RAM if I can but I am wary this might be a bad idea. So my questions pertaining to the memory are as follows:
1. ECC or non-ECC RAM with EPYC 7K62 and H11DSi ?
2. How much does RAM bandwidth matter for XMRig and how much do I need to not turn it into a bottleneck ? I'm hesitating in the 2400 MHz to 3200 MHz range.
3. Is having 6 channels populated with 3200 MHz RAM going to perform exactly the same as having 8 channels populated with 2400 MHz RAM because 6x3200 = 8x2400 or is there some reason unknown to me for which that might not be true ? Total amount of GBs of RAM doesn't matter. It's gonna be too much anyway. I'd only do it for the bandwidth.
4. 1Rx8 or 2Rx4 or it doesn't matter ? Any other thing I might need to know ?

The cooling:
The cooling didn't worry at all me until I read about the overheating VRM of the H11DSi. Now I'm not as confident in my selection but any other dual SP3 mobo is prohibitively expensive.
1. Is there some heatsink upgrade or even a water-cooling solution including the VRM I can buy to keep temperatures at a safe level ? I doubt the choices of water cooling solutions is as extensive for servers as it is for desktop PCs.
2. If I adopt the default fan+heatsink solution for this kind of rig, how loud can it be ? I'm not looking for something actually quiet but it kind of matters how loud it's really going to be.

The PSU:
Does this need a special power supply unit ? I think I already have a PSU that can do the job because it is a 1200W monster that has the 24-pin connector plus 2 8-pins CPU power connector. Is that all that is required ?

The case:
Is any E-ATX compatible case with good air flow sufficient ? This one (Montech AIR 903 BASE, E-ATX Mid Tower Case, High Airflow with Max Capacity, 3x 140mm PWM Fans Pre-installed, Tempered Glass Side Panel, Mesh Front, TYPE-C, Support 4090 GPUs, White - Newegg.com) doesn't seem too expensive.

The peripherals:
I rarely care about fluff and 25000 USB ports and other add-ons they usually put into consumer mobos but no audio is rather barebone if you ask me, lol ! Is the audio function of audio-video outputs from normal graphic cards fully functional within the graphic card alone or could I possibly need to purchase something extra just to get audio from this titanic machine ?

The OS:
Can I put any OS in there ? SHOULD I put any OS in there ? What's the advantages of a Windows "Server" Edition ? My only significant life experience with linux is the normie linux mint with cinnamon desktop btw and I would favor the kind of OS that will give me less trouble.

Power efficiency:
And last but not least, will the current combination of CPUs and mobo allow me to underclock and undervolt the CPUs ? I imagine great benefits from that including reducing the temperature, power consumption and component aging a great deal for what I believe will be a far lesser sacrifice in hash rate and I would really want to have the option to make my rig work in this more power efficient regime from time to time. In theory, any undervolting should increase the profit margin from mining crypto due to reduced energy cost per hash computed, right ? But I don't think the barebone mobo will give me that option so can any of you tell me if this is achievable regardless ?

Overall, is this idea sound ?
Any other pitfall I should be wary of ?

 

[Sean Ho]

To answer a few of your questions:

H11DSi uses DDR4 RDIMM (and LR, and 3DS); this has ECC built-in. It will not take desktop UDIMMs.

If RAM speed is more important than capacity I'd consider 16x 16GB 3200 sticks (or even 8GB sticks) to fill the memory channels. Yes, multiple channels are interleaved, a bit like RAID0. If budget is tight and latency is more important than bandwidth, 2400 would also be fine, as the CAS latency in ns is generally about the same. I am not familiar with XMRig's requirements.

Look for hardware revision 2.0 or later for Rome support.

VRM cooling would be a DIY thing, but it's certainly doable if you find that to be a bottleneck. The board+CPUs don't have "default" coolers; rackmount servers tend to use passive heatsinks together with banshee case fans that are powerful enough to lift small pets. But if you have the clearance to fit a big active tower cooler, it can be very quiet with the right fans (Arctic, Noctua, etc). The case is relevant, too; big towers can sometimes be worse for cooling due to air moving through the path of least resistance, leaving pockets of relatively slow-moving air in the case. Prebuilt tower workstations (e.g., Dell T-series) tend to have fancy airflow design and custom shrouds to optimise this.

750W PSU is probably sufficient if there won't be much else on the system. 1200W is fine.

You can run any OS you like; Linux would be appropriate for a server that mostly sits headless in the corner doing its thing. ssh in to administer it.

Underclocking/undervolting is generally not exposed by BIOSes on server boards.

Overall, if your primary purpose is to turn a profit from mining, I do not think it would be a prudent investment. But there can also be a lot of fun and experience gained from building a server like this; it's your money and your decision.

 

[MLG-443]

Yeah, it's not really a investment decision more than how can I get the most with the least money spent/lost in the end. Computers are rapidly depreciating assets.

I was also thinking 750W would be enough but PSUs of that caliber tend not to have 2 8-pin power connectors for the CPUs.

"Underclocking/undervolting is generally not exposed by BIOSes on server boards." Does that mean there is no way to undervolt the CPU or just that it will be harder or less straightforward ?

 

[MLG-443]

Or here's another idea, can CPUs be forced into a certain power state that is not the highest (which I think is called P0) ? You know how regular desktop PCs have dynamic control over their own voltage and frequency for optimization of the power consumption ? Does that happen with EPYC cpus and can it be controlled ?

 

[ca3y6]

Just built a similar server (same two CPU and motherboard also bought on ebay). I bought some Noctua NH-U9 TR4-SP3 fans hoping it would be silent. That was without counting that under load, every time the CPU gets throttled a little bit, the motherboard emits a loud beep. I haven't found a way to get rid of the beeps but I will take any suggestion! In any case plan for some strong cooling.

Also for the computer case, keep in mind the cpu fans will be blowing up instead of toward the back (due to orientation of server vs desktop sockets). So better to take a case that has a ventilation grid at the top (or even better a fan you can use to extract hot air).

And something you might or might not care about: most cases that claim to be compatible with EATX will require to sacrifice the internal 3.5 drives bays to make room for such a large MB.

The server consumes about 100w idle.

 

[MLG-443]

Thanks. is the VRM temperature really bad under load or is the CPU cooling a bigger concern ? I read a comment somewhere that apparently the VRM tends to overheat on this mobo.

 

[rtech]

xmrig likes low latency high frequency memory and you should fill up all DIMM slots for maximum perf gain.

 

[ca3y6]

Thanks. is the VRM temperature really bad under load or is the CPU cooling a bigger concern ? I read a comment somewhere that apparently the VRM tends to overheat on this mobo.

In the health event log, I only see warnings about the CPU being throttled, nothing related to VRM.

Will perhaps try a bunch of NH-U12S TR4-SP3 fans that have a significantly higher advertised TDP but not sure they fit in my chassis.

 

[MLG-443]

xmrig likes low latency high frequency memory and you should fill up all DIMM slots for maximum perf gain.

Thanks, I think that's what I'll do if my offers are accepted.

 

[Sean Ho]

Lower cTDP limits can be set to optimise for cooling, I suppose that'd be the modern equivalent of underclocking.

 

[MLG-443]

Lower cTDP limits can be set to optimise for cooling, I suppose that'd be the modern equivalent of underclocking.

Well, even if undervolting at will can't be done, at least I'll have this... Even if that probably doesn't make the mining more efficient.

 

[rtech]

cTDP might make mining more efficient from standpoint of cost per hash

 

[ca3y6]

The NH-U12S TR4-SP3 fans seem to have done the trick, even under near max load the CPU temperature remains around 65 deg celcius. Though throttling still happens but more rarely.

 

[MLG-443]

The NH-U12S TR4-SP3 fans seem to have done the trick, even under near max load the CPU temperature remains around 65 deg celcius. Though throttling still happens but more rarely.

Can you check VRM temp ?

 

[ca3y6]

Can you check VRM temp ?

A little over 80 deg celcius

 

[MLG-443]

A little over 80 deg celcius

Alright, I'm pulling the trigger. This build is happening ! Exchange rates have been favorable to CAD lately and I think I have a good enough deal all around to justify the cost with all the coupons and stuff. Let's hope the crypto currencies I will be mining wont crash at the wrong time.

 

[ca3y6]

So in the end the only way I found to eliminate the beep is to cap the CPU usage to 90% in the WIndows Advanced Power Options. 90% of 96 cores / 192 threads is still good enough.

 

[MichelleVern]

If you're looking for pre-built mining rigs or quality components to get started without the hassle of sourcing everything separately, mineshop.eu has a decent selection. They specialize in mining gear, so it's worth checking out if you'd rather not spend too much time piecing everything together. I’ve used them before for GPUs and risers, and the shipping was reliable. Power consumption is another big thing to watch—undervolt where possible.