Dual Epyc Poker Solver Workstation

[lost_newb]

Build’s Name: Aces Wired
Operating System/ Storage Platform: windows
CPU: Dual Epyc 7551 ES
Motherboard: Supermicro H11DS-B
Chassis: Bench table
Drives: whatever I have lying around
RAM: TBD
Add-in Cards:
Power Supply:TBD
Other Bits: Will watercool the chips as I have boxes full of fittings rads and a few pumps.

Usage Profile: Running Poker Sims…probably with some form of remote access

Other information:

Greetings all

I apologize in advance for this wall of text but I think some background is in order.

I’ve committed to this build and I am way, way, way out of my depth. Hoping I can get some help. Not looking to be completely spoon fed, as I’m putting several hours a day into learning what I can on the interwebs but definitely in need of some pointers.

I am a VERY experienced PC builder. My first PC build was done with my dad in 1983: plugging 16k RAM chips into an 8088 motherboard from Jameco. Those early systems probably cost us under $2k to build vs $3k for the same spec Compaq Portable or $5k for the IBM PC XT (and this is 1983 dollars, so substantial savings).

Over the next 15 years I built well over 100 PCs. Largely general use office PCs built around AMD 5x86 and Cyrix 6x86 systems for my dad’s rapidly growing accounting firm. The occasional server or high end graphics editing build, but those were rare.

By the late 90s the costs of the OS (no $3 activation keys back then) and the value in bundled software were squeezing my margins. Factor in warranty/reliability issues and my devotion to weed and Phish concerts, and any early similarities I held to Michael Dell went out the window. Over the past 20 years my builds have been 1 personal high end rig every couple of years and helping out friends and family.

My current personal build is an ultra gaming rig (even though at 47 I’m a mediocre at best gamer) with a i9-9900ks and a 2080ti on a Z390 Dark. For everyday use I’ve got the chip running all cores at 5.2Ghz at 1.344 volts and I’ve got 16Gb of RAM overclocked and super duper stable at 4400-16-16, with ultra tweaked secondary and tertiaries (though that's still a work in progress for very diminishing returns...just principle at this point). It's taking a hair more DRAM voltage than I'd really like so may scale back.

So i'm in at least the 95th percentile at PC building. But I’m getting into uncharted territory now.

A buddy of mine is a semi-professional poker player and he runs a solving program called piosolver. His budget has him limited to running a 3700k and I spent a lot of time getting his 64Gb RAM tuned for him (he’s got decent PC hardware skills but I'm better at RAM tuning). He showed me the basics of his software and I looked into it a bit. Apparently a lot of pros are running this thing on rented dedicated servers as it scales really well with additional cores and loves loves loves tons of RAM (like Terabytes of RAM). He’d been talking to me about helping him build a 3950x system with 128Gb and saying someday he hoped to go Threadripper so that build has been in the back of my mind for awhile now.

Last week my cousin told me he could loan me, for an unspecified amount of time, 2 Epyc 7551 Engineering Samples he has access to at work. I immediately thought of my poker buddy and after some discussion we’re going ahead with the build.

I’m COMPLETELY out of my depth, but hopeful I can muddle my way through.

We’re trying to complete this build as cheaply as possible. While we think we’ll have access to the chips for at least 6 months they can get yanked back at any time. So we want to buy other components that, if necessary, we can unload on ebay at as close to cost as possible.

My first sticking point is RAM. 72 hours ago I knew nothing about server RAM, and as of this writing I’m only *SLIGHTLY* more knowledgeable. So here are my questions and I apologize for how elementary some of them may be.

I’ve decided to initially get going with 256Gb and I’m looking on ebay for server pulls. Should I be looking at 16x16? 8x32? Something else?

In my gaming builds I exclusively use Samsung B-die. For these ECC is there anything like that? i.e. a specific chip I should either expressly be looking for or avoiding? I’m generally agnostic as to who is slapping the DIMMS together though I do tend to stick with recognizable names and avoiding the geil/oly stuff. Is it the same for server chips or does somebody do a particularly good or bad job of throwing the modules together?

How mix and match is the memory configuration going to be? I assume I can fairly freely add different capacity DIMMs, but if I change speeds they’ll run at the lowest speed?

For instance, If I were to start with 16x16Gb and then came across a smoking deal on some 64Gb Dimms could I pull 2 of the 16Gbs and run each CPU with 7x16Gb and 1x64Gb?

Would I be better off starting with 8x32Gb? Then I could just add rather than swap out and run each with 4x32Gb and 1x64Gb? Any issues with populating 5 out of 8 slots per cpu? The whole channel thing has me a little confused but hopefully I'll figure that out shortly.

Anything in particular I should be looking for in the RAM besides just registered ECC?

There’s an ebay listing for some 128Gb sticks that are engineering samples. I’m willing to roll the dice on ES for the CPUs because there’s no arguing with the price/performance when the numerator is zero but would rolling the dice on ES DIMMs off the bay just be crazy and/or stupid?

Finally a quick question on PSU. I’ve got a whole bunch of ATX PSUs lying around and am certainly planning on using one while this is on the bench. When I get it into a chassis would a server PSU be a worthwhile investment? I read somwhere they can have beefier chokes/caps etc. Is it worthwhile?

I’d also appreciate any links to resources I should be reading. Definitely looking to learn to fish rather than being handed a fish.

Thanks so much in advance, and again, my apologies for the rambling and the elementariness of the questions.

 

[BlueFox]

Each CPU has 8 channel memory, so you will want 16 DIMMs, or multiples of that. When it comes to server RAM, you're really only going to see name brand stuff, primarily Hynix, Micron, and Samsung. You'll want 3200mhz RAM for maximum performance and stick to one DIMM size. I would avoid ES RAM personally, but I have no first hand experience.

 

[TLN]

Professional poker player but only have budget for 3700x against other poker player with TERABYTES of RAM on dedicated servers? LOL
The only industry where such stuff(terabytes of memory/overclocked CPU) might be helpful is HFT, but usually those guys don't care about pricing. Can't think of anything else quite honestly.

Other Bits: Will watercool the chips as I have boxes full of fittings rads and a few pumps.

I am a VERY experienced PC builder.

When I get it into a chassis would a server PSU be a worthwhile investment? I read somwhere they can have beefier chokes/caps etc. Is it worthwhile?

 

[lost_newb]

Professional poker player but only have budget for 3700x against other poker player with TERABYTES of RAM on dedicated servers? LOL
The only industry where such stuff(terabytes of memory/overclocked CPU) might be helpful is HFT, but usually those guys don't care about pricing. Can't think of anything else quite honestly.

Please note that I said "semi-professional". He makes a living at it, but we're talking about a comfortable living for a kid in his 20s, And as I understand it he's relatively new at utilizing the solver. On his PC build he runs the solvers for 12-24 hours and then studies the results: if we can knock that time down by 60% let him run more sims allow him to use his 2700 strictly for call of duty I'll be happy.

I'm vague on this but my understanding is that multi-terabyte RAM use cases are if one were trying to run sims in real time with complicated pre-flop "trees." Again...the poker stuff is really outside of my ken.

We're not looking to compete against the high stakes pros sitting in front of $50k worth of hardware solving n real time....we're looking to outperform the guys renting a xeon online for $150/month running sims in the background and I would think that a dual 7551 should beat the snot out of a xeon 2630, right?

Maybe I'm wrong and the solutions are more RAM bound than core/thread bound, but I'm open to some experimentation.

Maybe this whole thing is a fool's errand which is why I'm not planning on dropping $20k into RAM to get this thing running on the bench.

If i drop a couple of grand and it doesn't work out and I resell the shit for $1500 I'll definitely have gotten $500 worth of entertainment out of the project.

 

[lost_newb]

Each CPU has 8 channel memory, so you will want 16 DIMMs, or multiples of that. When it comes to server RAM, you're really only going to see name brand stuff, primarily Hynix, Micron, and Samsung. You'll want 3200mhz RAM for maximum performance and stick to one DIMM size. I would avoid ES RAM personally, but I have no first hand experience.

Thanks for the prompt response.

I suppose I should have been a little clearer. When I said I'd settled on 256GB...that's what I envision as my initial set-up. Ultimately, if this thing works, I'd like to significantly upgrade the RAM.

Obviously most people running dual EPYCs don't have the budget limitations I do, and I'm sure I'll end up spending far more than I'd hoped before I headed down this rabbit hole, but I am to some degree, budhet constrained.

Lets say, hypothetically, that within a couple of weeks of getting this thing running with 256 Gb I'd want to go to 512Gb.

Performance wise I'd clearly be better off starting with 16x16 and then upgrading to 16x32 staying in an 8 channel config the whole time/

But from price/performance I'd imagine I'd be better off starting off with 8x32GB and then adding another 8x32Gb in a month?

I suppose questions regarding quad channel vs octal channel performance would be better directed at the software developer but would appreciate any thoughts you might have.

Also, you said "stick to one DIMM size." I'd been under the impression that different capacity DIMMs could be mixed and matched and thought that might be a means of going about a gradual upgrade path. Am I misinformed? Am I just asking for trouble mixing capcities? I'm fully prepared to accept your advice on sticking to 1 size, but would just like to understand the why.

Thanks

 

[ari2asem]

for memory stick speed i wouldn't go higher than 2666mhz, because your cpu is the limiting factor. epyc naples (your 7551 es) supports memory at 2666mhz. maybe you can put 3200-stick and get POST-ed, but don't expect you will profit of 3200mhz speed. your cpu memory controller is limited at 2666mhz.

another point for memory...per cpu you need at least 4 sticks. or a little better 8 sticks per cpu.

if you need that much amount of memory, then you could go for LRDIMM. LRDIMM are at max 128gb per stick supported on H11DSi. so, 128gb * 16 stick = 2 tb ram.
LRDIMM is load reduced dimm, they are ecc sticks.

good luck

 

[lost_newb]

for memory stick speed i wouldn't go higher than 2666mhz, because your cpu is the limiting factor. epyc naples (your 7551 es) supports memory at 2666mhz. maybe you can put 3200-stick and get POST-ed, but don't expect you will profit of 3200mhz speed. your cpu memory controller is limited at 2666mhz.

another point for memory...per cpu you need at least 4 sticks. or a little better 8 sticks per cpu.

if you need that much amount of memory, then you could go for LRDIMM. LRDIMM are at max 128gb per stick supported on H11DSi. so, 128gb * 16 stick = 2 tb ram.
LRDIMM is load reduced dimm, they are ecc sticks.

good luck

Thanks for that. I'd looked at my mobo specs but not the EPYC's. doh. Should amount to some savings and increase availability going down to 2666.

I've decided to just step up and go with a 16x32Gb initial config.

I'm very intrigued by some of the offerings on ebay by a seller called mem-store. The prices seem too good to be true, and my experience with those situations tends to be disappointing. But they've got 99.9% positive feedback. Anyone have any experience?

I'm specifically looking at this kit:
512GB 16x32GB DDR42666Mhz RDIMM Memory TSV for Giga Bytes MZ31AR0 AMD 7351p

512GB 16x32GB DDR4-2666Mhz RDIMM Memory TSV for Giga Bytes MZ31-AR0 AMD 7351p | eBay

kinda weird that the title lists Epyc but the description only references xeon.

There's a thread here on these (or highly similar) sticks.

https://forums.servethehome.com/index.php?threads/32gb-ddr4-2666-rdimms-85-ea.25203/

A guy in august couldn't get them to post on a h11ssl-nc, but someone else reported success in march using their 16x64Gb on an h11dsi.

As tempted as I am by the price, with my lack of experience I think I'm just begging for trouble using private label RAM.

open to feeback though and thanks everyone for all the help so far

 

[ari2asem]

i wouldn't buy that TVS memory. maybe very attractive price, but to many ppl have problems with these sticks.

i would avoid it if i was you