ES Xeon Discussion

[Andrix]

I have never messed with ES Xeons before (only retail units professionally), and I foolishly and made a purchase off ebay.

I purchased two Xeon 8592+ ES "Q2SR+" and a Gigabyte MS73-HB1 motherboard. In reading this thread, it is clear that I should have done a bit more research before I clicked the buy button. Oh well, live and learn right?

Well, you are clearly struck by the buyer's remorse and likely to change your mind about the purchase at least once.

I am guessing that the "Q2SR+" means that they could be any stepping after the Q2SR, and that is not a stepping.

"...Q2SR+ Gigabyte MS73-HB1..." most likely lacks a space and should be read as "...Q2SR and Gigabyte MS73-HB1...". You get to see some slopiness from hardware sellers on ebay quite a bit. In other words, I tend to believe that you bought specifically Q2SR and there is no such thing as Q2SR+.

I went with these CPU's for the on die AMX

And you should be getting it with Q2SRs unless they are broken. I elaborated on it in another thread.

So, How boned am I here? I know... I'm a dumbass that click buy before doing my research. Is this salvageable with modded bios, etc; or do I need to go shopping again?

I wouldn't be surprised if the motherboard arrives with already modified bios and the combination works fine as it. If it doesn't, you're sorted out apparently.

 

[RolloZ170]

I wouldn't be surprised if the motherboard arrives with already modified bios and the combination works fine as it.

i want to make clear that a mod. BIOS is not required, only if you want to use the actual official BIOS.
the mod. BIOS or the how to should have arrived in China by now, thought.
i have not kept it secret, but delayed to keep the EMR A0 ES prices low as long as possible.

 

[GadflyII]

Well, you are clearly struck by the buyer's remorse and likely to change your mind about the purchase at least once.


"...Q2SR+ Gigabyte MS73-HB1..." most likely lacks a space and should be read as "...Q2SR and Gigabyte MS73-HB1...". You get to see some slopiness from hardware sellers on ebay quite a bit. In other words, I tend to believe that you bought specifically Q2SR and there is no such thing as Q2SR+.


And you should be getting it with Q2SRs unless they are broken. I elaborated on it in another thread.


I wouldn't be surprised if the motherboard arrives with already modified bios and the combination works fine as it. If it doesn't, you're sorted out apparently.

After speaking to @RolloZ170, I am much more hopeful, I think this is going to work really well, If they do I am immediately buying a second setup, each will allow me to run 6 GPU's with 128 cores of AMX, two servers would let me run one training batch every ~22 days. For contrast, running one training batch with hosted setups would take ~30days and each batch costs about $50,000 - $80,000 in hardware fees. (which I obviously can't afford )

If anyone has any suggestions on where to find some cheap used memory. I would appreciate it. Looking for at least 8 modules to start, and will expand to 16 later. Most like go with some 48GB modules. I could most likely get away with 8x 32GB per socket, but it might be a little tight.

 

[Techiedudeon]

Just threw together a MS03-CE0 and Q2SR , Rolloz170 provided the bios, using it as a Debian firewall, no issues, upgraded from a dual 8280 machine. Max Steam downloading before would peak at 2.0/2.1GBps on a 3GBps internet pipe, now peaks at 2.8/2.9GBps

Using same nics as the dual 8280, watercooling the CPU, only 64GB of ram to start 4x16

Been up for a few days, no issues

Running with: for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n performance >| $CPUFREQ; done

Just testing stability with performance set in debian

Again no issues.

I picked up two DP Setups as well, will be testing a DP Plex Headend with a RTX 6000 Pro Blackwell (600watt version) for a static plex machine that will just connect to back end storage over dual connectx 7 dp 100gbits, not really pushing but stability will be interesting.

 

[custom90gt]

Just threw together a MS03-CE0 and Q2SR , Rolloz170 provided the bios, using it as a Debian firewall, no issues, upgraded from a dual 8280 machine. Max Steam downloading before would peak at 2.0/2.1GBps on a 3GBps internet pipe, now peaks at 2.8/2.9GBps

Using same nics as the dual 8280, watercooling the CPU, only 64GB of ram to start 4x16

Been up for a few days, no issues

Running with: for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n performance >| $CPUFREQ; done

Just testing stability with performance set in debian

Again no issues.

I picked up two DP Setups as well, will be testing a DP Plex Headend with a RTX 6000 Pro Blackwell (600watt version) for a static plex machine that will just connect to back end storage over dual connectx 7 dp 100gbits, not really pushing but stability will be interesting.

Out of total curiosity, why are you transcoding on a 9k GPU?

 

[Techiedudeon]

Out of total curiosity, why are you transcoding on a 9k GPU?

I like as good as I can possibly get I was using RTX 6000 quadros, but there was 3-5 second start latencies, tested a 4090, down to about 2, then a 5090 which downed it to almost instant (512GB DDR4 Transcode Ramdisk, all PCI 4 Raid 0s highpoint adapters), moving to all PCI 5 next week or week after once the new 2TBs of ddr5 shows up

Again, if you can why not, I found a both a 4090 and rtx 6000 quadro was a pos, if I can remove lag and latency, plus covers all the current codecs, why not? I have 12 6000 Blackwells sitting here at home atm to play with (Plan to build some AI sand boxes, but I want to build at least 2 with 4x Blackwells each).

OH at any given time I have 80-120 simultaneous playbacks, I let people access the library for fun, give content ideas etc, currently at over 1PB of data, a lot of 4k content, each to their own, I prefer "best" vs "good enough", what other peoples perceptions are, is not my reality

 

[custom90gt]

I like as good as I can possibly get I was using RTX 6000 quadros, but there was 3-5 second start latencies, tested a 4090, down to about 2, then a 5090 which downed it to almost instant (512GB DDR4 Transcode Ramdisk, all PCI 4 Raid 0s highpoint adapters), moving to all PCI 5 next week or week after once the new 2TBs of ddr5 shows up

Again, if you can why not, I found a both a 4090 and rtx 6000 quadro was a pos, if I can remove lag and latency, plus covers all the current codecs, why not? I have 12 6000s sitting here at home atm to play with.

OH at any given time I have 80-120 simultaneous playbacks, I let people access the library for fun, give content ideas etc, currently at over 1PB of data, a lot of 4k content

Hah sounds like you've got a fairly lucrative source of income to have that kind of money to play with hardware like that. I'd love to have that kind of hardware. Are you running spinning rust or is this all on SSDs? You really should have a build thread somewhere for us normal folk to drool over.

And 80-120 playbacks is pretty amazing. I was impressed when I hit 20 once lol.

 

[luckylinux]

Just threw together a MS03-CE0 and Q2SR , Rolloz170 provided the bios, using it as a Debian firewall, no issues, upgraded from a dual 8280 machine. Max Steam downloading before would peak at 2.0/2.1GBps on a 3GBps internet pipe, now peaks at 2.8/2.9GBps

Using same nics as the dual 8280, watercooling the CPU, only 64GB of ram to start 4x16

Been up for a few days, no issues

Running with: for CPUFREQ in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor; do [ -f $CPUFREQ ] || continue; echo -n performance >| $CPUFREQ; done

Just testing stability with performance set in debian

Again no issues.

I picked up two DP Setups as well, will be testing a DP Plex Headend with a RTX 6000 Pro Blackwell (600watt version) for a static plex machine that will just connect to back end storage over dual connectx 7 dp 100gbits, not really pushing but stability will be interesting.

What I'm mostly surprised about is that you need THAT much CPU power just to get 3 gbps Internet .

I mean ... ARM Dual Core Consumer Routers can quite easily manage 900-100 mbps (or close to it), do you really need a 64 Core / 128 Thread (granted it has a low Clock Speed but still) ?

My OPNSense is running virtualized in a Proxmox VE VM (without NIC Passthrough, just using Linux bridges) with an Intel Xeon E3 1240 v5 and can easily achieve around 800-900 mbps (not a huge Bottleneck, ISP Limit seems to be around 800-900 mbps rather than 1gbps).

 

[Techiedudeon]

Hah sounds like you've got a fairly lucrative source of income to have that kind of money to play with hardware like that. I'd love to have that kind of hardware. Are you running spinning rust or is this all on SSDs? You really should have a build thread somewhere for us normal folk to drool over.

And 80-120 playbacks is pretty amazing. I was impressed when I hit 20 once lol.

I tested with Qnaps, synologies etc, found they all kind of sucked, so for storage, I ended at Supermicro 4U SuperStorage Servers, picked up two, bought 60 15TB SSDs off ebay so I run two of those with DP 100GB uplinks to me Cisco Nexus 9504 core (which can go up to 400Gbit, but then I'm just wasting money for fun).

 

[Techiedudeon]

What I'm mostly surprised about is that you need THAT much CPU power just to get 3 gbps Internet .

I mean ... ARM Dual Core Consumer Routers can quite easily manage 900-100 mbps (or close to it), do you really need a 64 Core / 128 Thread (granted it has a low Clock Speed but still) ?

My OPNSense is running virtualized in a Proxmox VE VM (without NIC Passthrough, just using Linux bridges) with an Intel Xeon E3 1240 v5 and can easily achieve around 800-900 mbps (not a huge Bottleneck, ISP Limit seems to be around 800-900 mbps rather than 1gbps).

I can speed test anything and max out, enable BBR in debian and 90% of speed tests I get my 3GBps (Dual 3Gbps uplinks) and work has a dedicated 10Gbit installed but I dont push my personal stuff over that

I was one of those weird long ago earlier adopters of replacing ISP gear with my own, were talking back when we soldered NICs to make FTTH work, so my test cases are specific to "me" and people can disagree, example my old nics that I had to solder, sure i could achieve 3.2/3.3GBps throughput, but then test specific traffic (I like NZBGet, pretty much maxes everything out no matter what) but then try things like steam (my two main stations are a 9950x3d and a intel 285k, intel in this case loses and the AMD box for steam as an example pushes almost to the limit, I have 20k+ games so I don't like waiting more than a moment or two for them to download).

Also, test things like steam downloads, I know its stupid but also max our plex or live streams and see how far you get

And does it seem like a waste, again perceptions and realities, one of my users use to buy kaleidescape devices for their homes, we're not talking normal people money, were talking he spend "god like money on it" and quite frankly it was ok, but expensive and they didnt have good upgrade options and his Blu Ray library exceeded like 9000 disks, so eventually he couldnt keep his collections "life on demand" I offered plex, he was skeptical, built a demo in my house, he's loved it ever since (actually pays me to run a plex server at 100k a year for doing it), to which it was his idea to share to get more content ideas (luckily he paid for the install of a pair of generators and a huge UPS system lol).

Again, perceptions and realities, everyone's is different and considering his projects for his in house imax movie theaters cost more than Ill make in my life, he found it worth it to himself (example story)

OH so why use ES cpus, I use to just call up dell for servers and lease them, usually I'd get some good deals (my older R940XAs with 32x 15TB SSDs, from dell direct, I paid like 60k each), now I cant even get them to sell me something worthy for under 100-150k, so next best thing the leases alone don't make it worth the effort, before, the deals were good. Also noticed the dell setups have a lot of bus latency compared to a simple home build, might just be a me observation though.

 

[custom90gt]

I can speed test anything and max out, enable BBR in debian and 90% of speed tests I get my 3GBps (Dual 3Gbps uplinks) and work has a dedicated 10Gbit installed but I dont push my personal stuff over that

I was one of those weird long ago earlier adopters of replacing ISP gear with my own, were talking back when we soldered NICs to make FTTH work, so my test cases are specific to "me" and people can disagree, example my old nics that I had to solder, sure i could achieve 3.2/3.3GBps throughput, but then test specific traffic (I like NZBGet, pretty much maxes everything out no matter what) but then try things like steam (my two main stations are a 9950x3d and a intel 285k, intel in this case loses and the AMD box for steam as an example pushes almost to the limit, I have 20k+ games so I don't like waiting more than a moment or two for them to download).

Also, test things like steam downloads, I know its stupid but also max our plex or live streams and see how far you get

And does it seem like a waste, again perceptions and realities, one of my users use to buy kaleidescape devices for their homes, we're not talking normal people money, were talking he spend "god like money on it" and quite frankly it was ok, but expensive and they didnt have good upgrade options and his Blu Ray library exceeded like 9000 disks, so eventually he couldnt keep his collections "life on demand" I offered plex, he was skeptical, built a demo in my house, he's loved it ever since (actually pays me to run a plex server at 100k a year for doing it), to which it was his idea to share to get more content ideas (luckily he paid for the install of a pair of generators and a huge UPS system lol).

Again, perceptions and realities, everyone's is different and considering his projects for his in house imax movie theaters cost more than Ill make in my life, he found it worth it to himself (example story)

OH so why use ES cpus, I use to just call up dell for servers and lease them, usually I'd get some good deals (my older R940XAs with 32x 15TB SSDs, from dell direct, I paid like 60k each), now I cant even get them to sell me something worthy for under 100-150k, so next best thing the leases alone don't make it worth the effort, before, the deals were good. Also noticed the dell setups have a lot of bus latency compared to a simple home build, might just be a me observation though.

Hah that's quite an insane story. Makes sense now why spending 9k on a GPU for transcodes is within reason, same with having 60 15TB SSDs. As a physician, I'll never have that kind of money to burn on a system lol.

 

[Techiedudeon]

Hah that's quite an insane story. Makes sense now why spending 9k on a GPU for transcodes is within reason, same with having 60 15TB SSDs. As a physician, I'll never have that kind of money to burn on a system lol.

I have roughly 20 cards here to play and test with, it was about what was "fastest", it just won the race, I can probably accomplish the same with the Blackwell 4500 but just so happened to get 12 of the 6ks, though 90% of stuff is direct play, there's the odd time, depending on what device they're using, you run into transcoding regardless.

My lord of the brings blue rays are 160GB+ each for example, my "test subject" spend like 8 hours with me until he was able to play direct or transcode and not wait more than 1 second, its how he wanted it, but I mean if I had movie theaters in my house (were talking real ones not little ones) and used christie digital projectors (they cost $1M+ each), I'd probably be that picky as well

Stupid examples, the first plex head I built for him was a R940XA, 32Xssds (latest movies would be copied over to it, then copied over to the massive nas units after x amount of days), so all the latest stuff would be instant, but with the quad CPU setup (only bought because they didn't charge more for the quads over the duals, no idea why), there was a 2-3 second latency over a dual setup i had, personally think a SP beats out, but then the radarr and sonarr scans etc when running will use almost every core and causes 4-5 second latencies when running through the 12-14k movies and like 5000 tv series (tv series doesn't include reality tv, documentaries game shows, talk shows etc).

The move to PCI5 with the ES stuff seems to make sense to test if I actually get more from it and if things improved enough over the 82XX stuff that the upgrade is worth it if so, then buy the retail stuff and just build my own with water-cooling.

Picked up two Highpoint R7604A nvme cards, for just plex head end, have 2TBx4 in raid0 and another with 2TBx4 for plex only (Samsung 9100 Pros), should be an insanely bad ass setup, overkill and waste sure, but what would the bottle neck be then

Not sure what you do but I'm a double PhD, if we're talking money personally I did just take delivery of a new G63, a G580, Defender Octa and a new 2025 GTS 4 911 lol, so as I said, everyone's perceptions and realities are different

 

[custom90gt]

I have roughly 20 cards here to play and test with, it was about what was "fastest", it just won the race, I can probably accomplish the same with the Blackwell 4500 but just so happened to get 12 of the 6ks, though 90% of stuff is direct play, there's the odd time, depending on what device they're using, you run into transcoding regardless.

My lord of the brings blue rays are 160GB+ each for example, my "test subject" spend like 8 hours with me until he was able to play direct or transcode and not wait more than 1 second, its how he wanted it, but I mean if I had movie theaters in my house (were talking real ones not little ones) and used christie digital projectors (they cost $1M+ each), I'd probably be that picky as well

Stupid examples, the first plex head I built for him was a R940XA, 32Xssds (latest movies would be copied over to it, then copied over to the massive nas units after x amount of days), so all the latest stuff would be instant, but with the quad CPU setup (only bought because they didn't charge more for the quads over the duals, no idea why), there was a 2-3 second latency over a dual setup i had, personally think a SP beats out, but then the radarr and sonarr scans etc when running will use almost every core and causes 4-5 second latencies when running through the 12-14k movies and like 5000 tv series (tv series doesn't include reality tv, documentaries game shows, talk shows etc).

The move to PCI5 with the ES stuff seems to make sense to test if I actually get more from it and if things improved enough over the 82XX stuff that the upgrade is worth it if so, then buy the retail stuff and just build my own with water-cooling.

Picked up two Highpoint R7604A nvme cards, for just plex head end, have 2TBx4 in raid0 and another with 2TBx4 for plex only (Samsung 9100 Pros), should be an insanely bad ass setup, overkill and waste sure, but what would the bottle neck be then

Not sure what you do but I'm a double PhD, if we're talking money personally I did just take delivery of a new G63, a G580, Defender Octa and a new 2025 GTS 4 911 lol, so as I said, everyone's perceptions and realities are different

Pretty wild, certainly a 1% situation for sure. My only question now is, why do a cheap ES setup when an epyc with much faster single core speeds would likely benefit the odd occasions when you have to transcode audio or insert subtitles? Seems like an odd area to cheap out based on the other stuff (cars included).

 

[Techiedudeon]

Pretty wild, certainly a 1% situation for sure. My only question now is, why do a cheap ES setup when an epyc with much faster single core speeds would likely benefit the odd occasions when you have to transcode audio or insert subtitles? Seems like an odd area to cheap out based on the other stuff (cars included).

You missed my response on how I use to just call dell and they’d give good deals and now I can’t get anything worth a nickel to a reasonable price.

I’ll test AMD next but I don’t care about “efficiency” I like the best, in this case before I go buying 10-15k CPUs I want to see if there’s any major benefits over the older platforms, I’d prefer “best” vs efficient.

plus, as a firewall, it was cheap and easy, and it gives better performance, my old R760 firewall ran like 40k and in my tests, it wasn’t doing what I wanted, this little ES setup performs better and it’s a much easier thing. So I’ll run it for a few months and see how it plays out, reliability and consistency.

so if I buy a new server from dell, last quote was like 260k, I get it and the improvement is marginal, I just wasted money and for that money I can probably build 5 new ones even with all retail

also, I had less luck overall with AI testing, running llms and doing gpu and cpu handoffs, I always seem to have better luck with Intel, again might be specific to my perception and testing

 

[custom90gt]

You missed my response on how I use to just call dell and they’d give good deals and now I can’t get anything worth a nickel to a reasonable price.

I’ll test AMD next but I don’t care about “efficiency” I like the best, in this case before I go buying 10-15k CPUs I want to see if there’s any major benefits over the older platforms, I’d prefer “best” vs efficient.

plus, as a firewall, it was cheap and easy, and it gives better performance, my old R760 firewall ran like 40k and in my tests, it wasn’t doing what I wanted, this little ES setup performs better and it’s a much easier thing. So I’ll run it for a few months and see how it plays out, reliability and consistency.

so if I buy a new server from dell, last quote was like 260k, I get it and the improvement is marginal, I just wasted money and for that money I can probably build 5 new ones even with all retail

also, I had less luck overall with AI testing, running llms and doing gpu and cpu handoffs, I always seem to have better luck with Intel, again might be specific to my perception and testing

But you don't have "the best," you certainly have second place. An epyc or threadripper would be much faster. My guess is that it's just your luck with the difference in builds. You don't need to spend 260k on a server, if you can piece together an ES setup, you can put together a retail setup. Again, just a weird compromise for someone who has already spent so much money.

 

[Techiedudeon]

But you don't have "the best," you certainly have second place. An epyc or threadripper would be much faster. My guess is that it's just your luck with the difference in builds. You don't need to spend 260k on a server, if you can piece together an ES setup, you can put together a retail setup. Again, just a weird compromise for someone who has already spent so much money.

personal history I’ve always had bad luck with AMD lol I’ve always built the latest amd and Intel boxes for gaming and this was the first time amd won

For servers I’ve never had issues with Intel, not a compromise o just recall last epyc I played with cost me like 3 months running esxi vsan solutions.
But I’ll retest which is better for my scenarios

For vSans when I was testing, the Intel Xeons not only outperformed by a gross margin, but there was little to no tuning or tweaking, even VM performance itself was significantly better. There were firmware and bios updates we had to wait for, but by the time all these patches came out we had about 400 servers built and deployed and had to go with Intel, didn't have time to wait on AMD solutions.

In regards to Dell, no they just seemed to give better deals a few years ago vs now, its like their AI servers, even without good GPUs, they start at like 200-300k, when I did the R940XAs, with the 32x 15TB SSDs, 60k each was a steal

 

[GadflyII]

I can only speak in a large datacenter context, but for general compute, running VM's, and non-specialized workloads, AMD Eypc offer the most clocks per watt. For accelerated workloads, like heavy AVX512, or anything AI, Xeons are much faster than the Eypcs, by a significant margin; but consume a lot more power per clock.

I am just getting started setting up my own home servers for development and training of AI models, it really sucks having to pay for my own hardware .

I'd absolutely love to run a stack of RTX's Pro Blackwell's, I just can't swing the cost. I think I am going to have to just run 6x RTX 5090's. I am also looking at picking up and running some Xeon Max CPU's. As long as you run on linux (native, not WSL), you can sub-divide the numa nodes and pin the CPU threads from each compute tile to an HBM stack, then pin certain parts of the model to each sub-node. So you can keep the weights on the GPU, and move other parts to the CPU with HBM and accelerate it with AMX. eliminating cross talk across sockets, and the EMB connections to a bare minimum.

A lot slower than just running everything on GPU's, but a lot faster than running on pure DDR5 and/or CPU's without AMX.

 

[Techiedudeon]

You and I are the same lol

I’ll retest amds but we were pushing gpus in those systems as well, that was 2/3 years ago maybe things changed a lot but atm this is me playing around

I want to do some of the newer intel gear but starting at these as 1500 for a motherboard and two CPUs doesn’t hurt, so I can test and move up if the differences are astronomical

I lucked out and I need up with 8x 5090 Asus LCs they run great but for LLMs that vram, and with my h100s the blackwells perform overall better even with no sli

 

[UhClem]

... As long as you run on linux (native, not WSL), you can sub-divide the numa nodes and pin the CPU threads from each compute tile to an HBM stack, then pin certain parts of the model to each sub-node. ...

Such affinity-related operations should/need not be restricted to Linux. That same functionality appears to be available on Windows. I base this supposition on the fact that Intel's mlc program [Link], which has versions for both Windows & Linux, includes the following:

 

[custom90gt]

I can only speak in a large datacenter context, but for general compute, running VM's, and non-specialized workloads, AMD Eypc offer the most clocks per watt. For accelerated workloads, like heavy AVX512, or anything AI, Xeons are much faster than the Eypcs, by a significant margin; but consume a lot more power per clock.

I am just getting started setting up my own home servers for development and training of AI models, it really sucks having to pay for my own hardware .

I'd absolutely love to run a stack of RTX's Pro Blackwell's, I just can't swing the cost. I think I am going to have to just run 6x RTX 5090's. I am also looking at picking up and running some Xeon Max CPU's. As long as you run on linux (native, not WSL), you can sub-divide the numa nodes and pin the CPU threads from each compute tile to an HBM stack, then pin certain parts of the model to each sub-node. So you can keep the weights on the GPU, and move other parts to the CPU with HBM and accelerate it with AMX. eliminating cross talk across sockets, and the EMB connections to a bare minimum.

A lot slower than just running everything on GPU's, but a lot faster than running on pure DDR5 and/or CPU's without AMX.

Are those benchmarks between xeons and epycs with the current generation? AMD now does AVX-512 fyi. All of the testing I've seen shows that it's generally easier to list the few areas where a 128 core xeon 6980P 2P manages to beat the 64 core 9655 2P let alone the 128 core 9755 2P setup... The Xeon max stuff is cool for a variety of reasons though.

 

[Techiedudeon]

Such affinity-related operations should/need not be restricted to Linux. That same functionality appears to be available on Windows. I base this supposition on the fact that Intel's mlc program [Link], which has versions for both Windows & Linux, includes the following:
View attachment 45072

it’s not but red hat for example does it without much effort, I also find it less laggy in Linux distros, predominantly used in Linux so maturity