AM5 B650 2U Rack Workstation - seeking components! "El Finito"

[RDshift]

Hi! I'm a pretty experienced system builder looking to replace my aging Kaby Lake Xeon with AMD 7000/8000 generation. I plan to use this as my new primary utility and jump box. I'd like to get any input on specific part selections you'd recommend. Energy efficiency is top of mind, but due to all the connection needs, PCIe ports are going to be a challenge.
Bolded is points of interest where I'm most seeking input.

Goals:

• Uses Server-style B650/B650e motherboard
  • Best today is H13SAE-MF | Motherboards | Products | Supermicro due to PCIe switching options.
  • Must have IPMI
  • Must be right angle for optimal 2u airflow.
  • Should have but does not require at least 1x 80mm or higher M.2
  • Will use PWM fan hub if needed, nodding to the...
• CPU Heatsink
  • Active/Passive will depend upon chassis fans.
  • Assuming Dynatron for the moment, but would LOVE go to overboard with solid copper, vapor chamber etc. Cooling is key for efficiency of ongoing workloads, and the cooler it is, the more work I get for less power, so this is as important as the CPU.
  • El primo thermal goop, might go with newer-gen Carbon Thermal pads, reluctant to go liquid metal, but wouldn't be my first time.
• GPU Intel Arc Pro A40. PCIe lanes won't matter much. Likely headless for transcode only. Arc due to low power + AV1 performance. Will replace with Battlemage if reviews are even half-decent.
• Network:
  • Primary: 10GB SFP+ slot (NOT 10GbE). I'm looking for a PCIe 4.0 SFP+ NIC. 1 port is fine. SFP28 and SFP56 is fine.
  • IPMI
  • Built-in: management/remote access port only, resolving just the machine name to that MAC? Maybe route internet traffic via built in, and internal network /16 via 10GbE? Got options. If it's there, let's get a benefit!
• PCIe storage - PCIe 4.0 x4 minimum. May use Samsung server grade SSD for OS + Intel Optane for experimental hi-write work. Ebayyyyy!
• PCIe riser, active, to split the 8x PCIe 4.0 lanes I will get from the h13sae-mf. This is hard to find right now.
• 2U
  • Looking for case. A case that supports 2 slot HHHL + 3 slot 90 degree FHFL would be great, or some similar over designed options that can support both HHHL and FHFL.
  • No water cooling. Sound not an issue, reliability is.
• Energy efficiency. Will run non-x chips detuned in 35w mode.
• PSU: Don't need a lot of capacity
  • 550-750w seems sensible.
  • Must be Titanium (ish, I see you Cybernetics Labs <3). Will be on 24/7, so this will pay back. (50w a year of inefficiency is $65.75 at 0.15 per kW/h)
  • Prefer 1+1 redundant PSU in the Chassis.

What else do I need to make this best in class, doing all the things?

 

[NPS]

Your PSU plan seems about double what you really would need. PCI gen4 NICs are not that wide spread. I know only high performance (not low power) ones like ConnectX-6 LX or Intel E810. Marvell AQC113 could be a chip of interest but probably there are no SFP+ NICs based on that chip. Your plan with risers is very interesting. I would love to see this work out but at gen4 it will become very hard I guess. Server grade NVMe and Optane are not power efficient at idle as well... I somehow think limiting your CPU to 35W won't make such a big difference unless your CPU will be under significant load most of the time. Idle power of desktop class Ryzen is still not where I would like it to be... And it does not change by limiting TDP. Intel gen9 was and still is nice concerning idle power!

 

[RDshift]

I agree with you on the PSU and power thoughts. I'm purchasing higher because it's not much pricier, and supports future expansion - say a full-fat video card. I'm going for generally efficient, but not no-holds-barred - I should have known with this community hahaha. Since it's a utility and experimental box rather than a single task box, it still has to handle quite a variety of work and future demands as well, which leads into the heavy duty PCIe cards. It is regularly processing work above and beyond 1Gbit.
I strongly agree that the NIC will be quite tricky. I may compromise ($avings!) with a previous gen card, lean on M.2 storage, and expand further into PCIe later when the products are more mature.
This brings up a new awareness to look out for how much power an active riser card uses - since that adds to the power budget, and if it's a net loss, then I need to try to avoid it.
Does that make sense? I'm aware it can't be a super efficient box since I want it to do all the things. But I will detune where possible so my 24/7 workloads use less juice. I'm already doing OK with my entire stack only using around 400w currently.

 

[NPS]

What do you mean with "active riser card"? I think you don't need a PCIe switch. You need four devices each with x4 link? (I count: NVMe, Optane, NIC, GPU)

 

[RDshift]

Correct, I won't if I'm sticking to just the four out of the box on a B650.
I have two PCIe 3.0 x8 storage cards though - the Optane and a now-unused Samsung/Oracle F320 I was hoping to have host the OS. I'll probably do the Sammy since it's got capacity. I also have unused U.2 drives. I'm already at the limit of what it can support. I'd much prefer a 90° rotated x670, but you can't have it all, apparently.
Current Plan for PCIe Lanes:

• NVMe M.2 - Boot - Motherboard, probably CPU - 990 Pro 2TB or similar
• PCH Slot1 PCIe 4.0 x4 storage - 10GbE SFP+ NIC - OK in the x4 slot at 3.0 x4 because only one port will be in use
• CPU Slot_6 5.0 x16 in 3.0 x8 mode - Intel Arc Pro A40 LP - Transcode, so b/w not a problem
• CPU Slot_4 5.0 x16 in 3.0 x8 mode - Reuse retired Samsung F320 (or Optane, if the workload demands it)

Don't look at me for the slot numbering, that's how Supermicro printed the board!

 

[nexox]

Cooling isn't really that critical, big heat sink capacity means you can burn more power to get more performance, it doesn't make anything go faster at the same power level, just a little cooler, which is not really beneficial. Choose something that'll keep the CPU under about 70C at load, which, for a 35W CPU, shouldn't be difficult.

Do you have a good way to acquire the Arc Pro A40? Last I checked they still weren't sold individually to consumers.

 

[Sean Ho]

I'm not quite clear on your anticipated workload, but what you describe isn't particularly heavy. An older X11SCH-F (C246, 1151-2) may suffice: x16, x8, 2280, 22110. I'm not sure why you'd need to be transcoding AV1, and QSV can easily handle low-res transcodes. The total cost of the whole system including CPU and RAM would be much cheaper than B650. Do you need spinners?

 

[RDshift]

Yeah, I'm already using an X10 platform for Unraid, and a x11-equiv Dell R640 for further virtualization (usually off - see average 400w). Anime, Plex, and long-delayed plans to transcode footage I want to archive into a smaller size. The B650 platform will also work significantly better for Steam Link, without requiring as much power as leaving my gaming rig on. Or the flights of stairs between. I've agonized over it, and I have considered the numbers. B650 is the right fit, about 4x times my current CPU capabilities coming from the Xeon 1245 v6. I was somewhat tempted to single-socket Xeon it, but I can make up for everything with the video engine on the Arc.
Heatsink: I am leaning towards what @nexox said. Since I'm expecting spiky workloads rather than sustained, AND considering eco mode, just getting a solid cooler should super overpowered.

I think I really just need to keep my eyes open for developments on new NICs, pass on the riser complication unless it comes up organically, and actually, y'know, pull the trigger on buying parts.