LGA 1851 Arrow Lake "Servers"

[autoturk]

I just built a server with the Supermicro X14SAE-F mainboard (was a nightmare to get this early). I used an Intel Core Ultra 5 235 CPU because it's the low power version. (6 performance, 8 efficient - 65w base, 121w turbo) Works very well. On Chip graphics Quick Sync works well for transcode, and it idles at very low power. Installed 64gig of Micron 5600 DDR5 ECC memory.

CPU sits at 4~5 watts at idle via HWinfo.

I have 6 x 8TB Samsung QVO drives as storage in a "Raid 10" storage space, and 2 x Samsung 980 Pro NVME drives for boot and scratch space.

I have a Mellanox SFP+ Card running 10gig DAC cable to my switch, it seems to run cool but I would like this system to run low power so I'll check it's power draw.

Unfortunately, it will not wake up from sleep with the Mellanox card so I will need to revisit this.

There is one bug with this board - The fans. They spin up and spin down constantly. I have a noctua CPU fan, it spins at low speed, and I the other fans freak out and spin up to make up for it. It reports an error in the Mainboard BMC ASPEED2600 management sayingthe fan periodically goes to zero rpm hence the others spinning up. This cannot be fixed with adjusting the settings. Full speed, balanced, or Silent. I have been investigating some IPMI settings that can be manually sent to the chip but it's a bit of a pain. So far no luck. It runs that cool though, I have them unplugged and only the CPU fan running.

The south bridge is fairly warm though.

Happy to answer any questions.

Have you measured power consumption at the wall?

 

[nexox]

There is one bug with this board - The fans. They spin up and spin down constantly. I have a noctua CPU fan, it spins at low speed, and I the other fans freak out and spin up to make up for it.

This is standard Supermicro fan controller behavior, not a bug. The lower RPM limits are rather high, because server fans spin fast, and once a fan drops below those limits all fans get boosted to maximum then the speed eases off slowly to try to hit equilibrium, then a fan drops below the lower threshold and spins everything up again. On many boards you can alter thresholds with a simple command, but not all boards, sometimes you just have to give up RPM monitoring on the larger, slower fans.

 

[Alterra]

There is one bug with this board - The fans. They spin up and spin down constantly. I have a noctua CPU fan, it spins at low speed, and I the other fans freak out and spin up to make up for it. It reports an error in the Mainboard BMC ASPEED2600 management sayingthe fan periodically goes to zero rpm hence the others spinning up. This cannot be fixed with adjusting the settings. Full speed, balanced, or Silent. I have been investigating some IPMI settings that can be manually sent to the chip but it's a bit of a pain. So far no luck. It runs that cool though, I have them unplugged and only the CPU fan running.

This is standard Supermicro fan controller behavior, not a bug. The lower RPM limits are rather high, because server fans spin fast, and once a fan drops below those limits all fans get boosted to maximum then the speed eases off slowly to try to hit equilibrium, then a fan drops below the lower threshold and spins everything up again. On many boards you can alter thresholds with a simple command, but not all boards, sometimes you just have to give up RPM monitoring on the larger, slower fans.

On X13SRA-TF (also has the same AST2600 BMC, and exhibits the same behavior), I can set the fan lower critical RPM with a command such as sudo ipmitool sensor thresh FAN1 lcr 420. As long as the fan spins faster than this value, the other fans behave normally. In general it seems to take quite a bit of experimenting with settings to reach a good compromise of cooling and noise level. But Core Ultra 235 should not be difficult.

 

[JimmyBlack]

Have you measured power consumption at the wall?

hi mate

Did just then. I was RDP'd to it, and it was downloading 10gigs of data.

Sits on 45watts while doing this.

if I turn on the "Always use energy saver" it drops to 39-40 watts.

I'll monitor what it uses when its properly idling doing nothing at all (maybe when it sleeps the SSDs).

Power usage is impressive that's for sure.

 

[autoturk]

hi mate

Did just then. I was RDP'd to it, and it was downloading 10gigs of data.

Sits on 45watts while doing this.

if I turn on the "Always use energy saver" it drops to 39-40 watts.

I'll monitor what it uses when its properly idling doing nothing at all (maybe when it sleeps the SSDs).

Power usage is impressive that's for sure.

Thanks! You saved me experimenting with the Supermicro board in the hunt for better power savings. Enjoy!

 

[zzz111]

intel IDLE is better than AMDs. more core need faster supply, faster = more current.
a 14900KF idles at 5Watt (only the package)
compare the performance.

Core Ultra is higher than 14th gen. I have a 265k in a Z890 motherboard and it idles at 35W before I added my nic and storage. Definitely still better than AMD. My mITX 7700X pc idles at 60W with nothing but a boot drive and one fan.

W880 honestly feels like a solid platform (certainly better than w680) for homelab purposes. Good PCie layout + 8x4x4 bifurcation, and many more chipset PCIe lines (24, I believe) with no "if x connected, then y doesn't work" kind of tradeoffs present on typical consumer boards. Still have two free CPU fed x4 m.2 PCIe slots if I want to Frankenstein some more stuff. I'm pretty happy.

This is why I got the 265k, but a z890 motherboard for now. I hope there comes some more affordable W880 motherboards. They were over $600 when i last checked. $450 is quite high for the asus one, and I cant even find the asrock or gigabyte ones. I'm not a huge fan of the Asus one. 265k has been $250 at microcenter intermittently so that's not even a huge factor into price as the 235, 245k and 255k are also always around $250. Reviews have been so lacking for the new intel parts. I haven't seen idle power consumption for 245k or 255k yet.

if I turn on the "Always use energy saver" it drops to 39-40 watts.

That's surprisingly good. I wonder if the idle power consumption is lower than the higher end parts. I literally didn't know the 235 existed until this post. I still haven't gotten my mellanox connectx4 to use under 9W. Your card is also probably also a big chunk of that 40W

 

[Stovar]

I just built a server with the Supermicro X14SAE-F mainboard (was a nightmare to get this early). I used an Intel Core Ultra 5 235 CPU because it's the low power version. (6 performance, 8 efficient - 65w base, 121w turbo) Works very well. On Chip graphics Quick Sync works well for transcode, and it idles at very low power. Installed 64gig of Micron 5600 DDR5 ECC memory.

CPU sits at 4~5 watts at idle via HWinfo.

I have 6 x 8TB Samsung QVO drives as storage in a "Raid 10" storage space, and 2 x Samsung 980 Pro NVME drives for boot and scratch space.

I have a Mellanox SFP+ Card running 10gig DAC cable to my switch, it seems to run cool but I would like this system to run low power so I'll check it's power draw.

Unfortunately, it will not wake up from sleep with the Mellanox card so I will need to revisit this.

There is one bug with this board - The fans. They spin up and spin down constantly. I have a noctua CPU fan, it spins at low speed, and I the other fans freak out and spin up to make up for it. It reports an error in the Mainboard BMC ASPEED2600 management sayingthe fan periodically goes to zero rpm hence the others spinning up. This cannot be fixed with adjusting the settings. Full speed, balanced, or Silent. I have been investigating some IPMI settings that can be manually sent to the chip but it's a bit of a pain. So far no luck. It runs that cool though, I have them unplugged and only the CPU fan running.

The south bridge is fairly warm though.

Happy to answer any questions.

hi mate

Did just then. I was RDP'd to it, and it was downloading 10gigs of data.

Sits on 45watts while doing this.

if I turn on the "Always use energy saver" it drops to 39-40 watts.

I'll monitor what it uses when its properly idling doing nothing at all (maybe when it sleeps the SSDs).

Power usage is impressive that's for sure.

Awesome, thanks for providing those figures its almost impossible to get good reviews on any Intel 235 or W880 products at this point.

Can I just confirm those 45 watts were while your Nas/server was downloading 10gb of data?

And is it 39-40watts even when doing heavy downloads/transfers or nas/server usage also?

Either way that is impressive, I have had the gmktec G9 Nas with 4x4tb WD red nvmes peak around 30-35 watts when doing heavy file transfers/downloads in unraid and around 25 watts idle, but that product fried and burned to death ( I don't recommend it!) so am looking at building something more stable/reliable so aiming to go back to good ole desktop where cooling/thermal issues and stability exist.

Regarding the fan issue, maybe try the noctua volt fan adaptors there ones, or an manual fan controller as a quick possible fix. I can't stand noisy PCs myself, so my nas/pc build will be fanless, hopefully with the intel 235 cpu and a fanless ps and fanless cpu cooler.

 

[Stovar]

That's surprisingly good. I wonder if the idle power consumption is lower than the higher end parts. I literally didn't know the 235 existed until this post. I still haven't gotten my mellanox connectx4 to use under 9W. Your card is also probably also a big chunk of that 40W

There are newer Intel 235 series cpus coming this fall, see here and here, but I believe one can still get the Intel 235 cpu and bios power tweak it and disable I think its Turbo and possible downclock it, you should be able to get it the same power wattage levels as say the T series cpus.... so I have read, not 100% tested it but maybe an intel 235 owner can confirm this?

 

[RolloZ170]

There is one bug with this board - The fans. They spin up and spin down constantly. I have a noctua CPU fan, it spins at low speed

lower critical FAN RPM threshold.
i have tested Noctua vs min. 700 RPM FAN.
on short loads the 700 RM FAN stays there, but the Noctua goes up to max. RPM because the cooler fins got too hot while IDLE,
this is not how heatpipes work well.

 

[zzz111]

Awesome, thanks for providing those figures its almost impossible to get good reviews on any Intel 235 or W880 products at this point

The only testing I’ve seen for more than the 285k idle power was techpowerup. They have idle system power for their 245k, 265k being the same with 285k having 2W higher idle. Its a lil sus and I think its also running windows and they dont really give a good idea of what they use for each configuration. They dont even say if they used the air or liquid cooler

 

[Stovar]

The only testing I’ve seen for more than the 285k idle power was techpowerup. They have idle system power for their 245k, 265k being the same with 285k having 2W higher idle. Its a lil sus and I think its also running windows and they dont really give a good idea of what they use for each configuration. They dont even say if they used the air or liquid cooler

Some power watt and benchmarks for the intel 235 found in case you wish to check here

 

[Stovar]

I just put together a w880 server based with the following

• asus w880-ace se - it was available on Amazon
• Core ultra 265k
• 2 x 48gb ddr5 ECC
• 3 x kioxia cd6-r through bifurcated PCIe slot
• 1 x Optane 905p (u.2 thru m.2 slot adapter)
• Mellanox connect x4 lx

Pulling 60 watts from the wall at idle, so take away 28 watts or so if you don't care about the power hungry storage + mellanox card.

W880 honestly feels like a solid platform (certainly better than w680) for homelab purposes. Good PCie layout + 8x4x4 bifurcation, and many more chipset PCIe lines (24, I believe) with no "if x connected, then y doesn't work" kind of tradeoffs present on typical consumer boards. Still have two free CPU fed x4 m.2 PCIe slots if I want to Frankenstein some more stuff. I'm pretty happy.

Have you tried to disable turbo and downclock on the asus bios and maybe get those watts down?

I read on reddit, many suggest not to bother with the T variant cpus, there is an 235T cpu at 35 watts but with those bios tweaks you should be able to get similar watts possibly.

I can still considering the intel 235 or 235T cpu.

 

[RolloZ170]

Have you tried to disable turbo and downclock on the asus bios and maybe get those watts down?

without knowing the CPU is in real IDLE ?
you don't need any tweak to get 5W package power( all cores @800mhz, mesh/uncore down)

there is an 235T cpu at 35 watts

this is not better silicon, can just not clock that high. 'T' models are for cooling and VRM limited machines.
you can make any 'K' to a 'T' by BIOS settings(clock TDP limits)
for 'T' models intel uses silicon which needs too much voltage for high frequency range.

 

[autoturk]

Have you tried to disable turbo and downclock on the asus bios and maybe get those watts down?

I read on reddit, many suggest not to bother with the T variant cpus, there is an 235T cpu at 35 watts but with those bios tweaks you should be able to get similar watts possibly.

I can still considering the intel 235 or 235T cpu.

I haven't, I'll give it a shot, but most of my power consumption is from the PCIe peripherals.

I never quite understood the drive to get T series CPUs. My understanding is that they essentially just limit the top-end of the CPU by reducing the power limits, but don't do much in terms of idle power. It's basically the same silicon with a "speed limit"

 

[RolloZ170]

My understanding is that they essentially just limit the top-end of the CPU by reducing the power limits, but don't do much in terms of idle power. It's basically the same silicone with a "speed limit"

intel CPU start with base clock, until BIOS enables P/C states.
means a 'K' model will fry VRM stage and overloads the limited cooling of a tiny PC.

 

[zzz111]

I never quite understood the drive to get T series CPUs. My understanding is that they essentially just limit the top-end of the CPU by reducing the power limits, but don't do much in terms of idle power. It's basically the same silicon with a "speed limit"

I think price has a big part to play in it. I've seen some retail pcs that have both options in older generations and the price difference was usually close to $100 with maybe a cooler change. Lower stock power max also lets you have a smaller psu and cheaper other components probably.

 

[autoturk]

I think price has a big part to play in it. I've seen some retail pcs that have both options in older generations and the price difference was usually close to $100 with maybe a cooler change. Lower stock power max also lets you have a smaller psu and cheaper other components probably.

I can't say much about the cost difference, but the latter part is always possible even with a K series CPU if you just power limit it. It doesn't consume more power if it is power limited just by virtue of being a K series CPU.

 

[Stovar]

without knowing the CPU is in real IDLE ?
you don't need any tweak to get 5W package power( all cores @800mhz, mesh/uncore down)

this is not better silicon, can just not clock that high. 'T' models are for cooling and VRM limited machines.
you can make any 'K' to a 'T' by BIOS settings(clock TDP limits)
for 'T' models intel uses silicon which needs too much voltage for high frequency range.

Yes true regarding idle figures, but I meant more for his average usage watts of 60 watts. Its tricky due to using pcie devices/drives. Another W880 owner got enabled Always use energy saver in his bios and shaved an extra 5-6 watts, so maybe its worth some bios tweaking especially if it does not eat into the overall performance too much.

 

[Stovar]

I haven't, I'll give it a shot, but most of my power consumption is from the PCIe peripherals.

I never quite understood the drive to get T series CPUs. My understanding is that they essentially just limit the top-end of the CPU by reducing the power limits, but don't do much in terms of idle power. It's basically the same silicon with a "speed limit"

Cheers, yeah I don't fancy getting the 235T cpu its not even stocked my end so quite tricky and almost £50-60 cost more, but I was at the same time hoping to get a NAS that can pull 30-40ish watts from the wall during light to medium task and hopefully 20-30watts idle. I would be using all nvme storage myself WD 8TB drives, maybe 3 of them to start with.

Its been years since I had a desktop, so long forgotten all the bios tweak power settings but maybe its not as needed anymore. Always use energy saver mode was mentioned by on page 1 for another W880 user which saved 5-6 watts, must be PL1/2 and C-states and few other bios settings, there is some info here but more related to mini pcs here and here

 

[Stovar]

Still trying to get parts for my new workstation unraid build, spotted the Intel core ultra 5 225 going more cheaper and almost made the mistake of buying one since its a good £60-80 less then the ultra 5 235 cpu but it does not support ECC memory like the ultra 5 235 does so just be warned about trying to save money or cut corners.