Got some money to burn and was thinking about replacing my Poweredge R720

[Computer cats and more]

So I have a R720 that is basically a glorified NAS that I also occasionally edit videos on. Was thinking about upgrading to something faster for a variety of reasons. None of which are good. Specs of my current system:

• Dual E5-2670 v2 2.5GHz 10 core
• 256GB DDR3
• X520 networking
• 2x Tesla M40
• 13TB SSD

I know I can get more for my money if I do non-Dell, but I would prefer Dell. Been toying with a R730, but it is so loud once I put the Tesla in. I was also eyeballing the R740 but I don't think I could get a build I would be happy with for under $2000 used. Was curious what the experts here had to say. In all honesty I am leaning towards the R730xd route and soundproofing my "server room".

 

[BlueFox]

What are your actual wants/requirements? If you just want a NAS why bother with a energy hungry NAS and 500W of very old GPUs. If editing videos, you are probably far better off using the GPU of your local workstation.

 

[CyklonDX]

You might be able to snipe r740 for under 1.6k; But improvement ain't going to be revolutionary.
If you want big boost; you should look at the R7515

Dell PowerEdge R7515 12LFF + H730P 2X1100W/ 7402P 24Core 2.8GHz/4X DDR4 32G 2400 | eBay

1xEPYC 7402P(24Core 2.8GHz). 4xDDR4 32G 2400MHz. Note : If the label is a second-hand product, there may be wear and scratches on the appearance. The photos on the link are inventory photos, not actual photos.

www.ebay.com

ex.

Intel Xeon Gold 6138 @ 2.00GHz vs AMD EPYC 7402P vs Intel Xeon E5-2670 v2 @ 2.50GHz [cpubenchmark.net] by PassMark Software

(Gold 6138 is one of the better cpu's on Rx40 platform - skylake, without price being inflated to moon)


Potentially you should upgrade your Tesla cards, P100/P40 (100-200 usd), or V100's (500usd), or oem 3090 (800 usd)

P100 (219 usd)

Q0E21A HPE NVIDIA TESLA P100 PCIE 16GB COMPUTATIONAL ACCELERATOR 868585-001 | eBay

Cloud Storage Corp © 2020. Ending Soon. New Arrivals. QUICK LINKS. We are here 24/7, should you need any help.

www.ebay.com

V100 (509 usd)

Nvidia Tesla V100 GPU 16GB Accelerator Card HBM2 5120 Cuda Cores 1245 MHz 300W 757897041387 | eBay

Find many great new & used options and get the best deals for Nvidia Tesla V100 GPU 16GB Accelerator Card HBM2 5120 Cuda Cores 1245 MHz 300W at the best online prices at eBay! Free shipping for many products!

www.ebay.com

oem 3090 (815 usd)

Nvidia GeForce RTX 3090 Graphics Card (GDDR6 24GB VRAM) Dell OEM Alienware | eBay

Find many great new & used options and get the best deals for Nvidia GeForce RTX 3090 Graphics Card (GDDR6 24GB VRAM) Dell OEM Alienware at the best online prices at eBay! Free shipping for many products!

www.ebay.com

(Your 2x M40's could be replaced by single A4000 ~ or A5000)
A4000 (602 usd)

NVIDIA RTX A4000 16GB GDDR6 Graphics Cards, Like New 195122692223 | eBay

Find many great new & used options and get the best deals for NVIDIA RTX A4000 16GB GDDR6 Graphics Cards, Like New at the best online prices at eBay! Free shipping for many products!

www.ebay.com

A5000 (1k) ~ this one supports sriov, its very nice card.

NVIDIA RTX A5000 24GB GDDR6 Graphics Card for sale online | eBay

Find many great new & used options and get the best deals for NVIDIA RTX A5000 24GB GDDR6 Graphics Card at the best online prices at eBay! Free shipping for many products!

www.ebay.com

(Upgrading to 1xA5000 from 2x M40 going be revolutionary even in current platform)

 

[Computer cats and more]

What are your actual wants/requirements? If you just want a NAS why bother with a energy hungry NAS and 500W of very old GPUs. If editing videos, you are probably far better off using the GPU of your local workstation.

Basically looking to gain NVME support through PCI express bifurcation lol. Also maybe more cores. Power is cheap so the 1kW my rack constantly draws doesn't bother me. I like the reliability a server offers. Plus I can offload tasks to multiple VMs without tying up my personal machine.

 

[Computer cats and more]

You might be able to snipe r740 for under 1.6k; But improvement ain't going to be revolutionary.
If you want big boost; you should look at the R7515

Dell PowerEdge R7515 12LFF + H730P 2X1100W/ 7402P 24Core 2.8GHz/4X DDR4 32G 2400 | eBay

1xEPYC 7402P(24Core 2.8GHz). 4xDDR4 32G 2400MHz. Note : If the label is a second-hand product, there may be wear and scratches on the appearance. The photos on the link are inventory photos, not actual photos.

www.ebay.com

ex.

Intel Xeon Gold 6138 @ 2.00GHz vs AMD EPYC 7402P vs Intel Xeon E5-2670 v2 @ 2.50GHz [cpubenchmark.net] by PassMark Software

(Gold 6138 is one of the better cpu's on Rx40 platform - skylake, without price being inflated to moon)


Potentially you should upgrade your Tesla cards, P100/P40 (100-200 usd), or V100's (500usd), or oem 3090 (800 usd)

P100 (219 usd)

Q0E21A HPE NVIDIA TESLA P100 PCIE 16GB COMPUTATIONAL ACCELERATOR 868585-001 | eBay

Cloud Storage Corp © 2020. Ending Soon. New Arrivals. QUICK LINKS. We are here 24/7, should you need any help.

www.ebay.com

V100 (509 usd)

Nvidia Tesla V100 GPU 16GB Accelerator Card HBM2 5120 Cuda Cores 1245 MHz 300W 757897041387 | eBay

Find many great new & used options and get the best deals for Nvidia Tesla V100 GPU 16GB Accelerator Card HBM2 5120 Cuda Cores 1245 MHz 300W at the best online prices at eBay! Free shipping for many products!

www.ebay.com

oem 3090 (815 usd)

Nvidia GeForce RTX 3090 Graphics Card (GDDR6 24GB VRAM) Dell OEM Alienware | eBay

Find many great new & used options and get the best deals for Nvidia GeForce RTX 3090 Graphics Card (GDDR6 24GB VRAM) Dell OEM Alienware at the best online prices at eBay! Free shipping for many products!

www.ebay.com

(Your 2x M40's could be replaced by single A4000 ~ or A5000)
A4000 (602 usd)

NVIDIA RTX A4000 16GB GDDR6 Graphics Cards, Like New 195122692223 | eBay

Find many great new & used options and get the best deals for NVIDIA RTX A4000 16GB GDDR6 Graphics Cards, Like New at the best online prices at eBay! Free shipping for many products!

www.ebay.com

A5000 (1k) ~ this one supports sriov, its very nice card.

NVIDIA RTX A5000 24GB GDDR6 Graphics Card for sale online | eBay

Find many great new & used options and get the best deals for NVIDIA RTX A5000 24GB GDDR6 Graphics Card at the best online prices at eBay! Free shipping for many products!

www.ebay.com

(Upgrading to 1xA5000 from 2x M40 going be revolutionary even in current platform)

Thanks I will look into that. The R7515 would be an interesting alternative. I would have to look into getting a different NVME adapter solution than the one I already purchased. Been eyeballing the Tesla P40 but feel they are overpriced still. I'll probably strike once they hit $150.

 

[zac1]

Have you considered something like dual Epyc 7742s in an ASRock Rack ROME2D16-2T?

 

[tinfoil3d]

i've been on EPYC since 2019, and they do rock. The only issue for me is it's hard to rack it, enclosures like supermicro which aren't proprietary form-factor are rare to come by in my area(Japan), hence I have to either stack them up on ikea lack tables as-is(no case at all) or put those in tower cases onto some rack shelf. also because i already have a stock of 2U, 3U and noctua heatsinks for SP3 i need a big enclosure. if i was in the states, supermicro chassis are as cheap and as easily available as dirt. But not so much here.
Dell and Lenovo need either already "owned" EPYC or otherwise they'll overwrite some protection in those CPUs so that you could only use them in Dell/Lenovo systems from that point on. Just so you know. https://www.servethehome.com/amd-psb-vendor-locks-epyc-cpus-for-enhanced-security-at-a-cost/

 

[Computer cats and more]

Had some time to think about what I want. It basically comes down to I want to add some higher speed storage to my rack and more storage. I may relegate my Tesla M40 cards to my slower R720 and replace my main R720 with a R730xd. Then build a rack mount video editing PC with a ton of cores.

 

[CyklonDX]

you can acquire 12gbps sas ssd's (not sure if r720 had a 12gbps backplane, but should deliver decent boost in r/w performance)

 

[chrgrose]

Why is the R730 louder than the R720 when you install the M40s? Does it spin up the fans a lot when the PCI-E lanes suck power? You could probably get an R730 with 2x2699v4 for only ~$650.

 

[CyklonDX]

I would assume those are pre-configured fan speeds to that gen of graphics cards. It was ready for P100's and V100's class cards that ran much hotter. You should still be able to override the fan offset/max in idrac settings.

 

[Computer cats and more]

Why is the R730 louder than the R720 when you install the M40s? Does it spin up the fans a lot when the PCI-E lanes suck power? You could probably get an R730 with 2x2699v4 for only ~$650.

Preconfigured fan profiles as CyklonDX suggested. I tried setting it on the low fan profile through SSH, but the lowest it would go is 60% fan speed when idle which is ~12000 RPM. On my R720 60% is around 8000 RPM which is noticeably quieter. The nice thing about the R730 is it will automatically ramp fan speeds based on GPU load for my Telsa M40 unlike my R720. If I was willing to cool my server room I could close the door then noise wouldn't matter. Maybe I should fix the large portable AC I have for it.

 

[Terry Kennedy]

Why is the R730 louder than the R720 when you install the M40s? Does it spin up the fans a lot when the PCI-E lanes suck power? You could probably get an R730 with 2x2699v4 for only ~$650.

One of the BIOS settings is "run system characterization" and the choices are something like always/never/no reboot. That runs the fans up to full speed, measures actual temperatures, and then reduces the fan speed. I have mine set to "no reboot" which does it only when they're power cycled.

Another thing to get r730 fan speed under control is to select an appropriate power profile. I use OS DBPM which lets the operating system have some control over power usage (and thus fan speed).

Having said all that, if you're generating a lot of heat in that chassis the fans are going to be noisy. If you happen to get a single CPU or 8-slot drive backplane model, not all of the fans may be populated. Put the missing fans in and they should all run a little slower.

 

[CyklonDX]

i recalled there to be more settings, but ... racadm tool should offer setting custom max fan speed in % too.

 

[Computer cats and more]

One of the BIOS settings is "run system characterization" and the choices are something like always/never/no reboot. That runs the fans up to full speed, measures actual temperatures, and then reduces the fan speed. I have mine set to "no reboot" which does it only when they're power cycled.

Another thing to get r730 fan speed under control is to select an appropriate power profile. I use OS DBPM which lets the operating system have some control over power usage (and thus fan speed).

Having said all that, if you're generating a lot of heat in that chassis the fans are going to be noisy. If you happen to get a single CPU or 8-slot drive backplane model, not all of the fans may be populated. Put the missing fans in and they should all run a little slower.

It has all fans fortunately. I will have to see if it actually did the lifecycle hardware characterization process. Honestly it is just the server idle fan speed that annoys me.

 

[TonyArrr]

My R730XD does the “you installed disks/PCIe cards I don’t like so here is 70% fans” thing as well so I just use a script to manually control the fan speed. I’ll find the github link and post it, has been working fairly well with it since late 2021.

I’m like you, glorified NAS, video encoding, misc VMs

 

[CyklonDX]

iDRAC7_fan_control/fan_control.sh at master · brezlord/iDRAC7_fan_control

A simple script to control fan speeds on Dell generation 12 PowerEdge servers. - brezlord/iDRAC7_fan_control

github.com

This should work on r730's.

 

[TonyArrr]

Ooohhh that one looks pretty pretty good, might do a comparison later.
The one I’m using is

GitHub - spacelama/R710-Fan-Control: A fork of R710-IPMI-TEMP from NoLooseEnds/Scripts, generalised to a fan control daemon for Dell Poweredge servers. Has reported to work on R710s, R520, R730. Allows more flexible control of the fan throttling vs the vanilla iDrac control which tends to ramp the fans up to full velocity the moment you add non-Dell hardware.

A fork of R710-IPMI-TEMP from NoLooseEnds/Scripts, generalised to a fan control daemon for Dell Poweredge servers. Has reported to work on R710s, R520, R730. Allows more flexible control of the f...

github.com

 

[microwave]

My R730XD does the “you installed disks/PCIe cards I don’t like so here is 70% fans” thing as well so I just use a script to manually control the fan speed. I’ll find the github link and post it, has been working fairly well with it since late 2021.

I’m like you, glorified NAS, video encoding, misc VMs

You might be interested looking into this: How To Disable The Third-Party PCIe Card Default Cooling Response On PowerEdge 13G Servers. For my r720 the fan speeds jumped significantly when installing an m.2 nvme and increased again when installing an RX580 (regardless of temperature), disabling the 3rd party cooling response dropped the idle fan speeds enormously. This was of course after trying disabling the fan speed offset via racadm.

Here is what my fan speeds looked like in grafana while disabling the 3rd party cooling response via ipmi:

 

[Computer cats and more]

You might be interested looking into this: How To Disable The Third-Party PCIe Card Default Cooling Response On PowerEdge 13G Servers. For my r720 the fan speeds jumped significantly when installing an m.2 nvme and increased again when installing an RX580 (regardless of temperature), disabling the 3rd party cooling response dropped the idle fan speeds enormously. This was of course after trying disabling the fan speed offset via racadm.

Here is what my fan speeds looked like in grafana while disabling the 3rd party cooling response via ipmi: View attachment 30257

That is weird your R720 increased fan speed when you installed NVMe drives and later a RX580. I haven't ran those together in my system, but it had no reaction to them individually for me. Honestly the only thing that has influenced my R720 fan speeds was the Tesla K80 I had in it, and when I was doing heavy loads like FAH or gaming and video editing at the same time.