64GB LRDIMM DDR3 ECC ram for $20 or less

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

computably

New Member
Aug 8, 2022
23
11
3
Cisco is not paywalled but you need to register for download access.

cisco c240 m3 is fairly cheap, that 56$ one even has 128gb ram it looks like.
Not sure how great the m3 webpanel/management is tho.
Any docs on whether it supports 24x 64GB? The original docs I found say it only supports 32GB DIMMs for a max of 768GB.
 
  • Like
Reactions: Samir

tinfoil3d

QSFP28
May 11, 2020
882
407
63
Japan
Maybe it should if CPU says it supports 1.5T? What's your CPU you intend to use?
I remember a funny story when first gen i7 came out it was listed as max 16g. At the time 8GB DDR3 sticks were still in early adoption or not even existed yet. Few years after purchase I got my hands on 8G sticks and installed all 4, and that was over the official intel spec. And it worked flawlessly with up to a year of uptime, under moderate load, definitely 16G was in use most of the time.
 
  • Like
Reactions: Samir

computably

New Member
Aug 8, 2022
23
11
3
Maybe it should if CPU says it supports 1.5T? What's your CPU you intend to use?
I remember a funny story when first gen i7 came out it was listed as max 16g. At the time 8GB DDR3 sticks were still in early adoption or not even existed yet. Few years after purchase I got my hands on 8G sticks and installed all 4, and that was over the official intel spec. And it worked flawlessly with up to a year of uptime, under moderate load, definitely 16G was in use most of the time.
I snagged Cisco UCS C240 M3 Rack Server Intel Xeon E5-2609 v2 x 2 12GB RAM No HDDs | eBay which came with 2x E5-2609v2. According to Intel the CPU supports a max of 768GB Intel® Xeon® Processor E5-2609 v2 (10M Cache, 2.50 GHz) - Product Specifications | Intel which hopefully means per socket?
 
  • Like
Reactions: Samir

zachj

Active Member
Apr 17, 2019
161
106
43
Any docs on whether it supports 24x 64GB? The original docs I found say it only supports 32GB DIMMs for a max of 768GB.
you’re constrained by max supported ram as declared by intel and you’re also constrained by max number of ranks per channel.

Boards with 24 dimm slots are 3DPC and have the unfortunate ability to let you install memory in excess of the limits on ranks per channel.

on the hp z420 you could exceed the limit on ranks even with only 2DPC by trying to install 2 x 32gb rdimm.
 
  • Like
Reactions: Samir and tinfoil3d

computably

New Member
Aug 8, 2022
23
11
3
you’re constrained by max supported ram as declared by intel and you’re also constrained by max number of ranks per channel.

Boards with 24 dimm slots are 3DPC and have the unfortunate ability to let you install memory in excess of the limits on ranks per channel.

on the hp z420 you could exceed the limit on ranks even with only 2DPC by trying to install 2 x 32gb rdimm.
Any way to verify such limits? If Intel lists the E5-2609 v2 as supporting 768GB over 4 channels, that clearly means the CPU supports 64GB DIMMs, right? Unless this figure is referring to the max total across two sockets. Besides that the only foreseeable issue would be with the MB rather than CPU.
 
  • Like
Reactions: Samir

chinesestunna

Active Member
Jan 23, 2015
622
195
43
56
Any way to verify such limits? If Intel lists the E5-2609 v2 as supporting 768GB over 4 channels, that clearly means the CPU supports 64GB DIMMs, right? Unless this figure is referring to the max total across two sockets. Besides that the only foreseeable issue would be with the MB rather than CPU.
I don't know specifics here but 768GB could be achieved with 3DPC x 4 channels x 32GB DIMMs as well
 
  • Like
Reactions: Samir

computably

New Member
Aug 8, 2022
23
11
3
768gb on the v2 Xeon would only be possible using 64gb lr-dimms.

This answers all your questions: https://sp.ts.fujitsu.com/dmsp/Publications/public/wp-ivy-bridge-ep-memory-performance-ww-en.pdf

The ddr3 spec only supports 8 ranks per channel.
Actually, I only have one question, which is "Does this server I got (Cisco UCS C240 M3 Rack Server Intel Xeon E5-2609 v2 x 2 12GB RAM No HDDs | eBay) support 24x 64GB LRDIMMs?"

The doc you linked states that LRDIMMs would be supported using virtual rank reduction:

The DDR3 memory controllers of the Xeon E5-2600 v2 processors support a maximum of 8 ranks per memory channel. In configurations with LRDIMMs with more than 8 physical ranks the Rank Multiplication function of this DIMM type maps the physical ranks onto virtual ones. The reduction factor is set to 2 or 4 so that a maximum of 6 virtual ranks occur.
So 12 LRDIMMs with 8 physical ranks each could be mapped into only 24 logical ranks, i.e. 6 ranks per channel.
 
  • Like
Reactions: Samir

wildpig1234

Well-Known Member
Aug 22, 2016
2,247
495
83
49
Asus Z9PED16 official specs is only up to 512GB ram. But it accepted the 64GB lrdimm and worked fine with 1TB ram. I verified this personally using the 2696 v2 cpus.
 

Samir

Post Liker and Deal Hunter Extraordinaire!
Jul 21, 2017
3,337
1,496
113
49
HSV and SFO
I was able to successfully run 1.5TB ram with 2696 v2 and this lrdimm on supermicro x9dri MB
Nicely done!
I have an HP z420 my workplace gave to me for remote work from home. updated the bios to the latest today. it has a 1620 v1 cpu. I put in 4X64GB and it was able to boot into bios showing 256GB DDR3 LRDIMM under the total ram section in line 2. But interestingly, the ram section about the size of each dimm below show blank instead of displaying the size of each dimm.

I then put in a 1607 v2 cpu leaving in the 256GB and i got a red light warning. I guess as we suspected,1600 v2 support for lrdimm was removed. Then i went back to the 1620 with the 256GB and it boot fine again. But putting in 2 more 64GB it won't boot which i guess because 1620 v1 only supports 256GB ram.

So i put in a 2609 v2 with one 64GB stick. It does not boot at all. After a while the fan just speeded up to max and stay there. Remove the 64GB and replace with a 4GB rdimm and it booted fine.

So the result so far for me is that LRDIMM support on Z420 is incomplete. The saddest part is not able to get it to boot with a 2609 v2 with just even one 64GB stick... ;(
Thank you for running these tests! Sounds like there's not enough ranks or else you should have been able to add another 64GB module. With 4x 32GB modules in each bank, each being 4 ranks, I've got 16 ranks--the same as your 2x 64GB modules at 8 ranks each. I don't have any extra slots for memory, but it seems like that doesn't matter as you couldn't get another module to work (which presented 24 ranks to the cpu).

I was curious about the v2 LRDIMMs support drop as I only worked with that one individual so it could have been a fluke, but you've proven that it's a real issue on the z420 for sure. v1 cpus only if one wants 256GB of RAM. I run an E5-2630L as that's what came in it.
 
  • Like
Reactions: wildpig1234

wildpig1234

Well-Known Member
Aug 22, 2016
2,247
495
83
49
is there any specific setting you used? on bios what does the RAM config say?
No specific settings. Just loaded up all slots with the 64GB dimms. Bios did say 1.5TB. I used y cruncher to calculate pi out to like 250billion digits all in ram only.

You have problem getting it to work?
 

Filez

Active Member
Feb 18, 2019
111
93
28
I have 250gb ram and even though they are 1600mhz, bios will say they are operating 1333mhz and if I add more, it will even go down to 800mhz. And the RAMs are running in independent mode. I am running X9DRI-LNF4. Are you getting full RAM speed?
 
  • Like
Reactions: wildpig1234

wildpig1234

Well-Known Member
Aug 22, 2016
2,247
495
83
49
I have 250gb ram and even though they are 1600mhz, bios will say they are operating 1333mhz and if I add more, it will even go down to 800mhz. And the RAMs are running in independent mode. I am running X9DRI-LNF4. Are you getting full RAM speed?
That is known. And yes, the more i populate the rams, the slower the speed. If you find a setting that restore the full speed, I think everyone here would really be interested.

But i do believe that one quirk of these older MB is that the more ram you have the slower they get
 
  • Like
Reactions: Filez