NVDimm support on AMD Epyc

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

RageBone

Active Member
Jul 11, 2017
617
159
43
two possible issues between powergems.
Pinout of the cable / connectors.
And the Controller used in the Powergem.

I suspect that they have i2c wired up so that the NVDIMM can get telemetry from the Gem.
Through that, it could check for a specific controller used.

Datasheets for a a LTC3350:
Technically its the wrong manufacturer since the logo on the chip shown in the pictures refers to Linear Technology (LT) and not Analog Devices

@nasbdh9 you could do as a favor in grabbing a multimeter and measuring the pinout of the powergem connector going to the NVDIMM.
 

nasbdh9

Active Member
Aug 4, 2019
164
96
28
two possible issues between powergems.
Pinout of the cable / connectors.
And the Controller used in the Powergem.

I suspect that they have i2c wired up so that the NVDIMM can get telemetry from the Gem.
Through that, it could check for a specific controller used.

Datasheets for a a LTC3350:
Technically its the wrong manufacturer since the logo on the chip shown in the pictures refers to Linear Technology (LT) and not Analog Devices

@nasbdh9 you could do as a favor in grabbing a multimeter and measuring the pinout of the powergem connector going to the NVDIMM.
Sorry, I have sold all these boxes :oops:
 

Propaganda

Active Member
Dec 6, 2017
154
62
28
43
two possible issues between powergems.
Pinout of the cable / connectors.
And the Controller used in the Powergem.

I suspect that they have i2c wired up so that the NVDIMM can get telemetry from the Gem.
Through that, it could check for a specific controller used.

Datasheets for a a LTC3350:
Technically its the wrong manufacturer since the logo on the chip shown in the pictures refers to Linear Technology (LT) and not Analog Devices

@nasbdh9 you could do as a favor in grabbing a multimeter and measuring the pinout of the powergem connector going to the NVDIMM.
Analog Devices bought Linear Technology a few years ago.
 
  • Like
Reactions: RageBone

edgecase

New Member
Nov 4, 2021
1
0
1
I wonder, are any of these safe to use in production environments, while waiting for CXL products to hit the market? I'm not a fan of HPE's storage cost, but I would trust their NVIDMM implementation, and possibly Dell. If I buy a Tyan TS75-B8252 and some NVDIMMs and PowerGems, even if they are new-in-box, is there any vendor support? I wonder if there is a way I could test them to be confident they are reliable?
 

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
Of course you can test them, its ram + battery.
Mount as drive, write to it, power down box, remove any trace of power, power back up. Everything should still be there.

I doubt you get vendor support from HPE/Dell unless the system they were sold with is still under warranty, and basically no chance for any support on 3rd party.
I know Supermicro is not helping much (no market for them), but Micron was helpful, but not sure if that's the official stance or if i was lucky
 

eduncan911

The New James Dean
Jul 27, 2015
648
506
93
eduncan911.com
Are we still looking for validation? With the price of NVDIMMs dropping, I'm thinking of playing with the tech now.

I'm just learning about NVDIMMs (yeah, late to the party - was stuck in X9/X10/E5 V1-V4 land the last 9 years).

I have a few boards that may be compatible. So I'll need to pick a couple modules up.

I was focused on the Intel Optane PMem versions, as I'm mostly interested in the flash storage aspect for ZFS tunables (ZIL, persistent L2ARC, etc). Until this thread, I had no idea it was an AMD option!

Is there interest in these Intel NVDIMM support?

Does anyone want me to test a particular type in these boards (and can send me the sample stick).

I'm currently learning the ropes of bios hacking / unhiding options, and I have the following boards to tinker with:

AsRock Rack EPYCD8-2T (7001/7002 Epycs)

AsRock Rack X399D8U-2T (ThreadRipper 2, I have no idea if this supports it by my gawd the BIOS is full of more CPU options than my Asus overclocking boards it seems!)

Unknown (Wynnsynn?) Dual LGA3647 Skylake-SP with 4x NVDIMM slots flanking both CPU. It's the 1U Hyve 8x LFF chassis in the Deals section (or check my posts).

So if I run across NVDIMM options, I'll look into them as well.
 

eduncan911

The New James Dean
Jul 27, 2015
648
506
93
eduncan911.com
Do I need to install an NVDIMM to get options in BIOS to show? Are the options hidden if you don't have any Optane memory?

Also, please excuse my addition rookie questions but...

I've only read up on the Intel Optane DC Pem tech and different modes of operations.

And a massive shout-out to @Patrick and his deep dive and "middle ground" videos on Intel Optane NVDIMMs! Seriously, lately, and I mean every single time, I go to STH to use the search, or prefix / prioritize STH in my searches, hands down and blindly.

however, I hear name drops and Pem200 stuff mentioned, but either no videos or my search-foo is really bad. Other types of NVDIMM or types?

Would someone be able to elaborate on these types and their differences? Maybe an STH video/article I've missed?
 
  • Like
Reactions: Patrick

RolloZ170

Well-Known Member
Apr 24, 2016
5,155
1,547
113
Unknown (Wynnsynn?) Dual LGA3647 Skylake-SP with 4x NVDIMM slots flanking both CPU. It's the 1U Hyve 8x LFF chassis in the Deals section (or check my posts).
don't expect these 4 extra slots to be for NVDIMM. this is mostly only A2 / D2.
memory population guide(manual): populate A2 / D2 only if you don't have enough memory.
Note 2ng gen scalable(non 'M' non 'L') support 1TB of memory, 8 x 128GB does the job with all 6 channels populated.
4x 256GB will also be 1TB but only quad.channel operation.
 

eduncan911

The New James Dean
Jul 27, 2015
648
506
93
eduncan911.com
don't expect these 4 extra slots to be for NVDIMM. this is mostly only A2 / D2.
memory population guide(manual): populate A2 / D2 only if you don't have enough memory.
Note 2ng gen scalable(non 'M' non 'L') support 1TB of memory, 8 x 128GB does the job with all 6 channels populated.
4x 256GB will also be 1TB but only quad.channel operation.
What gives me a little hope is that the BIOS had NVDIMM options. At least, from what someone else posted. I'll find out today.
 

RolloZ170

Well-Known Member
Apr 24, 2016
5,155
1,547
113
What gives me a little hope is that the BIOS had NVDIMM options. At least, from what someone else posted. I'll find out today.
you can put NVDIMM-N with backup batterie and or flash storage in every memory slot. this memory should not be erased on POST memtest, any idea why?
 

gb00s

Well-Known Member
Jul 25, 2018
1,175
586
113
Poland
My Gigabyte MZ-AR0 v1 and v3 support NVDIMM-N. Settings are available once you populate NVDIMM-N on them. I just cant get it running together with LR-DIMM. Only RDIMM. Just being a 2666Mhz is no issue for EPYC, at least not for Naples and Rome. 7371, 7F32 and 7302P are running fine. No post issues. Even passing them through to vm's is working with Proxmox, not just as a 'slow' block device. Fun but quirky sometimes.
 
  • Like
Reactions: Rand__

Rand__

Well-Known Member
Mar 6, 2014
6,626
1,767
113
That u cant mix NVDimm-N and LRDimm was to be expected, but else brilliant news. There might be an upgrade path for me after all;)
 
  • Like
Reactions: gb00s

i386

Well-Known Member
Mar 18, 2016
4,220
1,540
113
34
Germany
Crucial CTA36ASS4G72XF1Z-2G9PR1AB aka Micron MTA36ASS4G72PF1Z-2G9PR1AB
32gb, 2933 MHz
 

gb00s

Well-Known Member
Jul 25, 2018
1,175
586
113
Poland
thanks, any not EOL and under $600 usd ? thought Intel Optane DC Persistent Memory is expensive...
You get 128Gb modules for 50-70EUR sometimes. There was a German seller on eBay Germany selling them. There's also a UK seller who throughs them out for cheap from time to time. On eBayKA someone sold 512Gb modules for 250Eur each just recently. Puff ... All gone.

Also @Rand__ Posted an eBay sale some days ago.