2011 v2 system not able to load graphic drivers when having 1TB+ ram

[wildpig1234]

I finally got 1TB ram for my 2011 v2 set up now that ddr3 ram becoming very cheap

I don't know what the issue is but for some reasons when i have 1TB or more ram in a dual cpu 2011 v2 system, the video driver just wont load. If I just have one cpu and fill it up with 512GB (X9DRi and Z9PED16) or even 768GB (X9DRi) which means maxing out the ram on that one cpu socket, the video drivers would still load. But if i ever load up 1TB with two cpu, then it won't load the video driver. I have enabled above 4G decoding in the bios.

This is the 64GB LR dimm that i use: Samsung 64GB (1 X 64GB) PC3L-10600L - 8RX4 - LRDIMM Ram Memory (M386B8G70DE0)

Samsung PC3l-10600l ECC LRDIMM Memory Module - 64GB (M386B8G70DE0-YH93Q) for sale online | eBay

Find many great new & used options and get the best deals for Samsung PC3l-10600l ECC LRDIMM Memory Module - 64GB (M386B8G70DE0-YH93Q) at the best online prices at eBay! Free shipping for many products!

www.ebay.com

I use the nvidia K420 on my board.

If someone also running a 2011 v2 with 1TB+ ram please let me know if you are getting the same problem. Thanks

 

[i386]

what cpu are you using?
according to intel arc v2 xeons support only 768GB ram: Intel® Xeon® Processor E5-2697 v2 (30M Cache, 2.70 GHz) Product Specifications

 

[RolloZ170]

2x 768GB = ???

 

[zir_blazer]

What do you mean with video driver? Are you talking about Option ROM on POST or after OS boots? Do you get video output on a monitor plugged into the card when it POST in that condition? What about BMC video output or remote? You need A LOT more details about that.

If anything, I would point my finger at some Firmware limit that conflicts with PCI MMIO assignment. Perhaps workaroundeable in Linux with some Kernel Parameters to reallocate PCI MMIO after booting, but Windows would be another matter. But neither matters if it doesn't work at POST time.

 

[wildpig1234]

i am talking about the windows driver inside windows. my setup is dual 2696 v2. the driver loads as long as I have less than a total of 1TB ram on the MB. Not using linux

I am thinking it's either something in the bios setting possibly? I did enable above 4G decoding but still nothing?

Or can this be some other limitation intrinsic to the 2011 v2 platform? This problem shows up on both the Asus Z9PE-D16 as well as two supermicro X9DAi and X9Dri.

 

[TRACKER]

have you tried different video card?

 

[wildpig1234]

have you tried different video card?

Tried nvidia 410 and k420

Was using win10 ltsc. installed win11. still same problem. latest nvidia driver

 

[mrpasc]

This is a common issue with older NVidias.
The NVIDIA Grid Kx, Tesla Kxx/Mxx, and Quadro Kxxxx/Mxxxx cards were designed with a memory addressing capability of 40bits, which means they are capable of addressing memory locations up to 1TB. If the system memory exceeds 1TB, the NVIDIA card may truncate memory addresses that exceed 1TB, causing the card to access incorrect memory locations.
More information and some options how to resolve that issue: see this:

NVIDIA GPU Adapters system memory addressing limitations - Lenovo Servers - Lenovo Support US

support.lenovo.com

 

[wildpig1234]

This is a common issue with older NVidias.
The NVIDIA Grid Kx, Tesla Kxx/Mxx, and Quadro Kxxxx/Mxxxx cards were designed with a memory addressing capability of 40bits, which means they are capable of addressing memory locations up to 1TB. If the system memory exceeds 1TB, the NVIDIA card may truncate memory addresses that exceed 1TB, causing the card to access incorrect memory locations.
More information and some options how to resolve that issue: see this:

NVIDIA GPU Adapters system memory addressing limitations - Lenovo Servers - Lenovo Support US

support.lenovo.com

I believe this is THE answer. Thank you so much! I Do have a P1000 card. Will test that out to confirm this. Hopefully the P1000 will load the driver.

Well, i guess nvidia never really consider how 1TB ram will be so available to the average STHer when they designed the K/M cards....lol Not sure why they think their K/M cards would not be used with a server or workstation with 1TB+ ram

It sucks that the only solution is to take out ram so that you will have less than 1TB system ram.....lol

 

[wildpig1234]

This is a common issue with older NVidias.
The NVIDIA Grid Kx, Tesla Kxx/Mxx, and Quadro Kxxxx/Mxxxx cards were designed with a memory addressing capability of 40bits, which means they are capable of addressing memory locations up to 1TB. If the system memory exceeds 1TB, the NVIDIA card may truncate memory addresses that exceed 1TB, causing the card to access incorrect memory locations.
More information and some options how to resolve that issue: see this:

NVIDIA GPU Adapters system memory addressing limitations - Lenovo Servers - Lenovo Support US

support.lenovo.com

P600 card does indeed load up driver! Nvs 510, another kepler based card, even though not on that list, doesn't load driver either....
So if you do want to run a GPU with 1TB+ you have to get some sort of a Pascal based card!

Seems like the cheapest single slot Pascal card is the P400 for around $40. anyone has a cheaper idea for a single slot passive cooled pascal card?

 

[tinfoil3d]

This is a common issue with older NVidias.
The NVIDIA Grid Kx, Tesla Kxx/Mxx, and Quadro Kxxxx/Mxxxx cards were designed with a memory addressing capability of 40bits, which means they are capable of addressing memory locations up to 1TB. If the system memory exceeds 1TB, the NVIDIA card may truncate memory addresses that exceed 1TB, causing the card to access incorrect memory locations.
More information and some options how to resolve that issue: see this:

NVIDIA GPU Adapters system memory addressing limitations - Lenovo Servers - Lenovo Support US

support.lenovo.com

Thank you, interesting insight!

 

[bayleyw]

it seems like on Windows you can use bcdedit to limit the addressable memory to exactly 1TB and get the driver to run? it seems like a hassle though, probably better to get a Pascal card.

my real question is what you're doing with 1TB of memory on Sandy Bridge Windows system...

 

[wildpig1234]

it seems like on Windows you can use bcdedit to limit the addressable memory to exactly 1TB and get the driver to run? it seems like a hassle though, probably better to get a Pascal card.

my real question is what you're doing with 1TB of memory on Sandy Bridge Windows system...

why would you want to limit your ram to 1TB using a bcdedit command? that would negate the reason to have 1TB+ in first place. lol...
sometimes there's no reason for us STHers to have 1TB+ ram other than just because we could when the price is affordable enough

Aside from that, the 1TB+ allows me to calculate pi to 250billion digits using only ram! which i just did a few days ago in 10 hrs on the dual 2696 v2

 

[bayleyw]

well, I think it limits the RAM to exactly 1TB - (noise) - in theory the card should be able to address exactly 2^40 bytes of RAM, but because a small part of the lower part of the address space is occupied by things which are not physical RAM you go over the limit and it breaks

 

[wildpig1234]

This is a common issue with older NVidias.
The NVIDIA Grid Kx, Tesla Kxx/Mxx, and Quadro Kxxxx/Mxxxx cards were designed with a memory addressing capability of 40bits, which means they are capable of addressing memory locations up to 1TB. If the system memory exceeds 1TB, the NVIDIA card may truncate memory addresses that exceed 1TB, causing the card to access incorrect memory locations.
More information and some options how to resolve that issue: see this:

NVIDIA GPU Adapters system memory addressing limitations - Lenovo Servers - Lenovo Support US

support.lenovo.com

it seems like on Windows you can use bcdedit to limit the addressable memory to exactly 1TB and get the driver to run? it seems like a hassle though, probably better to get a Pascal card.

my real question is what you're doing with 1TB of memory on Sandy Bridge Windows system...

Does anyone know if it's possible to add a key or value into the windows registry or bcdedit to limit a SPECIFIC device to a certain amount of ram space? If that is possible, then we can probably get the k/m cards to also work with 1TB+ system ram
But my guess is probably not possible to this ?

 

[zachj]

FYI I ran into this on a hp z6 G4 using an amd rx580.

the bios has an option to limit to “1tb” (slightly less actually) but I agree with those who said it’s useless to install more than 1tb if you’re just going to disable it.

the amd Radeon w5500 pro supports 1tb+ memory if you’re looking for an amd gpu instead of nvidia. I can’t comment on what else works.

 

[wildpig1234]

FYI I ran into this on a hp z6 G4 using an amd rx580.

the bios has an option to limit to “1tb” (slightly less actually) but I agree with those who said it’s useless to install more than 1tb if you’re just going to disable it.

the amd Radeon w5500 pro supports 1tb+ memory if you’re looking for an amd gpu instead of nvidia. I can’t comment on what else works.

Would be nice if we can have info also on what amd card has this same problem. For nvidia, it looks like everything going back from Maxwell won't work with more than 1TB.

That is a bit disappointing to know that rx480/580 also have that limitation because it's such a great budget card.

 

[zachj]

Would be nice if we can have info also on what amd card has this same problem. For nvidia, it looks like everything going back from Maxwell won't work with more than 1TB.

That is a bit disappointing to know that rx480/580 also have that limitation because it's such a great budget card.

Agreed I was frustrated something as recent as a 580 is impacted.

my wild guess is that anything Navi-based will probably work; there’s no reason to think the non-pro cards don’t support it at the hardware level. At worst I would expect if a non-pro Navi-based card doesn’t work it’d be because amd nurfed it for market segmentation reasons.

I did some googling trying to figure out when amd fixed it but it’s not as easy to find an obvious answer as it is for nvidia cards. The best I found was this: Graphics Core Next - Wikipedia. As per Wikipedia GCN5 added a larger memory address space, but that’s pretty vague.