Server design for 12 memory channels per CPU?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Y0s

Member
Feb 25, 2021
36
7
8
Hi,

Genoa will increase memory channels from 8 -> 12 (matching core count growth, disclosed in the recent AMD Financial Analyst Day). Current 2 socket x 16 DIMM slots already fill the width of standard 19" chassis (or 8 DIMMS per socket in 2U4N configurations). Where are the additional DIMM slots going to physically fit? Power per socket is also increasing, are we moving towards single socket servers?

Cheers, Y0s
 

i386

Well-Known Member
Mar 18, 2016
4,218
1,540
113
34
Germany
Another thing I didn't think of: I have seen a supermicro server on the cebit 2011/12 expo that had daughterboards with 8x dimms per socket installed, I think that system had 4 or 8 cpus total.
Are you saying there will be 1xDIMM per memory channel and we'll switch to CXL for expansion?
I think it would make sense to get as many dimms directly connected to cpu as possible and then use cxl if more ram is required (at the cost of higher latency).

@Patrick can give probably more input :D
 

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,059
1,478
113
Another thing I didn't think of: I have seen a supermicro server on the cebit 2011/12 expo that had daughterboards with 8x dimms per socket installed, I think that system had 4 or 8 cpus total.
24 DIMMs per CPU has been available for a while.

4 CPUs with 96 DIMMs: 8048B-TRFT | 4U | SuperServer | Products | Super Micro Computer, Inc.
8 CPUs with 192 DIMMs: 7088B-TR4FT | 7U | SuperServer | Products | Super Micro Computer, Inc.

As for space constraints, Supermicro has had 32 DIMMs in 1U systems without risers for some time, so having 24 won't be a problem: AS -1124US-TNRP | 1U | A+ Servers | Products | Super Micro Computer, Inc.
 

Y0s

Member
Feb 25, 2021
36
7
8
Supermicro has had 32 DIMMs in 1U systems without risers for some time, so having 24 won't be a problem
Yes, current standard server is 2 sockets x 8 memory channels x 2 DIMM slots per channel = 32 slots. Genoa will be 48 following the same scaling?

Will dual sockets move inline? 2U4N is going to be tight squeeze!

Cheers, Y0s
 

BlueFox

Legendary Member Spam Hunter Extraordinaire
Oct 26, 2015
2,059
1,478
113
Yes, current standard server is 2 sockets x 8 memory channels x 2 DIMM slots per channel = 32 slots. Genoa will be 48 following the same scaling?

Will dual sockets move inline? 2U4N is going to be tight squeeze!

Cheers, Y0s
I would expect 1 DIMM per channel, not two. 12 DIMMs per CPU has been available in the 2U quad node systems for a while and there's not room for anything beyond that. Don't see how that will be a problem with DIMMs going up to 256GB.
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Another thing I didn't think of: I have seen a supermicro server on the cebit 2011/12 expo that had daughterboards with 8x dimms per socket installed, I think that system had 4 or 8 cpus total.

I think it would make sense to get as many dimms directly connected to cpu as possible and then use cxl if more ram is required (at the cost of higher latency).

@Patrick can give probably more input :D
Like this perhaps? https://www.servethehome.com/supermicro-superserver-8048b-tr4ft-4p-e7-v3-big-iron/