Server with very slow memory.

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
W

WSmith

New Member
Jun 23, 2021
3
0
1
Hello,

I have a PowerEdge R910 Ubuntu 18 Server that I am testing software on. The server is fairly powerful compute wise (many CPUs) however I am hitting memory bandwidth bottlenecks that are seriously impacting the performance of my application.

I measured the server's memory bandwidth with "mbw" on ubuntu, which showed an average of 1.3Gb/s which seems extremely slow to me? My Desktop PC at home has about 8+ GB/s. My server has memory paging disabled when I run these mbw tests.

If I were to try to improve this bandwidth, where should I start with respect to server configuration or upgrading hardware? Is it as simple as just purchasing RAM with a higher clock speed?
 
N

nasi

Member
Feb 25, 2020
64
20
8
There seems to be something wrong. According to it's Datasheet PowerEdge R910 should have DDR3-1333 RAM. Wikipedia says it has 10.6GB/s bandwidth per module.
First of all you could check the details of your RAM with
Code: 
sudo dmidecode -t 17
Keep an eye on "configured memory speed" and how many modules are deployed/if all memory channels are used.
 
W

WSmith

New Member
Jun 23, 2021
3
0
1
Hey @nasi, thanks for giving some advice.

The output is as below (I realize this is a bit of an Information dump, so I apologies for that.)

From what I can tell, I'm not able to see which channels are being used, but all DIMM slots available are filled so it would seem to reason all channels are being used.

I don't have a "Configured Memory Speed" output, but lshw says they are 4GB DDR3 1600Mhz RAM

Code: 
Handle 0x1100, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 1
        Locator: DIMM A1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA77
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1101, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 1
        Locator: DIMM A2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC4
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1102, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 2
        Locator: DIMM A3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA79
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1103, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 2
        Locator: DIMM A4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817125
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1104, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 3
        Locator: DIMM A5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9AB
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1105, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 3
        Locator: DIMM A6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817127
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1106, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 4
        Locator: DIMM A7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817148
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1107, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 4
        Locator: DIMM A8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817115
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1108, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 5
        Locator: DIMM B1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781713B
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1109, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 5
        Locator: DIMM B2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817158
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x110A, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 6
        Locator: DIMM B3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781713C
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x110B, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 6
        Locator: DIMM B4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817157
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x110C, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 7
        Locator: DIMM B5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781713D
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x110D, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 7
        Locator: DIMM B6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817156
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x110E, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 8
        Locator: DIMM B7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781713E
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x110F, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 8
        Locator: DIMM B8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817155
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1110, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 9
        Locator: DIMM C1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9B4
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1111, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 9
        Locator: DIMM C2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC3
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1112, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 10
        Locator: DIMM C3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9B2
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1113, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 10
        Locator: DIMM C4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC1
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1114, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 11
        Locator: DIMM C5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9B3
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1115, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 11
        Locator: DIMM C6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC2
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1116, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 12
        Locator: DIMM C7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9B1
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1117, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 12
        Locator: DIMM C8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9B5
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1118, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 13
        Locator: DIMM D1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817120
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1119, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 13
        Locator: DIMM D2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817126
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x111A, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 14
        Locator: DIMM D3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781711F
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x111B, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 14
        Locator: DIMM D4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781711D
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x111C, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 15
        Locator: DIMM D5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817121
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x111D, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 15
        Locator: DIMM D6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 37817124
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x111E, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 16
        Locator: DIMM D7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781711E
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x111F, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 16
        Locator: DIMM D8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781711C
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1120, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 17
        Locator: DIMM E1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9AF
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1121, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 17
        Locator: DIMM E2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC0
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1122, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 18
        Locator: DIMM E3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC9
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1123, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 18
        Locator: DIMM E4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAD3
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1124, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 19
        Locator: DIMM E5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAD4
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1125, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 19
        Locator: DIMM E6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC6
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1126, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 20
        Locator: DIMM E7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DACD
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1127, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 20
        Locator: DIMM E8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA7B
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1128, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 21
        Locator: DIMM F1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DACE
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1129, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 21
        Locator: DIMM F2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAD0
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x112A, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 22
        Locator: DIMM F3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DACB
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x112B, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 22
        Locator: DIMM F4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781711B
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x112C, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 23
        Locator: DIMM F5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DACC
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x112D, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 23
        Locator: DIMM F6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA76
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x112E, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 24
        Locator: DIMM F7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DACA
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x112F, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 24
        Locator: DIMM F8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3781711A
        Asset Tag: 0F135163
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1130, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 25
        Locator: DIMM G1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAA4
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1131, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 25
        Locator: DIMM G2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAAA
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1132, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 26
        Locator: DIMM G3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAA5
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1133, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 26
        Locator: DIMM G4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAA9
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1134, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 27
        Locator: DIMM G5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAA6
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1135, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 27
        Locator: DIMM G6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAA8
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1136, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 28
        Locator: DIMM G7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAAB
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1137, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 28
        Locator: DIMM G8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC5
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1138, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 29
        Locator: DIMM H1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9AA
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x1139, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 29
        Locator: DIMM H2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9A9
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x113A, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 30
        Locator: DIMM H3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC8
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x113B, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 30
        Locator: DIMM H4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DAC7
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x113C, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 31
        Locator: DIMM H5
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782D9AC
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x113D, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 31
        Locator: DIMM H6
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA75
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x113E, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 32
        Locator: DIMM H7
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA78
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1

Handle 0x113F, DMI type 17, 28 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: 32
        Locator: DIMM H8
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous Registered (Buffered)
        Speed: 1600 MT/s
        Manufacturer: 802C80B3802C
        Serial Number: 3782DA7D
        Asset Tag: 0F140363
        Part Number: 9KSF51272PZ-1G6E2
        Rank: 1
 
Last edited:
R

RTM

Well-Known Member
Jan 26, 2014
956
359
63
Could NUMA be part of the issue?
Workloads that extend past the cores of one of the CPU's will need to use NUMA which is slow.

Other than that, I suppose someone will eventually have to say, that the server is OLD, and it might as well be me.
If you can I would suggest looking into replacing it.
There it has been said ;)
 
W

WSmith

New Member
Jun 23, 2021
3
0
1
Hello,

NUMA will definitely be a contributor with the applications I am testing, which do utilize many cores. But the MBW tool I used to measure memory bandwidth looks like it is just using a single core.

I wonder if CPU will be a bottleneck here? CPUs I have support a SMI link speed of 6.4GT/s, which is the max speed supported by R910 if I understand the datasheet correctly.

We definitely do need to replace this server at some point :D
 
N

nasi

Member
Feb 25, 2020
64
20
8
WSmith said:
From what I can tell, I'm not able to see which channels are being used, but all DIMM slots available are filled so it would seem to reason all channels are being used.
Click to expand...
Yes, there seems to be no kind of hardware configuration mistake or alike. At least if the speed of 1600MHz is used. Maybe you could double check that somehow (BIOS, IPMI ?).

For what it's worth I have tested "MBW" (nice tool btw) on my Laptop which has 2x8GB LPDDR3-1866 and Core i7-7500U with 4GT/s bus speed:
Code: 
$ mbw 2048
[...]
AVG    Method: MEMCPY    Elapsed: 0.24223    MiB: 2048.00000    Copy: 8454.708 MiB/s
[...]
AVG    Method: DUMB    Elapsed: 0.61930    MiB: 2048.00000    Copy: 3306.975 MiB/s
[...]
AVG    Method: MCBLOCK    Elapsed: 0.40422    MiB: 2048.00000    Copy: 5066.549 MiB/s
Your 4 socket 4-channel system should be much faster than that. And I guess bus speed is not the crucial factor in this case.
 
A

alex_stief

Well-Known Member
May 31, 2016
884
312
63
38
PMBW uses a wide variety of tests for many things related to memory bandwidth. It can also run tests on more than one core. A single core can not utilize all available memory bandwidth on most modern systems.
For simpler (and quicker) tests, STREAM is the the way to go.
 
  • Like
Reactions: nasi
You must log in or register to reply here.
Top Bottom