It was time to replace my aging Dell Poweredge 2900 and Synology 1512+, so I decided on a major upgrade and learned a few things along the way that I thought I'd pass on.
Specs:
Supermicro AS-1113S-WN10RT (1113S-WN10RT | 1U | A+ Servers | Products - Super Micro Computer, Inc.)
The memory, in particular, required some research. There is b-die and c-die memory from Samsung that is very similar but b-die is a 20nm process and c-die is 10nm.
AMD Document about the Epyc and memory: https://developer.amd.com/wp-content/resources/56301_1.0.pdf
B-Die specs: https://www.samsung.com/semiconduct..._8Gb_B_die_Registered_DIMM_Rev1.91_May.17.pdf
C-Die specs: https://www.samsung.com/semiconduct...4_8Gb_C_die_Registered_DIMM_Rev1.3_Feb.18.pdf
C-Die article: Samsung Starts Production of 8 Gb DDR4-3600 ICs Using 2nd Gen 10nm-Class Tech
Supermicro B-Die: https://store.supermicro.com/16gb-ddr4-2666-mem-dr416l-sl02-er26.html
Supermicro C-Die: https://store.supermicro.com/16gb-ddr4-2666-mem-dr416l-sl04-er26.html
B-Die is quite the rage for good performance when searching, but there is no reason why c-die, being a 10nm process, shouldn't outperform it in every way. The spec sheets, if I'm reading them correctly, seem to show a slight edge as well. If anyone knows anything different about this, please chime in.
I also found that the info regarding single rank vs dual rank when trying to run @2666 is limited. The AMD document above touches on it a bit, showing that single rank or LRDIMMs are the only ones running at 2666. Running dual rank at 2666 could perform better, but the data is mixed with different motherboards and BIOSes behaving differently when that is attempted. Overall, I found that running no less than 8 DIMMs and running at 2666 were both key to maximum performance, also partially due to the AMD infinity fabric operating at memory speed. Be aware of how the Epyc handles memory and creates NUMA nodes. This could affect performance depending on your application. It's one of the few downsides of these CPUs vs. Intel.
I only went with 6 of the 12TB drives since I don't need more storage at the moment and thus might as well lower the current running cost, future drive cost, and spread out the drive failures a bit. The Synology server will primarily be iSCSI for the Supermicro server.
The Intel SSD is quite a beast (Intel® SSD DC P4610 Series (1.6TB, 2.5in PCIe* 3.1 x4, 3D2, TLC)) both in speed and lifetime endurance.
Waiting for the Mikrotik CRS312-4C+8XG to come out, and that will allow me to do copper 10gig to the Supermicro (it has 2 10gig ports) and Synology.
The Synology will be here next week - The Supermicro hasn't shipped. I've been dealing with atacom.com, who have been great and are local to Supermicro, so that's handy. Getting distributors to respond is difficult and changing things around/asking questions is not a fast process so don't expect immediate satisfaction if you go down a similar road.
It's pretty awesome to see a tiny SSD replacing an entire 15k array that I have now, and stomp all over it in performance! Looking forward to getting all of it!
Specs:
Supermicro AS-1113S-WN10RT (1113S-WN10RT | 1U | A+ Servers | Products - Super Micro Computer, Inc.)
- AMD Epyc 7551P 32 core CPU
- 8x 16GB Samsung ECC Registered single rank c-die @ 2666MHz
- Intel DC P4610 1.6TB SSD
- 6x Seagate ST12000NM0007 (12TB Exos X12)
- RKS1317 rail kit
- Synology E10G18-T1 10GbE PCIe expansion card (copper)
- Crucial CT16G4WFD8266 16GB DDR4 2666MHz UDIMM
The memory, in particular, required some research. There is b-die and c-die memory from Samsung that is very similar but b-die is a 20nm process and c-die is 10nm.
AMD Document about the Epyc and memory: https://developer.amd.com/wp-content/resources/56301_1.0.pdf
B-Die specs: https://www.samsung.com/semiconduct..._8Gb_B_die_Registered_DIMM_Rev1.91_May.17.pdf
C-Die specs: https://www.samsung.com/semiconduct...4_8Gb_C_die_Registered_DIMM_Rev1.3_Feb.18.pdf
C-Die article: Samsung Starts Production of 8 Gb DDR4-3600 ICs Using 2nd Gen 10nm-Class Tech
Supermicro B-Die: https://store.supermicro.com/16gb-ddr4-2666-mem-dr416l-sl02-er26.html
Supermicro C-Die: https://store.supermicro.com/16gb-ddr4-2666-mem-dr416l-sl04-er26.html
B-Die is quite the rage for good performance when searching, but there is no reason why c-die, being a 10nm process, shouldn't outperform it in every way. The spec sheets, if I'm reading them correctly, seem to show a slight edge as well. If anyone knows anything different about this, please chime in.
I also found that the info regarding single rank vs dual rank when trying to run @2666 is limited. The AMD document above touches on it a bit, showing that single rank or LRDIMMs are the only ones running at 2666. Running dual rank at 2666 could perform better, but the data is mixed with different motherboards and BIOSes behaving differently when that is attempted. Overall, I found that running no less than 8 DIMMs and running at 2666 were both key to maximum performance, also partially due to the AMD infinity fabric operating at memory speed. Be aware of how the Epyc handles memory and creates NUMA nodes. This could affect performance depending on your application. It's one of the few downsides of these CPUs vs. Intel.
I only went with 6 of the 12TB drives since I don't need more storage at the moment and thus might as well lower the current running cost, future drive cost, and spread out the drive failures a bit. The Synology server will primarily be iSCSI for the Supermicro server.
The Intel SSD is quite a beast (Intel® SSD DC P4610 Series (1.6TB, 2.5in PCIe* 3.1 x4, 3D2, TLC)) both in speed and lifetime endurance.
Waiting for the Mikrotik CRS312-4C+8XG to come out, and that will allow me to do copper 10gig to the Supermicro (it has 2 10gig ports) and Synology.
The Synology will be here next week - The Supermicro hasn't shipped. I've been dealing with atacom.com, who have been great and are local to Supermicro, so that's handy. Getting distributors to respond is difficult and changing things around/asking questions is not a fast process so don't expect immediate satisfaction if you go down a similar road.
It's pretty awesome to see a tiny SSD replacing an entire 15k array that I have now, and stomp all over it in performance! Looking forward to getting all of it!
Last edited:
