EPYC3251D4I-2T build

csp-guy

Active Member
Jun 26, 2019
350
116
43
Hungary, Budapest
Hi Guys.

I successfully bought my new mini itx server motherboard, so I started to build my replacement server instead of HP N54L Microserver.

Motherboard: ASRock Rack > EPYC3251D4I-2T

RAM: 2x32GB SKHynix HMA84GR7MFR4N-UH ECC REG RDIMM (I will get another 2 modules, I will have 128GB total ram).

Updated: 128GB ram installed :)


HDD: 4xWD RED 5 TB.

SSD: 2x Intel 730 480GB, 1x Intel S3700 800GB, 1x Intel optane 900p 280GB.

BOOT: 2x Apacer 16GB satadom.

Total number of drives: 10

Controller card: IBM M1015 flashed to IT mode plus 2x sas cables

Coolers: 2x noctua FLX 60x20 mm front intake, and 1x Noctua 40x10 mm on SAS controller, 1x Noctua PWM 60x20 mm CPU cooling.

Case: Supermicro CSE-721TQ-250B.

Extra cables: USB 3.0 header to 2xUSB3.0 converter, USB to 9 pin converter cable (To convert USB 2.0 ports in front to fit this motherboard). Noctua Y power cable (Motherboard has only 3 cooler connectors).

Some sheet metal work: I upgraded the side ssd metal holder by cutting 2 pieces of sheet metal of 2.5 to 3.5 converter to pieces and using this solution I was able to put 3 ssd drives to the side of the case.

Drive test: freenas, smb share access from windows, file copy to Optane drive, the 10GBE connection was fully utilized.

I used bhyve to virtualize Centos, Windows etc, worked fine.

I had problems with bhyve, no thanks, it is simple freenas server.

I am satisfied.

Updated: boot drive has been expanded to mirror using another sandisk drive.

Update 2: I replaced the sandisk and the other boot drive to 2 pcs of apacer satadom. This solution requires less space.

Update 3: I updated the bios and BMC to latest, now 16 threads visible.

Update 4: 19/11/2020 - I drilled two the front metal plate of the case and with 2 screws a locked 2 pieces of 2.5" -> 3.5" converter bay below the hdd bays. Plus 2 sata disk is useable. There is 2 cooler near this spot, i do not expect extra warming...

Upgrade 5: 24/12/2020 - i removed 2 sata doms, because the oculink-u.2 cable has been arrived. I inserted intel optane 32GB m.2 nvme ssd to motherboard slot, fast, great. :) I have 4 sata cables without disk drive. I need to upgrade the chassis to 8 bay version.
 

Attachments

Last edited:

Aquatechie

New Member
Oct 29, 2015
28
7
3
Interesting build. That's a awful lot to pack in such a small space. Any concerns about airflow with all the extra cabling?
 

csp-guy

Active Member
Jun 26, 2019
350
116
43
Hungary, Budapest
No problem with the airflow.

Front coolers -> mobo and ram cooling.
Cpu has his own cooler fixed by zip ties.

The m1015 card is cool, too.

I felt some warm when i touched the 10gbe chipsets heatsink.

Ram is cool, too.

The trick was cable management near the psu by zip ties and around the rear cooler to make enough space for the airflow.
 

Aquatechie

New Member
Oct 29, 2015
28
7
3
If you don't mind my asking, from whom did you purchase your motherboard? My Google-fu isn't returning many vendors who actually carry the part. Thanks in advance!
 

csp-guy

Active Member
Jun 26, 2019
350
116
43
Hungary, Budapest
If you don't mind my asking, from whom did you purchase your motherboard? My Google-fu isn't returning many vendors who actually carry the part. Thanks in advance!
Hi,

I am hungarian. I have a friend who runs a small IT Business. He has registration at Asbis Hungary. Asbis is s distributor in EU. They ordered the motherboard from Asbis CZ’s storage. My friend gave me the motherboard with 3 year warranty.

Asrockrack EU is in netherlands, they ships from there.

Regards
 
Last edited:

csp-guy

Active Member
Jun 26, 2019
350
116
43
Hungary, Budapest
62CEBA19-0678-448D-95E3-1AA17960D4AE.jpeg Speed test ISO copy from Windows 10 x540 t1 card to server, the storage is optane 900pbshared directly as smb share.

Finally I attached the 900p as ZIL device over mirrored hdd drives.
 
Last edited:

nasi

Member
Feb 25, 2020
39
10
8
Are you using the original PSU of the SM Chassis? Do you know how much Power your build is consuming under "normal" (whatever that means for you) and maximum loads?
 

csp-guy

Active Member
Jun 26, 2019
350
116
43
Hungary, Budapest
I am using the original psu.

I can not generate full load on this cpu, becuse I am using it as freenas file server, I had virtualized servers using bhyve on this too, but under freenas the virtualization is not fully compatible, so currently it is a simple file server.

I have these components inside the chassis:

- 4 pcs of wd red 5 tb drives
- intel 900p 280GB u2 drive
- intel s3700 800GB drive
- 2 pcs of intel 730 480GB drive
- IBM M1015 HBA
- 2 x noctua 6cm*2cm front intake cooler
- 1 x noctua 6cm*2cm cpu cooler
- 1 x noctua 4cm*1cm hba cooler
- 1 x factory rear cooler
- 4x32Gb ECC ram modules
- 2 x128GB Sandisk (boot) ssd

Power off: 6 W
Power on, initilization: 75w
SAS card init: 120W
Freenas boot to login rompt: 66W, no CPU load
File up load from Hp t630 client 50MB speed / sec -> optane900p zil -> mirrored wd red drives: 83W


Sensor Monitoring
Status Sensor Name Reading
3VSB 3.32 V
5VSB 5.01 V
CPU1_Vcore 0.77 V
VSOC 0.84 V
VCCM ABCD 1.20 V
BAT 3.06 V
3V 3.32 V
5V 5.13 V
12V 12.00 V
PSU1 VIN Not Available
PSU2 VIN Not Available
PSU1 IOUT Not Available
PSU2 IOUT Not Available
CPU1 Temp 27.00 °C
MB Temp 36.00 °C
Card Side Temp 52.00 °C
Onboard LAN Temp 55.00 °C
DDR4_A1 Temp 36.00 °C
DDR4_A2 Temp 34.00 °C
DDR4_B1 Temp 35.00 °C
DDR4_B2 Temp 35.00 °C
FAN1 3100.00 RPM
FAN2 900.00 RPM
FAN3 900.00 RPM
PSU1 PIN Not Available
PSU2 PIN Not Available
PSU1 POUT Not Available
PSU2 POUT Not Available
ChassisIntr No event assertion
CPU_PROCHOT No event assertion
CPU_THERMTRIP No event assertion
PSU1 Status No event assertion
PSU1 AC lost Not Available
PSU2 Status No event assertion
PSU2 AC lost Not Available

Screen capture, htop, freenas upload 70MB/sec.
 

Attachments

Last edited:
  • Like
Reactions: nasi

Aquatechie

New Member
Oct 29, 2015
28
7
3
It appears to be the case. I couldn't find 2666 modules at the density I wanted, so I opted for the 2933 modules thinking the memory controller would simply downclock to 2666. BTW, going into the memory overclock settings in the BIOS and setting the clock speed to match a 2666 module ended up preventing the system from booting. Gonna have to play around some more to see if I can find the right combination of settings.
 

gyik

New Member
Jun 13, 2020
20
3
3
This is listed on the QVL, I will go with this kit and will see.
DDR42666ECC16GBKingstonKSM26ED8/16ME8CE75D9VPPMicron