PVE nodes- Supermico H11SSL-C & EPYC 7401P in progress

EluRex

Active Member
Apr 28, 2015
218
77
28
Los Angeles, CA
Build’s Name: RCluster (30 nodes)
Operating System/ Storage Platform: Proxmox VE 5.1
CPU: AMD Ryzen 7401P
Motherboard: Supermicro H11SSL-C
Chassis: RMC3116-670-HSE with SAS3 expander
Drives: (16) Seagate ST6000NM0034 SAS3 6TB
RAM: DDR4 2666mhz ECC Reg 64GB x4 and 64GB x8
Network Cards: MCX354A-FCBT in IB FDR14
Power Supply: DSP Dual 800w Platinum
SSD Cache: Samsung sm961 1TB (undecided)
Other Bits: 2 x Mellanox SX6036 FDR14 switch for HA purpose


AMD EPYC 7401P are really huge and they are really as described as Intel indicated, glued together and the slide installation method is kinda new to me

MVIMG_20180306_111026.jpg

MotherboardMVIMG_20180306_111935.jpg

some of the ram moduleMVIMG_20180122_193755.jpg

SAS3 HDDMVIMG_20180129_125617.jpg

MVIMG_20180129_125538.jpg
MVIMG_20180309_162300.jpg

SSD Cache with coolingMVIMG_20180308_104617.jpg

some modules are sm960 pro 1tb
MVIMG_20171228_134911.jpg

chassis 16 3.5" bay with 2 2.5 bay @back
MVIMG_20180306_111617.jpg
IMG_20180306_111619.jpg

mellanox nic
MVIMG_20171228_145953.jpg
one of the mellanox switch
MVIMG_20171227_135327.jpg

another 4 supermicro x11ssh-ctf for home lab nodes
MVIMG_20180303_093905.jpg
 
Last edited:

Evan

Well-Known Member
Jan 6, 2016
3,279
566
113
With all the $$ spend elsewhere isn’t the ssd a bit underdone, an enterprise one would have been nicer.
What’s going to run on that ?
Big cluster :)
 

EluRex

Active Member
Apr 28, 2015
218
77
28
Los Angeles, CA
With all the $$ spend elsewhere isn’t the ssd a bit underdone, an enterprise one would have been nicer.
What’s going to run on that ?
Big cluster :)
Most of the parts are purchased @ unbelievable low price except motherboard and cpu...

I am still undecided on SSD cache and so far only purchased a few for testing. Ideally Intel optane SSD would be great but its way too pricey. Unfortunately Samsung Z-NAND SSD is still no way to be found... good thing is SSD cache can be added after the cluster goes live

The cluster will run Proxmox VE 5.1 (kvm hypervisor) + ZFS/CEPH bluestore

@PigLover ,

The cluster will run Odoo ERP/PLM/MES/Workflow/Zimbra Enterprise Email Suite, Corporate CMS/WebRTC and hadoop. they are replacing nodes ranging 5 yrs old~ 10 yrs old servers... then more services will added to the cluster
 
Last edited: