projeKt NV24Me

[awedio]

Build Name: projekt NV24Me
Operating System/ Storage Platform:

Chassis: SuperMicro SC216BA-xxxxx
Backplane: Supermicro BPN-NVMe3-216N-S4 24-port 2U (SC216/219) NVMe direct-attach
Power Supply: SuperMicro

CPU: AMD EPYC 7F52 CPU processor 16c|32t
Motherboard: ASRock Rack ROMED16QM3
Memory: 16x HMA82GR7AFR8N-VK HYNIX 16GB (1X16GB) 2RX8 PC4-2666V DDR4
Boot Device (NVMe): 2x Intel Optane P1600X M.2 2280 118GB
Boot Device (SATA): 2x Intel DC3700 400GB
Bulk Storage: 24x xxxx NVMe

HBA: n/a
NIC: Mellanox OCP 3.0 ConnectX-5 EX MCX566A-CDAI 100GbE Dual-port QSFP28 PCIe 4.0 x16

Possible Future Additions

HBA:
Bulk Storage:
ZIL/SLOG

 

[zac1]

Sweet system! @DarthCirrus is doing something similar. Think you two are using the same chassis and backplane.

Also, nice boot drives.

 

[DarthCirrus]

Awesome board! I missed this one as I was doing my research; my plan was an H12SSW-NT before I got a Tyan S8026 from @zac1 but this is a fantastic board for storage with all 24 drives directly connected to the board! Why 4 boot drives? Have you decided on an OS yet? How about plan for what NVMe drives you’ll be using?

Pics/details to follow on my build soon!

 

[awedio]

@DarthCirrus How's your project coming along? (I'm now following along)

Why 4 boot drives? Have you decided on an OS yet?

4 boot drives was more of a note to remind me of the available options. (SATA or M.2)
No on the o/s, have you?

How about plan for what NVMe drives you’ll be using?

Not yet...need to replenish the pocket book!

 

[awedio]

added:
NIC: Mellanox OCP 3.0 ConnectX-5 EX MCX566A-CDAI 100GbE Dual-port QSFP28 PCIe 4.0 x16

 

[awedio]

add NVMe backplane

 

[DarthCirrus]

@DarthCirrus How's your project coming along? (I'm now following along)


4 boot drives was more of a note to remind me of the available options. (SATA or M.2)
No on the o/s, have you?


Not yet...need to replenish the pocket book!

Project is good, posted some pics in Zac’s thread about the motherboard and how it fits in the build. I’ll get to put the system through the paces in July, so will post a full write up then.

My plan is UnRAID, so booting off a flash drive. With all NVMe flash in RAIDZ1 I need to ask someone who knows things if there’s any benefit to a ZIL/SLOG or metadata special device using Optane…

Last thing to buy before July is a 100gbE ocp2.0 nic for reasonable…..

 

[awedio]

My plan is UnRAID, so booting off a flash drive. With all NVMe flash in RAIDZ1 I need to ask someone who knows things if there’s any benefit to a ZIL/SLOG or metadata special device using Optane…

Last thing to buy before July is a 100gbE ocp2.0 nic for reasonable…..

Which NVMe drives will you be using?

I just got a few Mellanox ConnectX-5 (dual port) OCP3.0 nics, each $200

 

[DarthCirrus]

Which NVMe drives will you be using?

I just got a few Mellanox ConnectX-5 (dual port) OCP3.0 nics, each $200

I have 8x PM1733’s to start. Unfortunately OCP3.0 isn’t backwards compatible with OCP 3.0, completely different design.