Looking for help in building small NVMe storage based system

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Leo Villacorte

New Member
Mar 26, 2018
4
0
1
49
Hello all


I'm looking to build a small NVMe system, preferably Micro-ATX at most. This would be to replace my current Dell T5600

Ideally looking for a motherboard with an x16 slot that can handle bifurcation to support a Dell quad port Ultra Speed card or something similar. I'm open to multiple on on-board M.2 slots as well.

https://www.amazon.com/Dell-Ultra-Speed-Drive-Quad-Adapter/dp/B0714MMD6M

I've been looking around and not seeing much and trying to figure out what boards can handle PCIe bifurcation is not all that easy.

I'm not looking for a crazy amount of storage and looking to make it flexible as I'm still up in the air on Windows or VMWare. I would like six to eight cores, and the ability to take a few 2.5 inch SSD's as well. Multiple 1Gbps NICs also.

I already have a ReadyNAS for mass storage. I want to built this for faster IOPS

Any help is appreciated


Thanks
 

Patrick

Administrator
Staff member
Dec 21, 2010
12,511
5,792
113
Why multiple 1GbE NICs if you do not mind me asking? You are going to want high-speed networking on a box like that.

My dream board is something like this: X11SPM-TPF | Motherboards | Products - Super Micro Computer, Inc.

This may be a good option X11SDV-8C-TP8F | Motherboards | Products - Super Micro Computer, Inc.

If you were going ATX, and were OK with 4 cores, this just hit the lab and is borderline insanely cool MB51-PS0 (rev. 1.0) | Server Motherboard - GIGABYTE B2B Service

If you were bound by licensing, Xeon Scalable would be my suggestion. That gives 8 core options and you can scale to higher clock speeds (Windows) or more cores per socket (VMware) easily in the future. The downside is that anything below mATX is too small for Xeon Scalable.
 
Last edited:

Leo Villacorte

New Member
Mar 26, 2018
4
0
1
49
This is for something at home not work. So out of my own pocket, not going to pay for 10GigE just yet. Plus my ReadyNAS 316 has only 1Gbps and link aggregation on it is pointless. I don't have a lot of stuff at home to come close to maxing out 1Gbps on a sustained basis.

Multiple nics is more for VMWare as I'd like to separate out Mgmt, Vm traffic and possibly make a pfsense vm on it.

Also size is a factor. My Dell T5600 is huge. I live in a small apartment


The X11SDV looks about right. I'll have to keep my eye open for that one.


Thanks!
 

BlueLineSwinger

Active Member
Mar 11, 2013
176
66
28
Seems silly to put together a NVMe-based storage box and not put at least a 10 Gb NIC in it. You may as well stick with SATA SSDs if you're only going to use 1 GB NICs.

Also, you may want to consider merging the functions of the ReadyNAS into the new box, and run the whole thing under FreeNAS or some other storage-focused distribution.
 

Leo Villacorte

New Member
Mar 26, 2018
4
0
1
49
Just so everyone can understand my use case. This is for at home! I have NOTHING on my network that can come close to sustained 1Gbps for long periods of time. Doing Link aggregation on 1GigE is more than enough for me.



ReadyNAS 316 is just mass storage. I dump everything there. Important stuff I have backed up locally to a USB drive and to AWS S3. I don't need snapshots of anything or have a need to get super sexy with the file system

I do move a lot of small and large files around locally on my T5600. Copies, moves, etc. I don't do a lot of network traffic on my LAN.

Some minor video editing for GoPro content. File dumps to the NAS.

I actually do more traffic on the WAN as I have FTTH from Sonic.net

It currently has six 250GB SATA III SSD's in raid 0 Which works rather well. But I'd like something faster with more capacity and something that has a smaller footprint.

I'm also thinking of virtualizing my workstation and pfsense box so that I can have more space on and under my desk


10GigE, is a nice to have. But I'm not going to go out and buy a 10GigE switch



Here's what my network looks like


Dell T5600
|
FTTH - pfsense - Meraki MS220-8P - Google Wifi - laptop/Phone/tablets, etc
|
Chromecast Ultra
 

Nizmo

Member
Jan 24, 2018
101
17
18
37
Soon the market will be defacto 10Gb with gigabit(1Gb) being the equivalent of the old 10/100 networks.

You can do peer-peer 10Gb with a DAC cable if you didn't want buy a switch.
 

Leo Villacorte

New Member
Mar 26, 2018
4
0
1
49
I have nothing else that can do 10 Gig

ReadyNAS 316 can barely do full 1 Gig.

Everything else on my network combined, even streaming 4k video on plex won't scratch 10Gig


I'm not going for LAN speed. I'm going for internal to the disk iops
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
I have nothing else that can do 10 Gig

ReadyNAS 316 can barely do full 1 Gig.

Everything else on my network combined, even streaming 4k video on plex won't scratch 10Gig


I'm not going for LAN speed. I'm going for internal to the disk iops

I think people are confused because your title is "NVME Storage based system" and you even state in the first post " I want to built this for faster IOPS ".

A storage system with large # available IOPs has to be utilized by something.

You didn't state this, other than "storage based system".

What it sounds like is you're wanting a hyper-converged system to run VMs on, and wish to utilize the fast disks for this locally. Is that what you're building?
 

Marsh

Moderator
May 12, 2013
2,644
1,496
113
We are trying to point out if there is no need for NVME SSD if you are staying with 1GB LAN.

Buy 2 or more 1.9TB SSD for $310 each or M1000 SSD for $270 each in the Great deals section, then call it the day.

2 SSD would saturate the 1GB LAN.
 

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
@Marsh he has 0 reason to go the LAN though?

If he wants VMs to be fast having NVME on host will surely do that.
 

Marsh

Moderator
May 12, 2013
2,644
1,496
113
I was just skimping through post #3 and post #5 , about budget and pFsense VM

Some of the VMs that was mentioned would just work fine on SATA SSD.

Previously, I spent too much time on Great Deal section , and accumulate have many TB fast SSD, but, my main server have 1 Intel 800 SSD hosting over 12 VMs ( at least 6 Windows VM are running ) at home.
 
  • Like
Reactions: cliffr

T_Minus

Build. Break. Fix. Repeat
Feb 15, 2015
7,625
2,043
113
I was just skimping through post #3 and post #5 , about budget and pFsense VM

Some of the VMs that was mentioned would just work fine on SATA SSD.

Previously, I spent too much time on Great Deal section , and accumulate have many TB fast SSD, but, my main server have 1 Intel 800 SSD hosting over 12 VMs ( at least 6 Windows VM are running ) at home.
oh yes, been there :D Ha ha ha!!
 

TomUK

Member
Aug 30, 2017
67
12
8
38
If it just a compact and fast home lab box I would go for an AMD Threadripper build if you want to do it cheaply.

The 8 core 1900X is cheap and fast, many of the boards do support ECC and as the boards have 8 dimm slots it's relatively cheap to get to 64GB of memory, or even 128GB if needed (unbuffered memory on TR). TR systems are quad channel so you'll just need a minimum of 4 memory modules to get the most out of it.

I have an ASUS TR board with 2 of their Hyper M2 x16 cards, this let's me have a total of 10 NVME M2 drives maximum (including 2 on motherboard), plus most of the Asus boards have 6 Sata3 ports and a single U2 port as well. The ASrock boards are also good and feature rich for the prices.

ESX support can be a bit hit and miss if your going VMWare - so do your research if that's your hypervisor of choice. Linux support seems fine.
 
Last edited: