Looking for advice on build, storage spaces server

[jcl333]

Build’s Name: Storage Spaces
Operating System/ Storage Platform: Server 2019 or ESXi 6.7u1 + Server 2019
CPU: Xeon Silver 4110 (approx $460)
https://www.servethehome.com/intel-xeon-silver-4110-linux-benchmarks-and-review/
Motherboard: Supermicro X11SPH-nCTF (approx $600)
https://www.servethehome.com/supermicro-x11sph-nctf-atx-storage-motherboard-review/
Chassis: Supermicro SC743AC-668B (approx $450.00)
Drives: 7x 10TB WD Easystore's (@$170.00, already purchased)
RAM: Probably 2x 16GB 2666, @ around $150/ea.
Add-in Cards: none at this time
Power Supply: included with case
Other Bits: CPU fan, Oculink cables, SAS cables, etc.

Usage Profile: Plan to use for Server 2019 storage spaces (not direct) and using "mirror accelerated double parity". If that doesn't go well, FreeNAS.

Other information:
- Plan to upgrade my old QNAP 1079 Pro w/9x 3TB REDs to backup this server. CPU, RAM, 10Gig NIC, etc.
- Looking at this 10Gig switch - TRENDnet 8-Port 10G EdgeSmart
https://www.amazon.com/gp/product/B07BV2VYVF/ref=ox_sc_act_title_5?smid=A1AQ06IHQGN1E5&psc=1
- I have an APC double conversion UPS, and my house is getting Telsa Powerwall 2's soon
- So, following the 3-2-1 rule, next will be figuring out a cloud or off-site option, maybe Backblaze

Hello all, I am putting together a server to utilize storage that features data integrity for long term storage. Mostly videos and pictures of my wife and kids. I shoot allot of high-band video, even my wedding was 1080p back in 2003, I rented the same camera they used for Star Wars E1 (it was years before I could even watch it). Today I am shooting 4K, HDR, and 360-degree/VR stuff. I also have about 12 other computers in the house that I will probably backup to this server.

Yes of course this is wicked overkill, I am an IT guy so I want something nice that will be fun to build and have. I also want to try out some of the great things I read about on this site. Right now I have the QNAP, and I back that up to a cobbled-together machine with 20-odd various drives, and it takes about a week if I do a full backup (such as when upgrading firmware). So I want to get to 10Gig which should be able to do it in a day or less maybe.

Here are some of my goals:
- High reliability: ECC RAM, high quality parts, etc.
- Medium price: Trying to keep things around $2,000 if I can, or as close as I can
- Future proofing / versatility: Lots of potential for future upgrades and applications
- Noise: This will be in the basement, I don't want a screamer but quiet is a plus

Here is some reasoning on some of my choices:
- Motherboard: Read the STH review above, I just think this board is amazing. Save money by getting the SAS controller and 10Gig on the motherboard. I considered socket 2011/1151 as well as XeonD options, but hard to get as much good stuff on one motherboard with those.
- CPU: Seems to be a good balance of price and performance for the Skylake series. In a few years I could upgrade substantially when these series of CPUs starts getting old. I looked at the ES one's on e-bay, is that a bad idea? Looks like it might be....
- Chassis: Nice because I can get 8x SAS3 drives, room for 5 more in open drive bays with a 5-in-3
- Why Server 2019 Storage Spaces: Largely because I want to try it. I am primarily a Windows admin at work, then I do allot of VMware. There are not many choices for storage with data integrity. I can deal with FreeNAS, but I have less experience and comfort with Linux/Unix OS's, and it looks like ZFS has some hassles, but is certainly more mature.

Here is where I need some help and advice:
- Enterprise SSDs: Need to get a pair of eSSD's, preferably NVMe, the main thing I need is power protection so that Server 2019 will enable their use for write caching, otherwise performance will be horrible with a parity array. I will likely have more than one volume, one that is a mirror that is also "RAID6" but fast, and then the "cold storage" one using parity. But, there are so many choices. These one's I found here on NewEgg seem like they would meet my requirements, and the price is good:
Intel Fultondale 3 DC P3600 2.5" 1.2TB PCI-Express 3.0 x4 MLC Solid State Drive - Newegg.com
But, am I missing something on these? Is there a better way to go? Then just need to figure out cabling to plug these into the Oculink ports on the MB.

- RAM: I don't need allot to start with. Debating if I should just pick up some cheap 8GB or even 4GB 2400 RAM and get 6x of them in order to populate all the channels. Or, just get a pair of 16GB 2666 modules so I can just add as my needs grow.

- Chassis: I really like that chassis because it has a "right size" power-supply for what I am doing with it so I actually get some good efficiency, and it has a SAS3 backplane. But admittedly these are hard to find. I am wondering if it is feasible to get one of the one's with a SAS2 backplane, and just upgrade it in the future? Then I could get one for even less money and just upgrade later since I don't need SAS3 right away.
Side question: If you have the "TQ" style or "A" style backplanes (e.g. no expander), and you use it with two different controllers, what would happen to the SES3/SGPIO? Would it get pissed off if say, 4 drives are on one controller and the other 4 drives were on another? Or just not work?

Purchasing: Finally, it would be nice to purchase all this from one place, even if I had to pay a little bit more for some parts, I tend to buy things with 1-2 year 0% financing if I can. I know I can't get everything, but I think NewEgg actually has most of the main items I need. I might even be able to have them order some that I don't. But if anyone has any advice on purchasing anything here, or better deals or anything, please let me know, it would be greatly appreciated. Maybe I should look at some of the custom white-box server vendors? I could do lease-to-own....

Thanks much

-JCL

 

[TomUK]

That's a nice spec, but might be overkill for storage spaces by quite a margin.

I hear what you're saying with regards to future expansion, and I like decent kit as much as the next nerd, but I think you could get away with a much smaller and cheaper setup that will still be more than powerful enough.

You could go for a Xeon-E based system and save a load of money on the CPU and Motherboard.

You could also get a much cheaper and smaller chassis in the form of the In-Win MS08 or the Silverstone CS380, or any case with loads of 5.25 bays.

My 2p

 

[jcl333]

That's a nice spec, but might be overkill for storage spaces by quite a margin.

I hear what you're saying with regards to future expansion, and I like decent kit as much as the next nerd, but I think you could get away with a much smaller and cheaper setup that will still be more than powerful enough.

You could go for a Xeon-E based system and save a load of money on the CPU and Motherboard.

You could also get a much cheaper and smaller chassis in the form of the In-Win MS08 or the Silverstone CS380, or any case with loads of 5.25 bays.

My 2p

Hello, thank you for the reply.

I hear you. I have not actually ordered this yet, mostly due to other demands on my time, but also because the price of this build does give me a little pause.

For an update:
- I have already nixed the case, I am currently planning on using one I already have. It doesn't have hot-swap bays but it does a good job at keeping the drives cool and it's really quiet.
- I did like the cases you suggested, especially the In-Win, hard to find 12G cases with SGPIO, one of the main reasons I was considering the SM cases, but they are not cheap.

I certainly would consider looking at Xeon-E or even Xeon-D, but so far what I find is just a trade-off:
- This motherboard has two high-end Intel 10Gig NICs, which would cost almost as much as this entire motherboard separately. Even a cheap 10Gig NIC is $100.
- Then you have the SAS controller, which would cost a fair bit unless you went with something lower end
- This MB has two OCuLink ports, which is really convenient for the two SSDs I am looking at. If I did not have these, I would probably switch to SATA/SAS SSDs, but I really want to use NVMe if possible. But in some cases looks like I could go U.2 or even adapters for M.2.

Looking at the selection of LGA 2066 from SM, none have either 10Gig or SAS, although they do have 5Gig and OCuLink..... Admittedly in my use case I would be able to use the regular SATA ports and only buy the SAS controller if I needed it.

Looks like Tyan makes some with the SAS controller, but no 10Gig.

Gigabyte has a nice one here:
MF51-ES0 (rev. 1.0) | Server Motherboard - GIGABYTE Global
But, very little in the way of reviews, and the only one I found for sale was also $600.

Looking at CPUs, they look kinda pricey to me unless I am missing something here. Higher clock speeds, but twice the price for an 8-core / 16-thread model.

Did you have some specific parts you were thinking of in Xeon-W that you think would be a good fit for my use case that would show the price savings you are indicating?

Thanks

-JCL

 

[JimmyBlack]

Hey mate, signed up just to reply to your post.

I Grabbed the Gigabyte MF51-ES0. It's trash.

It works okay for basic stuff, but:

- BOTH 10gbe ports are useless. They fail every few days. (They go offline, and you need to reboot the server to bring them back) I think the chipset overheats. I had to install a SFP+ card and use a SFP connection to my switch. No more problems.

- There is no bifurcation. It's broken, badly. I contacted gigabyte about it, and they basically told me to go away, don't care. I've updated the bios etc.

- There is no resizable bar, so when I dropped an Intel Arc A310 card into it for hardware decoding (Emby) it just alerts you non stop in the notification list until you turn it off.

The board works..... sort of. I'm stuck with it now. I am now looking at the Asus range but I want to know if the bifurcation works. Can't really see much testing. Only those random reviewers that don't really review.