Is my ZFS server too powerful? It only needs to saturate two Gbit NICs.

I'm in the process of designing a backup storage server. This box will serve as a backup to our primary backup server. The only things we need out of it are the following:

1) Be able to fully saturate two Gigabit network cards (250 MB/s)
2) Lots of storage space (100TB ish)

The box will be connected via iSCSI to my primary server. I plan on using rsync (or something similar) to copy data onto this box. I plan on using OI+Napp-IT as the OS and storage management interface. We will only be writing data to this server. Never reading or serving out data. The only time a read will happen is in the event of a major disaster.

With that in mind, i did some research and have speced out the following items:

Chassis - Supermicro SC847E16-R1400LBP
Motherboard - SuperMicro X9SCM-F-O
CPU - Intel Xeon E3-1230V2 3.3Ghz
Ram - 32GB DDR3 1600 Unbuffered Ram
HBA - LSI SAS 9207-8i Controller Card
HDD - 36 x 3TB (possibly 4TB) 5400 RPM 6Gb/s Sata drives (Model TBD)​

Now to my questions:

1) Will i be able to saturate both my NICs and get a full 250MB/s into this box?

Obviously my main bottleneck is the network. As far as i can tell, with the specs above, i should be able to get at least 250MB/s. Am i missing something? My understanding is that rsync uses asynchronous writes. So i shouldn't really need any sort of SSD for a ZIL. Please correct me if i'm wrong here. My understanding of the above system's performance is this:

|HDD x 36 = 27,000 MB/s| -------> |Expander = 3,000 MB/s| ------> |PCIe 3.0 x8 HBA = 8,000 MB/s| ------->|Motherboard/CPU/Ram = A lot| -----------> |Bonded NICS = 250MB/s| ------> Network data

Beyond the NICs, the biggest bottle neck appears to the be expander. And thats what i'm a little worried about. The SuperMicro E16 expander is a SAS2 6Gb/s backplane. According to SuperMicro tech support, the expander has a single SFF8087 port that has four 6Gb/s lanes. According to the internet, each 6Gb lane = 750 MB/s. That means the single sff8087 port is capable of 3,000 MB/s. Is this correct? Aside from the technical bugs i've read about regarding expanders and ZFS, will this bottleneck be an issue for ZFS or other system functions? (Silvering, healing, verification, de-fragmenting, etc)

2) Is the box i've speced out above too powerful?

While i realize a lot of you build home storage servers that run ESXi and do all sorts of awesome things, this box only needs to do one thing. It only needs to write data to the disks and verify that the data is correct. I want to make sure i have enough horsepower to do what I've outlined above, but i also want to make sure i'm not buying a Xeon proc when i only need an i3 proc. (Backblaze, for example, only puts an Core i3 proc in their pods.) I just need the box to be able to keep up with the NICs and make sure ZFS will be able to do its thing and keep the data from rotting.



Thanks for helping me out. This forum has a wealth of information. I have found so many great posts here with a lot of very helpful information. Any and all advice or suggestions are welcome. Once i get all the parts i am planning on posting up here for everyone to see. It should be a lot of fun and a good experience.

 

Thanks for the response. Sorry for the delay in responding. I got busy at work and haven't had time to read this thread.

I completely forgot about the overhead of the gig nic. I'll be happy with 200 MB/s. As for the origins of the data, most of it will be coming from different sources, so it should be fine. If i do bond the nics, i'll probably go with an LACP link agg setup that will bond the two nics into one big pipe. I'm assuming Solaris supports such this as i know linux/bsd/windows/mac all do.

I am planning on setting it up in a Raid-Z or possibly z2 setup. (Most likely 3 separate raid-z pools) The data should be fairly small but i'm not 100% sure on that one. I'll definitely look into it.

Features wise, i'm not planning on doing dedupe (not nearly enough memory for that). Compression, on the other hand, looks like a good idea. From what i've read it helps performance. Though i'm not very familiar with ZFS.

And i have verified compatibility. So far as i can tell, OI/Napp-IT supports my mobo, hba, and expander. I've also read several sucess stories with the backplane/expander and my hba. Though if i have problems i can always send the hba back and try something else.

 

Compression squeezes more data through a connection. If you can move 200MB/s of data over a connection, you effectively move more data if you can turn 400MB into 200MB for the journey.

 

I would be curious as to where you would be finding this RAM, most of the re-sellers I am familiar with have only 1 model of these and they are ridiculously expensive.

 

Kingston KVR1600D3D4R11S/8G on ebay

You need 4 though.

 

Those are registered and will not work. They need to be UDIMMs or Unbuffered if you will.

 

What about Kingston KVR16E11/8? unbuffered ecc. Still the KVR16E11/8 is expensive.

 

OUCH! That is expensive. Looks like I am sticking with 4x4gb modules. Looks almost like a poor justification to upgrade my motherboard and processor!

 

I thought there were 16GB kits. Just don't know p/n

 

Now that would be nice! I can't justify it since I just built a new server!