Hi I'm new and looking for some advice!

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Hamtastic

New Member
Aug 1, 2019
2
0
1
Hi All,
I'm looking for some advice on building a file server, I want to use it for plain old file storage, both for backups and archiving old stuff, and for fast external storage for my Mac Mini 2018 with 10gb ethernet.

I've so far cobbled together a selection of parts, without much rhyme or reason, so I don't mind spending a bit more to round out the system, what I have so far:

Intel Server board S3420GPV with a quad core xeon, (cant remember exactly what model)
32gb ECC DDR3
2x 120gb Sata SSD's (Boot drives?)
4x 240gb Sata SSD's (fast storage, caching?)
4x 2tb HDD's
Theres a old Dell SAS controller in there, A dell sas 6/ir , so as basic as it gets really, I do have a second one if needed.
A USB 3.0 Card
and a Aquantia 10gbe network card (not currently supported by FreeBSD sadly)

Ideally I want to directly connect the 10gbe straight to the Mac Mini to avoid a costly switch, and use the other two ports for the rest of the network.

I've juggled OS and configurations (Windows Server, Ubuntu etc) and file systems but not really found what to settle on.
Ideally I'd like one big storage pool with a single volume with some sort of caching, that I can access from both windows and mac os, and have it fast enough to keep the 10gbe nice and full.

I've looked ubuntu server with ZFS, but after reading all the guides I still don't know the best way of config the ZFS.
I have a MSDN sub so trying windows server is an option, but not sure storage spaces is worth the hassle.

Any advice would be good, or new ideas to try!
 

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
It pains me to ask this - what about swapping the Aquantia for a better card? Then you can just use FreeNAS which lets you use ZFS and fixes your FreeBSD issue. Ideas for that I'll leave below. Alternatively, Proxmox VE has ZFS that the installed does for you and you can now make ZFS arrays from the web interface. The easy interface isn't as good as FreeNAS but there's a file sharing container that can get you everything you need. That's more learning, but Linux is better. How is Aquantia support in Linux? No idea since we'd never use those NICs but if its good, Proxmox will support. If those fail, I'd just do Windows.

Maybe a Chinese X550 https://www.ebay.com/itm/New-Intel-X550-T1-10G-Ethernet-Server-Adapter-Converged-Network-Adapter/173708301183 $130

X540-t2 is $85 https://www.ebay.com/itm/DELL-INTEL-X520-T2-PCI-E-NETWORK-CARD/273822833199

If not, the X520-T2's are cheap at under $55 now https://www.ebay.com/itm/DELL-INTEL-X520-T2-PCI-E-NETWORK-CARD/273822833199 but they've got the fan.
 

Terry Wallace

PsyOps SysOp
Aug 13, 2018
197
118
43
Central Time Zone
Second MiniKnight ! Go with FreeNas for what you are describing. You can have your cache layered storage pool and it supports apple sharing out of the box for seamless Time machine backups. I personally use the connectx-3 cards but I'm going to a sfp switch. So an intel card is your best bet from ebay unless you want to get something like this

MikroTik
thats a 4 port 10gig switch new for $150. Then your mac, your fileserver are 10Gig and you still have a couple of ports left over for down the road :)

p.s. thats an sfp switch so you'd also need a 45$ sfp to rj45 adapter. I'm using one of those for my mac mini to get it on my 10gig backbone.
 
  • Like
Reactions: Hamtastic

Hamtastic

New Member
Aug 1, 2019
2
0
1
Fantastic replies guys, exactly what I wanted.
Thats given me some good options, leaning towards replacing the aquantia and going freenas, I think that switch is probably a good idea anyway, didn't know such a thing existed!
I'll almost definitely find a use for the Aquantia elsewhere, I got it cheap as a amazon warehouse deal anyway ;)
 

ttabbal

Active Member
Mar 10, 2016
743
207
43
47
Since you're leaning toward FreeNAS, there is no "best" way to configure ZFS. :) There are a number of trade-offs and you have to decide what works for you.

A couple things to know, just in case you haven't seen them..

The Dell controller is likely a RAID card and might not be the best choice for ZFS. You want a "dumb" card. The upside is that means it's a cheap card and nearly ideal cards for ZFS spinners are super cheap. H200/H310/etc..

raidz cannot have drives added or removed. If you create a 4 drive array, it will always be a 4 drive array. You can destroy it and use the drives to make a different size array, but obviously that loses data on the array. You can stripe more arrays in, with the same conditions.

You CAN replace drives with larger drives. Start with 4x2TB, move to 4x4TB. You have to do it one drive at a time, letting the reslivering process complete each time. This is slow, but it does work.

Since you are using 10Gbe, and seem interested in performance, I would suggest mirrors. Make 2 pairs and stripe them. Better performance, faster repair time, and easier to upgrade as you can do 2 at a time. Downside is you "lose" more space, always trade-offs.

Cache... well... You are going to hit rust eventually. You could use L2ARC with the SSDs, but I don't think it will be as effective as you expect. It also needs RAM to manage the cache, RAM that's no longer available to be used as cache. Particularly if you want that much. It's actually possible to slow performance of the overall system with L2ARC. If you want cache, look for small, really fast SSDs like Optane. Use a small part for SLOG, and some for L2ARC. Large slower SSDs are not a great choice for this.

You can use them, just create a second pool and use it as working space. I think I would do a plain stripe with them, no redundancy, and have a job on the server that creates backups to the main pool. Just accept that a failure means data loss and plan for it.

However, cache and log devices can be added and removed at will, so test it. Note that the L2ARC is not designed to be used as tiered storage, so trying to do that will require some manual, probably command line, tweaking. One thing to note is that by default, streaming reads are not cached. I've heard some people enable that and tweak the settings to allow things like media files to be cached and streamed from SSD.