First time Freenas build, sanity check & advice please!

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

Fixer

New Member
Jun 4, 2017
11
0
1
Santa Monica
www.linkedin.com
Build’s Name: Free Indie
Operating System/ Storage Platform: Freenas (latest, or your recommendation for latest stable version?)
CPU: 2x e5-2650 v2
Motherboard: SuperMicro X9DRI-LN4F+ Motherboard
Chassis: SuperMicro 846 - 24 bay w/SAS2-846EL1 Backplane
Drives:
boot:
SanDisk Ultra Fit 32GB
l2arc: Intel 750 Series AIC 400GB PCI-Express 3.0 x4 MLC SSD
SLOG: Intel OPTANE SSD 900p PCIe 280gb
data: 24x 4TB HGST enterprise drives (20x are from a previous LumaForge server, going to try to retain the data/volumes)
RAM: 8x 16gb PC3-10600R
Add-in Cards:
HBA: SuperMicro AOC-S2308L-L8I (flashed to IT mode)
NIC: Chelsio T540-CR - Network Adapter 10 Gigabit Ethernet
Power Supply: 2x 1200w SQ (Super Quiet) PSU
Other Bits:

Usage Profile: File Server w/VM's

Other information… VR/AR/VFX/CG production studio file server (smb) plus VM's for production tracking/database, pipeline/license server/render queue manager/dns/wiki, proxy transcoding/review streaming. For now there are remote artists delivering files via VPN to the storage server and kicking off renders. The pipeline will determine if the job renders locally (four dual xeon servers w/GTX 1080ti gpu's someday to be Titan V or GV100) or if we send to cloud based rendering via Deadline (render queue manager & cloud rendering w/usage based licensing). Render nodes are connected to the Freenas server via DAC.

Any gotchas to watch for in this build? The VM's will not be added until the SMB File Server is proven bulletproof.
 

CookiesLikeWhoa

Active Member
Sep 7, 2016
112
26
28
35
I would ditch the L2ARC and get more RAM since the board has 8 more RAM slots. RAM will do more for you than the L2ARC will if you plan to run a lot of VMs. (4K Random IO)

I personally like to keep my storage server physically separate from my hypervisors, but you can do an AIO for this with FreeNAS under ESXi or use Proxmox with a ZFS storage system. Just remember that ZFS likes RAM. What ever RAM you use for your VMs will take away from what you can give it for ARC. The less RAM it has the smaller the ARC is and the harder it will be to keep your working files in ARC.
 

Fixer

New Member
Jun 4, 2017
11
0
1
Santa Monica
www.linkedin.com
Make sure you get a rev of that board that supports e5v2 CPUs.
It is. Bought as a package from an Ebay vendor I have bought from in the past (Garland Computers). It came with dual e5-2690v2's that I am shifting to an ASRock EP2C602-4L/D16 to use as a render node. The e5-2650v2's that I am using came from the same vendor as part of a previous SuperMicro package that was upgraded to e5-2697's.
 
Last edited:

MiniKnight

Well-Known Member
Mar 30, 2012
3,072
973
113
NYC
I'm sorry to go here but that's a lot of CPU for FreeNAS. FN is good but it's not really what you want for virtualization.

You've got a great Proxmox node though.
 

Fixer

New Member
Jun 4, 2017
11
0
1
Santa Monica
www.linkedin.com
I'm sorry to go here but that's a lot of CPU for FreeNAS. FN is good but it's not really what you want for virtualization.

You've got a great Proxmox node though.

The only reason Freenas is in the mix is that the 20x 4TB drives have data on them currently, from a LumaForge system that was upgraded to 8TB drives. I am trying to save the data and use the same zfs volumes (LumaForge is based on Freenas).

Maybe as a short term fix I install Proxmox and then install Freenas as a VM there to mount the drives, then work on data reduction and migration to 4x 8TB archive drives, then I can kill the Freenas VM and use the 20x 4tb drives directly in Proxmox?
 

CookiesLikeWhoa

Active Member
Sep 7, 2016
112
26
28
35
The board has 24 dimm slots so I can go even further, but since I already have the Intel 750, would there be any harm in keeping the l2arc and later adding more ram?
Ah, if you already have that's fine. Figured you were buying it. Just remember if you do decided to go down the AIO that you should try to keep a 5 to 1 ratio of L2ARC to ARC or better to ensure you don't hurt performance.