New DemoEval FreeNAS Storage Build

[Patrick]

Recently I wanted to add some additional shared storage in the lab. I also noticed that post-Las Vegas shutdown I had tons of Pliant/ Sandisk Lightning SLC SAS drives. I decided to see what I had in the spare parts bin to build a new storage setup for the lab.

Head Node

• Intel S2600GZ system w/ 8x 2.5" drives
• 128 GB DDR3 RAM (16x8GB)
• 2x Intel Xeon E5-2690 V1
• 2x Intel S3500 160GB boot drives (WAY overkill but what else would I use them for?)
• 8x 400GB SLC SAS SSDs with a LSI SAS 9300-8i controller (flashed IBM)
• 1x Intel DC P3700 400GB NVMe
• 1x LSI SAS 9300-8e (to JBOD)
• 1x Intel XL710-QDA1 NIC (40GbE Mezzanine)
• 1x Chelsio T580 (2x 40GbE PCIe)

JBOD

• 45 bay Supermicro (iXsystems branded) SAS2 dual port expander JBOD.

Thus far it is extremely mundane with only 26 drives sans-NVMe. Adding another 50+ TB of disk tomorrow. I think it only has 50TB+ in it right now,

I also had to re-wire the S2600GZ machine for 12gbps SAS. I wanted to use the 1.6TB Toshiba PX03 12gbps SAS drives in it so it seemed like a good idea.

My plan is to have enough room left over to add higher capacity drives over time since it is a ZFS machine.







Hardware pics coming. I know this is all a bit eclectic but was just a collection of items I had available.

 

[MiniKnight]

I was gonna say pictures or it didn't happen but UI counts (BARELY). That is like a top 1% most powerful FreeNAS box ever lol.

Hey, why use Chelsio and Intel? Why not one or the other?

 

[Patrick]

Hey, why use Chelsio and Intel? Why not one or the other?

I almost threw a ConnectX-3 EN in there too. Fun which is faster article maybe?!?

Just want to see which I like better. The XL710 was a cheap ebay purchase but the Chelsio is a much more substantial card.

 

[whitey]

Holy hodgepodge :-D (jealous at the plethora of disks you have thrown at this)

Hope you have a disk layout strategy in mind for pool layouts...looks like you may very well be able to offer several tiers/classes of stg off of that HOG :-D

I spies me a 400 GB raid10 pool, a raidz2 4tb pool, mirrored 8tb pool, mirrored 1.6tb pool, use those babies (100-200GB ssd's) as cache dev's to accelerate 4tb pool if they are good SLOG candidates...if those 916gb disks are good L2ARC dev's use them there...400GB NVMe maybe a a SLOG for one of the pools...still some spares hangin' arnd...send those to me, you're not gonna need them hahahha

EDIT: I pray that you mapped those VERY methodically and KNOW w/out a shadow of a doubt what disk/serial maps to what slot. 'sas2ircu 0 display|less' can help w/ this. Also a place w/in FreeNAS UI on the description of the devices to document a slot number/identifier.

2cents, good show sir!

 

[Patrick]

@whitey doing that now. Adding serial numbers to labels. Ordered 3 more Dymo Rhino 1/4 rolls

 

[OBasel]

Aside from a new UI, FreeNAS 10 desperately needs wireframes and a way to show where drives are in a chassis. So hard right now without that.

Also, they don't have enclosure services to let you ID blink a drive right?

Sick build tho. Why not just do all in 1 enclosure?