DIY Oracle ZFS Storage Appliance?

[Moothusala]

Heya folks,

Potentially crazy question here. Is it possible to build/implement Oracle ZFS Storage Appliance using only freely (free as in beer, not necessarily free as in speech, but would be a bonus) available packages? (Linky to Oracles sales site about ZFS/SA: ZFS Storage Appliance - Features | Oracle) Other than sales slicks I can't seem to find anything about this online. This is just a theorycrafting project at the moment as I haven't leaned on my source for Sparc gear and I need to finish the remodel on what will become the server room to make the WAF factor a non-issue. (No way I can run 70+ dbA equipment in the laundry room, Oracle/Sun hardware is LOUD)

Intended application is mainly for labbing using various hypervisors (Proxmox, ESXi, Hyper-v, et al) and potentially moving "production" (home network) storage onto it assuming there's not a licensing issues with that (I haven't checked yet, I do know Oracle's licensing makes about as much sense as Microsoft's lol).

Reasons I'd like to explore this over the popular FreeNAS/OpenFiler:

• FreeNAS's forum community makes my blood boil
• Oracle's new dedupe/compression tech is supposed to be the bee's knees
• Oracle appears to have real "tiered" storage a-la Nimble/Compellent/etc. ("live" tiering, not scheduled tiering like MS Storage Spaces)
• I _may_ be able to source last gen Sparc based hardware fairly cheaply.
• The challenge of learning Solaris.

I'm always open to suggestions, and if there's something else to consider I'd love to hear about it. I've only recently heard of napp-it so I'm working on reading through what's out there on it.

Cheers!

--Moo

 

[MatrixMJK]

Just a few thoughts about this topic. I have and have used a lot of Sun/Oracle gear and it is all very solid and well planned and built, but expensive. We have several Oracle-ZFS appliances here at work and they are great. To get all the features you are talking about, you have to get it from Oracle.

The free and open source stuff is great too, we use it a lot as well, and I use it at home extensively.

A few things will be missing as compared to the HW/SW appliance from Oracle. No real tiering is available. The interface is cli for the most part, but can be alleviated using the EXCELLENT napp-it mentioned all over these forums. Or even Nexenta for smaller installs. De-dupe and compression are both there in the free/OSS versions, but read up on de-dupe especially, may not be worth the memory investment needed. Also it runs on x86 gear very well, Sparc not really needed unless you have the gear or want to use it specifically. You need to run it on Solaris if you want encryption, but it can be on hardware other than Oracle.

Using napp-it should make the 'learning solaris' part better. Make a visit to @gea 's site napp-it.org and read a little about what you can run it on (almost anything really). Even a VM to test it out. By the way, @gea the developer of napp-it, is a frequent posting member of these forums and very helpful.

Lots of build guides and info in this forum as well. Lots of people doing this kind of set up.

Just my $.02, I'm sure lots of others have good suggestions too.

 

[gigatexal]

are you hoping to bring an Oracle storage solution to your home lab? if so, you might be out of luck to Oracle only features without paying for the whole unit. That said if this were a production use case there might be alternatives but I am one who would rather an company like Oracle, or EMC, or Dell furnish the system that comes with a support team and a SLA rather than me being on call and saving all that coin going DIY.

 

[cperalt1]

In case you missed it from the sales brochure.

8x 15-core 2.8 GHz Intel® Xeon® processors

No need for the SPARC hardware. @gea would be a good source with Napp-It but depending on what you wanted it to do you could probably do most of what you want via DIY for home use as you can use Solaris at home without licensing fees. See also. How to Optimize Your Enterprise Storage with Oracle Solaris ZFS

 

[gigatexal]

well I guess you could pay for a solaris license and then bring your own hardware and lean on the community for support, would that work?

 

[Patrick]

• I _may_ be able to source last gen Sparc based hardware fairly cheaply.
• The challenge of learning Solaris.

It has been awhile since I have had a Sparc system (Ultra 10!). I would love to see a DIY build on Sparc hardware.

I do think that you can get a lot of the way there with napp-it.

Actually, if you could do 2 napp-it boxes plus a 4-6 node cluster that would be great. Proxmox might even be fun to start with. Their forums would generally make you want to go back to the FreeNAS forums but they have ZoL implemented and Ceph. You can pretty easily hook ZFS iSCSI storage to the cluster as well.

 

[whitey]

It has been awhile since I have had a Sparc system (Ultra 10!). I would love to see a DIY build on Sparc hardware.

I do think that you can get a lot of the way there with napp-it.

Actually, if you could do 2 napp-it boxes plus a 4-6 node cluster that would be great. Proxmox might even be fun to start with. Their forums would generally make you want to go back to the FreeNAS forums but they have ZoL implemented and Ceph. You can pretty easily hook ZFS iSCSI storage to the cluster as well.

Don't MAKE me 'whip out' my SPARC Ultra 2 Patrick haha (2 slot/cartridge style 400Mhz procs, 1 GB of memory...sh|||||t that thing was a monster back in the day)...

I'll probably get all nostalgic if I pull it out of the crawlspace...think I have a 'green screen' stuffed in there as well, sunray I and II's/etc. SRSS FTW, yeah I'm old and crusty at the tender age of 33 Seen pretty much a majority and dealt w/ the array of thin client/VDI tech over the last 15 years from SunRay, LTSP, Linux PXE booting, vSphere Autodeploy, Citrix provisioning (weakest here), View/Horizon, etc.

Ohh the memories and a trip down IT history/career path memory-lane :-D

 

[Moothusala]

Wow! This got more attention than I was expecting! As I suspected looks like Solaris Storage Appliance is a "product" (Boo!)

@Patrick I'll definitely do a write up if/when I get my hands on some Oracle gear (will probably be early next year) Even if it's a train wreck it'll be interesting.

@gigatexal This would be home lab only, we're a Dell shop at work (we're in the process of buying a new Compellent cluster there)

@cperalt1 Nice catch there, I would have expected this to be on M7 not x86. Makes me wonder if there's a reason even Oracle isn't running it on Sparc (other than those appliances are already 500k+ on x86 for minimum entry)

From some more digging into napp-it, I'll probably wind up using it on OmniOS for "production" stuff at the house. If/when I get some Oracle gear I think I'll try use it natively without napp-it or other interfaces. I have been doing some thinking about the tiering piece, what's the difference between live tiering like Compellent and Nimble compared to having an l2ARC that's larger than your readset/changeset and a ZIL that's larger than your changeset for a day/hour/whatever? (not sure how l2ARC/ZIL aging is handled, haven't gotten that far down in the docs yet)

Thanks again for all the input folks!

--Moo

 

[cperalt1]

Being that you are a Dell shop at work you might want to look into the Dell/Nexenta partnership.

Dell and Nexenta Software-defined Storage Solution

 

[Moothusala]

Being that you are a Dell shop at work you might want to look into the Dell/Nexenta partnership.

Dell and Nexenta Software-defined Storage Solution

I hadn't thought of that, do you know if Nexenta licenses transferable? (ie if I get my hands on a used XC box do I have to purchase licenses still?)

We use these bad daddies at work: Compellent SC8000 Controller - Data Center SAN As much as people knock on SANs the tooling in those has saved my bacon more than once (example: ~32tb restore/rollback from Cryptowall getting ahold of a privileged account, took about 45 seconds) and their raid morphing stuff (raid 10 -> raid 5 or triple mirror autonomously, in place, and on the fly) still amazes me even though it gives me nightmares thinking about what would happen if they patch in a bug to that part of the system.