Cockpit ZFS Manager

[optimans]

Cockpit ZFS Manager
An interactive ZFS on Linux admin package for Cockpit.

Use of this software is at your risk!

End-of-Life

This software has reached its end-of-life and is no longer maintained.

Requirements

• Cockpit: 201+
• NFS (Optional)
• Samba: 4+ (Optional)
• ZFS: 0.8+

Download

github.com/optimans/cockpit-zfs-manager

Screenshots



Screenshots continued: Post #3

 

[dragonme]

been running zfs on OS X since 2008 ish.. while I would not consider myself a massive power user, I would be willing to beta test a bit

I was a software engineer / quality assurance and IV&V director of test in a previous life

not familiar with the cockpit project.. link?

currently my OS X zfs server is somewhat sidelined and a backup of sorts

currently running a ESXI all in one

napp-it storage both direct to the network for NAS and to esxi for nfs backing

have several linux VMs running various tasks..

 

[optimans]

Screenshots continued

 

[optimans]

@dragonme Thanks for response, I have added the link as requested and some more information/screenshots.

 

[dragonme]

sounds like a great project.. and very timely now that joyent have abandoned solars as their head for zfs development and moved linux to the head spot moving forward as the lead operating system

my thoughts on zfs

thus far I have dabbled with many front ends, Sun's zfs storage server (one of the better ones), freenas, napp-it etc

one glaring issue with them all is that administration of ZFS from a GUI only interface at some point almost always leads an admin to do something stupid...

1> because its not always evident WHAT or HOW something is actually going to be done.. so graphically it all looks good but the action fails to do what the admin THOUGHT was going to happen.. and MANY things in ZFS are un-reversable are require the full destruction and rebuild at best or DATA LOSS at the other end

2> GUI administration tends to lower the bar and stagnates or eliminates the learning curve that I feel anyone managing a ZFS array needs to learn

SOLUTIONS

my suggestions then for any GUI front end to ZFS is that

1> any ACTION features of the GUI should show the actual terminal commands that the pending operation will accomplish and be editable so the admin can tweek it if necessary.

2> clearly warn when a pending activity will result in a irreversible data structure change or result in data destruction. and show a pre and post view of what is pending before execute is pressed.

3> monitoring .. I think that none of the current front ends really power use dtrace. sure they provide pretty graphs (some of them) but its usually rather simplistic raw data.. not really curated into a collection of graphs of analyzed raw data to help solve an actual bottleneck or other issue. Suns storage servers again are better here and it stands to reason .. they have been at the ZFS and dtrace game since the beginning since they invented both. and their ZFS skunkworks had some really great dtrace graphs and scripts.. I watch as they guy started yelling at a 36 drive shelf and the graphing dtrace screen was showing latency jitter on the drives in real time showing that the noise of his yelling was effecting the array .. powerful stuff

Lastly... on linux

I hate doing basic house keeping and setup of linux from the command line (my spoiled OS X upbringing) so a great suite of admin tools to properly configure and monitor the linux host (like webmin but better) would be a welcome breath of fresh air.
tuning of the underling operating system network for example is just as important as turning the knobs on tuning ZFS in order to fully realize high performance operation ...

I assume this runs on server linux without a gui front end but generate a web gui for Cockpit...

I look forward to giving it a spin...

bare metal and esxi ova .. can test both... and I can put it on some older commodity hardware (like most home users)

this will be a winner if its feature complete, fully free, and not cut down like other solutions and of course being the head for openzfs .. should remain as cross platform pool capable as possible... or again, warn a user before making or changing a pool feature flag that will limit it portability .. this is one of the most important benefits of ZFS.. being able to yank 36 drives out of one shelf on zfs and move it over to a completely different installation running a different operating system and import it like it was native...

 

[dragonme]

That aforementioned Linux migration added Delphix's impressive array of OpenZFS developers to the large contingent already working on ZFS on Linux. In November, the FreeBSD project announced its acknowledgment of the new de facto primacy of Linux as the flagship development platform for OpenZFS. FreeBSD did so by rebasing its own OpenZFS codebase on ZFS on Linux rather than Illumos. In even better news for BSD fans, the porting efforts necessary will be adopted into the main codebase of ZFS on Linux itself, with PRs being merged from FreeBSD's new ZoL fork as work progresses.

sorry .. old age is a bitch.. I meant Delphix.. joyent probably too as openzfs is pretty much lead by Delphix

 

[dragonme]

gea had a good post about it.. and rather than link it here it is..

but I have read in multiple places that Zol has now become the head for openZFS development

a final brake if you will from solaris and sun... its been a long time coming

when they dropped community support from opensolaris and denied access to zfs codebase and it was forked (for legal reasons) into the non compatible with solars openzfs.. the writing was on the wall... with Linux proliferating into the server space its only natural that it will begin to dominate the storage space as well

here are gas's comments


It's all about new ZFS features.
In the last years they were mainly developed from Illumos related firms like Delphix, Joyent or Nexenta where most ZFS developers landed when they left Oracle after overtaking Sun.

The common Open-ZFS platform ensures that new features could be part of ZFS in Free-BSD or ZoL

Now the situation is different.
The Linux platform is much much larger than the Unix variants Free-BSD or Solarish. ZFS is now also the leading filesystem on Linux with many firms developing around ZFS.

A typical example is ZFS encryption. While it is based on the OpenSolaris/Illumos bits where it was nearly ready (Oracle published it with Solaris 11) and not Linux to avoid GPL problems, it is finalized by Datto on ZoL and is now in the process of beeing ported to Illumos and Free-BSD.

Porting it to Illumos and then from Illumos to Free-BSD makes no sense. Porting form the OS of origin to a target OS is always needed and should be done directly.

OS of origin of new ZFS features is more and more ZoL and to a lesser extend Illumos.
After all this is good news for ZFS on all platforms and this means today Free-BSD, Illumos, OSX, propably Windows and Linux.

Up to now new ZFS features were mostly available first on Illumos and with a delay on others. Whats happening is that more and more are first published on ZoL and with a delay on the others.

 

[optimans]

I assume this runs on server linux without a gui front end but generate a web gui for Cockpit...

Yes, Cockpit has its own web service, can run headless; as the project continues, they are adding more and more controls to it, can even use Cockpit as a virtualization host to manage your VMs.

bare metal and esxi ova .. can test both... and I can put it on some older commodity hardware (like most home users)

This is exactly what I'm looking for. I've been testing and trashing real and virtual servers/disks on my end, but need different configurations to prove it works as should.

1> any ACTION features of the GUI should show the actual terminal commands that the pending operation will accomplish and be editable so the admin can tweek it if necessary.

May be able to add an expert/advanced mode that user can select so they can see command info if need be.

2> clearly warn when a pending activity will result in a irreversible data structure change or result in data destruction. and show a pre and post view of what is pending before execute is pressed.

Some of the modals do include warning text, however could probably add a switch toggle to critical actions (VDEV add/remove, disk offline/replace/labelclear, upgrade, etc) so they can't continue without acknowledging.

Ease of use as and cross compatibility is definitely part of the plan.

but I have read in multiple places that Zol has now become the head for openZFS development

I think this is great news, it shows that ZFS is very much alive, and the power of the community as a whole.

 

[dragonme]

I think ZFS still has a bright future...

most important of a file system is maturity and ZFS has more than a decade under its belt

I have been running ZFS on multiple platforms using cheap consumer grade drives in against standards single disk 8 drive raidz volumes and even 6-8 wide volumes running NO redundancy.. have not lost a single file in over 10 years

only question mark moving forward as openzfs has had to refactor and reverse engineer most of suns code... will the forward development be as thought out, stable, and capable... so far it looks good

 

[gea]

The case is diferent.

Up from ZFS v28/5 (OpenSource OpenSolaris) Oracle developped its own ZFS line Solaris 11,x. While native Oracle ZFS is still the fastest and most feature rich ZFS , Open-ZFS is improving its own fork of (Opensource) ZFS. No (illegal?) reverse engineering but independent software development.

 

[Elliott]

This looks great! Are you implementing this using ZFS Channel Program with JSON output?
Please consider contributing to the project, there is an open issue for ZFS support here:
ZFS support · Issue #3519 · cockpit-project/cockpit

 

[Raice]

I have been working on a ZFS manager package for the Cockpit project for last couple of months

Wow! Great project! How can I try it?

 

[Robertejes]

I'd like to try testing too. Have some QA experience with oVirt

Spoiler:

Just hiding it from everybody. I also have a good experience with testing of paraphrasing tool pages(for example, Paraphrasing Tool Online for Free). If you look at it, it works as business owner needs.

 

[optimans]

Wow! Great project! How can I try it?

Currently doing code clean up at the moment . Still hoping to release a beta in about a weeks time. Will update details on first post when available.

 

[optimans]

This looks great! Are you implementing this using ZFS Channel Program with JSON output?
Please consider contributing to the project, there is an open issue for ZFS support here:
ZFS support · Issue #3519 · cockpit-project/cockpit

Not using JSON yet, but will have a look at it, would probably make zpool status coding easier.

I had a look at the GitHub issue a little while ago; looks like its labelled as blocked, wonder if there will be any review in near future.

 

[Wolfstar]

Tossing a "pick me!" on here to track it. Just migrated to Ubuntu 18.04 off of OMV because if I'm going to manage ZFS on the command line anyhow, might as well do it on something with more options in general. I know that rules me out from a version perspective, but I don't mind updating to 19.04 or (when it comes out) 19.10 interim for testing.

 

[optimans]

Thought I'd better give a quick update:

Nearly finished with code work; bug fixes and trying to maximize compatibility. NFS UI support has been added plus more. Almost ready for final testing. Shouldn't be too much longer.

 

[alazare619]

Thought I'd better give a quick update:

Nearly finished with code work; bug fixes and trying to maximize compatibility. NFS UI support has been added plus more. Almost ready for final testing. Shouldn't be too much longer.

Do you have a github or gitlab...with this + cockpit in general thats everything thats needed for a roll your own very effecient nas..super excited i'd love to give it a try and submit any potential PR's against as needed

 

[optimans]

Do you have a github or gitlab...with this + cockpit in general thats everything thats needed for a roll your own very effecient nas..super excited i'd love to give it a try and submit any potential PR's against as needed

Yes, have got a Github account, and will be publishing to it when ready. Will admit Github is new area for me so will need to get my head around it.

When I started making this, it was purely for personal use (trying to get away from Solaris), but now with the apparent interest generated within this forum, it looks like it could be the start of something greater than that I first thought.

 

[Fukuzawa Katsuo]

big fan for zfs！
I am a practitioner of geographic information science, and have used zfs as a storage system to complete graphics rendering for a couple of years.
using CLI to manage pool or filesystem, Which is of course quite cumbersome.
glad to help you test, hope to see your great work.