Cockpit ZFS Manager

optimans

Member
Feb 20, 2015
61
77
18
Is there a technical reason why this depends on Cockpit >= 201+?
PatternFly V4 was introduced to Cockpit at 201. The html content is static at the moment and has been coded to PF4.

I will admit, web development is not my forte; has been a learning curve for me. Plan is to learn React next year and try to migrate to it.

My roadmap for next year in no particular order (except #1):
  • Full documentation
  • SMART info
  • SAS2 and SAS3 information - disk location for taking disk(s) online/offline/replace
  • LEDmon
  • React?
  • Pool history and events
  • IOstat?
  • Status and event notifications?
  • Bookmark and checkpoint support
  • Datatables/search/filtering for datasets
  • Snapshot automation?
  • Samba maintenance
 

optimans

Member
Feb 20, 2015
61
77
18
Thanks for all the feedback guys! Glad to hear is working well. More features coming next year.

Has anyone tried trashing their systems with this yet? Would like to be able to remove the pre-release warning soon if working well.

Thanks and Merry Christmas everyone!
 

Revoman

New Member
May 23, 2019
5
1
3
I recently updated my server to CentOS Stream with two ZFS pools. One that has been around for close to 10 years using spinny disks, the other is a 4x1tb NVME RaidZ pool. Updating to the cockpit-preview release wasn't too difficult and installing this package was very simple.
The interface is clean, organized and intuitive.
When enabling the "Trim" feature on my nvme pool the interface appeared to hang. I'm sure the command was just taking its own time returning (a slow/old cpu and not trimming for 6 months might have added to the lag). Wondering if there was a way to add a pseudo progress bar that indicated the process was still running.

So far my pool hasn't blown up. But I haven't done any major tasks with it yet.
 

optimans

Member
Feb 20, 2015
61
77
18
Hi @Revoman, thanks for the feedback!

When you enabled the automatic TRIM property, did the modal close? The modal should stay open with a spinner and status text until it receives done/fail from the process. Might be too late now but did the console logging display 'in progress' or 'complete' status? Should of set the property quickly and been done; according to documentation, it will TRIM periodically. I don't think it initiates TRIM straight after setting the property.

Do you have the same issue when starting TRIM on the storage pool or one of the disks?

What version of Cockpit do you have? I had issues with 205 on CentOS 8.0 and processes would timeout at random (not execute); had to restart the cockpit.socket service to fix. Versions before and after seem to be fine.
 

Revoman

New Member
May 23, 2019
5
1
3
Trying to use this add-on to create a new pool. I have 2 pools in my environment, and only some the disks are showing the warning symbol as being used. aka, all disks from the first pool show, only some from the 2nd pool show as used.

CentOS Stream
Cockpit v209
ZoL 0.8.2

upload_2019-12-27_13-38-26.png

upload_2019-12-27_13-38-49.png

upload_2019-12-27_13-40-25.png
 

optimans

Member
Feb 20, 2015
61
77
18
@Revoman, can you please echo the following commands to seperate txt files, will try to emulate your setup (PM if need):

Code:
zpool status -p -t
zpool status -P
lsblk -o label,model,mountpoint,name,partuuid,phy-sec,rota,serial,size,type,uuid,vendor,wwn -J -b
blkid -o full
ls /dev/disk/by-id/
I see you have custom dev naming for the tank pool, will need to look at whats going on for the fast pool too. Disks that are mounted, are part of in use pools and importable pools should not display in the list; however they are there.

Will have a look into this.
 

optimans

Member
Feb 20, 2015
61
77
18
Hey All, new update available: 0.2.0.233

Add quota property to file systems
Adds support for multiple disk identifiers:
  • Block Device (/dev),
  • Disk (/dev/disk/by-id),
  • Hardware Path (/dev/disk/by-path),
  • Virtual Device Mapping (/dev/disk/by-vdev) [Configured via /etc/zfs/vdev_id.conf]
  • WWN (/dev/disk/by-id)
Create Storage Pool / Import Storage Pool / Disk Attach / Disk Replace / Add Virtual Device modals will display WWN as selected; turn the switch off to choose from the drop down for identifier type. If an identifier is not available, it will fall back to block device:

upload_2020-2-20_1-22-38.png upload_2020-2-20_1-22-59.png upload_2020-2-20_1-23-15.png upload_2020-2-20_1-23-21.png

Import Storage Pool will display which identifiers are available for each pool and selectable via the drop down. Falls back to block device if unable to import with selected identifier:

upload_2020-2-20_1-23-28.png

Status with Virtual Device Mapping:

upload_2020-2-20_1-23-36.png

Disk detection has been rewritten, and should solve the issues @Revoman was having. Fingers crossed!

Still pre-release; so make sure to backup all data before playing with this update.
 

Wolfstar

Active Member
Nov 28, 2015
159
83
28
47
@optimans I've finally got this set up and I absolutely love it. No problems so far with the latest version. I've been able to create a couple of test pools with no problems, SMB support was flawless for me, and it recognized my main pool had NFS enabled correctly.

I would like to request a couple of features; let me know if you want me to open issues for them on GitHub.

1. Support for adding VDEVs/L2ARC/SLOG devices to existing pools. If I'm missing a way to expand a pool, please feel free to smack me - I looked but didn't see it. (Doesn't bother me to do CLI, but the option would be nifty.)

2. I don't know if this is part of what you meant by datatables/search/filtering for datasets, but the ability to view per-dataset snapshots by default - preferably in a tree format much like the zpool and dataset configuration - would be glorious. I'm using ZFS-Auto-Snapshot, and especially for my VM and Backups datasets, there are TONS of them. Opening the snapshots tab isn't quite useless, but I currently have 855 snapshots, so it's painful.

Thanks for your hard work on this - it's by far the best web UI out there for ZFS on Linux right now IMO.
 

optimans

Member
Feb 20, 2015
61
77
18
@optimans I've finally got this set up and I absolutely love it. No problems so far with the latest version. I've been able to create a couple of test pools with no problems, SMB support was flawless for me, and it recognized my main pool had NFS enabled correctly.
@Wolfstar Glad to hear it is working well for you. Did you end up moving to Ubuntu 19.10 or upgrading Cockpit?

1. Support for adding VDEVs/L2ARC/SLOG devices to existing pools. If I'm missing a way to expand a pool, please feel free to smack me - I looked but didn't see it. (Doesn't bother me to do CLI, but the option would be nifty.)
You can add and remove virtual devices via the UI.

On the Status tab, there is the actions menu for the storage pool and Add Virtual Device menu item. Here you should be able to add as per requirements including Special virtual devices. (Everything should work, but make sure your data is backed up first).

upload_2020-3-8_3-0-14.png upload_2020-3-8_3-1-7.png

Storage Pool auto expand property is configured via Configure Storage Pool. Automatically expand storage pool when larger devices are added

upload_2020-3-8_3-3-23.png upload_2020-3-8_3-4-10.png

If you are setting a disk Online, you will have the option to expand the disk to use all available space

upload_2020-3-8_3-5-49.png

Still got to do the documentation.

2. I don't know if this is part of what you meant by datatables/search/filtering for datasets, but the ability to view per-dataset snapshots by default - preferably in a tree format much like the zpool and dataset configuration - would be glorious. I'm using ZFS-Auto-Snapshot, and especially for my VM and Backups datasets, there are TONS of them. Opening the snapshots tab isn't quite useless, but I currently have 855 snapshots, so it's painful.
Please raise an issue on GitHub for this. Will look into this. I was thinking datatables to try and make it easier, but having another nested list view for each dataset may be a better way to go about it.

Thanks for your hard work on this - it's by far the best web UI out there for ZFS on Linux right now IMO.
Thank You!!! :):) This is awesome to hear, and cheers for the recommendation on Reddit.

Can't believe how much interest this has generated: 531 clones and 25 release downloads since end of November.

Also thanks to everyone for likes and feedback too.
 
  • Love
Reactions: djbe

Wolfstar

Active Member
Nov 28, 2015
159
83
28
47
@Wolfstar Glad to hear it is working well for you. Did you end up moving to Ubuntu 19.10 or upgrading Cockpit?
I ended up upgrading to 19.10. I considered upgrading Cockpit and the assorted components, but I wasn't sure how much of ZFS 0.8.2 had been backported by Ubuntu, and having to compile that with DKMS defeats the purpose of moving to Ubuntu from Debian in the first place. (For anyone considering that, it worked nearly flawlessly. My only issue was that the pool didn't import at first, and then when I did the manual import it defaulted back to /dev/sdX).

You can add and remove virtual devices via the UI.

On the Status tab, there is the actions menu for the storage pool and Add Virtual Device menu item. Here you should be able to add as per requirements including Special virtual devices. (Everything should work, but make sure your data is backed up first).
Yeah, there they are. I was defaulting to look under "Configure Storage Pool", rather than the actions menu in the Status tab. If it were me, I'd move the Add Virtual Device option to the same menu as Configure; everything about the Actions menu it's on speaks to maintenance functions, whereas the main pool option is about configurations of the entire pool. Either way, I know where it is now so that works fine for me. :)

Please raise an issue on GitHub for this. Will look into this. I was thinking datatables to try and make it easier, but having another nested list view for each dataset may be a better way to go about it.
Done, as requested. :)

Thank You!!! :):) This is awesome to hear, and cheers for the recommendation on Reddit.

Can't believe how much interest this has generated: 531 clones and 25 release downloads since end of November.

Also thanks to everyone for likes and feedback too.
You're quite welcome and you deserve it. Honestly, ease of manipulation for ZFS from a GUI is probably one of the (admittedly, many - looking at you CDDL) things holding ZFS on Linux adoption back. It became a pretty major choice for OpenMediaVault when someone with little experience with coding (and sadly no time to maintain) managed to hack together a plugin for it, and it was never a tight and clean implementation. I even tried to fix or change a couple of things myself, but I have neither the skills nor the time to acquire them to do more than a simple spot-fix.

So seriously, glad to see it and I'm looking forward to using it moving forward. :)
 

Marshall Simmons

Active Member
Feb 18, 2015
132
25
28
42
Hello
Thanks for the great piece of software.
This may be a simple question, but I'm having trouble with it.

I have a pool named datastore
I have two filesystems on top of it named media and appsisos
I have shared both via smb successfully

My question is even though I've set the share name to Media, the name of the share when looking from another computer is datastore_media
Am I missing something in the gui configuration that would preface my share names with the name of the pool?

Thanks so much!
 

optimans

Member
Feb 20, 2015
61
77
18
Hi Marshall,

Thanks for the great piece of software.
Thanks for the feedback, 0.3 will be available soon!

I have a pool named datastore
I have two filesystems on top of it named media and appsisos
I have shared both via smb successfully

My question is even though I've set the share name to Media, the name of the share when looking from another computer is datastore_media
Am I missing something in the gui configuration that would preface my share names with the name of the pool?
Is Cockpit ZFS Manager (CZM) managing the Samba share?
Did you import or create the file system after installing the package?

If CZM manages the shares and the share was automatically created (eg Import / Create File System / Mount), it will check if the share name is already in use and use {StoragePool}_{FileSystem} as fail safe.

If you go to Configure Samba share, you can rename the share:

upload_2020-4-24_9-21-28.png

An error message will appear if the share name is already in use:

upload_2020-4-24_9-21-42.png

If CZM is not managing the share, then ZFS will manage the name of the share automatically and changes made are not persistent via this method.

Let me know how you go
 

Marshall Simmons

Active Member
Feb 18, 2015
132
25
28
42
Hi Marshall,



Thanks for the feedback, 0.3 will be available soon!



Is Cockpit ZFS Manager (CZM) managing the Samba share?
Did you import or create the file system after installing the package?

If CZM manages the shares and the share was automatically created (eg Import / Create File System / Mount), it will check if the share name is already in use and use {StoragePool}_{FileSystem} as fail safe.

If you go to Configure Samba share, you can rename the share:

View attachment 13698

An error message will appear if the share name is already in use:

View attachment 13699

If CZM is not managing the share, then ZFS will manage the name of the share automatically and changes made are not persistent via this method.

Let me know how you go
I've used CZM to manage and create the shares and the pool.

So based off of your information, I kept the main pool "Datastore" and deleted all other filesystems

and then I created a filesystem on it called "AV"
I then enabled samba share, named the share "Media" and deleted the comments.
When I look at the share from another computer, I actually see two shares - one called "Datastore_AV" and one correctly called "Media"
 

optimans

Member
Feb 20, 2015
61
77
18
Thanks for the update. Is usershares configured in smb.conf? Sounds like this could be your issue:

optimans/cockpit-zfs-manager

Samba

ZFS always creates shares in /var/lib/samba/usershares folder when ShareSMB property is enabled. This is also the case even if Cockpit ZFS Manager is managing the shares. To avoid duplicate shares of the same file system, it is recommended to configure a different usershares folder path if required or to disable usershares in the Samba configuration file.

If enabled, Cockpit ZFS Manager manages shares for the file systems only. Samba global configuration will need to be configured externally.
 

Marshall Simmons

Active Member
Feb 18, 2015
132
25
28
42
Thanks for the update. Is usershares configured in smb.conf? Sounds like this could be your issue:

optimans/cockpit-zfs-manager
Based off of what you said, i did a little bit of digging and was able to find the command "net usershare" which allowed me to "net usershare list" and "net usershare delete Datastore_AV" which resolved the issue.

Could you point me in the right direction of which part of the smb.conf file I need to comment out to disable usershare creation?

Thanks so much for holding my hand through this.
 

optimans

Member
Feb 20, 2015
61
77
18
No problem.

Edit the smb.conf file:

Code:
$ sudo nano /etc/samba/smb.conf
In the [Global] section, look for usershare properties and comment out with #

Code:
 $ sudo systemctl restart smb
(smbd for Debian/Ubuntu)

If you don't see any usershare properties, you may need to delete the /var/lib/samba/usershares folder

Restart the server to make sure the Datastore_AV share is not re-created by ZFS on boot.
 
  • Like
Reactions: MiniKnight

rootpeer

Member
Oct 19, 2019
65
15
8
Hello! Sorry for the dumb question.

How can I get this running on CentOS 8.1? The latest cockpit package available is 196.3.
 

optimans

Member
Feb 20, 2015
61
77
18
Hi @rootpeer,

No problem, you will need to add the Cockpit Preview repo to get the latest version.

Here are the steps required (From the CentOS 8.0 Guide):

Code:
$ sudo dnf remove cockpit\*
$ sudo dnf config-manager --add-repo https://copr.fedorainfracloud.org/coprs/g/cockpit/cockpit-preview/repo/epel-8/group_cockpit-cockpit-preview-epel-8.repo

$ sudo dnf install cockpit cockpit-storaged setroubleshoot-server
$ sudo systemctl enable --now cockpit.socket
 
  • Like
Reactions: rootpeer

rootpeer

Member
Oct 19, 2019
65
15
8
Hi @rootpeer,

No problem, you will need to add the Cockpit Preview repo to get the latest version.

Here are the steps required (From the CentOS 8.0 Guide):

Code:
$ sudo dnf remove cockpit\*
$ sudo dnf config-manager --add-repo https://copr.fedorainfracloud.org/coprs/g/cockpit/cockpit-preview/repo/epel-8/group_cockpit-cockpit-preview-epel-8.repo

$ sudo dnf install cockpit cockpit-storaged setroubleshoot-server
$ sudo systemctl enable --now cockpit.socket
Great! Thank you very much you are a legend! This project makes so much sense!
 
  • Like
Reactions: optimans

optimans

Member
Feb 20, 2015
61
77
18
Hey All, new minor update available: 0.3.0.299
  • Display snapshots in lists by file system on the snapshots tab
  • Display file systems and snapshots count
  • UI improvements
If you have 0.2 installed, you will receive an update alert when you login to Cockpit.

upload_2020-4-27_1-23-31.png

Still pre-release; so make sure to backup all data before playing with this update.

Hope you are all staying safe during this pandemic!
 
  • Like
Reactions: EluRex and Wolfstar