Napp-it cs web-gui for (m)any ZFS server or servergroups

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
Update: feb 20, first beta with groups

groups.png

Napp-it for ZFS on Windows now supports groups to manage several Windows ZFS servers. This will be extended later to non-Windows servers like OmniOS or Linux but BSD, OSX or SmartOS are also planned options

You can also play with it for first steps with *BSD, Linux, Illumos, OSX or Solaris ZFS server.
Please report insights.

Download (unzip and replace /xampp/[nappit]/[data] ([names] can be modified)
https://napp-it.org/doc/downloads/napp-it_nightly.zip

unzip folder [napp-it]
- stop Apache and background services (cmd windows)
- replace c.\xampp\[napp-it] with newer version.

newest infos:
https://www.napp-it.org/doc/downloads/napp-it on windows.pdf


update 21.2.
Socketserver
I have added code to detect client OS (BSD,Linux,OSX)
napp-it now displays os:hostname as header ex W10:hostname and not only hostname

Tests on OmniOS,. I have included Perl modules not preinstalled on OmniOS
On OmniOS the socketserver often stops after requests, need time to investigate
 
Last edited:
  • Like
Reactions: odditory, Aluminat, nasbdh9 and 1 other person
O

odditory

Moderator
Dec 23, 2010
384
69
28
Interesting project. I tested this last night, was able to create a Raidz2 with 12x14TB HDD's in the GUI on Win11, did some basic performance testing. Was also able to import the zpool into a TrueNAS instance, as well as go the other way and take a TrueNAS-created zpool and import it into the Windows version. Obviously nothing technically advanced and basic-bitch zfs featureset, the interest was confirming that portability actually worked without gotchas or hiccups. No probs.

I realize performance tuning will be an ongoing process and the more immediate goal is getting functionality nailed down. That makes sense. I was just reading THIS thread on ZFS vs NTFS. I'll be continuing to evaluate this project with future updates, because a better multi-drive DAS solution for Windows has been sorely lacking forever, since Microsoft's solutions (Storage Spaces) aren't trustworthy or performant for lots of different reasons (Win10/Win11 Storage Spaces being gimped compared to server editions, windows updates breaking or removing functionality without notice, I/O bugs that can kill SS performance, the list is long).

@gea I admire your dedication, you've been around the storage/enthusiast/homelab community a long time -- both here and places like [H]ardforum going all the way back to 2010, with millions of views on your napp-it/ZFS threads. Not everyone realizes how much work you put into your projects, consistently, relentlessly over time. Plus all the techsupport answers, thousands of posts. It's actually staggering. Vielen dank.
 
  • Like
Reactions: Bronko, itronin, Aluminat and 1 other person
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
Thanks.

Open-ZFS pools are quite compatible between *BSD, Illumos, Linux, OSX or Windows. Only Solaris with native ZFS is not compatible. Problems can mainly happen if a required feature is not available on the target machine ex draid or raid-z expanded pools are currently not available on Illumos.

Napp-it cs (client/server edition) can be a perfect solution for everyone in a multi OS or multiserver environment as it is intended to manage any ZFS server or ZFS appliance or replicate filesystems between them.

I hope that Open-ZFS 2.2+ on Windows is stable soon (data should be already save, BSD or crashes can happen, already faster than ReFS) This is really something I missed in the past.
 
Last edited:
  • Like
Reactions: Aluminat and odditory
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
It really excites me when I sit in front of a Windows desktop and manage my OmniOS ZFS servers with same performance like on a local napp-it GUI or via Putty. A simple select and napp-it switches to another ZFS server.

Works like a charm beside large files. A zfs get all produces around 1 MB data and this seems too much for the socketserver in sync mode or even filetransfer mode with tmp files. Will need to check async socket connections. ZFS GUI remote control with zfs/zpool functions or cli commands via napp-it cmd form works on OmniOS and propably on BSD/Linux/OSX. Disk detection must be next per OS family.

New name for napp-it for ZFS on Windows is napp-it cs (client/server edition) as it should be able to manage (m)any ZFS server.
 
Last edited:
  • Like
Reactions: odditory
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
napp-it cs can now manage OmniOS, Linux and Windows ZFS servers

I have tested with Proxmox but any Linux ZFS server or appliance should basicall work
You only need Perl and ssh root access.

triple.PNG

Infos: napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris , ZFS Server for Windows
Proxmox: napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris : Linux

current nightly and Xampp2 are up to date

There remains a lot of work,
the basic problems are solved.
 
  • Like
Reactions: Aluminat, odditory, nasbdh9 and 2 others
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
OK, I can confirm that ZFS on OSX can be managed by napp-it cs as a group member.
As I have no real Mac hardware, I can only test the software, not disk detection
or reading smartvalues. But it looks very promising.

The last one that is missing now is SmartOS
where a Perl would be needed on persistent storage.

napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris : Downloads

napp-it.org - ZFS NAS SAN Server mit einer durch den Benutzer anpassbaren Weboberfläche // All In One server = virtualisierter lauffähiger ZFS-Server
www.napp-it.org www.napp-it.org
 
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
There is a new Open-ZFS 2.2 rc.13 for Windows

Jorgen Lundman: " I threw in some code that looks up ashift to use instead of 512. If you can, check out OpenZFSOnWindows-debug-2.2.99-13-gfddfb6aeb5.exe. I did the most obvious places, but there are quite a few ways to query that.

*** Please update, the trim bug might be corrupting pools! ***

Now trim is disabled by default, to check it works (on test pools right?) change
HLM/System/ControlSet001/Services/OpenZFS
windows_enable_trim to 1. "

Care: ashift is not a pool but a vdev property. Different ashift in a pool is bad but can happen if you add vdevs without forcing ashift manually.

github.com

Releases · openzfsonwindows/openzfs

OpenZFS on Linux and FreeBSD. Contribute to openzfsonwindows/openzfs development by creating an account on GitHub.
github.com github.com
 
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
I was asked if the socketserver that allows remote management via web-gui of nearly any ZFS server can be copied to
any location ex desktop on OSX or a SMB share. The answer is yes, no special location required (avoid spaces in path).

"Install" socketserver (= a simple copy)
Serversoftware is on your napp-it cs /xampp/napp-it/data/tools/socket

- copy folder c:/xampp/napp-it/data/tools/socket to your ZFS server /any_location/socket
- insert an auth string to /path_to/server.auth
- edit /socket/config.pl (timeout or allowed_ip, optional)
- start socketserver via: sudo perl /path_to/server.pl (Console)

On Windows cs
- add the server to the group in menu Extensions > Servergroup

As every setup is a little different, report insights with different systems or appliances.
 
B

BobTB

Member
Jul 19, 2019
81
20
8
gea said:
The last one that is missing now is SmartOS
where a Perl would be needed on persistent storage.

napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris : Downloads

napp-it.org - ZFS NAS SAN Server mit einer durch den Benutzer anpassbaren Weboberfläche // All In One server = virtualisierter lauffähiger ZFS-Server
www.napp-it.org www.napp-it.org
Click to expand...
Perhaps create an illumos zone in SmartOS with perl, it will be persistant in this case, right? Or even a zone running OmniOS. Do you think this would work?
 
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
I have reports that "it just works" as SmartOS already has Perl preinstalled.
Just copy the socketserver to a persistent folder (ex /opt/customs or anywhere on the ZFS pool)
and start: perl /abspath_to/socket/server.pl

If you get problems about missing perl modules
Edit server.pl and look for require "/path/config.pl"; and add in the line behind

use lib "path_to_socketfolder/CGI";

as I got a report that the autoconfig of "use lib" in config.pl has problems on SmartOS

napp-it // webbased ZFS NAS/SAN appliance for OmniOS, OpenIndiana and Solaris : Downloads

napp-it.org - ZFS NAS SAN Server with user editable web-gui // All In One server = virtualized ready to run ZFS-Server
www.napp-it.org www.napp-it.org

www.hardwareluxx.de

[Übersicht] - napp-it cs, Web-GUI für fast jeden ZFS Server oder Servergruppen

napp-it cs kann jetzt OmniOS, Proxmox und Windows ZFS Server managen OI und Solaris sollte auch gehen. Andere Linuxe und ZFS appliances auch. Man braucht lediglich Perl und ssh root Zugang. Infos: https://www.napp-it.de/downloads/windows_en.html Proxmox...
www.hardwareluxx.de www.hardwareluxx.de
(use Chrome to translate on the fly)
 
B

BobTB

Member
Jul 19, 2019
81
20
8
It actually works. Running this on SmartOS (Triton Headnode)

1709495753684.png

The problem I have it that when trying to run any command, I get the following error

1709495823415.png

The server can be added, and reports as this. What can I do?

1709495886689.png
 
B

BobTB

Member
Jul 19, 2019
81
20
8
Sorry, it seems that there was some bug, if I named it X1, it did not work. After renaming it as test, it works! :) This is amazing! Very nice work.

I will finally be able to use this with SmartOS / Triton. Now all we need is send/receive replication!

1709496360105.png
 
Last edited:
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
BobTB said:
Now all we need is send/receive replication!
Click to expand...
Send / receive from any member to any member is next step.
(can last a little as the regular napp-it replication script needs some extra work for the cs edition)
 
Last edited:
  • Like
Reactions: BobTB
A

Aluminat

Member
Jul 5, 2019
54
23
8
How can I start apache2 minimized?
1709616830502.png
Nomarl started make this window and but I can't minimized to tray system.
 
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
Whole startup is initiated via "C:\xampp\napp-it2\data\start.bat" with Apache startup in C:\xampp\ apache_start.bat
As it already starts minimized (minimized in taskbar) you can only comment out start in apache_start.bat and start as service.
 
  • Like
Reactions: Aluminat
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
Open-ZFS for BSD, Linux and OSX (rc) is now at 2.2.3 with improvements around trim and many bugfixes.
github.com

Release zfs-2.2.3 · openzfs/zfs

Supported Platforms Linux: compatible with 3.10 - 6.7 kernels FreeBSD: compatible with releases starting from 12.2-RELEASE Changes ZTS: Skip cross-fs bclone tests if FreeBSD < 14.0 #15901 [zfs-2...
github.com github.com

Open-ZFS for Windows is 2.2.2 but as far as I know Jorgen Lundman
2.2.3 on Windows (rc) will be out soon

OpenZFS on Illumos is separate as it has its own repository
with integration of Open-ZFS improvements after an additional review
 
Last edited:
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
Besterino is best..

Tests with Window SMB direct rdma/ vhdx/iscsi vs local performance:
SMB performance 11GB/s read and 5GB/s write on 100G nic, quite the same as local performanc with lowest CPU load.

This makes Windows superiour for any high performance workload (ex 4k+ video editing)
with ntfs, ReFS or ZFS on Windows

www.hardwareluxx.de

[Sammelthread] - Microsoft Hyper-V Stammtisch

So, es wird ernst(er). Für die nächste Testphase muss die Testumgebung erweitert werden für den Zugriff durch meinen Hauptsystem. Dafür hab ich nach langer Zeit mal meinen Haupt-Server physisch geöffnet, wieder 4 NVME SSDs nachgesteckt und eine der beiden ConnectX-5. AMD's Epyc-Plattform mit...
www.hardwareluxx.de www.hardwareluxx.de
(use Chrome to translate on the fly)
 
Last edited:
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
New release of ZFS on Windows-2.2.3rc1, it is fairly close to upstream OpenZFS-2.2.3
github.com

Releases · openzfsonwindows/openzfs

OpenZFS on Linux and FreeBSD. Contribute to openzfsonwindows/openzfs development by creating an account on GitHub.
github.com github.com
  • Redone CPUID probe, please verify it is correct
  • Tunable: "zfs_bclone_wait_dirty" set to wait instead of error from clonefile.
  • DDT work (memory reduction, speedup)
  • zfs list: add '-t fs' and '-t vol' options
  • Add slow disk diagnosis to ZED
  • Add 'zpool status -e' flag to see unhealthy vdevs
  • BRT: Fix FICLONE/FICLONERANGE shortened copy
  • (zpool: Add slot power control, print power status) not on macOS
  • spa: make read/write queues configurable
  • Allow block cloning across encrypted datasets
  • Add libfetch for https://
 
G

gea

Well-Known Member
Dec 31, 2010
3,163
1,195
113
DE
Small update of the remote_cs_server software as replication from any to any server requires more work

This howto requires server software up from Mar 16. 2024
########################################################

Client /server web-gui for (m)any ZFS Server

is a client server configuration with a frontend web-gui (currently Windows web application on Apache) and a
server backend that can run on nearly any ZFS server or appliance (BSD, Linux, Illumos, OSX, Solaris or Windows)

This folder "remote_cs_server" contains the server backend with the startup script "start_server_as_admin.pl" and "cs_backend.pl" with the server part and server.auth with the authorisation key.

cs_backend.pl is a background task that runs with admin permissions to process and return data ex for zfs and zpool commands

client.pl
this script is an example to request data from the server if you want to add your own addition or menu

Client/server setup should work on any server or appliance with Perl installed and access to a root console
based on *BSD, *Linux (Debian/Proxmox), OSX, Solaris/Illumos and Windows 10/11/Server

Server Setup
Upload folder remote_cs_server to any location of a ZFS server
(ex /pool/filesystem or desktop or a system folder like /var/web-gui)
via wget, SMB, WinSCP or any other method

Optionally
edit cs_backend.pl (WinSCP, midnight commander or other editor) and
adjust allowed_ip (restrict clients), timeout (client timeout), max_connections (concurrent number of clients)

Start server backend from console as root or admin
perl /path_to_this_folder/start_server_as_admin.pl or with a Windows ZFS server
/xampp/perl/bin/perl.exe /path_to_this_folder/start_server_as_admin.pl

If you use a Putty remote console to start the server:
When Putty disconnects the server stopps, use only for tests.


Client setup (Windows)
- Copy Xampp portable (With Apache and Perl) to c:\ to get c:\xampp
- Copy the folder with the client webapp ex web-gui or napp-it to c:\xammpp (to get c:\xampp\napp-it)
- Start the frontend web-app from a administrator powershell terminal:
/xampp/perl/bin/perl.exe /xampp/napp-it/data/start_zfs-gui_as_admin.bat

- Open a browser with adresss http(s)://localhost
- Add all ZFS server to menu ZFS Servergroup.
- Select the server that you want to manage in the main menu from list.
 
  • Like
Reactions: Aluminat
You must log in or register to reply here.
Top Bottom