ZFS No Available Space

[Weighted Cube]

So recently my ESXi drive which hold my VMs died and stupid me didn't have a backup. So I lost my OmniOS VM that was handling my ZFS pool. So to get everything up again, I figured I would try out ZFSonLinux and just import my pool. Upon importing my pool though I found that I couldn't write anything to the pool and that napp-it was saying I had 0 available space when I definitely did prior. zpool status says I have free space, but zfs list and df -h say I have 0. I've tried importing back into a new OmniOS VM and now I'm getting the same issue. I've tried looking into quotas/reservation but those don't seem to be the issue here. Any ideas would be greatly appreciated.

root@storage:/root# zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool  39.8G  9.59G  30.2G         -    14%    24%  1.00x  ONLINE  -
tank   13.6T  13.4T   275G         -     9%    98%  1.00x  ONLINE  -
root@storage:/root# zfs list
NAME                            USED  AVAIL  REFER  MOUNTPOINT
rpool                          11.1G  27.4G    26K  /rpool
rpool/ROOT                     3.05G  27.4G    23K  legacy
rpool/ROOT/omnios              3.05G  27.4G  2.46G  /
rpool/ROOT/omnios-backup-1      206K  27.4G  1.78G  /
rpool/ROOT/omniosvar             23K  27.4G    23K  legacy
rpool/ROOT/pre_napp-it-16.11f   116K  27.4G  1.76G  /
rpool/dump                     6.00G  27.4G  6.00G  -
rpool/export                     46K  27.4G    23K  /export
rpool/export/home                23K  27.4G    23K  /export/home
rpool/swap                     2.06G  28.9G   550M  -
tank                           10.7T      0   192K  /tank
tank/media                        6.80T      0  6.80T  /tank/media
tank/share                     3.87T      0  3.87T  /tank/share
root@storage:/root# df -h
Filesystem         Size  Used Avail Use% Mounted on
rpool/ROOT/omnios   30G  2.5G   28G   9% /
swap               6.2G  304K  6.2G   1% /etc/svc/volatile
swap               6.2G  112K  6.2G   1% /tmp
swap               6.2G   56K  6.2G   1% /var/run
rpool/export        28G   23K   28G   1% /export
rpool/export/home   28G   23K   28G   1% /export/home
rpool               28G   26K   28G   1% /rpool
tank               192K  192K     0 100% /tank
tank/media            6.8T  6.8T     0 100% /tank/media
tank/share         3.9T  3.9T     0 100% /tank/share

 

[Bronko]

May be you can delete some snapshots:

(Can't)Delete files when no space left on disk!!! (Nischal Rao)

 

[WeightedCube]

May be you can delete some snapshots:

(Can't)Delete files when no space left on disk!!! (Nischal Rao)

Unfortunately I don't have any snapshots for me to delete. Another note though is when I delete files from the pool, the available space doesn't change and still stays at 0.

 

[gea]

If a ZFS pool is completely full, you have the options

- delete snaps if you have one (see napp-it menu snapshots)
- remove a reservation (napp-it does a reservation per default, see menu ZFS filesystems > pool level)
- truncate a large file ex see ZFS Cant rm: No space left on device - SurlyJake Blog

example cat /dev/null > mybigfile or echo 1>./some_file

 

[Bronko]

napp-it does a reservation per default, see menu ZFS filesystems > pool level

How does you set reservations on pool level "overflow protection", because it is an option in napp-it on Create a Pool, but it is a zfs dataset option:

zfs set reservation=

 

[gea]

"Overflow protection" in napp-it is a 10% fres reservation on the pool level/root filesystem.
This has the effect that available capacity for any dataset below pool is reduced by 10%

You can check or modify Fres in menu ZFS Filesystems
see Fres 100G in this example

 

[Bronko]

This is what have understand in all the years @gea
But the magic behind is: after pool creation napp-it creates a first dataset with same pool name and sets refreservation on it?

zpool create tank ...
zfs create -o refreservation=100k tank ...

 

[gea]

No, this is not how ZFS and reservation works.
The key aspect is:

- A ZFS pool itself is a ZFS filesystem
It is a "special" filesystem only as it has zfs and zpool properties

- Reservation is a guarantee that the affected filesystem can use this space.
This has the effect that "available" for all other filesystems is reduced by resevations

So the needed commands are

zpool create tank ...
zfs set refreservation=100k tank

 

[Bronko]

ok, my fault... thanks a lot!