Snaps are broken after restarting ubuntu 20.04 on zfs

ghostbuster91 · Sep 16, 2020

Hi, I’ve been using snap for at least over a year now without any problems and it was great.

Recently I decided to reinstall my system to ubuntu 20.04 but with ZFS as a file system. Since then I have weird problems with snap. After a restart I cannot launch any of my applications installed via snap. They also appear as broken in snap list output.

Code:

snap list               
Name                    Version  Rev   Tracking         Publisher   Notes
canonical-livepatch              95    latest/stable    canonical✓  broken
core                             9804  latest/stable    canonical✓  broken
core18                           1885  latest/stable    canonical✓  broken
gnome-3-34-1804                  36    latest/stable/…  canonical✓  broken
gtk-common-themes                1506  latest/stable/…  canonical✓  broken
intellij-idea-ultimate           245   latest/stable    jetbrains✓  broken
skype                            148   latest/stable    skype✓      broken
snap-store                       467   latest/stable/…  canonical✓  broken
snapd                            8790  latest/stable    canonical✓  broken
–

Code:

lla /var/lib/snapd/snaps
.rw------- 9,5M root  7 wrz  1:25 canonical-livepatch_95.snap
.rw-------  57M root 31 lip 18:35 core18_1880.snap
.rw-------  58M root  7 wrz  9:13 core18_1885.snap
.rw------- 101M root  7 wrz  1:25 core_9804.snap
.rw------- 267M root 31 lip 18:35 gnome-3-34-1804_36.snap
.rw-------  65M root 31 lip 18:35 gtk-common-themes_1506.snap
.rw------- 755M root  7 wrz 11:09 intellij-idea-ultimate_245.snap
drwxr-xr-x    - root 10 lip 15:59 partial
.rw------- 188M root  9 wrz 22:45 skype_147.snap
.rw------- 188M root 11 wrz 18:56 skype_148.snap
.rw-------  52M root 31 lip 18:35 snap-store_467.snap
.rw-------  31M root 31 lip 18:35 snapd_8542.snap
.rw-------  31M root  7 wrz  9:13 snapd_8790.snap

Code:

service snapd status
● snapd.service - Snap Daemon
     Loaded: loaded (/lib/systemd/system/snapd.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-12 07:27:56 CEST; 9min ago
TriggeredBy: ● snapd.socket
   Main PID: 32026 (snapd)
      Tasks: 22 (limit: 38174)
     Memory: 20.8M
     CGroup: /system.slice/snapd.service
             └─32026 /usr/lib/snapd/snapd

wrz 12 07:27:57 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "core" at revision 9804: cannot find installed snap "core" at revision 9804: missing file /snap/core/9804/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "snapd" at revision 8790: cannot find installed snap "snapd" at revision 8790: missing file /snap/snapd/8790/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "canonical-livepatch" at revision 95: cannot find installed snap "canonical-livepatch" at revision 95: missing file /snap/canonical-livepatch/95/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "skype" at revision 148: cannot find installed snap "skype" at revision 148: missing file /snap/skype/148/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "snap-store" at revision 467: cannot find installed snap "snap-store" at revision 467: missing file /snap/snap-store/467/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "core" at revision 9804: cannot find installed snap "core" at revision 9804: missing file /snap/core/9804/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "gnome-3-34-1804" at revision 36: cannot find installed snap "gnome-3-34-1804" at revision 36: missing file /snap/gnome-3-34-1804/36/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "gtk-common-themes" at revision 1506: cannot find installed snap "gtk-common-themes" at revision 1506: missing file /snap/gtk-common-themes/1506/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "core18" at revision 1885: cannot find installed snap "core18" at revision 1885: missing file /snap/core18/1885/meta/snap.yaml
wrz 12 07:29:25 precision-7530 snapd[32026]: snapmgr.go:298: cannot read snap info of snap "intellij-idea-ultimate" at revision 245: cannot find installed snap "intellij-idea-ultimate" at revision 245: missing file /snap/intellij-idea-ultimate/245/meta/snap.yaml

Code:

mount | grep snap
rpool/ROOT/ubuntu_i8m3zy/var/snap on /var/snap type zfs (rw,noatime,xattr,posixacl)

Code:

zfs list 
NAME                                               USED  AVAIL     REFER  MOUNTPOINT
bpool                                              352M  1,41G       96K  /boot
bpool/BOOT                                         350M  1,41G       96K  none
bpool/BOOT/ubuntu_i8m3zy                           350M  1,41G      182M  /boot
rpool                                             66,9G   390G      192K  /
rpool/ROOT                                        9,23G   390G      192K  none
rpool/ROOT/ubuntu_i8m3zy                          9,23G   390G     3,73G  /
rpool/ROOT/ubuntu_i8m3zy/srv                       192K   390G      192K  /srv
rpool/ROOT/ubuntu_i8m3zy/usr                      1,89M   390G      192K  /usr
rpool/ROOT/ubuntu_i8m3zy/usr/local                1,70M   390G     1,37M  /usr/local
rpool/ROOT/ubuntu_i8m3zy/var                      4,12G   390G      192K  /var
rpool/ROOT/ubuntu_i8m3zy/var/games                 192K   390G      192K  /var/games
rpool/ROOT/ubuntu_i8m3zy/var/lib                  3,88G   390G     2,75G  /var/lib
rpool/ROOT/ubuntu_i8m3zy/var/lib/AccountsService  1,56M   390G      280K  /var/lib/AccountsService
rpool/ROOT/ubuntu_i8m3zy/var/lib/NetworkManager   4,19M   390G      360K  /var/lib/NetworkManager
rpool/ROOT/ubuntu_i8m3zy/var/lib/apt              78,5M   390G     65,3M  /var/lib/apt
rpool/ROOT/ubuntu_i8m3zy/var/lib/dpkg              107M   390G     51,3M  /var/lib/dpkg
rpool/ROOT/ubuntu_i8m3zy/var/log                   247M   390G     26,9M  /var/log
rpool/ROOT/ubuntu_i8m3zy/var/mail                  192K   390G      192K  /var/mail
rpool/ROOT/ubuntu_i8m3zy/var/snap                 2,14M   390G      728K  /var/snap
rpool/ROOT/ubuntu_i8m3zy/var/spool                1,34M   390G      288K  /var/spool
rpool/ROOT/ubuntu_i8m3zy/var/www                   192K   390G      192K  /var/www
rpool/USERDATA                                    15,1G   390G      192K  /
rpool/USERDATA/kghost_c729ro                      15,1G   390G     9,72G  /home/kghost
rpool/USERDATA/root_c729ro                         700K   390G      244K  /root
rpool/swap2                                       42,5G   433G      116K  -
rpool/var                                          988K   390G      192K  /var
rpool/var/lib                                      796K   390G      192K  /var/lib
rpool/var/lib/docker                               604K   390G      604K  /var/lib/docker

It looks that something isn't remounting correctly, but I don't know how to debug it further. Any ideas?

aboron · Dec 24, 2020

Sorry for a really late reply, but I just started reading the forums here.

It is my understanding the snaps rely on the overlay filesystem feature, and there isn't a backend to support that on zfs, yet.
This affects using docker on zfs backing also.

Bug here: Bug #1718761 “It's not possible to use OverlayFS (mount -t overl...” : Bugs : zfs-linux package : Ubuntu

deviantintegral · Dec 30, 2020

I'm in the middle of rebuilding a server myself and my plan was to migrate my existing lxd containers to a zfs-on-root setup. Of course, lxd highly recommends using snap to install it. Thanks for calling this out, as I'd be sure to hit this at the worst point.

I'm kind of amazed that snaps don't work given the docs create a separate dataset for /var/snap, as you have above. Did you use the installer to set up zfs or by following the openzfs docs? If the latter, I wonder if the Ubuntu installer does something different.

Could creating a zvol and formatting it as ext4 or some other filesystem for /var/snap work around this?

deviantintegral · Dec 31, 2020

I just did an install inside a VM using the desktop installer, with the zfs option enabled. Snaps are working fine after reboots - in fact, gnome itself is installed via snap. I was also able to install lxd and skype as well.

I wonder if the tmpfs mount for /run/snapd is a problem?

Code:

$ mount | grep snap
rpool/ROOT/ubuntu_h11sc3/var/snap on /var/snap type zfs (rw,relatime,xattr,posixacl)
/var/lib/snapd/snaps/lxd_18772.snap on /snap/lxd/18772 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gtk-common-themes_1506.snap on /snap/gtk-common-themes/1506 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core18_1880.snap on /snap/core18/1880 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gnome-3-34-1804_36.snap on /snap/gnome-3-34-1804/36 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/snap-store_467.snap on /snap/snap-store/467 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/core_10577.snap on /snap/core/10577 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/skype_161.snap on /snap/skype/161 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/snapd_8542.snap on /snap/snapd/8542 type squashfs (ro,nodev,relatime,x-gdu.hide)
tmpfs on /run/snapd/ns type tmpfs (rw,nosuid,nodev,noexec,relatime,size=805372k,mode=755)
nsfs on /run/snapd/ns/lxd.mnt type nsfs (rw)

Search

Snaps are broken after restarting ubuntu 20.04 on zfs

ghostbuster91

New Member

aboron

New Member

deviantintegral

New Member

deviantintegral

New Member