Move away from unsafe FAT on USB sticks or disks, use ZFS

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.
G

gea

Well-Known Member
Dec 31, 2010
3,172
1,197
113
DE
As we now have ZFS on every mainstream OS from BSD over OSX to Windows, I start using ZFS instead FAT on USB sticks or disks as the ultimate solution for data security on such unsecure media to have full checksum verification of data or redundancy with copies=2

There is no automount of ZFS removeable pools like you have with FAT, but I simply name all pools on USB sticks or disks "usb". A simple "zpool import -f usb" and the stick/disk is there. On Windows I place a usbmount.bat on the desktop. A simple right mouse click as admin and the stick is mounted.

Export pool usb prior remove
 
Last edited:
  • Like
Reactions: T_Minus, name stolen and itronin
N

nimajneb

New Member
Apr 14, 2024
13
4
3
I occasionally plug in two USB storage devices at the same time, does naming them "usb" cause a conflict? I've never purposely plugged in two storage devices with the same drive name so I don't know.
 
G

gea

Well-Known Member
Dec 31, 2010
3,172
1,197
113
DE
If you plug in two disks with ZFS pools and same name on them and call zpool import, you may not be able to import based on name but only based on unique uid and a different name on second import. No other problems to be expected. A unplug without prior export may be a bigger problem as this usually requires a reboot to remove the pool suspended state of a pool.
 
N

nimajneb

New Member
Apr 14, 2024
13
4
3
gea said:
If you plug in two disks with ZFS pools and same name on them and call zpool import, you may not be able to import based on name but only based on unique uid and a different name on second import. No other problems to be expected. A unplug without prior export may be a bigger problem as this usually requires a reboot to remove the pool suspended state of a pool.
Click to expand...
Interesting thanks. I was thinking about ZFS on a USB storage (I think a SSD driving in a USB -> SATA adapter) the other day when reformatting one.
 
G

gea

Well-Known Member
Dec 31, 2010
3,172
1,197
113
DE
unwind-protect said:
I hesitate about this. ZFS can be very unamused on a sudden USB cable connection problem and the like. Keep in mind there is no fsck for ZFS.
Click to expand...
There is no fschk or chkdsk on ZFS because you do not need and there is nothing that can be fixed by such a tool.

A sudden crash or disconnect on ext/fat/ntfs can result in incomplete atomic writes with a damaged filesystem as result. Fschk or chkdsk are there to repair filesystem problems as good as possible. Due the lack of checksums, this is a best guess approach. Years ago a chkdsk on my mailserver gives a result like scrumbled eggs. Fschk or chkdsk have no options to validate good data. (This was the initial point for me switching to ZFS).

ZFS has Copy on Write. This means that any atomic write operation (ex write data and update metadata or write a raid stripe sequentially over several disks) is done completely or discarded at all, leaving ZFS in the state prior a write efford.

On all problems even silent data corruptions on disks, ZFS detects the problem during read due bad data or metadata checksums and repair on the fly (self healing).

This is why there is no fschk on ZFS. A sudden disconnect or crash during write is uncritical to ZFS
ZFS avoids problems that can be fixed with fschk by filesystem design.

Only point for raidless ZFS pools:
If a disk fails in a raid pool, the disk goes offline, the pool is degraded waiting for a disk replace or error reset if the disk comes back. No dataloss.

If a disk fails in a single disk pool, the pool state is "suspended" as there are incomplete and already commited writes ro the writing app, waiting for disk (dataloss) This is not critical for data on disk or the ZFS filesystem but as there is no way to complete writes, Linux/Unix waites forever. Currently you cannot force a pool export (with a dataloss for commited writes) but needs a reboot to regain access to the pool. Ths behaviour is more Unix related and ZFS is a Unix filesystem.
 
Last edited:
unwind-protect

unwind-protect

Active Member
Mar 7, 2016
424
157
43
Boston
It really isn't quite that clear cut. Reorderings from hardware (or rather, the firmware controlling that hardware) can happen and would leave a ZFS in an inconsistent state on a random disconnect. Even when ZFS only ever issued write sequences that as originally ordered were incorruptible. USB firmware can do random damage on top.

I'm not saying ZFS on USB is a bad idea, just that there isn't total safety on USB no matter what you do.
 
G

gea

Well-Known Member
Dec 31, 2010
3,172
1,197
113
DE
If a USB firmware does write data to disk "on its own" it is buggy, nothing to do with ZFS.
In any other cases, a sudden disconnect is uncritical, on other filesystems a single sudden disconnect is critical often on first occurance.

There is only one case that ZFS cannot control, this is a disc/controller cache situation when the disk or controller confirms a write that is not on disk when the system crashes or the disk disconnects. I would not expect a USB controller to behave like this. The disk cache is under control of the OS or ZFS. A single SSD without powerloss protection can be more critical in a raidless configuration but far less with ZFS compared with older filesystems on same SSD.

As all digital actions are not parallel but sequential (write block by block or stripe per stripe over a raid), there is no 100% guarantee that nothing can go wrong (only death is 100%). But regarding crash/disconnect behaviour ZFS is state of the art (no other filesystem behaves better). If you worry about sudden disconnects, you have all reasons with fat,ext or ntfs.
 
Last edited:
G

gea

Well-Known Member
Dec 31, 2010
3,172
1,197
113
DE
There are three questions about this:

- Does this affects ZFS more or less than any other filesystem (is it more secure to use ZFS than other options with USB)
- Does the controller commits a write when data is not on disk (this is the real question and such data caching should not happen even when a USB controller tries to "optimize" writes.)
-Is this a real danger (has a relevant statistical risk compared to other risks of dataloss). If so you would avoid USB for any use.
 
G

gea

Well-Known Member
Dec 31, 2010
3,172
1,197
113
DE
I can agree that USB is not as reliable as Sata or SAS, mainly due driver and controller problems with some of them, so USB disks for a regular fileserver instead Sata/SAS is nonsense.

But I have not seen statistics to condemn them in general for any use case. USB is the primary option for removeable disks and sometimes you use them for exact this reason ex for disaster backup (protection against fire,flash,theft,ransomware) that you can remove and place in a save location.
 
You must log in or register to reply here.
Top Bottom