Booting from ZFS: How to Best Handle Swap Space?

[mattlach]

Hey all,

So I plan on booting a server I am putting together from a ZFS mirror of two small SSD's, but no other drives.

There are a few different ways one can handle swap space in this setup. I was hoping to get some input from you guys on how to best do this. I have identified two methods I can think of below, but are there others?

Option 1: Using Zvol's

1.) Create the ZFS mirror using 100% of the SSD's

2.) Create a Zvol on the pool.

3.) Format Zvol as a linux swap device

Benefits:
- Swap space is also mirrored, meaning system is less likely to crash if a drive fails.
- ZFS gets to natively manage the entire drives, which it likes to do.

Drawbacks:
- Snapshots of root file system will include swap data?
- Swap exists on non-native block device in a file?


Option 2: Using Separate Partitions

1.) Create equal sized partitions on eeach drive for root file system.

2.) Mirror partitions above using ZFS

3.) Create swap partitions on bare drives

Benefits:
- More swap space
- Swap space is on bare metal on disk.

Drawbacks:
No swap redundancy.


Option 3: ???


How would you set this up, and why?


Appreciate any input.

 

[Patrick]

Here is a Proxmox VE rpool/ swap setup if that helps:

rpool                     53.5G   307G    96K  /rpool
rpool/ROOT                4.09G   307G    96K  /rpool/ROOT
rpool/ROOT/pve-1          4.09G   307G  4.09G  /
rpool/swap                49.4G   355G  1012M  -

 

[PigLover]

When booting from ZFS I've always preferred to use the whole disk for the rpool and swap on a zvol. This lets ZFS function as your volume manager and keeps everything very, very simple (for example, you don't have to worry about partitioning the drive, ensuring partition aliment, etc).

I don't see the snapshot issue as a really big deal. If snaps are your concern then you can just snap "rpool/ROOT" insteal of "rpool".

 

[mattlach]

Here is a Proxmox VE rpool/ swap setup if that helps:

rpool                     53.5G   307G    96K  /rpool
rpool/ROOT                4.09G   307G    96K  /rpool/ROOT
rpool/ROOT/pve-1          4.09G   307G  4.09G  /
rpool/swap                49.4G   355G  1012M  -

When booting from ZFS I've always preferred to use the whole disk for the rpool and swap on a zvol. This lets ZFS function as your volume manager and keeps everything very, very simple (for example, you don't have to worry about partitioning the drive, ensuring partition aliment, etc).

I don't see the snapshot issue as a really big deal. If snaps are your concern then you can just snap "rpool/ROOT" insteal of "rpool".

Ahh, OK.

I've never played around with zvols before, but it looks like it creates it in a separate dataset. That's good. I pictured it essentially just being an image file.

 

[mattlach]

When booting from ZFS I've always preferred to use the whole disk for the rpool and swap on a zvol. This lets ZFS function as your volume manager and keeps everything very, very simple (for example, you don't have to worry about partitioning the drive, ensuring partition aliment, etc).

I don't see the snapshot issue as a really big deal. If snaps are your concern then you can just snap "rpool/ROOT" insteal of "rpool".

Upon further thought, could this ever lead to a counter productive scenario where the ZFS ARC in RAM is used to cache data swapped to disk due to lack of RAM?

Is there a dataset/zvol setting that tells ZFS to minimize the ARC footprint of a certain dataset or zvol?

 

[Rhinox]

Option 3: do not use swap at all.

 

[mattlach]

Option 3: do not use swap at all.

I equip my servers with enough RAM that swap shouldn't be necessary, but I still feel it is better to err on the side of safety just in case...

 

[gea]

For Solaris, the swap size recomendation is
1GB with a system of 4GB RAM or 1/4 of physical memory

Planning for Swap Space - Oracle Solaris Administration: Devices and File Systems

 

[mattlach]

For Solaris, the swap size recomendation is
1GB with a system of 4GB RAM or 1/4 of physical memory

Planning for Swap Space - Oracle Solaris Administration: Devices and File Systems

That is good to know. I'll have to look up the recommendation for a headless Ubuntu Linux server.

I have to admit I'm a bit embarrassed that I've just been using the old Windows "swap shouLd be twice the size of system RAM" rule of thumb, even though I've mostly been using Linux for over 15 years.

I never thought to look if it was any different for other operating systems.

That being said, I vaguely remember the Ubuntu and Linux Mint installers committing a huge amount of disk space to swap by default as well.

Maybe this is a server vs desktop thing, not a *nix vs Windows thing?

 

[cperalt1]

ZestyZapus/ReleaseNotes - Ubuntu Wiki newest version = no swap partition