BASTRIX SharedLVM for Proxmox VE 9 — Thin Pools and Thick Generations over shared FC/iSCSI SAN

[bastrix]

Hi everyone,

I am publishing BASTRIX SharedLVM, a free and open-source community storage
plugin for Proxmox VE 9.


What is it for?

BASTRIX SharedLVM is for Proxmox clusters that need to keep using an existing shared FC/iSCSI SAN while gaining snapshots, rollback, resize, backup and live migration.

It integrates with the native Proxmox Storage API and uses standard Linux LVM, device-mapper, multipath and cluster-locking components—without proprietary SAN integrations, third-party kernel modules or external locking services.

It does not use SAN snapshots or move LUNs between hosts. Proxmox manages the VM lifecycle, while the plugin manages per-VM volumes inside the shared VG in Thin or Thick Generations mode.

If you already use Ceph, NFS or a suitable vendor-native plugin, you probably do not need it.

The project provides two explicitly selectable allocation modes over an
existing dedicated shared LVM volume_STA group:

• Thin
One LVM-thin pool per VM, providing space-efficient allocation, snapshots,
rollback and guarded autogrow.

• Thick Generations
Fully allocated VM disks with independent, fully materialized snapshot
generations and deterministic physical-space reservation.

Both modes are included in the same Debian package. They can be configured as
separate Proxmox storage IDs over the same pinned shared VG and used side by
side. An ordinary Proxmox Storage Move can convert a disk from Thin to Thick
Generations or in the opposite direction.

Thick Generations uses persistent dm-clone only as a temporary transition
mechanism during snapshot and rollback materialization. Once materialization
finishes, the active disk and every retained snapshot are independent ordinary
linear LVs. There is no permanent snapshot or backing-volume chain in the
steady-state guest data path.

RC5.4 TG12 has been qualified on the Proxmox VE 9.2.x release line with Storage
API 14 and 15. Qualification included:

• Thin and Thick allocation, snapshot, rollback, resize and deletion;
• cross-node reconstruction and live migration;
• Thin-to-Thick and Thick-to-Thin Storage Move with block-hash verification;
• native Proxmox backup and cross-mode restore;
• Veeam HotAdd backup of Thin, Thick and mixed guests;
• supported-console Veeam restores to Thin, Thick and mixed layouts;
• two-node and three-node quorum and fail-closed behavior;
• installation and reinstallation across a three-node API 14/15 cluster;
• crash and partial-transaction recovery testing;
• a clean four-hour dual-mode endurance run;
• 156 Python tests and 236 Perl tests;
• reproducible Debian package builds.

The design remains deliberately safety-focused:

• storage identity, ownership, quorum and cluster locking are positively
verified before mutation;
• missing storage is treated as UNAVAILABLE, never as empty;
• foreign, legacy, unknown or ambiguous objects are not adopted or deleted;
• partial failures preserve data rather than attempting speculative cleanup;
• unknown block devices are never initialized automatically;
• the plugin performs no automatic metadata repair.

The plugin does not configure SAN connectivity, multipath, fencing or quorum.
It does not replace the storage array's availability and data-protection
mechanisms. Existing guest I/O during complete path loss remains governed by
QEMU, device-mapper, multipath and the underlying SAN.

RC5.4 TG12 is still a pre-release. It should first be validated on disposable
storage matching the intended production SAN, HBA, firmware, multipath and
failure policies. Multipathed iSCSI and a virtual Linux FCoE laboratory target
were exercised; representative physical enterprise FC hardware was not
available and is therefore not claimed as qualified.

GitHub:

GitHub - delltech1/proxmox-sharedlvmthin: Snapshot-capable shared FC/iSCSI SAN storage for Proxmox VE 9 clusters, with Thin Pools and Thick Generations over an existing shared LVM VG.

Snapshot-capable shared FC/iSCSI SAN storage for Proxmox VE 9 clusters, with Thin Pools and Thick Generations over an existing shared LVM VG. - delltech1/proxmox-sharedlvmthin

github.com

RC5.4 TG12 release and DEB:

Release SharedLvmThin RC5.4-TG12 · delltech1/proxmox-sharedlvmthin

SharedLvmThin RC5.4 TG12 release notes RC5.4 TG12 is the first public dual-mode release candidate. One Debian package supports explicit thin and thick-generations storage definitions over an existi...

github.com

Allocation modes:

https://github.com/delltech1/proxmox-sharedlvmthin/blob/experimental/thick-generations/docs/allocationocation-modes.md

Installation guide:

proxmox-sharedlvmthin/docs/installation.md at experimental/thick-generations · delltech1/proxmox-sharedlvmthin

Snapshot-capable shared FC/iSCSI SAN storage for Proxmox VE 9 clusters, with Thin Pools and Thick Generations over an existing shared LVM VG. - delltech1/proxmox-sharedlvmthin

github.com

Feedback, reproducible issue reports, GitHub Stars and anonymized compatibility
results for Proxmox VE, SAN arrays, HBAs, multipath configurations and firmware
versions are very welcome.

Please do not include credentials, private IP addresses, WWIDs or other
sensitive infrastructure identifiers in public reports.

 

[kapone]

 

[bastrix]

Project update — RC5.4 TG24 milestone (Proxmox Clusters)

A newer experimental release, RC5.4 TG24, is now available. It combines the established per-VM Thin mode with the optional Thick Generations mode and has reached an important qualification milestone.

Thick Generations explores a different trade-off from the Proxmox VE 9 snapshot-as-volume-chain preview:

PVE volume chains:
separate layered qcow2 volumes → active backing chain

BASTRIX Thick Generations:
temporary dm-clone transition → fully materialized independent linear LV
PVE volume chains prioritize fast and space-efficient snapshots. Thick Generations requires additional capacity and background materialization time, but completed snapshots and active generations do not retain a permanent backing chain. This is an architectural comparison, not a performance claim.

TG24 adds and qualifies:

• Thin and Thick modes within the same shared VG;
• snapshot, rollback, resize and multi-disk lifecycle handling;
• Thin-to-Thick and Thick-to-Thin Storage Move;
• migration, native HA relocation and backup/restore workflows;
• transaction-scoped persistent recovery metadata;
• bounded installation preflight with full read-only diagnostics available separately;
• concurrent qualification with 150 Thin and 150 Thick test VMs.

TG24 remains a pre-release and is not a universal SAN or enterprise-scale certification. Please test it first on disposable storage matching your intended SAN, multipath, fencing and workload configuration.

Release and DEB:

Release SharedLvmThin RC5.4 TG24 — Thin + Thick Generations · delltech1/proxmox-sharedlvmthin

Updated documentation:

GitHub - delltech1/proxmox-sharedlvmthin: Snapshot-capable shared FC/iSCSI SAN storage for Proxmox VE 9 clusters, with Thin Pools and Thick Generations over an existing shared LVM VG.

Feedback and sanitized hardware compatibility results are very welcome. Please do not publish private addresses, credentials, WWIDs or organization-specific storage identities.

 

[bastrix]

Update: RC5.4.1 TG25 is now available

TG25 is a maintenance and compatibility hotfix for the TG24 dual-mode milestone. It preserves the existing Thin and Thick Generations storage formats while improving PVE package-update, initramfs and post-reboot recovery safety.

It also adds the mandatory read-only sharedlvmthin compat-check for validating Storage API compatibility, required commands, services, quorum, initramfs contents and storage recovery state.

The release was tested with Proxmox VE 9 Storage API 14 and 15, including package upgrades, cold reboots, Thin/Thick lifecycle operations and cross-node migrations. It remains a release candidate—please validate it on disposable storage matching your SAN and multipath configuration.

Release and DEB:
Release SharedLvmThin RC5.4.1-TG25 · delltech1/proxmox-sharedlvmthin

Installation guide:
proxmox-sharedlvmthin/docs/installation.md at main · delltech1/proxmox-sharedlvmthin

Feedback and hardware compatibility reports are welcome.

 

[bastrix]

Update — RC5.31 TG52 released
BASTRIX SharedLVM RC5.31 TG52 is now available as a new experimental pre-release for Proxmox VE clusters using shared FC/iSCSI SAN storage.
What is new?

- Thick Generations now supports both Eager Zeroed and Lazy Zeroed provisioning.
- Added a standalone Thick-only DEB package.
- The original DUAL package continues to provide both Thin and Thick modes.
- Eager and Lazy volumes may share the same dedicated Thick VG.
- Thin and Thick storage must use physically separate, dedicated shared VGs.
- Additional fail-closed protection was added for cluster-wide mutations, upgrades, reboots, recovery and package-profile replacement.
- Compatibility coverage includes Proxmox VE Storage API 14 and 15.
- A deliberately constrained two-node Proxmox VE cluster mode remains available with explicit warnings.
- The release candidate passed 1,962 Python tests and 1,224 Perl tests, together with reproducible-build, package-profile parity and privacy checks.

SharedLVM is intended specifically for Proxmox VE clusters where every participating node sees the same FC/iSCSI LUN and shared LVM VG.
It is not intended for arbitrary local VGs or for VGs presented independently to different nodes.

Package options

Both package profiles use the same Thick implementation and on-disk format:

- pve-sharedlvmthin — DUAL Thin and Thick package.
- pve-sharedlvmthin-thick — standalone Thick-only package.

The two packages are mutually exclusive. The same package and version must be installed on every participating Proxmox VE node.

Experimental status
Both Thin and Thick Generations remain experimental.

Use only with disposable Proxmox VE hosts, disposable shared SAN storage and disposable guest data. This release is not production-ready, certified, commercially supported or warranted.
Independent tested backups, correct fencing, quorum, multipath configuration and a tested recovery procedure remain the operator’s responsibility.
Testing cannot cover every SAN array, firmware version, kernel, LVM/device-mapper version, multipath policy, cluster topology, workload or failure ordering.

Links

- RC5.31 TG52 release and DEB packages:
Release SharedLvmThin RC5.31-TG52 · delltech1/proxmox-sharedlvmthin

 

[bastrix]

BASTRIX SharedLVM RC5.97 TG53 — API14–16 bridge and stronger upgrade protection
Hi everyone,

BASTRIX SharedLVM RC5.97 TG53 is now available as an experimental pre-release for Proxmox VE 9 clusters using shared FC/iSCSI SAN storage.

This release retains the existing Thin and Thick functionality and extends the compatibility, package-preservation and update-control workflows.

What does SharedLVM provide?
SharedLVM integrates with the native Proxmox Storage API, using standard Linux LVM, device-mapper, multipath and Proxmox cluster components.

Available provisioning modes:

• Thin: separate per-VM LVM-thin pools with guarded autogrow.
• Thick Eager Zeroed: fully allocated, initialized volumes.
• Thick Lazy Zeroed: fully allocated volumes with guarded lazy initialization.

The project provides snapshots, rollback, disk growth, native Proxmox backup/restore and supported Storage Move workflows.

Thin and Thick use physically separate dedicated shared VGs. Eager and Lazy volumes may share a dedicated Thick VG.

Both package profiles remain available:

• pve-sharedlvmthin — DUAL Thin and Thick.
• pve-sharedlvmthin-thick — standalone Thick-only.

They share the same Thick implementation and on-disk format. The packages are mutually exclusive.

Administrator-controlled update protection
SharedLVM provides three explicit update policies, strengthened by the compatibility and settlement work in this release:

1. Freeze

Holds the monitored storage-stack packages and blocks ordinary changes to that stack.

This is intended for administrators who want to retain a known environment until they deliberately schedule maintenance. Unrelated updates can still proceed where their dependencies allow.

Held packages can include security fixes, so administrators must review them rather than leave the stack frozen indefinitely.

2. Qualified Auto

Permits only an exact manifest-qualified update transition.

This is an admission policy—not an automatic updater. It does not independently download upgrades, start APT or reboot hosts. A newer version number alone does not establish compatibility.

3. Warn / explicit authorization

An unqualified watched update requires explicit authorization for the specific proposed package transaction.

After installation, runtime qualification must be settled before storage mutations are permitted again. This is not a blanket bypass for subsequent updates.

These controls help prevent accidental storage-stack changes. They do not remove the administrator’s authority or make a forced/manual package replacement safe.

What RC5.97 adds

• Lifecycle adapters for Proxmox Storage API 14, 15 and 16, including API16 RAW-format contracts.
• Exact package-family and operation-specific compatibility checks.
• A plugin-first bridge workflow: update SharedLVM first, then deliberately update the Proxmox stack one node at a time.
• Stronger package preservation and executor fencing.
• Qualification tied to the actual running kernel, boot and loaded runtime.
• Improved recovery/reference checks and read-only health reporting.
• Additional native-task and guest-protocol regression tooling.

Plugin replacement does not require restarting QEMU, deactivating guest volumes, reloading disk mappings or restarting multipath. Live guest I/O continued during the exercised plugin upgrades. Management-service refresh can briefly affect the Proxmox API/web interface.

A host reboot remains a planned maintenance operation: evacuate workloads and verify runtime, SSH and storage health before returning the node to service.

Migration behavior remains state-aware
Materialized Thick volumes use the normal shared-storage live-migration contract.

Thin workloads can use the Materialized Migration Bridge: online Storage Move to Thick, normal live migration, then Storage Move back to target-owned Thin pools. Direct simultaneous cross-node activation of the same thin pool is refused.

Active, unmaterialized Lazy volumes require the documented materialization/preflight workflow before online migration.

Testing
The release source suite completed:

• 2,647 Python tests: 2,594 passed and 53 declared skips.
• 1,691 Perl tests.
• Reproducible DUAL/Thick-only builds, shared-payload parity, package identity and privacy checks.

Current-build lab testing included clean API14 package/profile cycles, live plugin upgrades with write/read/SHA witnesses, an API15→16 package transition, kernel reboot and storage reconnection, and three-disk RAM snapshot/rollback verification.

Release notes include the exact exercised Proxmox package and kernel versions.

Experimental status
Thin, Thick Eager and Thick Lazy remain experimental.

Use only disposable Proxmox VE hosts, disposable shared storage and disposable guest data. This is not production-ready, certified, commercially supported or warranted. Maintain independently tested backups and recovery procedures.

Links

• RC5.97 TG53 release, both DEBs and SHA256SUMS
• Project
• Installation guide
• Compatibility information
• Risk and support boundary

Sanitized testing reports, code reviews and reproducible failure cases are welcome.