This article is intended for administrators looking to explore alternative configurations whereby the HA is in the application stack and not the Hypervisor/Storage.
This enables use of Proxmox VM snapshots with block storage, provided by SvSAN enabling caching and data-at-reset encryption functionality however excludes VM High Availability so will require application level clustering.
Note: All images are clickable for enlarging, or can be opened in a new tab.
Resolution/Information
What is Proxmox
Proxmox is an open-source virtualization management platform known as Proxmox Virtual Environment (Proxmox VE). It is built on Debian Linux and Kernel-based Virtual Machine (KVM) for full virtualization and combined with using iSCSI storage administrators can create a more robust and flexible virtualization environment with enhanced storage capabilities, making it a versatile tool for various needs whether this is on the Edge or Enterprise and a viable alternative to Broadcom and other Hypervisors.
What is SvSAN
StorMagic SvSAN is a tried and tested software-defined storage solution that is typically used for highly available shared storage using only two servers over iSCSI. It is designed to provide simple, cost-effective hyperconverged infrastructure without compromising on reliability or performance. It is lightweight and has the ability to operate with minimal hardware requirements (1 vCPU and 1GB of memory per node from version 6.7 2 vCPU and 2 GB of memory). Uses a lightweight witness service to monitor the cluster's "heartbeat" and ensure proper operations - such as a raspberry pi.
SvSAN is particularly well-suited for edge computing, remote offices, and distributed IT environments where traditional storage solutions may be impractical or cost-prohibitive. It allows organizations to maintain high availability and protect critical data across various locations, from headquarters to remote sites.
Combined together a 2-node High Availability Proxmox cluster can be built with 2 Servers and a lightweight StorMagic witness and Proxmox qdevice such as a Raspberry pi. For details on how to build a Proxmox Cluster with SvSAN please see these guides.
LVM vs LVM-thin
LVM, or Logical Volume Management, is a storage device management technology that provides flexible administration of disk space in Linux systems. It creates an abstraction layer between physical storage devices and the operating system, allowing for more dynamic and efficient management of storage resources.
Key Components:
Physical Volumes (PVs): These are the actual storage devices or partitions that LVM manages.
Volume Groups (VGs): PVs are combined into VGs, which act as pools of storage space.
Logical Volumes (LVs): These are created from VGs and function like partitions, but with greater flexibility.
The key difference between LVM and LVM-thin is that LVM-thin cannot be used as Shared Storage so cannot be used in a standard HA Hyper converged Cluster. But LVM-thin supports live snapshots for KVM virtual machines
However this article is not about configuring a standard HA solution as a requirement for shared iscsi Storage is an LVM must be used. This means that if there is a requirement to use snapshots from proxmox this is at present not possible with an LVM.
This is where the use of LVM-thin comes into play with regards to using snapshots when a shared storage is not required ie the Applications running on the LVM-thin have their own built HA.
However additional features from SvSAN can be used with LVM-thin such as Caching (IO acceleration) or Encryption at rest.
How to Configure
Follow the steps for a normal SvSAN deployment with Proxmox up until this stage:
Create an VG (Volume Group) on the multipath device
Note it is a choice at this stage if mirrored storage is desired or if just simple non-mirrored Targets bearing in mind that the HA is factored/needed to be provided in the Guest application.
If using a standard LVM after creating the Volume Group the LVM would be added using the Proxmox GUI.
However when using LVM-thin first the LVM needs to be created and converted as per the following steps:
In the example below the following has been created:
- Physical Volume:
pvcreate /dev/mapper/mpatha- Volume Group:
vgcreate vg-vm-lvmthin1 /dev/mapper/mpatha- Logical Volume
lvcreate -L 40G -n lvmthin1 vg-vm-lvmthin1- Conversion of LVM to LVM-thin
lvconvert --type thin-pool vg-vm-lvmthin1/lvmthin1When working with LVM-Thin the metadata size for the snapshots needs to be considered.
Source: "man" for "lvmthin":
Size of pool metadata LV
The amount of thin metadata depends on how many blocks are shared between thin LVs (i.e. through snapshots). A thin pool with many snapshots may need a larger metadata LV. Thin pool metadata LV sizes can be from 2MiB to approximately 16GiB.
When an LVM command automatically creates a thin metadata LV, the size is specified with the --poolmetadatasize option. When this option is not given, LVM automatically chooses a size based on the data size and chunk size.
Once the LVM-thin has been created the size of the metadata can be adjusted with the following command:
To see the existing allocation the following command can be used:
lvs -aThis can then be extended with the following command which will extend the poolmetadatasize for the LVM-thin by 2GiB (note the max is approximately 16 Gib).
lvextend --poolmetadatasize +2G vg-vm-lvmthin1/lvmthin1_tmetaOnce converted to LVM-thin this can then me mapped to the Host using the Proxmox GUI
Datacenter>Storage>Add LVM-Thin
Fill in the required details and click Add.
- ID
- Volume group
- Thin Pool
- Nodes
- Enable
Then the VMs can be created on the Storage as normal and once running there should be the option in the Snapshots tab to Take Snapshot.
Taking Snapshots is very intuitive select the Take Snapshot and fill in the details/options and then confirm with the Take Snapshot button.
The metadata size etc and use of the LVM-thin can then be checked as before:
root@mc-proxmox8-01:~# lvs -a
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
lvmthin1 vg-vm-lvmthin1 twi-aotz-- <95.82g 19.31 1.07
[lvmthin1_tdata] vg-vm-lvmthin1 Twi-ao---- <95.82g
[lvmthin1_tmeta] vg-vm-lvmthin1 ewi-ao---- <2.09g
[lvol0_pmspare] vg-vm-lvmthin1 ewi------- <2.09g
snap_vm-102-disk-0_Snap1 vg-vm-lvmthin1 Vri---tz-k 32.00g lvmthin1 vm-102-disk-0
vm-102-disk-0 vg-vm-lvmthin1 Vwi-aotz-- 32.00g lvmthin1 55.68
vm-102-state-Snap1 vg-vm-lvmthin1 Vwi-a-tz-- <4.49g lvmthin1 15.22 For further information about enabling SSD Caching(Io acceleration) and Encryption please see:
See Also
https://stormagic.com/doc/svsan/6-3-U1/en/Content/global/home-landing.htm
https://stormagic.com/doc/svkms/svkms_2.6.5/Content/Global_Topics/Home.htm
Comments
0 comments
Article is closed for comments.