This article is intended for administrators wishing to create a 2x node + witness High Availability cluster utilizing Proxmox Virtual Environment (https://www.proxmox.com/en/proxmox-virtual-environment/overview) and StorMagic SvSAN (https://stormagic.com/svsan/).

Please ensure the VSA VM is stored on local, non-shared disks on the proxmox host. e.g. local-lvm or local zfs datastores.

Note: All images are clickable for enlarging, or can be opened in a new tab

Information

Guide Summary/shortcuts
Create the SvSAN Storage Controller Virtual Machines via the SvSAN VSA OVA
Configure the VSA Networking
Configure the VSA Storage
Add the RDM sym link/SAN Storage
Example 1 - Multiple disk device add
Example 2 - single RDM/Pool add
Boot and Configure the VSA VM
First Boot Wizard

Guide Summary/shortcuts

This multipart guide will walk through the process to deploy 2x hyperconverged Proxmox VE nodes utilizing SvSAN virtualized block storage and a lightweight witness node, such as a Raspberry Pi.

https://support.stormagic.com/hc/en-gb/articles/17158984245789-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-0-5-Creating-nested-VMs-for-Proxmox-on-VMware

https://support.stormagic.com/hc/en-gb/articles/15088839377053-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-1-Installing-Proxmox

https://support.stormagic.com/hc/en-gb/articles/15567317393181-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-2-Preparing-the-Hosts

https://support.stormagic.com/hc/en-gb/articles/16198574072733-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-3-Creating-the-Proxmox-Cluster

https://support.stormagic.com/hc/en-gb/articles/16204170204061-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-4-VSA-Deployment

https://support.stormagic.com/hc/en-gb/articles/16343256511261-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-5-Storage-Provisioning

https://support.stormagic.com/hc/en-gb/articles/16347739183005-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-6-Create-Guests-and-Enable-HA

https://support.stormagic.com/hc/en-gb/articles/27437315756573-Deploying-a-2x-node-witness-Proxmox-and-SvSAN-HA-cluster-Part-7-Enabling-healthy-login-and-cluster-restart

Create the SvSAN Storage Controller Virtual Machines via the SvSAN VSA OVA

Note: Repeat the below steps on both hyperconverged nodes.

Note: Note: In the example below, 900 is the VM ID and local is the destination storage. Change these values as required for the environment. Ensure each VSA uses a unique VM ID.

With the local storage volume selected, select Import, select the previously uploaded VSA OVA, and select Import.

Alternatively, the VSA OVA can be extracted and imported from the Proxmox CLI.

To import via the Proxmox CLI:

mkdir -p /root/stormagic-import
tar -xf /var/lib/vz/import/StorMagic_VSA_1.0-bios.ova -C /root/stormagic-import
cd /root/stormagic-import
ls -l

Once extracted, import the OVF into Proxmox:

qm importovf 900 /root/stormagic-import/StorMagic_VSA_1.0-bios.ovf local

Give the VSA VM a name, changing from the default StorMagic-SvSAN and confirm it has 1vCPU and 2GB RAM (up to 32GB if utilizing SSD/Memory caching)
https://support.stormagic.com/hc/en-gb/articles/5887719016989-SvHCI-SvSAN-Caching

Ensure the imports complete successfully:

On both VSAS, with the VSA VM selected, select Options and enable start at boot:

Configure the VSA Networking

Note: Repeat the below steps on both hyperconverged nodes/VSAs.

Modify net1 to attach to a storage NIC

Modify both net0 and net1 from the default vmxnet3 to virtio NIC drivers.

Note that the SvSAN VSA includes drivers for vmxnet3, virtio and E1000E.

Technical Services have seen different network throughputs on different systems and advise validating network performance per the below KB:

https://support.stormagic.com/hc/en-gb/articles/8116960680861-How-to-test-and-validate-network-performance-in-an-SvHCI-or-SvSAN-Environment

add additional logical vNICs to the previously created bridges, if required

Ensuring to attach the NIC to the correct bridge/network

Configure the VSA Storage

On both VSAs, with the VSA VM selected, select Hardware, and add a Hard Disk (the VSA Journal Disk)

Create the disk as 20GB, to be the Journal for change block tracking and logs.

Modify the SCSI Controller from the Default "LSI 53C895A" over to VirtIO SCSI

Modify the VSA storage controller to virtIO

It is also possible to add a disk image, equivalent to a VMDK in VMware, as SAN storage that we would add here as scsi3.

However in this example we'll utilize an RDM equivalent symbolic link to the physical disk, to utilize as SAN storage.

Add the RDM sym link/SAN Storage

Note: Repeat the below steps on both hyperconverged nodes.

The below equivalent to vmkfstools in VMware, documented below:

https://stormagic.com/doc/svsan/6-7/en/Content/allocating-additional-storage-vs.htm

It is not possible to add an RDM link disk via Proxmox GUI, like you can in VMware or Hyper-V.

The below third party video details disk passthru also.

https://www.youtube.com/watch?v=U-UTMuhmC1U

Most SvSAN customers will typically have either

1. some form of boot storage and some form of SAN storage. e.g. RAID1 boot (such as BOSS/NS204 etc) and separate SAN (RAID10, RAID5, RAID50 etc)
SvSAN supports both hardware RAID and can also support software RAID over individual disks:

https://support.stormagic.com/hc/en-gb/articles/17597617525917-SvHCI-SvSAN-and-Software-RAID

2. or split virtual disks

https://support.stormagic.com/hc/en-gb/articles/5665208916381-SvSAN-Leveraging-Split-Virtual-Disks-Logical-Volumes-with-HPE-Servers

https://support.stormagic.com/hc/en-gb/articles/9128058771741-SvSAN-Leveraging-Split-Virtual-Disks-Logical-Volumes-with-Dell-Servers

https://support.stormagic.com/hc/en-gb/articles/5896861868317-SvSAN-Leveraging-Split-Virtual-Disks-Logical-Volumes-with-Lenovo-Servers

In both cases above the host will see the boot as /dev/sda and the storage to be used for the SAN as /dev/sdb

If there are other disks the letters will increment through the alphabet being /dev/sdc etc

NVMe based storage, may appear differently. Just be careful here and ensure nothing else is using the disks.

Commands used:

ls -n /dev/disk/by-id/

/sbin/qm set [VM-ID] -virtio2 /dev/disk/by-id/[DISK-ID]

Example 1 - Multiple disk device add

These Edgeline systems have 2x 3.87TB NVMe drives that, in this example, we'll map to the SvSAN VSA and software RAID1 to provide internal node resiliency, and then the SvSAN synchronous mirroring across the VSAs/nodes providing cross-node resiliency.

This architecture enables 2x fault domains.

The below putty sessions are from the Edgeline systems demonstration the symbolic link creation.

This example was run on hardware with 2x NVMe's assigning to SvSAN to be used via software RAID1 for the pool storage

If mapping up multiple disks for software RAID, note the VM ID

101 and the virtio id virtio2

example - multiple RDM/pools disk add (from HPE Edgelines with 2x Samsung NVMe for SvSAN software RAID1)

root@proxmox-1:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part3 - ../../sda3
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-name-pve-root - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:27 dm-name-pve-swap - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-uuid-LVM-rk53mp1TcHumnHnAxU6G12FI7KjFvXMK4Rbm0N5hFIyDERfHbfFdxEcmxTz6OycW - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:27 dm-uuid-LVM-rk53mp1TcHumnHnAxU6G12FI7KjFvXMKhnwdeMkQJemP4yiqT2lZtLmNLrGJpSFm - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 lvm-pv-uuid-VWXWt8-8cgk-hxo9-FOcX-c1yz-U8KL-zC43FB - ../../sda3
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-eui.343646304e6108110025385900000001 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-eui.343646304e6108360025385900000001 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610811 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610836 - ../../nvme1n1
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 wwn-0x5707c18100925c1a - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part3 - ../../sda3
root@proxmox-1:~#
root@proxmox-1:~# /sbin/qm set 100 -virtio2 /dev/disk/by-id/nvme-eui.343646304e6108110025385900000001
update VM 100: -virtio2 /dev/disk/by-id/nvme-eui.343646304e6108110025385900000001
root@proxmox-1:~# /sbin/qm set 100 -virtio3 /dev/disk/by-id/nvme-eui.343646304e6108360025385900000001
update VM 100: -virtio3 /dev/disk/by-id/nvme-eui.343646304e6108360025385900000001
root@proxmox-1:~#


root@proxmox-2:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part3 - ../../sda3
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-name-pve-root - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:24 dm-name-pve-swap - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-uuid-LVM-ng5R1ctgy041DlLIagXtSqn6454eaenqF4sEOW9d2RM315ejVzxpOUlu9ayhn3YK - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:24 dm-uuid-LVM-ng5R1ctgy041DlLIagXtSqn6454eaenquJLJNeWfSD4Hg5Q9faLlAv9PWZielnX8 - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 lvm-pv-uuid-WGMruL-XZfV-7zPW-3BVp-Qzqg-Twrd-PS8qic - ../../sda3
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-eui.343646304e6102800025385900000001 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-eui.343646304e6107270025385900000001 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610280 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610727 - ../../nvme0n1
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 wwn-0x5707c18100925646 - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part3 - ../../sda3
root@proxmox-2:~# /sbin/qm set 101 -virtio2 /dev/disk/by-id/nvme-eui.343646304e6102800025385900000001
update VM 101: -virtio2 /dev/disk/by-id/nvme-eui.343646304e6102800025385900000001
root@proxmox-2:~# /sbin/qm set 101 -virtio3 /dev/disk/by-id/nvme-eui.343646304e6107270025385900000001
update VM 101: -virtio3 /dev/disk/by-id/nvme-eui.343646304e6107270025385900000001
root@proxmox-2:~#

Once completed successfully this will display in the GUI per the below:

Example 2 - single RDM/Pool add

This example was completed on a system nested on VMware, hence the disk name.

Note: below sda and sdb are seen twice. once with a SCSI label:

lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb

and once with a wwn label:

lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb

For simplicities sake, and to keep this architecture similar to VMware, where SvSAN maps to the naa. device (e.g. scsi) it is recommended to use the SCSI option

root@proxmox1:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 ata-VMware_Virtual_IDE_CDROM_Drive_00000000000000000001 -> ../../sr0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-name-pve-root -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-name-pve-swap -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-uuid-LVM-397ePpY9Yy0rnlBAaao02wNyCEIqmouhKzBkyr2czUFry9Raklzym2p2zynHSDnB -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-uuid-LVM-397ePpY9Yy0rnlBAaao02wNyCEIqmouhnvouNGymAO9TNqIkjunt0nK3NogmNjfr -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 lvm-pv-uuid-mEFIaM-ZrA9-Ulan-3XhS-i1DL-hbFZ-9iS3dp -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb


root@proxmox1:~# /sbin/qm set 100 -virtio2 /dev/disk/by-id/scsi-36000c29fd24a75bc12f86ebaa096dbb6
update VM 100: -virtio2 /dev/disk/by-id/scsi-36000c29fd24a75bc12f86ebaa096dbb6
root@proxmox1:~#

Noting the disk ID ending dbb6

Boot and Configure the VSA VM

The VSA will boot up like an blank deployed OVA in VMware with DHCP on the NICs.

Per the below Console walkthrough KB:

https://support.stormagic.com/hc/en-gb/articles/18860298535709-SvHCI-SvSAN-Console-usage

Management:

and Storage interface:

It is also advised to set DNS, NTP and a network hostname such that the VSA will license correctly when in the first boot wizard, being able to route out to and resolve https://licensing.stormagic.com.

Offline license files may be used for closed/air gapped environments per the below KB:

https://support.stormagic.com/hc/en-gb/articles/6397884188829-SvHCI-SvSAN-Online-and-Offline-Licensing