This article is intended for administrators wishing to create a 2x node + witness High Availability cluster utilizing Proxmox Virtual Environment (https://www.proxmox.com/en/proxmox-virtual-environment/overview) and StorMagic SvSAN (https://stormagic.com/svsan/).
Note: All images are clickable for enlarging, or can be opened in a new tab
Information
Guide Summary
This multipart guide will walk through the process to deploy 2x hyperconverged Proxmox VE nodes utilizing SvSAN virtualized block storage and a lightweight witness node, such as a Raspberry Pi.
Create the SvSAN Storage Controller Virtual Machines
Note: Repeat the below steps on both hyperconverged nodes.
Note: Ensure to create VSA2 with a different VM ID - per the previous mention of unable to join when there’s VMs
Create a VM
Ensure to specify to start at boot.
Set the install ISO to the uploaded SvSAN file.
Use default system settings.
Modify the scsi0 boot disk down from the default size to 1GB smallest (potentially SvSAN will move to a 2GB boot disk in the future)
Add an additional scsi1 disk to be the journal, specifying 20GB in size. This is utilized for Change Block Tracking.
It is possible to add a disk image, equivalent to a VMDK in VMware, as SAN storage that we would add here as scsi3.
However in this example we'll utilize an RDM sym link to the physical disk post SvSAN VSA VM creation.
Specify >1GB RAM max 32GB for memory caching
Attach the default NIC to vmbr0, selecting a VirtIO paravirtuzalied NIC driver
Review and confirm the summary on completing the wizard
Modify the VM Version/Proxmox version
In order to support Proxmox 8, set the VM to be compatible with Proxmox 7.
Add the secondary, and any additional, network interfaces
Note: Repeat the below steps on both hyperconverged nodes/VSAs.
Ensuring to attach the NIC to the correct bridge/network
Add the RDM sym link
Note: Repeat the below steps on both hyperconverged nodes.
The below equivalent to vmkfstools in VMware, documented below:
https://stormagic.com/doc/svsan/6-3-P2/en/Content/allocating-additional-storage-vs.htm
It is not possible to add an RDM link disk via Proxmox GUI, like you can in VMware or Hyper-V.
https://www.youtube.com/watch?v=U-UTMuhmC1U
Most SvSAN customers will typically have either
1. some form of boot storage and some form of SAN storage. e.g. RAID1 boot and RAID5 SAN
2. or split virtual disks
In both cases above the host will see the boot as /dev/sda and the storage to be used for the SAN as /dev/sdb
If there are other disks the letters will increment through the alphabet being /dev/sdc etc
NVMe based storage, may appear differently. Just be careful here and ensure nothing else is using the disks.
Commands used:
ls -n /dev/disk/by-id/
/sbin/qm set [VM-ID] -virtio2 /dev/disk/by-id/[DISK-ID]
Example 1 - Multiple disk device add
This example was run on hardware with 2x NVMe's assigning to SvSAN to be used via software RAID1 for the pool storage
If mapping up multiple disks for software RAID, note the VM ID
101
and the virtio id virtio2
example- multiple RDM/pools disk add (from HPE Edgelines with 2x Samsung NVMe for SvSAN software RAID1)
root@proxmox-1:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0 9 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part3 - ../../sda3
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-name-pve-root - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:27 dm-name-pve-swap - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-uuid-LVM-rk53mp1TcHumnHnAxU6G12FI7KjFvXMK4Rbm0N5hFIyDERfHbfFdxEcmxTz6OycW - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:27 dm-uuid-LVM-rk53mp1TcHumnHnAxU6G12FI7KjFvXMKhnwdeMkQJemP4yiqT2lZtLmNLrGJpSFm - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 lvm-pv-uuid-VWXWt8-8cgk-hxo9-FOcX-c1yz-U8KL-zC43FB - ../../sda3
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-eui.343646304e6108110025385900000001 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-eui.343646304e6108360025385900000001 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610811 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610836 - ../../nvme1n1
lrwxrwxrwx 1 0 0 9 Oct 10 09:45 wwn-0x5707c18100925c1a - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part3 - ../../sda3
root@proxmox-1:~#
root@proxmox-1:~# /sbin/qm set 100 -virtio2 /dev/disk/by-id/nvme-eui.343646304e6108110025385900000001
update VM 100: -virtio2 /dev/disk/by-id/nvme-eui.343646304e6108110025385900000001
root@proxmox-1:~# /sbin/qm set 100 -virtio3 /dev/disk/by-id/nvme-eui.343646304e6108360025385900000001
update VM 100: -virtio3 /dev/disk/by-id/nvme-eui.343646304e6108360025385900000001
root@proxmox-1:~#
root@proxmox-2:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0 9 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part3 - ../../sda3
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-name-pve-root - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:24 dm-name-pve-swap - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-uuid-LVM-ng5R1ctgy041DlLIagXtSqn6454eaenqF4sEOW9d2RM315ejVzxpOUlu9ayhn3YK - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:24 dm-uuid-LVM-ng5R1ctgy041DlLIagXtSqn6454eaenquJLJNeWfSD4Hg5Q9faLlAv9PWZielnX8 - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 lvm-pv-uuid-WGMruL-XZfV-7zPW-3BVp-Qzqg-Twrd-PS8qic - ../../sda3
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-eui.343646304e6102800025385900000001 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-eui.343646304e6107270025385900000001 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610280 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610727 - ../../nvme0n1
lrwxrwxrwx 1 0 0 9 Oct 10 09:45 wwn-0x5707c18100925646 - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part3 - ../../sda3
root@proxmox-2:~# /sbin/qm set 101 -virtio2 /dev/disk/by-id/nvme-eui.343646304e6102800025385900000001
update VM 101: -virtio2 /dev/disk/by-id/nvme-eui.343646304e6102800025385900000001
root@proxmox-2:~# /sbin/qm set 101 -virtio3 /dev/disk/by-id/nvme-eui.343646304e6107270025385900000001
update VM 101: -virtio3 /dev/disk/by-id/nvme-eui.343646304e6107270025385900000001
root@proxmox-2:~#
And in the GUI below:
Example 2 - single RDM/Pool add
This example was completed on a system nested on VMware, hence the disk name.
Note: below sda and sdb are seen twice. once with a SCSI label:
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
and once with a wwn label:
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
For simplicities sake, and to keep this architecture similar to VMware, where SvSAN maps to the naa. device (e.g. scsi) it is recommended to use the SCSI option
root@proxmox1:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 ata-VMware_Virtual_IDE_CDROM_Drive_00000000000000000001 -> ../../sr0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-name-pve-root -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-name-pve-swap -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-uuid-LVM-397ePpY9Yy0rnlBAaao02wNyCEIqmouhKzBkyr2czUFry9Raklzym2p2zynHSDnB -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-uuid-LVM-397ePpY9Yy0rnlBAaao02wNyCEIqmouhnvouNGymAO9TNqIkjunt0nK3NogmNjfr -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 lvm-pv-uuid-mEFIaM-ZrA9-Ulan-3XhS-i1DL-hbFZ-9iS3dp -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
root@proxmox1:~# /sbin/qm set 100 -virtio2 /dev/disk/by-id/scsi-36000c29fd24a75bc12f86ebaa096dbb6
update VM 100: -virtio2 /dev/disk/by-id/scsi-36000c29fd24a75bc12f86ebaa096dbb6
root@proxmox1:~#
Noting the disk ID ending dbb6
Install the SvSAN VSA VM
Power on the VSA VM, open the console and complete the installer wizard
Complete a default to a standard install
Select to install to the 1GB boot disk, sda
and confirm
Once completed, press Enter to reboot
Configure the VSA VM
The VSA will boot up like an blank deployed OVA in VMware with DHCP on the NICs.
password = ‘password’
Either via the VSA VM console login and set IPs,
Set static IP and DNS below via the console, unless set via DHCP, so that when in the wizard the VSA can route out to and resolve https://licensing.stormagic.com in order to license.
Complete the standard default boot wizard via the WebGUI
Accept the EULA:
Proceed through the wizard:
License the VSA VM
Specify hostname and domain name
Set a password
Set an NTP Server, in order license VSA time must be within an hour of correct, with NTP also configurable via the VSA VM console
Skip setting an iSNS server
Review the IP addresses configured, these can be modified later
Select a JBOD pool - The below shows an already created one due to detecting an existing pool signature having reinstalled to the same disk
Login and view the VSA, such as Network configuration and traffic flags
and storage assigned to manage
See Also
Comments
0 comments
Article is closed for comments.