SvSAN & Proxmox 9 - Deployment Guide

HPE Edgeline el4000

Cartridge 1 iLO = 10.10.194.1/24
Cartridge 2 iLO = 10.10.194.2/24

HPE M510 Cartridge
Intel Xeon CPU D-1548 8C@2.00GHz
64GB RAM, 2x 32GB DIMMS
1x120GB SATA SSD as boot disk
2x 4TB (3840GiB) NVMe SSDs
- Samsung MZ1LB3T8HMLA-00007

This example runbook enables the deployment of 2x servers running Proxmox on separate boot storage,  and 2x SvSAN VSAs enabling shared storage with software RAID1 over 2x 3.84TB NVMe m.2 adapters similar to the below diagram:

• Install using the Proxmox ISOs
• Proxmox install splash screen

• Promox install EULA

• Proxmox OS disk selection

• Proxmox OS disk selection - selecting the 512GB boot m.2

• Proxmox install location and time zone settings

• Promox install root password configuration

• Promox install hostname and management IP address settings

• Proxmox install Installation config summary

• Proxmox installation progress

• Proxmox Console view

• Promox Console view

• Proxmox VM webGUI access on node IP address on port 8006

• Should the Proxmox WebGUI not be accessible, the networking may be modified via the console (such as iLO) to modify /etc/network/interfaces, per the below, should the wrong NIC have been selected or gateway details wrongly entered or similar.

Note: Repeat the below steps on both hyperconverged nodes.

• Delete the LVM on the “SAN disk”, equivalent to deleting a local VMFS in VMware, on the disk to be assigned to the SvSAN VSA. e.g. the storage we’ll use for the SAN is currently consumed as “Local Storage”

• Confirm

• Next enable storing of VMs on the boot “Local Disk”

• Enable VM storage on the local datastore, leaving the SAN storage unformatted to be handed to the StorMagic VSA later

Note: Repeat the below steps on both hyperconverged nodes.

• Under Content select “Disk image”, "ISO Image", and "Import" in addition to the other selected items

• Such that it now looks like:

Note: Repeat the below steps on both hyperconverged nodes.

• Expand the file system local boot root file system to have room for the VSA VM:
• The below example shows the increase of /dev/mapper/pve-root from a small percentage to the full size of the disks and uses the commands:

root@edgeline01:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 12G 0 12G 0% /dev
tmpfs 2.4G 904K 2.4G 1% /run
/dev/mapper/pve-root 69G 2.5G 63G 4% /
tmpfs 12G 46M 12G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/fuse 128M 16K 128M 1% /etc/pve
tmpfs 2.4G 0 2.4G 0% /run/user/0
root@edgeline01:~#
root@edgeline01:~#
root@edgeline01:~# lvremove /dev/pve/data
Do you really want to remove active logical volume pve/data? [y/n]: y
Logical volume "data" successfully removed.
root@edgeline01:~#
root@edgeline01:~#
root@edgeline01:~# lvresize -l +100%FREE /dev/pve/root
Size of logical volume pve/root changed from <69.75 GiB (17855 extents) to <231.00 GiB (59135 extents).
Logical volume pve/root successfully resized.
root@edgeline01:~#
root@edgeline01:~#
root@edgeline01:~#
root@edgeline01:~#
root@edgeline01:~# resize2fs /dev/mapper/pve-root
resize2fs 1.46.5 (30-Dec-2021)
Filesystem at /dev/mapper/pve-root is mounted on /; on-line resizing required
old_desc_blocks = 5, new_desc_blocks = 13
The filesystem on /dev/mapper/pve-root is now 26944512 (4k) blocks long.
root@edgeline01:~#

• This increase will display in the GUI per the below:

Note: Repeat the below steps on both hyperconverged nodes.

• This guide utilizes the public/unsubscribed repo to get setup.  It is recommended if utilizing these technologies in production that Proxmox Support and the Enterprise repo is utilized.
• Change the repository from the Enterprise repo per the below screenshot to Public/unsubscribed.

• The below message will be displayed:

• Add the No-Subscription Repository

• If the open-source repo isn’t added the upgrade will fail similar to the below:

• Disable BOTH the enterprise repo's for hypervisor and Ceph, for this build however in production Proxmox Support and Enterprise repo usage is strongly recommended.

• Go to Updates and refresh - this will perform an apt update - e.g. see what updates the repo has available

• If prompted for console encoding typically select UTF-8 (the default)

• Again the below message will be displayed:

• The below will display showing the apt update

• Go to Updates and Upgrade - this will perform an apt upgrade - e.g. pull and apply the available updates
• A list of available updates will be displayed

• A console box will display per the below to proceed with the upgrade

Note: Repeat the below steps on both hyperconverged nodes.

Note: The VSA OVA is downloadable via the standard https:support.stormagic.com portal

• Select the Local storage volume, Import and select Upload

• The upload dialog box will then appear

• Browse to the SvSAN VSA OVA, available for download via our support portal, https://support.stormagic.com

• This will upload

• Ensure the upload is successful.

Once the VSA OVA has been successfully uploaded, extract the OVA from the Proxmox CLI.

mkdir -p /root/stormagic-import
tar -xf /var/lib/vz/import/StorMagic_VSA_1.0-bios.ova -C /root/stormagic-import
cd /root/stormagic-import
ls -l

Note: The ls -l command can be used to confirm that the OVA contents, including the OVF and associated virtual disk file(s), have been successfully extracted.

The extracted OVF will be imported in the next section when creating the StorMagic VSA virtual machine.

• ISOs can also be uploaded via WinSCP or similar to the below location:

/var/lib/vz/template/iso/

• WinSCP ISO Upload

• The Proxmox q-device acts as a lightweight witness service for the Proxmox compute cluster.
• SvSAN also leverages a different witness service for the StorMagic storage cluster. 
• In this example we're running both of these on the same machine, being a Raspberry Pi4, however StorMagic also has use cases running both in an LXC container on a QNAP as a potential alternative.

Note: The qdevice root password must be open over SSH and match the Proxmox root password

• Change the root password with the below:

sudo passwd root

• and enable root ssh login via:

vi /etc/ssh/sshd_config

• un hashing/commenting “permit root login” and change to “yes”

root@raspberrypi:~# sudo nano /etc/ssh/sshd_config 
root@raspberrypi:~# passwd root 
New password: 
Retype new password: 
passwd: password updated successfully 
root@raspberrypi:~# sudo systemctl restart ssh 
root@raspberrypi:~#

Or utilizing the below:

sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
sudo systemctl restart ssh

sudo apt install corosync-qnetd

e.g.

root@raspberrypi:~# sudo apt install corosync-qnetd 
Reading package lists... Done 
Building dependency tree... Done 
Reading state information... Done 
The following package was automatically installed and is no longer required:     libfuse2 
Use 'sudo apt autoremove' to remove it. 
The following additional packages will be installed:     libnss3-tools 
The following NEW packages will be installed:  corosync-qnetd libnss3-tools 
0 upgraded, 2 newly installed, 0 to remove and 76 not upgraded. 
Need to get 940 kB of archives. After this operation, 4,280 kB of additional disk space will be used. 
Do you want to continue? [Y/n] y 
Get:1 http://raspbian.raspberrypi.org/raspbian bullseye/main armhf libnss3-tools armhf 2:3.61-1+deb11u3 [882 kB] 
Get:2 http://raspbian.mirror.uk.sargasso.net/raspbian bullseye/main armhf corosy nc-qnetd armhf 3.0.1-1 [57.9 kB] 
Fetched 940 kB in 5s (197 kB/s) Selecting previously unselected package libnss3-tools.
(Reading database ... 106411 files and directories currently installed.) 
Preparing to unpack .../libnss3-tools_2%3a3.61-1+deb11u3_armhf.deb ... 
Unpacking libnss3-tools (2:3.61-1+deb11u3) ... 
Selecting previously unselected package corosync-qnetd. Preparing to unpack .../corosync-qnetd_3.0.1-1_armhf.deb ... 
Unpacking corosync-qnetd (3.0.1-1) ... 
Setting up libnss3-tools (2:3.61-1+deb11u3) ... 
Setting up corosync-qnetd (3.0.1-1) ... 
Creating /etc/corosync/qnetd/nssdb Creating new key and cert db password file contains no data 
Creating new noise file /etc/corosync/qnetd/nssdb/noise.txt 
Creating new CA   

Generating key. This may take a few moments...  
Is this a CA certificate [y/N]? 
Enter the path length constraint, enter to skip [<0 for="for" unlimited="unlimited" path]:="path]:">  Is this a critical extension [y/N]?   

Generating key. This may take a few moments...  

Notice: Trust flag u is set automatically if the private key is present. 
QNetd CA certificate is exported as /etc/corosync/qnetd/nssdb/qnetd-cacert.crt 
Created symlink /etc/systemd/system/multi-user.target.wants/corosync-qnetd.service → /lib/systemd/system/corosync-qnetd.service. 
/usr/sbin/policy-rc.d returned 101, not running 'start corosync-qnetd.service'
Processing triggers for systemd (245.4-4ubuntu3.20) ...
Processing triggers for man-db (2.9.4-2) ...  
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
root@raspberrypi:~# 

• Install the SvSAN Witness Service to the Ubuntu 24.04 based qdevice per the below KB:

https://support.stormagic.com/hc/en-gb/articles/12525906976669-Installing-the-StorMagic-Witness-used-for-SvSAN-6-7-or-SvHCI-2-3-to-Ubuntu-Linux-24-04-x64

• Ensure time is correct on both Proxmox nodes and the qdevice

root@raspberrypi:/opt/StorMagic/Witness/bin#sudo apt-get install ntp 
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo ufw allow 123/udp 
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo systemctl restart ntp 
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo systemctl status ntp 
root@raspberrypi:/opt/StorMagic/Witness/bin#ntpq -p
root@raspberrypi:/opt/StorMagic/Witness/bin#date

Tue 29 Oct 10:29:18 UTC 2025

Note: This must be done before VSA, or any VM, is create otherwise you’ll receive the below:

detected the following error(s): * this host already contains virtual guests TASK ERROR: Check if node may join a cluster failed!
detected the following error(s): 
* this host already contains virtual guests 
TASK ERROR: Check if node may join a cluster failed!

• The above error is related to the VM IDs. e.g. if you have 2x VMs or VSAs provisioned first, they’ll default to VM ID 100.
• To join both nodes to a cluster this would need to be correct, which Proxmox doesn’t handle.
• They still block even if different VM IDs.
• This can be worked around via the below:
• Backup and remove the VM conf files from /etc/pve/nodes/edgeline01/qemu-server
• Then add back in after cluster join.
• This is similar to the remove from inventory and register vm to inventory functionality within VMware.

Proxmox Join Cluster Failed - This host already contains virtual guests!

• Define a cluster name, and select the network to utilize, in our case a teamed vmbr0:

• Confirm the task completes OK:

• Select Join Information and copy the info to node2:

• And once completed they'll display as per the below, noting that the pages may need a refresh due to certificate changes:

Cluster Manager - Proxmox VE - Corosync External Vote Support section

• Install the corosync-qdevice package on all nodes e.g. edgeline01 and edgeline02

apt install corosync-qdevice

Note: All nodes in the cluster will need to be configured, online and utilizing the same root password to cluster correctly

root@edgeline01:~# pvecm qdevice setup 10.10.130.220
/bin/ssh-copy-id: INFO: Source of key(s) to be install                                                         ed: "/root/.ssh/id_rsa.pub"
/bin/ssh-copy-id: INFO: attempting to log in with the                                                          new key(s), to filter out any that are already install                                                         ed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installe                                                         d -- if you are prompted now it is to install the new                                                          keys
root@10.10.130.220's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh -i /root/.ssh/id_rsa 'root@10.10.130.220'"
and check to make sure that only the key(s) you wanted were added.


INFO: initializing qnetd server
Certificate database (/etc/corosync/qnetd/nssdb) already exists. Delete it to initialize new db

INFO: copying CA cert and initializing on all nodes

node 'edgeline01': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'edgeline01': Creating new key and cert db
node 'edgeline01': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'edgeline01': Importing CA
node 'edgeline02': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'edgeline02': Creating new key and cert db
node 'edgeline02': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'edgeline02': Importing CA
INFO: generating cert request
Creating new certificate request


Generating key.  This may take a few moments...

Certificate request stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.crq

INFO: copying exported cert request to qnetd server

INFO: sign and export cluster cert
Signing cluster certificate
Certificate stored in /etc/corosync/qnetd/nssdb/cluster-proxmox-cluster.crt

INFO: copy exported CRT

INFO: import certificate
Importing signed cluster certificate
Notice: Trust flag u is set automatically if the private key is present.
pk12util: PKCS12 EXPORT SUCCESSFUL
Certificate stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.p12

INFO: copy and import pk12 cert to all nodes

node 'edgeline01': Importing cluster certificate and key
node 'edgeline01': pk12util: PKCS12 IMPORT SUCCESSFUL
node 'edgeline02': Importing cluster certificate and key
node 'edgeline02': pk12util: PKCS12 IMPORT SUCCESSFUL
INFO: add QDevice to cluster configuration

INFO: start and enable corosync qdevice daemon on node 'edgeline01'...
Synchronizing state of corosync-qdevice.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable corosync-qdevice
Created symlink '/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service' - '/usr/lib/systemd/system/corosync-qdevice.service'.

INFO: start and enable corosync qdevice daemon on node 'edgeline02'...
Synchronizing state of corosync-qdevice.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable corosync-qdevice
Created symlink '/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service' - '/usr/lib/systemd/system/corosync-qdevice.service'.
Reloading corosync.conf...
Done
root@edgeline01:~#

• If you see an error:

command 'ssh -o 'BatchMode=yes' -lroot 10.10.130.220 corosync-qdevice-net-certutil
  -m -c /etc/pve/qdevice-net-node.p12' failed: exit code 255

• ssh between the nodes first as root 'ssh root@10.10.130.220' accepting the cert, then re-attempt the qdevice add operation
• Check the status of the cluster

pvecm status

• Example output below

root@edgeline01:~# pvecm status
Cluster information
-------------------
Name:             proxmox-cluster
Config Version:   3
Transport:        knet
Secure auth:      on

Quorum information
------------------
Date:             Tue Nov  4 12:32:57 2025
Quorum provider:  corosync_votequorum
Nodes:            2
Node ID:          0x00000001
Ring ID:          1.9
Quorate:          Yes

Votequorum information
----------------------
Expected votes:   3
Highest expected: 3
Total votes:      3
Quorum:           2
Flags:            Quorate Qdevice

Membership information
----------------------
    Nodeid      Votes    Qdevice Name
0x00000001          1    A,V,NMW 10.10.194.3 (local)
0x00000002          1    A,V,NMW 10.10.194.4
0x00000000          1            Qdevice
root@edgeline01:~#

• It is also possible to see the q-device IP via the corosync config

root@edgeline01:~# cat /etc/pve/corosync.conf
logging {
  debug: off
  to_syslog: yes
}

nodelist {
  node {
    name: edgeline01
    nodeid: 1
    quorum_votes: 1
    ring0_addr: 10.10.194.3
  }
  node {
    name: edgeline02
    nodeid: 2
    quorum_votes: 1
    ring0_addr: 10.10.194.4
  }
}

quorum {
  device {
    model: net
    net {
      algorithm: ffsplit
      host: 10.10.130.220
      tls: on
    }
    votes: 1
  }
  provider: corosync_votequorum
}

totem {
  cluster_name: proxmox-cluster
  config_version: 3
  interface {
    linknumber: 0
  }
  ip_version: ipv4-6
  link_mode: passive
  secauth: on
  version: 2
}

root@edgeline01:~#

Note: Repeat the below steps on both hyperconverged nodes.

Note: Ensure to create VSA2 with a different VM ID - per the previous mention of unable to join when there’s VMs

• With the local storage volume selected, select import, select the previously uploaded VSA OVA and select Import

• Give the VSA VM a name, changing from the default StorMagic-SvSAN and confirm it has 1vCPU and 2GB RAM (up to 32GB if utilizing SSD/Memory caching)
  https://support.stormagic.com/hc/en-gb/articles/5887719016989-SvHCI-SvSAN-Caching

• Ensure the imports complete successfully:

• On both VSAS, with the VSA VM selected, select Options and enable start at boot:

Note: Repeat the below steps on both hyperconverged nodes/VSAs.

• Modify net1 to attach to a storage NIC 

• Modify both net0 and net1 from the default vmxnet3 to virtio NIC drivers.
• Note that the SvSAN VSA includes drivers for vmxnet3, virtio and E1000E.
• Technical Services have seen different network throughputs on different systems and advise validating network performance per the below KB:

https://support.stormagic.com/hc/en-gb/articles/8116960680861-How-to-test-and-validate-network-performance-in-an-SvHCI-or-SvSAN-Environment

• add additional logical vNICs to the previously created bridges, if required

• Ensuring to attach the NIC to the correct bridge/network

• On both VSAs, with the VSA VM selected, select Hardware, and add a Hard Disk (the VSA Journal Disk)

• Create the disk as 20GB, to be the Journal for change block tracking and logs.

• Modify the SCSI Controller from the Default "LSI 53C895A" over to VirtIO SCSI

• Modify the VSA storage controller to virtIO

• It is also possible to add a disk image, equivalent to a VMDK in VMware, as SAN storage that we would add here as scsi3.
• However in this example we'll utilize an RDM equivalent symbolic link to the physical disk, to utilize as SAN storage.

Note: Repeat the below steps on both hyperconverged nodes.

• The below equivalent to vmkfstools in VMware, documented below:

https://stormagic.com/doc/svsan/6-7/en/Content/allocating-additional-storage-vs.htm

• It is not possible to add an RDM link disk via Proxmox GUI, like you can in VMware or Hyper-V.
• The below third party video details disk passthru also.

https://www.youtube.com/watch?v=U-UTMuhmC1U

• Most SvSAN customers will typically have either
• Some form of boot storage and some form of SAN storage. e.g. RAID1 boot (such as BOSS/NS204 etc) and separate SAN (RAID10, RAID5, RAID50 etc)SvSAN supports both hardware RAID and can also support software RAID over individual disks:

https://support.stormagic.com/hc/en-gb/articles/17597617525917-SvHCI-SvSAN-and-Software-RAID

• Or split virtual disks

https://support.stormagic.com/hc/en-gb/articles/5665208916381-SvSAN-Leveraging-Split-Virtual-Disks-Logical-Volumes-with-HPE-Servers

https://support.stormagic.com/hc/en-gb/articles/9128058771741-SvSAN-Leveraging-Split-Virtual-Disks-Logical-Volumes-with-Dell-Servers

https://support.stormagic.com/hc/en-gb/articles/5896861868317-SvSAN-Leveraging-Split-Virtual-Disks-Logical-Volumes-with-Lenovo-Servers

• In both cases above the host will see the boot as /dev/sda and the storage to be used for the SAN as /dev/sdb
• If there are other disks the letters will increment through the alphabet being /dev/sdc etc
• NVMe based storage, may appear differently. Just be careful here and ensure nothing else is using the disks.

Commands used:

ls -n /dev/disk/by-id/

/sbin/qm set [VM-ID] -virtio2 /dev/disk/by-id/[DISK-ID]

root@proxmox-1:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K24291DCC1U-part3 - ../../sda3
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-name-pve-root - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:27 dm-name-pve-swap - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-uuid-LVM-rk53mp1TcHumnHnAxU6G12FI7KjFvXMK4Rbm0N5hFIyDERfHbfFdxEcmxTz6OycW - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:27 dm-uuid-LVM-rk53mp1TcHumnHnAxU6G12FI7KjFvXMKhnwdeMkQJemP4yiqT2lZtLmNLrGJpSFm - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 lvm-pv-uuid-VWXWt8-8cgk-hxo9-FOcX-c1yz-U8KL-zC43FB - ../../sda3
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-eui.343646304e6108110025385900000001 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-eui.343646304e6108360025385900000001 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610811 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:27 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610836 - ../../nvme1n1
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 wwn-0x5707c18100925c1a - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925c1a-part3 - ../../sda3
root@proxmox-1:~#
root@proxmox-1:~# /sbin/qm set 100 -virtio2 /dev/disk/by-id/nvme-eui.343646304e6108110025385900000001
update VM 100: -virtio2 /dev/disk/by-id/nvme-eui.343646304e6108110025385900000001
root@proxmox-1:~# /sbin/qm set 100 -virtio3 /dev/disk/by-id/nvme-eui.343646304e6108360025385900000001
update VM 100: -virtio3 /dev/disk/by-id/nvme-eui.343646304e6108360025385900000001
root@proxmox-1:~#


root@proxmox-2:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 ata-IM2S33D4_2K242L1DE2TY-part3 - ../../sda3
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-name-pve-root - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:24 dm-name-pve-swap - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:40 dm-uuid-LVM-ng5R1ctgy041DlLIagXtSqn6454eaenqF4sEOW9d2RM315ejVzxpOUlu9ayhn3YK - ../../dm-1
lrwxrwxrwx 1 0 0 10 Sep 28 18:24 dm-uuid-LVM-ng5R1ctgy041DlLIagXtSqn6454eaenquJLJNeWfSD4Hg5Q9faLlAv9PWZielnX8 - ../../dm-0
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 lvm-pv-uuid-WGMruL-XZfV-7zPW-3BVp-Qzqg-Twrd-PS8qic - ../../sda3
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-eui.343646304e6102800025385900000001 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-eui.343646304e6107270025385900000001 - ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610280 - ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Sep 28 18:24 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610727 - ../../nvme0n1
lrwxrwxrwx 1 0 0  9 Oct 10 09:45 wwn-0x5707c18100925646 - ../../sda
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part1 - ../../sda1
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part2 - ../../sda2
lrwxrwxrwx 1 0 0 10 Oct 10 09:45 wwn-0x5707c18100925646-part3 - ../../sda3
root@proxmox-2:~# /sbin/qm set 101 -virtio2 /dev/disk/by-id/nvme-eui.343646304e6102800025385900000001
update VM 101: -virtio2 /dev/disk/by-id/nvme-eui.343646304e6102800025385900000001
root@proxmox-2:~# /sbin/qm set 101 -virtio3 /dev/disk/by-id/nvme-eui.343646304e6107270025385900000001
update VM 101: -virtio3 /dev/disk/by-id/nvme-eui.343646304e6107270025385900000001
root@proxmox-2:~#

lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb

lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb

root@proxmox1:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 ata-VMware_Virtual_IDE_CDROM_Drive_00000000000000000001 -> ../../sr0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-name-pve-root -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-name-pve-swap -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-uuid-LVM-397ePpY9Yy0rnlBAaao02wNyCEIqmouhKzBkyr2czUFry9Raklzym2p2zynHSDnB -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 dm-uuid-LVM-397ePpY9Yy0rnlBAaao02wNyCEIqmouhnvouNGymAO9TNqIkjunt0nK3NogmNjfr -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 lvm-pv-uuid-mEFIaM-ZrA9-Ulan-3XhS-i1DL-hbFZ-9iS3dp -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-36000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-36000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 scsi-SVMware_Virtual_disk_6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11 -> ../../sda
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Mar 24 18:14 wwn-0x6000c296a1fedf604ddc9bfff9978a11-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 9 Mar 24 18:14 wwn-0x6000c29fd24a75bc12f86ebaa096dbb6 -> ../../sdb


root@proxmox1:~# /sbin/qm set 100 -virtio2 /dev/disk/by-id/scsi-36000c29fd24a75bc12f86ebaa096dbb6
update VM 100: -virtio2 /dev/disk/by-id/scsi-36000c29fd24a75bc12f86ebaa096dbb6
root@proxmox1:~#

• The VSA will boot up like an blank deployed OVA in VMware with DHCP on the NICs.

• Per the below Console walkthrough KB:

https://support.stormagic.com/hc/en-gb/articles/18860298535709-SvHCI-SvSAN-Console-usage

• Login to the VSA using the default password of ‘password’ and configure the IP addresses statically.
• Management:

• and Storage interface:

• It is also advised to set DNS, NTP and a network hostname such that the VSA will license correctly when in the first boot wizard, being able to route out to and resolve https://licensing.stormagic.com.
• Offline license files may be used for closed/air gapped environments per the below KB:

https://support.stormagic.com/hc/en-gb/articles/6397884188829-SvHCI-SvSAN-Online-and-Offline-Licensing

• Complete the standard default boot wizard via the WebGUI
• Accept the EULA:

• Proceed through the wizard:

• License the VSA VM

https://support.stormagic.com/hc/en-gb/articles/5355376684445-SvHCI-SvSAN-Licensing-model

https://support.stormagic.com/hc/en-gb/articles/6397884188829-SvHCI-SvSAN-Online-and-Offline-Licensing

• Once licensed certain features will display TRUE or if using an evaluation key display FALSE but have a evaluation time remaining:

• Specify hostname and domain name, if not previously set via the console:

• Set a password

• Complete the wizard:

• Once logged in with your own password explore the node WebGUI such as Devices:

• And create a pool ready to proceed to Storage Provisioning:

Note: Repeat the below steps on both hyperconverged nodes.

• There have been issues, working nested, in the past with IQNs being the same.
• Confirm they’re different, however maybe you want to rename to something more friendly.

nano /etc/iscsi/initiatorname.iscsi

• In our systems we have the below:
• host1

root@edgeline01:~# cat /etc/iscsi/initiatorname.iscsi 
## DO NOT EDIT OR REMOVE THIS FILE! 
## If you remove this file, the iSCSI daemon will not start. 
## If you change the InitiatorName, existing access control lists 
## may reject this initiator. The InitiatorName must be unique 
## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames. 
InitiatorName=iqn.1993-08.org.debian:01:48aa658c4691

• host2

root@edgeline02:~# cat /etc/iscsi/initiatorname.iscsi 
## DO NOT EDIT OR REMOVE THIS FILE! 
## If you remove this file, the iSCSI daemon will not start. 
## If you change the InitiatorName, existing access control lists 
## may reject this initiator. The InitiatorName must be unique 
## for each iSCSI initiator. Do NOT duplicate iSCSI InitiatorNames. 
InitiatorName=iqn.1993-08.org.debian:01:bea18b518bc6

• Should they be edited restart the iSCSI daemon via systemctl

root@edgeline01:~# systemctl restart iscsid.service

• These IQNs can be manually added to the VSA via the WebGUI or will be picked up automatically later with a rescan and then need to be added to the target ACL.

https://stormagic.com/doc/svsan/6-7/en/Content/datastore-create-manually.htm

Note: Repeat the below steps on both hyperconverged nodes.

https://pve.proxmox.com/wiki/ISCSI_Multipath

apt-get install multipath-tools

• For SvSAN with Linux initators and multipath configure the below:

https://support.stormagic.com/hc/en-gb/articles/6435122249501-SvSAN-Linux-iSCSI-Initiators

• Create the below folder path:

mkdir /etc/multipath/conf.d/

• Create the file per the below (on both hosts):

nano /etc/multipath/conf.d/StorMagc.conf

• To contain

defaults {
    polling_interval     2
    max_polling_interval 4
    find_multipaths      "yes"
}

blacklist {
    devnode "^(ram|raw|loop|fd|md|sr|scd|st|sda)[0-9]*"
}

devices {
    device {
        user_friendly_names  "yes"
        no_path_retry        5
        detect_checker       "no"
        path_checker         "tur"
        path_grouping_policy "group_by_prio"
        detect_prio          "no"
        prio                 "alua"
        prio_args            "exclusive_pref_bit"
        hardware_handler     "1 alua"
        vendor               "StorMagc"
        product              "iSCSI Volume"
    }
}

• and restart multipath

systemctl restart multipathd.service
multipath -r

Note: Repeat the below steps on both hyperconverged nodes.

• iscsiadm is a commandline tool similar to the iSCSI Control panel (iscsicpl) in Hyper-V and is documented : - https://linux.die.net/man/8/iscsiadm
• The below section will use the below flags to log the host initiators into the Target.

-m or --mode
-T or --target
-p or --portal
-l or --login
-n node.startup -v automatic

• Via the commandline on each Proxmox host run the below to discover (search out on the network) to the SvSAN VSA iSCSI IP addresses (not the Proxmox hosts IP addresses). 
• This is the equivalent to adding the IPs to iscsicpl (iSCSI Control Panel) in Microsoft Hyper-V.
• e.g. go out and see what targets the host is allowed to see or not.
• In the above example the target has a blank Access Control List (ACL).  The method used auto populates the IQNs to the SvSAN VSAs, to save copy and paste.

• The below command will return iscsiadm: No portals found as expected behaviour. This is due to the Proxmox node IQN not yet being present on the SvSAN VSAs or not yet being in the Target ACL.

root@edgeline01:~# iscsiadm -m discovery -t st -p 192.168.1.5
iscsiadm: No portals found
root@edgeline01:~# iscsiadm -m discovery -t st -p 192.168.1.6
iscsiadm: No portals found

root@edgeline02:~# iscsiadm -m discovery -t st -p 192.168.1.5
iscsiadm: No portals found
root@edgeline02:~# iscsiadm -m discovery -t st -p 192.168.1.6
iscsiadm: No portals found

• Having run the above browse to the VSA WebGUI, and select the Initiators page, noting that the IQNs will now be displayed having triggered the Proxmox nodes to scan out with the above command

• Next select the Targets page and select the target.
• Add the initiators, by selecting Initiators on the Target page...

• Add both Proxmox node IQNs to the target Access Control list (ACL).

• Run the command again and, all being correct, the available targets the Proxmox node initiator may login to will now be displayed, 
• host1

root@edgeline01:~# iscsiadm -m discovery -t st -p 192.168.1.5
192.168.1.5:3260,1 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
192.168.1.6:3260,3 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
root@edgeline01:~# iscsiadm -m discovery -t st -p 192.168.1.6
192.168.1.6:3260,3 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
192.168.1.5:3260,1 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
root@edgeline01:~#

• host2

root@edgeline02:~# iscsiadm -m discovery -t st -p 192.168.1.5
192.168.1.5:3260,1 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
192.168.1.6:3260,3 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
root@edgeline02:~# iscsiadm -m discovery -t st -p 192.168.1.6
192.168.1.6:3260,3 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
192.168.1.5:3260,1 iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror
root@edgeline02:~#

Login session to the hosts

• Now log each host into the SvSAN presented iSCSI target, ensuring each Proxmox node has the correct number of expected sessions to each SvSAN VSA.
• host1

root@edgeline01:~# iscsiadm -m node -T iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror -p 192.168.1.5 -l -n node.startup -v automatic
Login to [iface: default, target: iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror, portal: 192.168.1.5,3260] successful.

root@edgeline01:~# iscsiadm -m node -T iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror -p 192.168.1.6 -l -n node.startup -v automatic
Login to [iface: default, target: iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror, portal: 192.168.1.6,3260] successful.
root@edgeline01:~#

• host2

root@edgeline02:~# iscsiadm -m node -T iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror -p 192.168.1.5 -l -n node.startup -v automatic
Login to [iface: default, target: iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror, portal: 192.168.1.5,3260] successful.

root@edgeline02:~# iscsiadm -m node -T iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror -p 192.168.1.6 -l -n node.startup -v automatic
Login to [iface: default, target: iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror, portal: 192.168.1.6,3260] successful.
root@edgeline02:~#

• Check the expected number of sessions are correct in the SvSAN VSA webGUI

• The below example shows how to remove/logout a session, for troubleshooting purposes.

root@edgeline02:~# iscsiadm -m node -T iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror -p 192.168.1.5 --logout
root@edgeline02:~# iscsiadm -m node -T iqn.2006-06.com.stormagic:8661170200000018.m0svsanmirror -p 192.168.1.5 --op=delete

• host1

root@edgeline01:~# multipath -ll
mpatha (20003398661170001) dm-2 StorMagc,iSCSI Volume
size=3.5T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 6:0:0:0 sdb 8:16 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  `- 7:0:0:0 sdc 8:32 active ready running

• host2

root@edgeline02:~# multipath -ll
mpatha (20003398661170001) dm-2 StorMagc,iSCSI Volume
size=3.5T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 6:0:0:0 sdb 8:16 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  `- 7:0:0:0 sdc 8:32 active ready running

• host1

root@edgeline01:~# cat /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/20003398661170001/

• host2

root@edgeline02:~# cat /etc/multipath/wwids
# Multipath wwids, Version : 1.0
# NOTE: This file is automatically maintained by multipath and multipathd.
# You should not need to edit this file in normal circumstances.
#
# Valid WWIDs:
/20003398661170001/

• When listing the disks with the following command ls -n /dev/disk/by-id/ we now see an mpath disk.
• In this example dm-name-mpatha as dm-2 that is the map of dm-uuid-mpath-20003398661170001 -> ../../dm-2, the same WWID in the WWIDs file.
• Host 1

root@edgeline01:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0  9 Jan  2 09:20 ata-SSD_512GB_AA20231025512G651778 -> ../../sda
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 ata-SSD_512GB_AA20231025512G651778-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 ata-SSD_512GB_AA20231025512G651778-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 ata-SSD_512GB_AA20231025512G651778-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 dm-name-mpatha -> ../../dm-2
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 dm-name-pve-root -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 dm-name-pve-swap -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 dm-uuid-LVM-H2J26vqGLDfuYNblvXYnYJNrbpX0mirTMCRZfEzBZnfhkfxcz7yhwnWvv6vfBiFu -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 dm-uuid-LVM-H2J26vqGLDfuYNblvXYnYJNrbpX0mirTQ07jpOAHTty4U9Kv3TQa10EHtBy8V31y -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 dm-uuid-mpath-20003398661170001 -> ../../dm-2
lrwxrwxrwx 1 0 0 10 Jan  2 09:20 lvm-pv-uuid-KsbYC9-tEH1-11TE-yB3r-kyTQ-us1d-dGjFgc -> ../../sda3
lrwxrwxrwx 1 0 0 13 Jan  2 09:20 nvme-eui.343646304e6108110025385900000001 -> ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:20 nvme-eui.343646304e6108360025385900000001 -> ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:20 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610811 -> ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:20 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610811_1 -> ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:20 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610836 -> ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:20 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610836_1 -> ../../nvme1n1
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 scsi-20003398661170001 -> ../../dm-2
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 wwn-0x0003398661170001 -> ../../dm-2
root@edgeline01:~#

Host 2

root@edgeline02:~# ls -n /dev/disk/by-id/
total 0
lrwxrwxrwx 1 0 0  9 Jan  2 09:18 ata-SSD_512GB_AA20231025512G651754 -> ../../sda
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 ata-SSD_512GB_AA20231025512G651754-part1 -> ../../sda1
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 ata-SSD_512GB_AA20231025512G651754-part2 -> ../../sda2
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 ata-SSD_512GB_AA20231025512G651754-part3 -> ../../sda3
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 dm-name-mpatha -> ../../dm-2
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 dm-name-pve-root -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 dm-name-pve-swap -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 dm-uuid-LVM-p3CX3QeVLAcxSjIwKVYndFvzijwzxrsfdLR25zUqtPksIaMM7eMDZEyTc6BpLAK1 -> ../../dm-1
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 dm-uuid-LVM-p3CX3QeVLAcxSjIwKVYndFvzijwzxrsfX82XHSZigh79E3IdY3FYpxqWag8k8S4x -> ../../dm-0
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 dm-uuid-mpath-20003398661170001 -> ../../dm-2
lrwxrwxrwx 1 0 0 10 Jan  2 09:18 lvm-pv-uuid-xiRydb-7Osr-SdPb-7v6c-32SD-kyz2-lizJoh -> ../../sda3
lrwxrwxrwx 1 0 0 13 Jan  2 09:18 nvme-eui.343646304e6102800025385900000001 -> ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:18 nvme-eui.343646304e6107270025385900000001 -> ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:18 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610280 -> ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:18 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610280_1 -> ../../nvme1n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:18 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610727 -> ../../nvme0n1
lrwxrwxrwx 1 0 0 13 Jan  2 09:18 nvme-SAMSUNG_MZ1LB3T8HMLA-00007_S46FNY0N610727_1 -> ../../nvme0n1
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 scsi-20003398661170001 -> ../../dm-2
lrwxrwxrwx 1 0 0 10 Jan  2 09:22 wwn-0x0003398661170001 -> ../../dm-2

It is also possible, however not necessary to create a disk alias for the disk per the below settings in the multipath.conf on each node: 

root@edgeline01:~# cat /etc/multipath/conf.d/StorMagc.conf
defaults {
    polling_interval     2
    max_polling_interval 4
    find_multipaths      "yes"
}

blacklist {
    devnode "^(ram|raw|loop|fd|md|sr|scd|st|sda)[0-9]*"
}

devices {
    device {
        user_friendly_names  "yes"
        no_path_retry        5
        detect_checker       "no"
        path_checker         "tur"
        path_grouping_policy "group_by_prio"
        detect_prio          "no"
        prio                 "alua"
        prio_args            "exclusive_pref_bit"
        hardware_handler     "1 alua"
        vendor               "StorMagc"
        product              "iSCSI Volume"
    }
}

multipaths {
  multipath {
        wwid "20003398661170001"
        alias mpatha
  }
}

• From one host, noting the disk alias to select, in this instance being mpatha:

root@edgeline01:~# pvcreate /dev/mapper/mpatha
  Physical volume "/dev/mapper/mpatha" successfully created.

root@edgeline01:~# vgcreate vg-svsan-storage /dev/mapper/mpatha
  Volume group "vg-svsan-storage" successfully created

• Further SvSAN disks will be mpathb mpathc etc

• Connect to a node in the cluster, select the Datacenter object, select Storage, select Add and select LVM to add the storage to the nodes and make it usable.

• Under ID: specify a name for the datastore, in this example "svsan-storage", set the base storage as the previous created volume group "vg-svsan-storage", and ensure to select "Shared".
• Should shared not be selected when a guest VM is migrated between hosts, the data will also be copied rather than purely the Virtual Machine memory.

• Via the same process to create the VSA VM, upload the Guest OS install ISO, via the GUI or SCP/WinSCP to the below path: /var/lib/vz/template/iso/
• The below copies the install ISO from node1 (10.10.194.3) to node2 (10.10.94.4)

scp /var/lib/vz/template/iso/ubuntu-24.04.3-desktop-amd64.iso root@10.10.194.4:/var/lib/vz/template/iso/ubuntu-24.04.3-desktop-amd64.iso

Note: the storage on the below showing as Local is the storage of the ISO not the VM

• Set the install ISO and guest OS type

• Specify the graphics and storage controller, noting that different operating systems include different virtual hardware drivers and performance may vary depending on what is utilized:

• Select the storage for the boot disk to sit on, e.g the shared storage

• Select the disk size, in this example 32GiB

• Specify the CPUs and cores

• Specify the memory, in this example 4096MiB:

• Specify the network bridge for the VM to attach to and the virtual NIC hardware as VirtIO or VMXnet3, noting that different operating systems include different virtual hardware drivers and performance may vary depending on what is utilized

• Review the Summary and Finish to create the VM

• Install and set a static IP:

• Via the same process to create the VSA VM, upload the Guest OS install ISO, via the GUI or SCP/WinSCP to the below path: /var/lib/vz/template/iso/
• The below copies the install ISO from node1 (10.10.194.3) to node2 (10.10.94.4)

scp /var/lib/vz/template/iso/en-us_windows_server_2022_x64_dvd_620d7eac.iso root@10.10.194.4:/var/lib/vz/template/iso/en-us_windows_server_2022_x64_dvd_620d7eac.iso

• Installing a Windows VM on KVM requires a few extra steps which aren’t required for Linux VMs. This is a quick overview of those extra steps.

• Two ISOs, copied to both hosts

• A Windows installation ISO

• The VirtIO Drivers ISO downloadable from the below URL:
  https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso

Creating the Windows VM:

• Create the VM as a standard VM in Proxmox, noting that different operating systems include different virtual hardware drivers and performance may vary depending on what is utilized.

• Attach the Windows installation ISO and the Virtio ISO to the VM you are creating.

• The VM will boot into the Windows installer, due to Windows being the first attached ISO.
• Proceed to the “Which type of installation do you want” screen selecting Custom

• No disks will be visible, with a warning "We couldn't find any drives. To get a storage driver, click Load driver."

• If the necessary driver is not shown select Rescan.

• Select the driver for the version of Windows selecting Next to install the driver.

• The drive should now display as a selectable location to install to.  

• If the drive does not appear then select refresh, select the drive to attach.

• Select load driver again.

• Select Browse, browsing to the NetKVM folder,

• Select the OS, the AMD64 folder and select OK

• Select next.

• This will bring you back to the empty drive screen once more.

• Select Load Driver.

• Select the driver for the Windows version needed.

• Detail here the needed ISO driver and how to get/use

https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso

• Select the guests to protect with HA

• Failover time ~2min 30s

• This next step/setup ensures that iSCSI storage provided by SvSAN virtual appliances is reconnected at boot and that all required Proxmox HA resources are properly configured to start automatically, in the event of a cluster down scenario. 

Note: The following configuration needs to be done on every pve cluster member.

• The below script will check the path count and login to paths if not the expected number.

Modify the $count value to your expected number of paths.

nano /root/iscsi_login.sh

#!/bin/bash
#check Multipath
#set -x
/usr/sbin/iscsiadm -m session
ret=$?
if [ $ret -ne 0 ]; then
/usr/sbin/iscsiadm -m node -l
else
for i in `/usr/sbin/multipath -ll | grep StorMagc | awk '{print $1}'`
do
count=`/usr/sbin/multipath -ll $i | grep sd | wc -l`
        echo $count
#modify count to expected number of sessions e.g. 2 or 4 usually
        if [ $count = 2 ]
        then echo "ALL OK"
        else
        echo "rescan iSCSI login targets"
        /usr/sbin/iscsiadm -m node -l
        fi
done
fi

• Next chmod the script file to enable the execute bit

root@edgeline01:~# chmod +x /root/iscsi_login.sh

• Set the script to run via crontab using the -e for edit switch
• crontab -e
• If this is the first time running you will be prompted for an editor - choose option1

root@edgeline01:~# crontab -e
no crontab for root - using an empty one
Select an editor.  To change later, run select-editor again.
  1. /bin/nano        <---- easiest
  2. /usr/bin/vim.tiny

• Add the below lines to the crontab

*/5 * * * * /root/iscsi_login.sh > /dev/null 2>&1
@reboot /root/iscsi_login.sh

• Such that it looks like:

# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').
#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h  dom mon dow   command
*/5 * * * * /root/iscsi_login.sh > /dev/null 2>&1
@reboot /root/iscsi_login.sh