This article is intended for administrators wishing to create a 2x node + witness High Availability cluster utilizing Proxmox Virtual Environment (https://www.proxmox.com/en/proxmox-virtual-environment/overview) and StorMagic SvSAN (https://stormagic.com/svsan/).
Note: All images are clickable for enlarging, or can be opened in a new tab
Information
- Guide Summary/shortcuts
- Provision a qdevice/SvSAN witness
- Enable root SSH on the qdevice/SvSAN witness machine
- Install the qdevice
- Install the SvSAN Witness service
- Set the Time Configuration
- Create the Cluster
- Join the nodes
- Add the Corosync to the Proxmox nodes
- Add the qdevice to the cluster
Guide Summary/shortcuts
This multipart guide will walk through the process to deploy 2x hyperconverged Proxmox VE nodes utilizing SvSAN virtualized block storage and a lightweight witness node, such as a Raspberry Pi.
Provision a qdevice/SvSAN witness
The Proxmox q-device acts as a lightweight witness service for the Proxmox compute cluster.
SvSAN also leverages a different witness service for the StorMagic storage cluster.
In this example we're running both of these on the same machine, being a Raspberry Pi4, however StorMagic also has use cases running both in an LXC container on a QNAP as a potential alternative.
Enable root SSH on the qdevice/SvSAN witness machine
Note: The qdevice root password must be open over SSH and match the Proxmox root password
The below example utilizes and validated on Ubuntu Server 24.04 or Raspbian Buster. Ubuntu Desktop and other versions may differ slightly.
Change the root password with the below:
sudo passwd rootand enable root ssh login via:
vi /etc/ssh/sshd_configun hashing/commenting “permit root login” and change to “yes”
root@raspberrypi:~# sudo nano /etc/ssh/sshd_config
root@raspberrypi:~# passwd root
New password:
Retype new password:
passwd: password updated successfully
root@raspberrypi:~# sudo systemctl restart ssh
root@raspberrypi:~#Or utilizing the below:
sudo sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
sudo systemctl restart sshInstall the qdevice
sudo apt install corosync-qnetde.g.
root@raspberrypi:~# sudo apt install corosync-qnetd
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required: libfuse2
Use 'sudo apt autoremove' to remove it.
The following additional packages will be installed: libnss3-tools
The following NEW packages will be installed: corosync-qnetd libnss3-tools
0 upgraded, 2 newly installed, 0 to remove and 76 not upgraded.
Need to get 940 kB of archives. After this operation, 4,280 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://raspbian.raspberrypi.org/raspbian bullseye/main armhf libnss3-tools armhf 2:3.61-1+deb11u3 [882 kB]
Get:2 http://raspbian.mirror.uk.sargasso.net/raspbian bullseye/main armhf corosy nc-qnetd armhf 3.0.1-1 [57.9 kB]
Fetched 940 kB in 5s (197 kB/s) Selecting previously unselected package libnss3-tools.
(Reading database ... 106411 files and directories currently installed.)
Preparing to unpack .../libnss3-tools_2%3a3.61-1+deb11u3_armhf.deb ...
Unpacking libnss3-tools (2:3.61-1+deb11u3) ...
Selecting previously unselected package corosync-qnetd. Preparing to unpack .../corosync-qnetd_3.0.1-1_armhf.deb ...
Unpacking corosync-qnetd (3.0.1-1) ...
Setting up libnss3-tools (2:3.61-1+deb11u3) ...
Setting up corosync-qnetd (3.0.1-1) ...
Creating /etc/corosync/qnetd/nssdb Creating new key and cert db password file contains no data
Creating new noise file /etc/corosync/qnetd/nssdb/noise.txt
Creating new CA
Generating key. This may take a few moments...
Is this a CA certificate [y/N]?
Enter the path length constraint, enter to skip [<0 for="for" unlimited="unlimited" path]:="path]:"> Is this a critical extension [y/N]?
Generating key. This may take a few moments...
Notice: Trust flag u is set automatically if the private key is present.
QNetd CA certificate is exported as /etc/corosync/qnetd/nssdb/qnetd-cacert.crt
Created symlink /etc/systemd/system/multi-user.target.wants/corosync-qnetd.service → /lib/systemd/system/corosync-qnetd.service.
/usr/sbin/policy-rc.d returned 101, not running 'start corosync-qnetd.service'
Processing triggers for systemd (245.4-4ubuntu3.20) ...
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for libc-bin (2.31-0ubuntu9.9) ...
root@raspberrypi:~# Install the SvSAN Witness service
Install the SvSAN Witness Service to the Ubuntu 24.04 based qdevice per the below KB:
Set the Time Configuration
Ensure time is correct on both Proxmox nodes and the qdevice
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo apt-get install ntp
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo ufw allow 123/udp
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo systemctl restart ntp
root@raspberrypi:/opt/StorMagic/Witness/bin#sudo systemctl status ntp
root@raspberrypi:/opt/StorMagic/Witness/bin#ntpq -p
root@raspberrypi:/opt/StorMagic/Witness/bin#date
Tue 29 Oct 10:29:18 UTC 2025Create the Cluster
Note: This must be done before VSA, or any VM, is create otherwise you’ll receive the below:
detected the following error(s): * this host already contains virtual guests TASK ERROR: Check if node may join a cluster failed!
detected the following error(s):
* this host already contains virtual guests
TASK ERROR: Check if node may join a cluster failed!The above error is related to the VM IDs. e.g. if you have 2x VMs or VSAs provisioned first, they’ll default to VM ID 100.
To join both nodes to a cluster this would need to be correct, which Proxmox doesn’t handle.
They still block even if different VM IDs.
This can be worked around via the below:
Backup and remove the VM conf files from /etc/pve/nodes/edgeline01/qemu-server
Then add back in after cluster join.
This is similar to the remove from inventory and register vm to inventory functionality within VMware.
Proxmox Join Cluster Failed - This host already contains virtual guests!
Join the nodes
Define a cluster name, and select the network to utilize, in our case a teamed vmbr0:
Confirm the task completes OK:
Select Join Information and copy the info to node2:
And once completed they'll display as per the below, noting that the pages may need a refresh due to certificate changes:
Add the Corosync to the Proxmox nodes
Cluster Manager - Proxmox VE - Corosync External Vote Support section
Install the corosync-qdevice package on all nodes e.g. edgeline01 and edgeline02
apt install corosync-qdeviceAdd the qdevice to the cluster
Note: All nodes in the cluster will need to be configured, online and utilizing the same root password to cluster correctly
root@edgeline01:~# pvecm qdevice setup 10.10.130.220
/bin/ssh-copy-id: INFO: Source of key(s) to be install ed: "/root/.ssh/id_rsa.pub"
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already install ed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installe d -- if you are prompted now it is to install the new keys
root@10.10.130.220's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh -i /root/.ssh/id_rsa 'root@10.10.130.220'"
and check to make sure that only the key(s) you wanted were added.
INFO: initializing qnetd server
Certificate database (/etc/corosync/qnetd/nssdb) already exists. Delete it to initialize new db
INFO: copying CA cert and initializing on all nodes
node 'edgeline01': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'edgeline01': Creating new key and cert db
node 'edgeline01': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'edgeline01': Importing CA
node 'edgeline02': Creating /etc/corosync/qdevice/net/nssdb
password file contains no data
node 'edgeline02': Creating new key and cert db
node 'edgeline02': Creating new noise file /etc/corosync/qdevice/net/nssdb/noise.txt
node 'edgeline02': Importing CA
INFO: generating cert request
Creating new certificate request
Generating key. This may take a few moments...
Certificate request stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.crq
INFO: copying exported cert request to qnetd server
INFO: sign and export cluster cert
Signing cluster certificate
Certificate stored in /etc/corosync/qnetd/nssdb/cluster-proxmox-cluster.crt
INFO: copy exported CRT
INFO: import certificate
Importing signed cluster certificate
Notice: Trust flag u is set automatically if the private key is present.
pk12util: PKCS12 EXPORT SUCCESSFUL
Certificate stored in /etc/corosync/qdevice/net/nssdb/qdevice-net-node.p12
INFO: copy and import pk12 cert to all nodes
node 'edgeline01': Importing cluster certificate and key
node 'edgeline01': pk12util: PKCS12 IMPORT SUCCESSFUL
node 'edgeline02': Importing cluster certificate and key
node 'edgeline02': pk12util: PKCS12 IMPORT SUCCESSFUL
INFO: add QDevice to cluster configuration
INFO: start and enable corosync qdevice daemon on node 'edgeline01'...
Synchronizing state of corosync-qdevice.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable corosync-qdevice
Created symlink '/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service' - '/usr/lib/systemd/system/corosync-qdevice.service'.
INFO: start and enable corosync qdevice daemon on node 'edgeline02'...
Synchronizing state of corosync-qdevice.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable corosync-qdevice
Created symlink '/etc/systemd/system/multi-user.target.wants/corosync-qdevice.service' - '/usr/lib/systemd/system/corosync-qdevice.service'.
Reloading corosync.conf...
Done
root@edgeline01:~#If you see an error:
command 'ssh -o 'BatchMode=yes' -lroot 10.10.130.220 corosync-qdevice-net-certutil
-m -c /etc/pve/qdevice-net-node.p12' failed: exit code 255ssh between the nodes first as root 'ssh root@10.10.130.220' accepting the cert, then re-attempt the qdevice add operation
Check the status of the cluster
pvecm statusExample output below
root@edgeline01:~# pvecm status
Cluster information
-------------------
Name: proxmox-cluster
Config Version: 3
Transport: knet
Secure auth: on
Quorum information
------------------
Date: Tue Nov 4 12:32:57 2025
Quorum provider: corosync_votequorum
Nodes: 2
Node ID: 0x00000001
Ring ID: 1.9
Quorate: Yes
Votequorum information
----------------------
Expected votes: 3
Highest expected: 3
Total votes: 3
Quorum: 2
Flags: Quorate Qdevice
Membership information
----------------------
Nodeid Votes Qdevice Name
0x00000001 1 A,V,NMW 10.10.194.3 (local)
0x00000002 1 A,V,NMW 10.10.194.4
0x00000000 1 Qdevice
root@edgeline01:~#It is also possible to see the q-device IP via the corosync config
root@edgeline01:~# cat /etc/pve/corosync.conf
logging {
debug: off
to_syslog: yes
}
nodelist {
node {
name: edgeline01
nodeid: 1
quorum_votes: 1
ring0_addr: 10.10.194.3
}
node {
name: edgeline02
nodeid: 2
quorum_votes: 1
ring0_addr: 10.10.194.4
}
}
quorum {
device {
model: net
net {
algorithm: ffsplit
host: 10.10.130.220
tls: on
}
votes: 1
}
provider: corosync_votequorum
}
totem {
cluster_name: proxmox-cluster
config_version: 3
interface {
linknumber: 0
}
ip_version: ipv4-6
link_mode: passive
secauth: on
version: 2
}
root@edgeline01:~#See Also
Comments
0 comments
Article is closed for comments.