SvHCI - Evaluators Guide For Testing

Mark Christie
  • Updated

StorMagic Banner

Evaluate SvHCI HA Failure Scenarios & Performance

This guide explains how to deploy a two-node StorMagic SvHCI environment, validate high availability and resiliency using failure scenarios, test data encryption with SvKMS, and benchmark storage performance using a reference hardware platform.

Audience: Administrators Product: SvHCI Topic: Evaluation & Failure Testing Includes: HA, Encryption & Performance Testing
Note: All images are clickable for enlarging or can be opened in a new browser tab.
SvHCI Failure Testing & Benchmarking Guide

Configure a two-node SvHCI environment and validate high availability, resiliency, encryption, and storage performance using the evaluation scenarios contained in this guide.

Introduction

This Evaluator’s Guide describes how to perform SvHCI failure testing and benchmarking using an example hardware platform and SvHCI configuration.

The first section describes configuring and deploying the infrastructure, including:

  • SvHCI hosts and clusters
  • Configuring the hardware
  • Deploying SvHCI
  • Configuring the virtual networking
  • Clustering the systems with a witness
  • Creating a test virtual machine
  • Configuring SvHCI event notifications

The following sections describe various failure scenarios that verify the environment is configured correctly to provide the highest possible resiliency and redundancy while demonstrating the expected behaviour for each failure condition.

These scenarios are grouped into two categories:

  • Lights On – Single failure scenarios where guest VMs continue running due to the resiliency built into the platform.
  • Lights Off – Multiple failure scenarios where guest VMs are expected to become unavailable.

Additional sections cover data encryption using StorMagic SvKMS, encryption failure testing, performance benchmarking with and without caching, troubleshooting guidance, and rollback procedures.

StorMagic Support may be contacted at support@stormagic.com or via the StorMagic Support Portal.

↑ Back to top

Lab Environment & Infrastructure Deployment

The evaluation environment used throughout this guide is based on HPE ProLiant DL325 Gen11 systems.

Example SvHCI hardware platform

Example Hardware Configuration

  • Server: HPE ProLiant DL325 Gen11
  • 1 × AMD EPYC 9224 2.5GHz 24-Core Processor
  • 48 Logical Processors
  • 256 GB RAM (8 × 32 GB)
  • HPE NS204i-u Gen11 NVMe Boot Device for Hypervisor & VSA
  • HPE MR216i-o Gen11 Storage Controller
  • 5 × 1.92 TB HPE SATA Read Intensive SSDs

Complete the environment in the following order:

  1. Install SvHCI.
  2. Complete the First Boot Wizard.
  3. Configure the virtual networking.
  4. Deploy a Witness.
  5. Create the Cluster.
  6. Create the Storage Pool.
  7. Create five Ubuntu benchmark virtual machines.

Each benchmark VM should contain an operating system disk together with approximately 100 GB of benchmark storage.

Configured SvHCI networking

Deployment References

  • Install SvHCI
  • Complete the First Boot Wizard
  • Configure Virtual Networking
  • Deploy a Witness
  • Create a Cluster
  • Create a Storage Pool
  • Create Ubuntu Guest VMs

↑ Back to top

Video Resources

The following webinar demonstrates many of the evaluation scenarios covered throughout this guide.

Hardware Failure (HA Failover)

WAN Drop (Witness Connectivity Failure)

Storage Failure

Cluster Shutdown

↑ Back to top
Lights On Failure Scenarios

The following scenarios simulate failures that may occur in a production environment while guest virtual machines continue to run. These tests validate that the cluster has been configured correctly to provide the highest possible levels of resiliency and redundancy.

During each scenario, guest virtual machines remain online ("Lights On") while the underlying infrastructure automatically tolerates the failure.

Scenario Failure Description Expected Result
1 Mirror Network Failure Disconnect the preferred mirror network between cluster nodes. Guest virtual machines remain online while mirror traffic automatically reroutes across the remaining network.
2 WAN / Witness Failure Disconnect connectivity to the SvHCI Witness. Cluster operation continues while guest virtual machines remain available.
3 Storage Device Failure Simulate the loss of a storage device. Storage remains available through mirroring while alerts are generated.
4 Host Network Failure Disconnect one or more production network interfaces. Guest connectivity continues using the remaining available interfaces.
5 Live Migration Migrate guest virtual machines between cluster members. Guest workloads continue running without interruption.
6 Host Failure Power off a single SvHCI node. Protected guest virtual machines automatically restart on the surviving cluster node.
↑ Back to top
Lights Off Failure Scenarios

The following scenarios intentionally create multiple simultaneous failures that exceed the redundancy available within the cluster. During these tests, guest virtual machines are expected to become unavailable ("Lights Off").

These scenarios demonstrate system behaviour during catastrophic events and the recovery process once connectivity or hardware has been restored.

Scenario Failure Description Expected Result
1 Complete Mirror Network Failure Disconnect all mirror network connectivity between the cluster nodes. Guest virtual machines stop because mirrored storage communication is no longer possible.
2 Power Failure Power off both SvHCI hosts. All guest virtual machines stop until power is restored and the cluster recovers.
3 Dual Host Failure Introduce failures that make both hosts unavailable. Cluster services become unavailable until at least one host is recovered.
4 Complete Storage Failure Remove or disconnect all storage devices required by the storage pool. Guest virtual machines stop because no storage replicas remain available.
Recovery: After restoring hardware, networking, or storage connectivity, verify cluster health before powering guest virtual machines back on.
↑ Back to top
Data Encryption & SvKMS Overview

StorMagic SvHCI supports encrypted virtual machine storage through integration with StorMagic SvKMS, providing centralized key management for encrypted storage pools.

Encryption keys are stored externally within SvKMS and are retrieved by the SvHCI cluster when encrypted storage resources are accessed.

The following sections demonstrate encryption behaviour during normal operation together with failure scenarios involving SvKMS availability.

Best Practice: Deploy SvKMS independently from the SvHCI cluster to ensure encryption keys remain available during infrastructure maintenance or host failures.
↑ Back to top
Encryption Failure Scenarios (Lights On)

These scenarios validate encrypted SvHCI storage while guest virtual machines remain operational.

Scenario Failure Description Expected Result
1 SvKMS Network Loss Disconnect network connectivity to the SvKMS server. Running encrypted guest virtual machines continue operating using cached encryption keys.
2 SvKMS Shutdown Power off the SvKMS appliance. Existing encrypted workloads continue operating without interruption.
↑ Back to top
Encryption Failure Scenarios (Lights Off)

These scenarios demonstrate how encrypted SvHCI storage behaves when multiple failures occur simultaneously and encrypted volumes can no longer retrieve the required keys from the key management server.

Scenario Failure Description Expected Result
1 SvKMS Offline During Cluster Restart Both SvHCI nodes restart while the SvKMS server is unavailable. Encrypted storage remains locked until the key server becomes available again and the encryption keys can be retrieved.
2 SvKMS Restored Reconnect or power on the SvKMS server after the cluster has restarted. Encrypted volumes automatically unlock, mirrored storage is restored, and normal cluster operation resumes.
Note: Recovery of encrypted storage requires successful communication with the configured key management server before encrypted targets can be brought online.
↑ Back to top
Performance Benchmarking

This section demonstrates how to benchmark the evaluation environment using FIO (Flexible I/O Tester). FIO measures storage bandwidth, IOPS, latency, and response times using configurable workloads.

Benchmark Preparation

  1. Create Ubuntu guest virtual machines.
  2. Add both an operating system disk and a benchmark disk.
  3. Install FIO.
  4. Execute the desired benchmark workloads.

The example environment consists of five Ubuntu virtual machines simultaneously running 4 KB random read workloads using a queue depth of 32 and four jobs per virtual machine.

Job Name Read Bandwidth (KB/s) Read IOPS Average Read Latency (µs)
vm1-4vcpus-reads-32qd-4jobs 10796916 89,970 47292
vm1-4vcpus-writes-32qd-4jobs 6756420 56,298 18761
vm2-4vcpus-reads-32qd-4jobs 11442576 95,354 14155
vm2-4vcpus-writes-32qd-4jobs 6373564 53,109 14671
vm3-4vcpus-reads-32qd-4jobs 11512968 95,940 13008
vm3-4vcpus-writes-32qd-4jobs 6832208 56,932 13997
vm4-4vcpus-reads-32qd-4jobs 11507000 95,890 15186
vm4-4vcpus-writes-32qd-4jobs 7169252 59,743 14190
vm5-4vcpus-reads-32qd-4jobs 12496424 104,135 10437
vm5-4vcpus-writes-32qd-4jobs 8008460 66,736 13201

Performance benchmarking datastore example

↑ Back to top
See Also
↑ Back to top
↑ Back to top
SvHCI • High Availability • Failure Testing • Performance Benchmarking • Data Encryption

Related articles

Comments

0 comments

Article is closed for comments.