SvHCI & SvSAN Caching

Storage can be assigned to an SvSAN VSA using:

• VMDK storage on VMware.
• VHDX storage on Hyper-V.
• Raw Device Mapping.
• Pass-through LUNs, typically backed by a hardware RAID volume.

VMware DirectPath assignment of M.2 adapters is also supported for edge-focused hardware, with SvSAN VSAs providing Software RAID 0, RAID 1 or RAID 10.

SvSAN caching uses persistent SSD or flash storage to service reads and writes, together with volatile memory for read-only caching.

SvSAN provides synchronously mirrored targets and disks. Whether caching is enabled or not, I/O must complete to both VSAs before it is acknowledged to the initiator or guest virtual machine.

This prevents data loss if a hypervisor host fails.

An SvSAN VSA consumes one vCPU and 1 GB of RAM when performing basic mirroring. From SvSAN 6.7.x, the base memory allocation is 2 GB. A second vCPU may be added for environments with heavy I/O.

SvSAN caching is typically used with HDD-backed storage rather than SSD-backed storage.

SvSAN supports both read and write caching. When the backing storage uses read-intensive SSDs, it may be advantageous to use a write-intensive or mixed-use SSD for the cache tier.

The following example illustrates a configuration that can be deployed within two 1U servers:

Position in the Software Stack

The SvSAN caching module sits beneath the mirroring layer in the software stack.

SvSAN does not actively keep both cache tiers synchronized because doing so could reduce performance. Each node may therefore be at a different stage of write-back.

These differences are visible on the Targets page as dirty cache values, which may differ between partner nodes.

Cache Resiliency

A single SSD can be used as a cache disk. However, if a partner hypervisor node remains offline for an extended period, the remaining single SSD becomes a potential single point of failure.

Write I/O must complete to storage, making write activity straightforward to capture in cache.

SvSAN services 100% of write I/O through the cache tier, providing a 100% write cache hit rate.

Writes are then sequentialized, or write-coalesced, and flushed to the backing storage as each 1 GB segment is filled.

Aside from the sequentialization and flushing process, the SSD provides the full write performance.

Dirty Cache Data

Dirty cache data is data that remains in cache and has not yet been flushed to the backing storage.

Some dirty cache data will always be present because SvSAN flushes data in 1 GB segments.

The cache remains resilient across nodes because I/O must complete to both VSA cache tiers before the operation completes.

Each VSA cache tier may be at a different stage of write-back because the backing storage or mirrored storage may be experiencing different loads.

Clean Cache Data

Clean cache data is data that has already been flushed to the backing storage but remains in cache until the space is needed for reuse.

Frequently written data is often read again, so retaining clean data in cache can improve subsequent read performance.

Unlike write I/O, it is not possible to know in advance which data from the backing storage will be read.

For example, in an environment with 50 TB of data, SvSAN must determine which portions of that data should be retained in cache.

By default, SvSAN uses the following mechanisms.

1 Read Ahead

Read ahead operates by detecting when a large sequential dataset is being read and pulling that data into the SSD cache.

2 Predictive Storage Caching

Predictive storage caching tracks data that is read and moves it into memory and SSD cache based on read frequency over time.

Frequently accessed workload data is promoted into cache, serviced from SSD, and retained because it is likely to be read again.

A workload such as a nightly backup that runs once every 24 hours has minimal effect on the overall read cache hit rate.

3 Memory Data Pinning

Memory data pinning works by effectively recording read activity on the VSAs for a defined period.

Data read during that period is then pinned in memory so future requests for the same data are guaranteed cache hits.

This is most effective for predictable, short-duration bursts of heavy random read I/O, such as a VDI boot storm.

All workloads vary significantly. The following example is from a retail environment running containerized workloads across approximately 3,000 two-node SvSAN clusters.

StorMagic recommends sizing the cache using the following two primary guidelines.

1 Allocate Approximately 10% of Backing Storage

As a general rule, allocate cache capacity equal to approximately 10% of the total RAID-protected backing storage.

For example, an environment with 50 TB of usable backing storage would typically use around 5 TB of cache.

2 Size for Daily Data Change Rate

Cache sizing should also consider the amount of data that changes during a typical day.

This information can often be collected from storage assessment tools provided by the hardware vendor, including:

• HPE Assessment Foundry 
• Lenovo OneIQ 
• Dell Live Optics 

SSDs are available from different hardware vendors and are marketed using several workload classifications.

HPE Examples

Lenovo Examples

SvSAN uses cache for both read and write operations, so both performance characteristics and endurance ratings are important.

One of the most important endurance specifications is Drive Writes Per Day (DWPD).

SvSAN provides graphical visibility into cache hit rates and overall cache performance through the management interface.

The following example is taken from a production retail customer utilizing SvSAN caching on an approximately 7 TB datastore.

This example shows:

• 100% write cache hit rate (382.05 GB written to SSD).
• 1.61 TB (88%) of reads serviced from approximately 14 GB of RAM.
• Approximately 200 GB (11%) of reads serviced from SSD cache.
• Approximately 20 GB (1%) of read misses.

This workload resulted in an average latency of less than 1 ms over a 24-hour period.

Additional production examples are shown below.

• StorMagic Product Documentation