Version: 7.5.10 (latest)

Protecting SUSE Virtualization (Harvester) VMs and Images

SUSE Virtualization (Harvester) leverages KubeVirt to manage Virtual Machines (VMs) within a Kubernetes-native environment. Veeam Kasten provides advanced functionality for protecting VMs and associated image configuration and data.

Limitations

For current limitations and compatibility notes, see SUSE Virtualization (Harvester) VM Backup and Restore Support Limitations.

Backup and Restore of Virtual Machines and Images

In the general case, backup and restore of VMs within the same SUSE Virtualization (Harvester) cluster can be performed using the standard KubeVirt VM workflows supported by Veeam Kasten (see VM Protection). This requires the same VM images to be present on the cluster where the restore is performed.

However, when migrating a VM between different clusters or restoring a VM whose original image is missing, the following challenges may arise:

How to restore a VM that is linked to a missing image.
How to protect VM image data.
How to migrate VM image data between clusters.

Some of these issues can be addressed by uploading the images to a different cluster and recreating the VirtualMachineImage CRDs with the original names manually (see SUSE Virtualization (Harvester) documentation). However, manual migration may be difficult due to cluster-specific constraints or administrative restrictions.

To address such cases, Veeam Kasten offers two features for handling image-related issues during VM migration:

The ability to override VM image references during restore.
Support for backing up, restoring, and migrating images.

Configuration Requirements

To enable consistent backup, restore, and migration workflows in Veeam Kasten, ensure that VM image volumes and images are properly annotated:

VM image volumes should include the harvesterhci.io/imageId annotation pointing to the source image.

Example:

$ kubectl get pvc customer-test-vm-disk-0-mwos0 -n default -o yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    harvesterhci.io/imageId: vm-images/image-h5zfv
    ...
  name: customer-test-vm-disk-0-mwos0
  namespace: default
  ...
spec:
  ...
status:
  ...

VM images should include the harvesterhci.io/storageClassName annotation referencing the default storage class used during image creation.

Example:

$ kubectl get vmimage image-h5zfv -n vm-images -o yaml            
apiVersion: harvesterhci.io/v1beta1
kind: VirtualMachineImage
metadata:
  annotations:
    harvesterhci.io/storageClassName: harvester-longhorn
    ...
  name: image-h5zfv
  namespace: vm-images
  ...
spec:
  ...
status:
  ...

These annotations are automatically added when an image or a VM is created using the SUSE Virtualization (Harvester) UI. When using custom YAML specifications, ensure these annotations are included.

Restoring VMs with Overriding Image References

To restore a VM by changing its image references, perform the following steps:

Manually identify the names of VM image PVCs in the restored namespace. This can be done by inspecting the harvesterhci.io/volumeClaimTemplates annotation of a restored VM:

$ kubectl get virtualmachine customer-test-vm -n default -o yaml
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
    harvesterhci.io/volumeClaimTemplates: |
      [
        {
          "metadata": {
            "name": "customer-test-vm-disk-0-mwos0",
            "annotations": {
              "harvesterhci.io/imageId": "vm-images/image-h5zfv"
            }
          },
          "spec": {
            ...
          }
        },
        {
          "metadata": {
            "name": "customer-test-vm-disk-1-dkltt",
            "annotations": {
              "harvesterhci.io/imageId": "vm-images/image-592jl"
            }
          },
          "spec": {
            ...
          }
        },
        {
          "metadata": {
            "name": "customer-test-vm-disk-2-hralv"
          },
          "spec": {
            ...
          }
        }
      ]
    ...
  labels:
    harvesterhci.io/creator: harvester
    harvesterhci.io/os: ubuntu
  name: customer-test-vm
  namespace: default
...

PVCs for VM images are annotated with harvesterhci.io/imageId.

If a VM was created manually and lacks the annotation, identify the image PVCs by inspecting the associated storage classes:

$ kubectl get pvc -n default -o custom-columns=NAME:.metadata.name,STORAGECLASS:.spec.storageClassName
NAME                            STORAGECLASS
customer-test-vm-disk-0-mwos0   longhorn-image-h5zfv
customer-test-vm-disk-1-dkltt   longhorn-image-592jl
customer-test-vm-disk-2-hralv   harvester-longhorn

In the example above the customer-test-vm-disk-0-mwos0 and customer-test-vm-disk-1-dkltt are VM image PVCs.

Identify the names and namespaces of replacement images to be used.

After gathering the information, pass it to a Restore Policy or manual Restore Action using the overrideVolumeImages parameter:

kind: Policy
apiVersion: config.kio.kasten.io/v1alpha1
metadata:
  name: vm-restore-test
  namespace: kasten-io
spec:
  frequency: "@onDemand"
  selector: {}
  actions:
    - action: import
      importParameters:
        receiveString: exampleReceiveString
        profile:
          name: profile1
          namespace: kasten-io
    - action: restore
      restoreParameters:
        virtualMachineParameters:
          harvester:
            overrideVolumeImages:
              customer-test-vm-disk-0-mwos0:
                name: image-h5zfv
                namespace: vm-images
              customer-test-vm-disk-1-dkltt:
                name: image-592jl
                namespace: vm-images

By default, the restore process checks whether the checksums of the provided VM images match the originals.

If a checksum mismatch is expected, integrity verification can be skipped by setting ignoreChecks: true:

kind: Policy
apiVersion: config.kio.kasten.io/v1alpha1
metadata:
  name: vm-restore-test
  namespace: kasten-io
spec:
  actions:
    ...
    - action: restore
      restoreParameters:
        virtualMachineParameters:
          harvester:
            overrideVolumeImages:
              customer-test-vm-disk-0-mwos0:
                name: image-h5zfv
                namespace: vm-images
                ignoreChecks: true
              ...

VM Image Migration

The default way of creating VMs on SUSE Virtualization (Harvester) is using Image Volumes based on VM images that are standard KVM-compatible images in qcow2, raw, or ISO format, used as templates for creating VMs. A VM image can be created by uploading via URL, uploading a local file, or creating it from volumes.

When creating an image, a special image storage class is created and used for further provisioning of VM image volumes from an image.

Veeam Kasten simplifies migration and protection of VM images by enabling policies and actions on namespaces containing VM images, treating them as Kubernetes applications.

Note

VM image data is backed up only via an Export to a remote location. Namespace snapshots include only metadata. VM image data cannot be restored from local RestorePoints.

During export, VM images are converted into volumes and backed up using block mode export.

During restore, data is imported into a new PV and re-exported into a new VM image.

Note

The PV for VM image data is created with the image's virtual size rounded to the nearest GiB. As a result, restored image size and checksums may differ.

Note

To use a restored VM image for VM restore, disable integrity checks using ignoreChecks: true.

Example transformation of a VirtualMachineImage YAML specification during migration from a source cluster to a destination cluster:

Source cluster:

$ kubectl get virtualmachineimage image-h5zfv -n vm-images -o yaml
apiVersion: harvesterhci.io/v1beta1
kind: VirtualMachineImage
metadata:
  annotations:
    harvesterhci.io/storageClassName: harvester-longhorn
  labels:
    harvesterhci.io/image-type: raw_qcow2
    harvesterhci.io/imageDisplayName: ubuntu-24.04
    harvesterhci.io/os-type: ubuntu
  finalizers:
  - wrangler.cattle.io/vm-image-controller
  name: image-h5zfv
  namespace: vm-images
  ...
spec:
  checksum: ""
  displayName: ubuntu-24.04
  pvcName: ""
  pvcNamespace: ""
  retry: 3
  sourceType: download
  storageClassParameters:
    migratable: "true"
    numberOfReplicas: "3"
    staleReplicaTimeout: "30"
  url: https://cloud-images.ubuntu.com/minimal/releases/noble/release-20250228/ubuntu-24.04-minimal-cloudimg-amd64.img
status:
  size: 247595008
  virtualSize: 3758096384
  storageClassName: longhorn-image-h5zfv
  ...

Destination cluster:

$ kubectl get virtualmachineimage image-h5zfv -n vm-images -o yaml
apiVersion: harvesterhci.io/v1beta1
kind: VirtualMachineImage
metadata:
  annotations:
    harvesterhci.io/storageClassName: harvester-longhorn
  labels:
    harvesterhci.io/image-type: raw_qcow2
    harvesterhci.io/imageDisplayName: ubuntu-24.04
    harvesterhci.io/os-type: ubuntu
  finalizers:
  - wrangler.cattle.io/vm-image-controller
  name: image-h5zfv
  namespace: vm-images
  ...
spec:
  checksum: ""
  displayName: ubuntu-24.04
  pvcName: k10-image-export-pvc-image-h5zfv-84kfk
  pvcNamespace: vm-images
  retry: 0
  sourceType: export-from-volume
  storageClassParameters:
    migratable: "true"
    numberOfReplicas: "3"
    staleReplicaTimeout: "30"
  url: ""
status:
  size: 4294967296
  virtualSize: 4294967296
  storageClassName: longhorn-image-h5zfv
  ...

$ kubectl get pvc k10-image-export-pvc-image-h5zfv-84kfk -n vm-images
NAME                                     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         VOLUMEATTRIBUTESCLASS   AGE
k10-image-export-pvc-image-h5zfv-84kfk   Bound    pvc-4c7af3f2-034c-4196-8fb3-01ce0a8ad615   4Gi        RWX            harvester-longhorn   <unset>                 4d18h

Block Mode Exports of Images and Image Volumes

Backing up a VM image volume with an associated PVC in Block Mode or a VM image by default does not require the special annotation if the storage class used for VM image creation is already annotated. If the storage class used for VM image creation is not annotated, the annotation can be applied to the image storage class itself.

Limitations​

Backup and Restore of Virtual Machines and Images​

Configuration Requirements​

Restoring VMs with Overriding Image References​

VM Image Migration​

Block Mode Exports of Images and Image Volumes​

Limitations

Backup and Restore of Virtual Machines and Images

Configuration Requirements

Restoring VMs with Overriding Image References

VM Image Migration

Block Mode Exports of Images and Image Volumes