Migrating Applications

Introduction

The ability to move an application across clusters is an extremely powerful feature that enables a variety of use cases including Disaster Recovery (DR), Test/Dev with realistic data sets, and performance testing in isolated environments.

In particular, the K10 platform is being built to support application migration and mobility in a variety of different and overlapping contexts:

  • Cross-Namespace: The entire application stack can be migrated across different namespaces in the same cluster (covered in restoring applications).

  • Cross-Cluster: The application stack is migrated across non-federated Kubernetes clusters.

  • Cross-Account: Mobility can additionally be enabled across clusters running in different accounts (e.g., AWS accounts) or projects (e.g., Google Cloud projects).

  • Cross-Region: Mobility can further be enabled across different regions of the same cloud provider (e.g., US-East to US-West).

  • Cross-Cloud: Finally, mobility can be enabled across different cloud providers (e.g., AWS to Azure).

Mobility Configuration

Some additional infrastructure configuration is needed before migration between two clusters can be enabled.

  • Required: Object storage configuration

  • Use-Case Specific: Cross-account and Kanister configuration

Object Storage Configuration

For two non-federated clusters to share data, we need to create an object storage bucket (e.g., k10-migrate) that will be used to store data about the application restore points that have been selected for migration between clusters. The source cluster needs to have write permissions to this bucket, and the destination cluster needs to have read permissions. The appropriate credentials can be configured using Export Profile and Import Profile as described below.

Additional Kanister Configuration

The Kanister framework, described later, stores application-level artifacts in an object store. For Kanister-enabled applications, in addition to the profile configuration above, both clusters need to use a Kanister profile that refers to the same object storage bucket. This can be accomplished by creating a Kanister profile via the K10 dashboard or via the CRD-based Profiles API.

Note

The requirement for both clusters to have access to the Kanister object storage bucket will go away in an upcoming release in favor of only using the object storage bucket configured above for mobility.

Cross-Cloud Configuration

When migrating applications across different cloud providers, including hybrid environments, K10 should have support for storage system snapshots on the originating side.

Apart from the Import and Export profiles, a Kanister profile also needs to be created as the data belonging to cross-cloud portable snapshots is stored as Kanister artifacts.

Customer-Provided Encryption Keys

Currently, data exported by K10 for cross-cloud snapshot migration is encrypted via the use of the AES-256 algorithm. It is highly recommended that you specify your own encryption key. This can be done via the creation of a Kubernetes secret in the kasten-io namespace after install:

$ kubectl create secret generic k10-artifact-key \
   --namespace kasten-io \
   --from-literal=key=<key>

Note

If the above key is modified after artifacts have been created, old data will not be readable without support intervention.

Cross-Account Configuration

The following configuration is only required if cross-account migration within the same cloud-provider is needed.

AWS

If cross-account volume migration is desired, K10 needs an IAM user created within the destination account (e.g., k10migrate). The destination AWS account ID is also needed. During the creation of the migration profile, the username should be specified as <AWS Account ID>/<User Name>

In an AWS environment, K10 grants access to snapshots stored in the source account to the destination account. If the destination cluster is in a different region, K10 will make a cross-region copy of the underlying EBS snapshots before sharing access.

Google Cloud

Currently, due to underlying provider limitations, optimized migration is only supported among different clusters in the same project. For migration across clusters running within different accounts, please follow the documentation for cross-cloud migration.

Import/Export Profiles

To enable the export and import of applications, you are required to create a profile on each side of the transfer. Profile creation can be accessed from the Settings icon in the top-right corner of the dashboard or via the CRD-based Profiles API.

../_images/migration_profiles_none.png

Export Profile

To move data to another cluster, simply click Create Profile on the profiles page and select Export. We recommend using the target cluster name as the profile name to make it easier to access later.

As mentioned earlier, exporting data requires an object storage location. You are therefore required to pick an object storage provider, a region for the bucket, and the bucket name. If a bucket with the specific name does not exist, it will be created in the specified region.

Further, if the source and destination clusters are in different regions of the same cloud provider, it is essential that the region selected in the profile is the same region as where the destination cluster is located, as K10 uses that to determine the region where the artifacts (e.g., snapshot copies) should be copied to.

If migration is desired across Kubernetes clusters in different clouds or between on-premises and public cloud environments, you need to enable the infrastructure portability option when creating an export profile.

../_images/migration_profiles_export.png

If you are using an S3 compatible bucket that is not hosted by one of the supported cloud providers, you will need to also specify an S3 endpoint URL (accessible from both clusters) and optionally disable SSL verification. Disabling SSL verification is only recommended for test setups.

../_images/migration_profiles_export_s3generic.png

When you hit Validate and Save, the config profile will be created and you will see a profile similar to the following:

../_images/migration_profiles_example.png

Import Profile

To import data into a cluster, we also need to create a import profile on the destination cluster. Similarly to the export profile on the source cluster, you will need to specify S3 provider, endpoint information and credentials that will allow read access to the bucket.

../_images/migration_profiles_import.png

Exporting Applications

The workflow for exporting applications into another cluster is very similar to the workflow followed for restoring applications. You simply go over to the Applications page, click on the dropdown, and select Export Application.

../_images/applications_protected_manual.png

After a restore point is selected, you will be presented with information about the application snapshot being migrated (e.g., time, originating policy, artifacts) and, more importantly, the option to configure the destination by selecting a migration profile.

../_images/migration_export.png

A migration profile, as defined above, specifies where the exported data and metadata will be stored. This profile contains the name of the cluster that we are migrating data to and the location of an object storage bucket (e.g., using AWS S3, Google Cloud Storage, or any other S3-compliant object store) to export the needed data.

Simply select any of the migration profiles you might have previously created and then click on Export. After confirming the export, you will be presented with a confirmation dialog. It is essential that you copy the text block presented here because it will be required to securely create an import policy on the destination cluster.

../_images/migration_textblock.png

Importing Applications

Importing an application snapshot is again very similar to the policy workflow for protecting applications. From the Policies page, simply select Create New Policy.

../_images/policies_migration_import.png

To import applications, you need to select Import for the action. While you need to also specify a frequency for the import, this simply controls how often to check the shared object storage location. If data within the object store is not refreshed at the same frequency, no duplicate work will be performed.

You will also need to paste the text block presented by the destination cluster during export to allow us to create the data export/import relationship for this application across these clusters. Finally, similar to the export workflow, the permissions profile also need to be selected but, as mentioned earlier, the destination cluster only needs read and list permissions on all the data in the shared object store.

After the Create Policy button is clicked, the system will start scheduling jobs based on the policy definition. If new data is detected, it will be imported into the cluster, automatically associated with the application stack already running, and be made available as a restore point. The normal workflow for restoring applications can be followed and, when given the choice, simply select a restore point that is tagged with the policy name used above to create the import policy.

Migration Considerations

While migrating applications across clusters, regions, and clouds is significantly simpler with K10, there are still other considerations to be aware of that might impact your migration workflow. The below sections cover these considerations in detail for a smoother migration process.

Non-Application Resources

While the K10 platform will protect all resources that are discovered in the application namespace, it intentionally does not gather resources that are not part of an application namespace but only found at the cluster level. Examples of such resources include Custom Resource Definitions or CRDS (but not the actual Custom Resources or CRs), Storage Classes, or Cluster Role Bindings. We assume that these resources will be deployed as a part of cluster creation by the administrator or the cluster deployment process. This is of particular importance when migrating applications to a new cluster as the absence of these resources could cause application restore failures.

Related to the issue of cluster-wide resources, there are Kubernetes resources that are only found in a namespace but have a resource conflict with other applications in the same cluster. While generally discouraged for production usage, a commonly observed resource that falls in this category is the NodePort service. Once claimed by an application, a subsequent request for the same NodePort will conflict and be disallowed. Whenever possible, K10 attempts to work around such limitations by resetting such settings on restore to a cluster-provided non-conflicting value. For example, with NodePort, K10 will allow Kubernetes to pick a port from the default port range allocated to the NodePort service.

Finally, applications sometimes will have dependencies that are external to the cluster (e.g., DNS entries) that might not be visible to Kubernetes or K10. However, it is possible to work with such external resources through the use of Kanister blueprints.

Availability and Failure Zones

For migrations across Kubernetes clusters that are spread across different availability (or failure) zones, K10, for stateful applications in the destination cluster, tries to maintain the same fault independence as the source cluster. However, this can be hard or not possible when either the destination cluster has fewer availability zones than the source cluster or does not have nodes deployed in all availability zones.

To help work around these issues, K10 adopts a number of techniques to simplify the redeployment process. As far as possible, K10 will attempt to only place volumes in availability zones where it has already discovered worker nodes belonging to the destination cluster.

In the case of migration in the same region, K10 will attempt to first place volumes in the same zones that the source cluster had selected. If that is not possible, it will assign volumes to other availability zones where it has found worker nodes.

For cross-region migration, K10 will first attempt to map availability zones between the source and destination regions and, assuming the worker nodes are found in those zones, provision volumes according to that mapping. If a mapping is not possible (e.g., the source cluster used more available zones than the destination cluster has available), it will provision volumes into one of the other discovered zones in the destination cluster.

In the unlikely case that K10 is unable to either find worker nodes or discover zones, it will fallback to assigning volumes to zones that it knows exist in that region.

Finally, note that the above approach can potentially run into scheduling issues where the destination cluster might have either insufficient or no compute resources in a given zone. If you run into this issue, the simplest solution is to provision more worker nodes across your desired available zones.