Air-Gapped Install
For environments that are connected to the Internet, one needs access to three repositories to install Veeam Kasten:
- The Helm repository that contains the Veeam Kasten chart
- The container registry that contains the Veeam Kasten container images
- Upstream repositories to install Veeam Kasten dependencies (e.g., Prometheus)
However, if an air-gapped installation is required, it is possible to
use your own private container registry to install Veeam Kasten. While
this can always be done manually, the k10tools image command makes it easier
to automate the process.
An air-gapped installation has no outbound connectivity. If the cluster can reach the internet through an HTTP(S) proxy instead, install Veeam Kasten normally and configure the proxy as described in Configuring an HTTP(S) Proxy.
Air-Gapped Veeam Kasten Installation
If the Veeam Kasten container images are already available in a private repository, the below instructions can be used to install in an air-gapped environment. If needed, support for uploading images to a private image registry is documented below.
Fetching the Helm Chart for Local Use
To fetch the most recent Veeam Kasten Helm chart for local use, run the
following command to pull the latest Veeam Kasten chart as a compressed
tarball (.tgz) file into the working directory.
$ helm repo update && \
helm fetch kasten/k10
If you need to fetch a specific version, please run the following command:
$ helm repo update && \
helm fetch kasten/k10 --version=<k10-version>
Installing Veeam Kasten with Local Helm Chart and Container Images
If the Veeam Kasten container images were uploaded to a registry at
repo.example.com, an air-gapped installation can be performed by
setting global.airgapped.repository=repo.example.com as shown in the
below command:
$ kubectl create namespace kasten-io
$ helm install k10 k10-9.0.11.tgz --namespace kasten-io \
--set global.airgapped.repository=repo.example.com
Installing Veeam Kasten with Disconnected OpenShift Operator
To install Veeam Kasten with an OpenShift operator in an air-gapped cluster, follow the steps under offline operator install.
Running Veeam Kasten Within a Local Network
To run Veeam Kasten in a network without the ability to connect to the
internet, Veeam Kasten needs to be installed in an air-gapped mode with
the helm value metering.mode=airgap as shown in the command below:
$ kubectl create namespace kasten-io
$ helm install k10 k10-9.0.11.tgz --namespace kasten-io \
--set metering.mode=airgap
If metering.mode=airgap is not set in an offline cluster, some
functionality will be disabled. A message warning that Veeam Kasten is
"Unable to validate license" will be displayed in the web based user
interface. Errors containing messages "Could not get google bucket for
metrics", "License check failed" and "Unable to validate license"
will be logged.
If the metering service is unable to connect to the internet for 24 hours, the metering service will restart.
Providing Credentials if Local Container Repository is Private
If the local repository that has been provided as the value of
global.airgapped.repository is private, credentials for that
repository can be provided using secrets.dockerConfig and
global.imagePullSecret flags, as below, with the helm install
command.
--set secrets.dockerConfig=$(base64 -w 0 < ${HOME}/.docker/config.json) \
--set global.imagePullSecret="k10-ecr"
Our Helm chart creates a secret with the name k10-ecr with the value
that has been provided for secrets.dockerConfig. That's why we are
providing secret name k10-ecr as value of global.imagePullSecret.
Enabling Kanister Operations on Kubernetes 1.35 or Later
On Kubernetes version 1.35 or later, a pod must present its own registry
credentials to use a container image, even one already present on the node.
Some Veeam Kasten worker pods run in the application namespace, where the
k10-ecr secret does not exist, and so cannot pull images from a private
repository.
When you use a private repository, this affects the following operations:
- Backup and restore of applications protected by a Kanister blueprint
- Backup of volumes that are not attached to a running workload
- Execution hooks whose blueprint creates a pod
The affected operations can be enabled in following two ways.
Manually copy image pull secrets to application namespaces
To enable these operations, copy the k10-ecr secret into each protected
namespace and attach it to the default service account of that namespace.
For more about image pull secrets, see
Pull an Image from a Private Registry.
$ export APP_NAMESPACE=<application-namespace>
$ kubectl get secret k10-ecr --namespace=kasten-io --output=json \
| jq '{apiVersion, kind, metadata: {name: .metadata.name}, type, data}' \
| kubectl apply --namespace=${APP_NAMESPACE} --filename=-
$ kubectl patch serviceaccount default \
--namespace=${APP_NAMESPACE} \
--patch='{"imagePullSecrets":[{"name":"k10-ecr"}]}'
-
If you provided a name other than
k10-ecrforglobal.imagePullSecret, use that name instead. -
The patch replaces the
imagePullSecretsfield of the service account instead of merging into it. If thedefaultservice account already lists image pull secrets, include them in the patch as well. - To restore an application into a namespace that does not exist yet, create and configure that namespace before running the restore.
Configure Veeam Kasten to create image pull secret copies
Alternatively, Veeam Kasten can be configured to create the copies of image pull
secrets in application namespaces for the duration of operation.
To enable this, set flag global.copyImagePullSecretsToAppNamespaces to true
with helm install command as follows.
--set global.copyImagePullSecretsToAppNamespaces=true
When configured, Veeam Kasten creates copies of secrets specified using
global.imagePullSecrets in application namespaces. The secrets are automatically
deleted when operation finishes.
The automatic creation of temporary image pull secrets is currently supported for following scenarios:
- Backup and restore of applications protected by a Kanister blueprint
- Backup of volumes that are not attached to a running workload
Enabling Veeam Kasten Primer Operations on Kubernetes 1.35 or later
Veeam Kasten Primer operations are also affected by issue described in Enabling Kanister Operations on Kubernetes 1.35 or Later. Following sections describe steps to enable various primer operations.
Enabling operations by primer script and k10tools primer gvs-cluster-check subcommand
Veeam Kasten Primer script k10_primer.sh offers pre-flight checks as described in
Installation Pre-flight Checks,
CSI Pre-flight Checks, and
Veeam Kasten Primer Block Mount Check.
To let Veeam Kasten to be able to perform these operations correctly, k10-ecr secret must be present
in default namespace and default service account must be patched to use the k10-ecr secret.
If an operation allows passing a custom namespace and service account, make sure namespace being passed
contains k10-ecr secret and the service account must be configured to use the k10-ecr. Steps mentioned
in Manually copy image pull secrets to application namespaces can help
setting this up. Make sure values for namespace and service account are adjusted accordingly.
Currently, only following k10tools primer subcommands will work by adding secrets to namespace and
patching service account.
k10tools primer storage check csik10tools primer storage check blockmountk10tools primer gvs-cluster-check
Cloud provider integration check subcommands k10tools primer storage check <awsebs|azure|gcepd> do not
work with method described above.
Enabling storageclass validation from Veeam Kasten UI
Veeam Kasten UI provides way to validate storageclass from Settings -> System Information page.
In air-gapped environments on Kubernetes v1.35 and above, the default service account of the namespace
where Veeam Kasten is installed (default kasten-io), needs to be configured to use k10-ecr secret. Refer to
instructions in Manually copy image pull secrets to application namespaces to configure
service account to use secret. Make sure to adjust namespace name to kasten-io (or the namespace where Veeam Kasten
is installed) while executing the command.
Preparing Veeam Kasten Container Images for Air-Gapped Use
There are multiple ways to use a private repository including setting up a caching or proxy image registry that points to the Veeam Kasten image repositories using tools such as JFrog Artifactory. However, if images need to be manually uploaded or an automated upload pipeline is required to add Veeam Kasten images into your private repository, the following documentation should help.
To see all available commands and flags for running k10tools image please
run the following:
$ docker run --rm gcr.io/kasten-images/k10tools:9.0.11 image --help
The following commands operate against the latest version of Veeam Kasten (9.0.11).
k10tools image is only supported for versions 7.5.0+ of Veeam Kasten and must match the version you're installing.
For older version, please refer to their documentation: https://docs.kasten.io/<version>/install/offline.html.
List Veeam Kasten Container Images
The following command will list all images used by the current Veeam Kasten version (9.0.11). This can be helpful if there is a requirement to tag and push Veeam Kasten images into your private repository manually instead of using the Kasten provided tool documented below.
$ docker run --rm gcr.io/kasten-images/k10tools:9.0.11 image list
Copy Kasten Images into a Private Repository
The following command will copy the Veeam Kasten container images into your
specified registry. If the destination image tag should be different than the
Veeam Kasten version, then the --dst-image-tag can be used to specify a new
image tag.
The following example uses a repository located at repo.example.com.
$ docker run --rm -v $HOME/.docker:/home/kio/.docker gcr.io/kasten-images/k10tools:9.0.11 image copy --dst-registry repo.example.com
This command will use your local docker config if the private registry requires authentication.
The credsStore field in the $HOME/.docker/config.json is used to
specify the credential store. This is typically an external credential
store requiring an external helper and it may not be usable from within
the docker container. Please refer to the docker documentation
for more information.
Alternatively, k10tools image provides authentication mechanisms such as
passing a username and password (--dst-username and --dst-password
flags) or a bearer token (--dst-token flag). Please refer to
the help flag for more information.
After running the previous command, use the
instructions above to install Veeam Kasten via images
uploaded to repo.example.com.
Copy Kasten Images to/from a Filesystem Directory
Network limitations may limit the ability to directly copy images into a private repository. Alternatively, images can be copied to the local filesystem and then pushed to a repository separately. This requires downloading the k10tools binary.
The following example copies the images to a directory images. This
directory can then be used to upload to a private repository located at
repo.example.com.
:substitutions:
$ k10tools image copy --dst-path images
$ k10tools image copy --src-path images --dst-registry repo.example.com
Using Iron Bank Veeam Kasten Container Images
If you want to use the Iron Bank hardened Veeam Kasten images in an air-gapped
environment, execute the above commands but replace
image with ironbank image:
:substitutions:
$ docker run --rm gcr.io/kasten-images/k10tools:9.0.11 ironbank image list
$ docker run --rm -v $HOME/.docker:/home/kio/.docker gcr.io/kasten-images/k10tools:9.0.11 ironbank image copy --dst-registry repo.example.com
This ensures the images are pulled from Registry1.
You must be logged in to the docker registry locally for this process
to function correctly. Use docker login registry1.dso.mil --username "${REGISTRY1_USERNAME}" --password-stdin with your Registry1 CLI secret as
the password to login.
Alternatively, provide credentials using the methods described above.