Advanced Install Options

FREE K10 Edition and Licensing

By default, K10 comes with an embedded free Edition license. The free edition license allows you to use the software on a cluster with at most 50 worker nodes for one month and then 10 nodes after the one month period. The free edition license embedded in each release is valid for 12 months after which an update to the latest released version will be required which will extend the license. You can remove the node restriction by updating to Enterprise Edition and obtaining the appropriate license from the Kasten team.

Using A Custom License During Install

To install a license provided by Kasten that removes the node restriction, please add the following to any of the helm install commands. K10 dynamically retrieves the license key and a pod restart is not required.

...
          --set license=<license-text>

Changing Licenses

To add a new license to K10, a secret needs to be created in the K10 namespace (default is kasten-io) with the requirement that the license text be set in a field named license. To do this from the command line, run:

$ kubectl create secret generic <license-secret-name> \
    --namespace kasten-io \
    --from-literal=license="$(echo '<base64_encoded_license>' | base64 --decode)"

Multiple license secrets can exist simultaneously and K10 will check if any are valid. This license check is done periodically and so, no K10 restarts are required if a different existing license becomes required (e.g., due to a cluster expansion or an old license expiry) or when a new license is added.

The resulting license will look like:

 apiVersion: v1
 data:
   license: Y3Vz...
 kind: Secret
 metadata:
   creationTimestamp: "2020-04-14T23:50:05Z"
   labels:
     app: k10
     app.kubernetes.io/instance: k10
     app.kubernetes.io/managed-by: Helm
     app.kubernetes.io/name: k10
     helm.sh/chart: k10-4.5.1
     heritage: Helm
     release: k10
   name: k10-custom-license
   namespace: kasten-io
 type: Opaque

Similarly, old licenses can be removed by deleting the secret that contains it.

$ kubectl delete secret <license-secret-name> \
    --namespace kasten-io

License Grace period

If the license status of the cluster becomes invalid (e.g., the licensed node limit is exceeded), the ability to perform manual actions or creating new policies will be disabled but your previously scheduled policies will continue to run for 50 days. The displayed warning will be look like:

By default, K10 provides a grace period of 50 days to ensure that applications remain protected while a new license is obtained or the cluster is brought back into compliance by reducing the number of nodes. K10 will stop the creation of any new jobs (scheduled or manual) after the grace period expires.

If the cluster's license status frequently swaps between valid and invalid states, the amount of time the cluster license spends in an invalid status will be subtracted from subsequent grace periods.

Manually Creating or Using an Existing Service Account

The following instructions can be used to create a new Service Account that grants K10 the required permissions to Kubernetes resources and the use the given Service Account as a part of the install process. The instructions assume that you will be installing K10 in the kasten-io namespace.

# Create kasten-io namespace if have not done it yet.
$ kubectl create namespace kasten-io

# Create a ServiceAccount for k10 k10-sa
$ kubectl --namespace kasten-io create sa k10-sa

# Create a cluster role binding for k10-sa
$ kubectl create clusterrolebinding k10-sa-rb \
    --clusterrole cluster-admin \
    --serviceaccount=kasten-io:k10-sa

Following the SA creation, you can install K10 using:

$ helm install k10 kasten/k10 --namespace=kasten-io \
    --set rbac.create=false \
    --set serviceAccount.create=false \
    --set serviceAccount.name=k10-sa

Using Red Hat certified upstream container images

The upstream container images (e.g., Ambassador, Dex) that K10 uses, by default, are not Red Hat certified images. If it's required to run the Red Hat certified version of those images, this helm flag can be used:

--set global.upstreamCertifiedImages=true

Pinning K10 to Specific Nodes

While not generally recommended, there might be situations (e.g., test environments, nodes reserved for infrastructure tools, or clusters without autoscaling enabled) where K10 might need to be pinned to a subset of nodes in your cluster. You can do this easily with an existing deployment by using a combination of NodeSelectors and Taints and Tolerations.

The process to modify a deployment to accomplish this is demonstrated in the following example. The example assumes that the nodes you want to restrict K10 to has the label selector-key: selector-value and a taint set to taint-key=taint-value:NoSchedule.

$ cat << EOF > patch.yaml
spec:
  template:
    spec:
      nodeSelector:
        selector-key: selector-value
      tolerations:
      - key: "taint-key"
        operator: "Equal"
        value: "taint-value"
        effect: "NoSchedule"
EOF

$ kubectl get deployment --namespace kasten-k10 | awk 'FNR == 1 {next} {print $1}' \
  | xargs -I DEP kubectl patch deployments DEP --namepsace kasten-k10 --patch "$(cat patch.yaml)"

Using Trusted Root Certificate Authority Certificates for TLS

Note

For temporary testing of object storage systems that are deployed using self-signed certificates signed by a trusted Root CA, it is also possible to disable certificate verification if the Root CA certificate is not easily available.

If the S3-compatible object store configured in a Location Profile was deployed with a self-signed certificate that was signed by a trusted Root Certificate Authority (Root CA), then the certificate for such a certificate authority has to be provided to K10 to enable successful verification of TLS connections to the object store.

Similarly, to authenticate with a private OIDC provider whose self-signed certificate was signed by a trusted Root CA, the certificate for the Root CA has to be provided to K10 to enable successful verification of TLS connections to the OIDC provider.

Multiple Root CAs can be bundled together in the same file.

Install Root CA in K10's namespace

Assuming K10 will be deployed in the kasten-io namespace, the following instructions will make a private Root CA certificate available to K10.

Note

The name of the Root CA certificate must be custom-ca-bundle.pem

# Create a ConfigMap that will contain the certificate.
# Choose any name for the ConfigMap
$ kubectl --namespace kasten-io create configmap custom-ca-bundle-store --from-file=custom-ca-bundle.pem

To provide the Root CA certificate to K10, add the following to the Helm install command.

--set cacertconfigmap.name=<name-of-the-configmap>

Install Root CA in application's namespace when using Kanister sidecar

If you either use K10's Kanister sidecar injection feature for injecting the Kanister sidecar in your application's namespace or if you have manually added the Kanister sidecar, you must create a ConfigMap containing the Root CA in the application's namespace and update the application's specification so that the ConfigMap is mounted as a Volume. This will enable the Kanister sidecar to verify TLS connections successfully using the Root CA in the ConfigMap.

Assuming that the application's namespace is named test-app, use the following command to create a ConfigMap containing the Root CA in the application's namespace:

Note

The name of the Root CA certificate must be custom-ca-bundle.pem

# Create a ConfigMap that will contain the certificate.
# Choose any name for the ConfigMap
$ kubectl --namespace test-app create configmap custom-ca-bundle-store --from-file=custom-ca-bundle.pem

This is an example of a VolumeMount that must be added to the application's specification.

- name: custom-ca-bundle-store
  mountPath: "/etc/ssl/certs/custom-ca-bundle.pem"
  subPath: custom-ca-bundle.pem

This is an example of a Volume that must be added to the application's specification.

- name: custom-ca-bundle-store
  configMap:
    name: custom-ca-bundle-store

Troubleshooting

If K10 is deployed without the cacertconfigmap.name setting, validation failures such as the one shown below will be seen while configuring a Location Profile using the web based user interface.

In the absence of the cacertconfigmap.name setting, authentication with a private OIDC provider will fail. K10's logs will show an error x509: certificate signed by unknown authority.

If you do not install the Root CA in the application namespace when using a Kanister sidecar with the application, the logs will show an error x509: certificate signed by unknown authority when the sidecar tries to connect to any endpoint that requires TLS verification.

Running K10 Containers as a Specific User

K10 service containers run with UID and fsGroup 1000 by default. If the storage class K10 is configured to use for its own services requires the containers to run as a specific user, then the user can be modified.

This is often needed when using shared storage, such as NFS, where permissions on the target storage require a specific user.

To run as a specific user (e.g., root (0), add the following to the Helm install command:

--set services.securityContext.runAsUser=0 \
--set services.securityContext.fsGroup=0 \
--set prometheus.server.securityContext.runAsUser=0 \
--set prometheus.server.securityContext.runAsGroup=0 \
--set prometheus.server.securityContext.runAsNonRoot=false \
--set prometheus.server.securityContext.fsGroup=0

Other SecurityContext settings for the K10 service containers can be specified using the --set service.securityContext.<setting name> and --set prometheus.server.securityContext.<setting name> options.

Using Kubernetes Endpoints for Service Discovery

The K10 API gateway uses Kubernetes DNS to discover and route requests to K10 services.

If Kubernetes DNS is disabled or not working, K10 can be configured to use Kubernetes endpoints for service discovery.

To do this, add the following to the Helm install command:

--set apigateway.serviceResolver=endpoint

Configuring Prometheus

Prometheus is an open-source system monitoring and alerting toolkit. K10 is bundled with Prometheus chart version 14.1.1 . Any of the Prometheus Chart values can be used during the K10 helm installation.

When passing value from the command line, the value key has to be prefixed with the prometheus. string:

--set prometheus.server.persistentVolume.storageClass=default.sc

When passing values in a YAML file, all prometheus settings should be under the prometheus key:

# values.yaml
# global values - apply to both K10 and prometheus
global:
  persistence:
    storageClass: default-sc
# K10 specific settings
auth:
  basicAuth: enabled
# prometheus specific settings
prometheus:
  server:
    persistentVolume:
      storageClass: another-sc

Complete List of K10 Helm Options

The following table lists the configurable parameters of the K10 chart and their default values.

Parameter

Description

Default

eula.accept

Whether to enable accept EULA before installation

false

eula.company

Company name. Required field if EULA is accepted

None

eula.email

Contact email. Required field if EULA is accepted

None

license

License string obtained from Kasten

None

rbac.create

Whether to enable RBAC with a specific cluster role and binding for K10

true

scc.create

Whether to create a SecurityContextConstraints for K10 ServiceAccounts

false

services.dashboardbff.hostNetwork

Whether the dashboardbff pods may use the node network

false

services.executor.hostNetwork

Whether the executor pods may use the node network

false

serviceAccount.create

Specifies whether a ServiceAccount should be created

true

serviceAccount.name

The name of the ServiceAccount to use. If not set, a name is derived using the release and chart names.

None

ingress.create

Specifies whether the K10 dashboard should be exposed via ingress

false

ingress.class

Cluster ingress controller class: nginx, GCE

None

ingress.host

FQDN (e.g., k10.example.com) for name-based virtual host

None

ingress.urlPath

URL path for K10 Dashboard (e.g., /k10)

Release.Name

ingress.annotations

Additional Ingress object annotations

{}

ingress.tls.enabled

Configures a TLS use for ingress.host

false

ingress.tls.secretName

Specifies a name of TLS secret

None

global.persistence.enabled

Use PVS to persist data

true

global.persistence.size

Default global size of volumes for K10 persistent services

20Gi

global.persistence.catalog.size

Size of a volume for catalog service

global.persistence.size

global.persistence.jobs.size

Size of a volume for jobs service

global.persistence.size

global.persistence.logging.size

Size of a volume for logging service

global.persistence.size

global.persistence.metering.size

Size of a volume for metering service

global.persistence.size

global.persistence.storageClass

Specified StorageClassName will be used for PVCs

None

global.airgapped.repository

Specify the helm repository for offline (airgapped) installation

''

secrets.awsAccessKeyId

AWS access key ID (required for AWS deployment)

None

secrets.awsSecretAccessKey

AWS access key secret

None

secrets.awsIamRole

ARN of the AWS IAM role assumed by K10 to perform any AWS operation.

None

secrets.googleApiKey

Non-default base64 encoded GCP Service Account key file

None

secrets.azureTenantId

Azure tenant ID (required for Azure deployment)

None

secrets.azureClientId

Azure Service App ID

None

secrets.azureClientSecret

Azure Service APP secret

None

secrets.azureResourceGroup

Resource Group name that was created for the Kubernetes cluster

None

secrets.azureSubscriptionID

Subscription ID in your Azure tenant

None

secrets.azureResourceMgrEndpoint

Resource management endpoint for the Azure Stack instance

None

secrets.azureADEndpoint

Azure Active Directory login endpoint

None

secrets.azureADResourceID

Azure Active Directory resource ID to obtain AD tokens

None

secrets.vsphereEndpoint

vSphere endpoint for login

None

secrets.vsphereUsername

vSphere username for login

None

secrets.vspherePassword

vSphere password for login

None

secrets.dockerConfigPath

Use --set-file secrets.dockerConfigPath=path_to_docker_config.yaml to specify docker config for image pull

None

cacertconfigmap.name

Name of the ConfigMap that contains a certificate for a trusted root certificate authority

None

clusterName

Cluster name for better logs visibility

None

metering.mode

Control license reporting (set to airgap for private-network installs)

None

metering.reportCollectionPeriod

Sets metric report collection period (in seconds)

1800

metering.reportPushPeriod

Sets metric report push period (in seconds)

3600

externalGateway.create

Configures an external gateway for K10 API services

false

externalGateway.annotations

Standard annotations for the services

None

externalGateway.fqdn.name

Domain name for the K10 API services

None

externalGateway.fqdn.type

Supported gateway type: route53-mapper or external-dns

None

externalGateway.awsSSLCertARN

ARN for the AWS ACM SSL certificate used in the K10 API server

None

auth.basicAuth.enabled

Configures basic authentication for the K10 dashboard

false

auth.basicAuth.htpasswd

A username and password pair separated by a colon character

None

auth.basicAuth.secretName

Name of an existing Secret that contains a file generated with htpasswd

None

auth.k10AdminGroups

A list of groups whose members are granted admin level access to K10's dashboard

None

auth.k10AdminUsers

A list of users who are granted admin level access to K10's dashboard

None

auth.tokenAuth.enabled

Configures token based authentication for the K10 dashboard

false

auth.oidcAuth.enabled

Configures Open ID Connect based authentication for the K10 dashboard

false

auth.oidcAuth.providerURL

URL for the OIDC Provider

None

auth.oidcAuth.redirectURL

URL to the K10 gateway service

None

auth.oidcAuth.scopes

Space separated OIDC scopes required for userinfo. Example: "profile email"

None

auth.oidcAuth.prompt

The type of prompt to be used during authentication (none, consent, login or select_account)

select_account

auth.oidcAuth.clientID

Client ID given by the OIDC provider for K10

None

auth.oidcAuth.clientSecret

Client secret given by the OIDC provider for K10

None

auth.oidcAuth.usernameClaim

The claim to be used as the username

sub

auth.oidcAuth.usernamePrefix

Prefix that has to be used with the username obtained from the username claim

None

auth.oidcAuth.groupClaim

Name of a custom OpenID Connect claim for specifying user groups

None

auth.oidcAuth.groupPrefix

All groups will be prefixed with this value to prevent conflicts

None

auth.openshift.enabled

Enables access to the K10 dashboard by authenticating with the OpenShift OAuth server

false

auth.openshift.serviceAccount

Name of the service account that represents an OAuth client

None

auth.openshift.clientSecret

The token corresponding to the service account

None

auth.openshift.dashboardURL

The URL used for accessing K10's dashboard

None

auth.openshift.openshiftURL

The URL for accessing OpenShift's API server

None

auth.openshift.insecureCA

To turn off SSL verification of connections to OpenShift

false

auth.openshift.useServiceAccountCA

Set this to true to use the CA certificate corresponding to the Service Account auth.openshift.serviceAccount usually found at /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt

false

auth.ldap.enabled

Configures Active Directory/LDAP based authentication for the K10 dashboard

false

auth.ldap.restartPod

To force a restart of the authentication service pod (useful when updating authentication config)

false

auth.ldap.dashboardURL

The URL used for accessing K10's dashboard

None

auth.ldap.host

Host and optional port of the AD/LDAP server in the form host:port

None

auth.ldap.insecureNoSSL

Required if the AD/LDAP host is not using TLS

false

auth.ldap.insecureSkipVerifySSL

To turn off SSL verification of connections to the AD/LDAP host

false

auth.ldap.startTLS

When set to true, ldap:// is used to connect to the server followed by creation of a TLS session. When set to false, ldaps:// is used.

false

auth.ldap.bindDN

The Distinguished Name(username) used for connecting to the AD/LDAP host

None

auth.ldap.bindPW

The password corresponding to the bindDN for connecting to the AD/LDAP host

None

auth.ldap.bindPWSecretName

The name of the secret that contains the password corresponding to the bindDN for connecting to the AD/LDAP host

None

auth.ldap.userSearch.baseDN

The base Distinguished Name to start the AD/LDAP search from

None

auth.ldap.userSearch.filter

Optional filter to apply when searching the directory

None

auth.ldap.userSearch.username

Attribute used for comparing user entries when searching the directory

None

auth.ldap.userSearch.idAttr

AD/LDAP attribute in a user's entry that should map to the user ID field in a token

None

auth.ldap.userSearch.emailAttr

AD/LDAP attribute in a user's entry that should map to the email field in a token

None

auth.ldap.userSearch.nameAttr

AD/LDAP attribute in a user's entry that should map to the name field in a token

None

auth.ldap.userSearch.preferredUsernameAttr

AD/LDAP attribute in a user's entry that should map to the preferred_username field in a token

None

auth.ldap.groupSearch.baseDN

The base Distinguished Name to start the AD/LDAP group search from

None

auth.ldap.groupSearch.filter

Optional filter to apply when searching the directory for groups

None

auth.ldap.groupSearch.nameAttr

The AD/LDAP attribute that represents a group's name in the directory

None

auth.ldap.groupSearch.userMatchers

List of field pairs that are used to match a user to a group.

None

auth.ldap.groupSearch.userMatchers.userAttr

Attribute in the user's entry that must match with the groupAttr while searching for groups

None

auth.ldap.groupSearch.userMatchers.groupAttr

Attribute in the group's entry that must match with the userAttr while searching for groups

None

auth.groupAllowList

A list of groups whose members are allowed access to K10's dashboard

None

services.securityContext

Custom security context for K10 service containers

{"runAsUser" : 1000, "fsGroup": 1000}

services.securityContext.runAsUser

User ID K10 service containers run as

1000

services.securityContext.runAsGroup

Group ID K10 service containers run as

1000

services.securityContext.fsGroup

FSGroup that owns K10 service container volumes

1000

injectKanisterSidecar.enabled

Enable Kanister sidecar injection for workload pods

false

injectKanisterSidecar.namespaceSelector.matchLabels

Set of labels to select namespaces in which sidecar injection is enabled for workloads

{}

injectKanisterSidecar.objectSelector.matchLabels

Set of labels to filter workload objects in which the sidecar is injected

{}

injectKanisterSidecar.webhookServer.port

Port number on which the mutating webhook server accepts request

8080

gateway.insecureDisableSSLVerify

Specifies whether to disable SSL verification for gateway pods

false

genericVolumeSnapshot.resources.[requests|limits].[cpu|memory]

Resource requests and limits for Generic Volume Snapshot restore pods

{}

prometheus.server.enabled

If false, K10 Prometheus server will not be created. k10 dashboard will not function properly if this option is set to false

true

prometheus.server.persistentVolume.enabled

If true, K10 Prometheus server will create a Persistent Volume Claim

true

prometheus.server.persistentVolume.size

K10 Prometheus server data Persistent Volume size

30Gi

prometheus.server.persistentVolume.storageClass

StorageClassName used to create Prometheus PVC. Setting this option overwrites global StorageClass value

""

prometheus.server.retention

(optional) K10 Prometheus data retention

"30d"

prometheus.server.baseURL

(optional) K10 Prometheus external url path at which the server can be accessed

/k10/prometheus/

prometheus.server.prefixURL

(optional) K10 Prometheus prefix slug at which the server can be accessed

/k10/prometheus/

grafana.enabled

(optional) If false Grafana will not be available

true

grafana.prometheusPrefixURL

(optional) URL for Prometheus datasource in Grafana (must match prometheus.server.prefixURL)

/k10/prometheus/

resources.<podName>.<containerName>.[requests|limits].[cpu|memory]

Overwrite default K10 container resource requests and limits

varies by container

route.enabled

Specifies whether the K10 dashboard should be exposed via route

false

route.host

FQDN (e.g., .k10.example.com) for name-based virtual host

""

route.path

URL path for K10 Dashboard (e.g., /k10)

/

route.annotations

Additional Route object annotations

{}

route.labels

Additional Route object labels

{}

route.tls.enabled

Configures a TLS use for route.host

false

route.tls.insecureEdgeTerminationPolicy

Specifies behavior for insecure scheme traffic

Redirect

route.tls.termination

Specifies the TLS termination of the route

edge

apigateway.serviceResolver

Specifies the resolver used for service discovery in the API gateway (dns or endpoint)

dns

limiter.genericVolumeSnapshots

Limit of concurrent generic volume snapshot create operations

10

limiter.genericVolumeCopies

Limit of concurrent generic volume snapshot copy operations

10

limiter.genericVolumeRestores

Limit of concurrent generic volume snapshot restore operations

10

limiter.csiSnapshots

Limit of concurrent CSI snapshot create operations

10

limiter.providerSnapshots

Limit of concurrent cloud provider create operations

10

cluster.domainName

Specifies the domain name of the cluster

cluster.local

kanister.backupTimeout

Specifies timeout to set on Kanister backup operations

45

kanister.restoreTimeout

Specifies timeout to set on Kanister restore operations

600

kanister.deleteTimeout

Specifies timeout to set on Kanister delete operations

45

kanister.hookTimeout

Specifies timeout to set on Kanister pre-hook and post-hook operations

20

kanister.checkRepoTimeout

Specifies timeout to set on Kanister checkRepo operations

20

kanister.statsTimeout

Specifies timeout to set on Kanister stats operations

20

kanister.efsPostRestoreTimeout

Specifies timeout to set on Kanister efsPostRestore operations

45

awsConfig.assumeRoleDuration

Duration of a session token generated by AWS for an IAM role. The minimum value is 15 minutes and the maximum value is the maximum duration setting for that IAM role. For documentation about how to view and edit the maximum session duration for an IAM role see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use.html#id_roles_use_view-role-max-session. The value accepts a number along with a single character m(for minutes) or h (for hours) Examples: 60m or 2h

''

awsConfig.efsBackupVaultName

Specifies the AWS EFS backup vault name

k10vault

vmWare.taskTimeoutMin

Specifies the timeout for VMWare operations

60

encryption.primaryKey.awsCmkKeyId

Specifies the AWS CMK key ID for encrypting K10 Primary Key

None