Advanced Install Options

FREE K10 Edition and Licensing

By default, K10 comes with an embedded free Edition license. The free edition license allows you to use the software on a cluster with at most 50 worker nodes for one month and then 10 nodes after the one month period. The free edition license embedded in each release is valid for 12 months after which an update to the latest released version will be required which will extend the license. You can remove the node restriction by updating to Enterprise Edition and obtaining the appropriate license from the Kasten team.

Using A Custom License During Install

To install a license provided by Kasten that removes the node restriction, please add the following to any of the helm install commands. K10 dynamically retrieves the license key and a pod restart is not required.

...
          --set license=<license-text>

Changing Licenses

To add a new license to K10, a secret needs to be created in the K10 namespace (default is kasten-io) with the requirement that the license text be set in a field named license. To do this from the command line, run:

$ kubectl create secret generic <license-secret-name> \
    --namespace kasten-io \
    --from-literal=license="$(echo '<base64_encoded_license>' | base64 --decode)"

Multiple license secrets can exist simultaneously and K10 will check if any are valid. This license check is done periodically and so, no K10 restarts are required if a different existing license becomes required (e.g., due to a cluster expansion or an old license expiry) or when a new license is added.

The resulting license will look like:

 apiVersion: v1
 data:
   license: Y3Vz...
 kind: Secret
 metadata:
   creationTimestamp: "2020-04-14T23:50:05Z"
   labels:
     app: k10
     app.kubernetes.io/instance: k10
     app.kubernetes.io/managed-by: Helm
     app.kubernetes.io/name: k10
     helm.sh/chart: k10-4.0.5
     heritage: Helm
     release: k10
   name: k10-custom-license
   namespace: kasten-io
 type: Opaque

Similarly, old licenses can be removed by deleting the secret that contains it.

$ kubectl delete secret <license-secret-name> \
    --namespace kasten-io

License Grace period

If the license status of the cluster becomes invalid (e.g., the licensed node limit is exceeded), the ability to perform manual actions or creating new policies will be disabled but your previously scheduled policies will continue to run for 50 days. The displayed warning will be look like:

By default, K10 provides a grace period of 50 days to ensure that applications remain protected while a new license is obtained or the cluster is brought back into compliance by reducing the number of nodes. K10 will stop the creation of any new jobs (scheduled or manual) after the grace period expires.

If the cluster's license status frequently swaps between valid and invalid states, the amount of time the cluster license spends in an invalid status will be subtracted from subsequent grace periods.

Manually Creating or Using an Existing Service Account

The following instructions can be used to create a new Service Account that grants K10 the required permissions to Kubernetes resources and the use the given Service Account as a part of the install process. The instructions assume that you will be installing K10 in the kasten-io namespace.

# Create kasten-io namespace if have not done it yet.
$ kubectl create namespace kasten-io

# Create a ServiceAccount for k10 k10-sa
$ kubectl --namespace kasten-io create sa k10-sa

# Create a cluster role binding for k10-sa
$ kubectl create clusterrolebinding k10-sa-rb \
    --clusterrole cluster-admin \
    --serviceaccount=kasten-io:k10-sa

Following the SA creation, you can install K10 using:

$ helm install k10 kasten/k10 --namespace=kasten-io \
    --set rbac.create=false \
    --set serviceAccount.create=false \
    --set serviceAccount.name=k10-sa

Using Red Hat certified upstream container images

The upstream container images (e.g., Ambassador, Dex) that K10 uses, by default, are not Red Hat certified images. If it's required to run the Red Hat certified version of those images, this helm flag can be used:

--set global.upstreamCertifiedImages=true

Pinning K10 to Specific Nodes

While not generally recommended, there might be situations (e.g., test environments, nodes reserved for infrastructure tools, or clusters without autoscaling enabled) where K10 might need to be pinned to a subset of nodes in your cluster. You can do this easily with an existing deployment by using a combination of NodeSelectors and Taints and Tolerations.

The process to modify a deployment to accomplish this is demonstrated in the following example. The example assumes that the nodes you want to restrict K10 to has the label selector-key: selector-value and a taint set to taint-key=taint-value:NoSchedule.

$ cat << EOF > patch.yaml
spec:
  template:
    spec:
      nodeSelector:
        selector-key: selector-value
      tolerations:
      - key: "taint-key"
        operator: "Equal"
        value: "taint-value"
        effect: "NoSchedule"
EOF

$ kubectl get deployment --namespace kasten-k10 | awk 'FNR == 1 {next} {print $1}' \
  | xargs -I DEP kubectl patch deployments DEP --namepsace kasten-k10 --patch "$(cat patch.yaml)"

Using Trusted Root Certificate Authority Certificates for TLS

Note

For temporary testing of object storage systems that are deployed using self-signed certificates signed by a trusted Root CA, it is also possible to disable certificate verification if the Root CA certificate is not easily available.

If the S3-compatible object store configured in a Location Profile was deployed with a self-signed certificate that was signed by a trusted Root Certificate Authority (Root CA), then the certificate for such a certificate authority has to be provided to K10 to enable successful verification of TLS connections to the object store.

Similarly, to authenticate with a private OIDC provider whose self-signed certificate was signed by a trusted Root CA, the certificate for the Root CA has to be provided to K10 to enable successful verification of TLS connections to the OIDC provider.

Multiple Root CAs can be bundled together in the same file.

Install Root CA in K10's namespace

Assuming K10 will be deployed in the kasten-io namespace, the following instructions will make a private Root CA certificate available to K10.

Note

The name of the Root CA certificate must be custom-ca-bundle.pem

# Create a ConfigMap that will contain the certificate.
# Choose any name for the ConfigMap
$ kubectl --namespace kasten-io create configmap custom-ca-bundle-store --from-file=custom-ca-bundle.pem

To provide the Root CA certificate to K10, add the following to the Helm install command.

--set cacertconfigmap.name=<name-of-the-configmap>

Install Root CA in application's namespace when using Kanister sidecar

If you either use K10's Kanister sidecar injection feature for injecting the Kanister sidecar in your application's namespace or if you have manually added the Kanister sidecar, you must create a ConfigMap containing the Root CA in the application's namespace and update the application's specification so that the ConfigMap is mounted as a Volume. This will enable the Kanister sidecar to verify TLS connections successfully using the Root CA in the ConfigMap.

Assuming that the application's namespace is named test-app, use the following command to create a ConfigMap containing the Root CA in the application's namespace:

Note

The name of the Root CA certificate must be custom-ca-bundle.pem

# Create a ConfigMap that will contain the certificate.
# Choose any name for the ConfigMap
$ kubectl --namespace test-app create configmap custom-ca-bundle-store --from-file=custom-ca-bundle.pem

This is an example of a VolumeMount that must be added to the application's specification.

- name: custom-ca-bundle-store
  mountPath: "/etc/ssl/certs/custom-ca-bundle.pem"
  subPath: custom-ca-bundle.pem

This is an example of a Volume that must be added to the application's specification.

- name: custom-ca-bundle-store
  configMap:
    name: custom-ca-bundle-store

Troubleshooting

If K10 is deployed without the cacertconfigmap.name setting, validation failures such as the one shown below will be seen while configuring a Location Profile using the web based user interface.

In the absence of the cacertconfigmap.name setting, authentication with a private OIDC provider will fail. K10's logs will show an error x509: certificate signed by unknown authority.

If you do not install the Root CA in the application namespace when using a Kanister sidecar with the application, the logs will show an error x509: certificate signed by unknown authority when the sidecar tries to connect to any endpoint that requires TLS verification.

Running K10 Containers as a Specific User

K10 service containers run with UID and fsGroup 1000 by default. If the storage class K10 is configured to use for its own services requires the containers to run as a specific user, then the user can be modified.

This is often needed when using shared storage, such as NFS, where permissions on the target storage require a specific user.

To run as a specific user (e.g., root (0), add the following to the Helm install command:

--set services.securityContext.runAsUser=0 \
--set services.securityContext.fsGroup=0 \
--set prometheus.server.securityContext.runAsUser=0 \
--set prometheus.server.securityContext.runAsGroup=0 \
--set prometheus.server.securityContext.runAsNonRoot=false \
--set prometheus.server.securityContext.fsGroup=0

Other SecurityContext settings for the K10 service containers can be specified using the --set service.securityContext.<setting name> and --set prometheus.server.securityContext.<setting name> options.

Using Kubernetes Endpoints for Service Discovery

The K10 API gateway uses Kubernetes DNS to discover and route requests to K10 services.

If Kubernetes DNS is disabled or not working, K10 can be configured to use Kubernetes endpoints for service discovery.

To do this, add the following to the Helm install command:

--set apigateway.serviceResolver=endpoint

Configuring Prometheus

Prometheus is an open-source system monitoring and alerting toolkit. K10 is bundled with Prometheus chart version 14.1.1 . Any of the Prometheus Chart values can be used during the K10 helm installation.

When passing value from the command line, the value key has to be prefixed with the prometheus. string:

--set prometheus.server.persistentVolume.storageClass=default.sc

When passing values in a YAML file, all prometheus settings should be under the prometheus key:

# values.yaml
# global values - apply to both K10 and prometheus
global:
  persistence:
    storageClass: default-sc
# K10 specific settings
auth:
  basicAuth: enabled
# prometheus specific settings
prometheus:
  server:
    persistentVolume:
      storageClass: another-sc

Complete List of K10 Helm Options

The following table lists the configurable parameters of the K10 chart and their default values.

Parameter	Description	Default
eula.accept	Whether to enable accept EULA before installation	false
eula.company	Company name. Required field if EULA is accepted	None
eula.email	Contact email. Required field if EULA is accepted	None
license	License string obtained from Kasten	None
rbac.create	Whether to enable RBAC with a specific cluster role and binding for K10	true
scc.create	Whether to create a SecurityContextConstraints for K10 ServiceAccounts	false
services.dashboardbff.hostNetwork	Whether the dashboardbff pods may use the node network	false
services.executor.hostNetwork	Whether the executor pods may use the node network	false
serviceAccount.create	Specifies whether a ServiceAccount should be created	true
serviceAccount.name	The name of the ServiceAccount to use. If not set, a name is derived using the release and chart names.	None
ingress.create	Specifies whether the K10 dashboard should be exposed via ingress	false
ingress.class	Cluster ingress controller class: nginx, GCE	None
ingress.host	FQDN (e.g., k10.example.com) for name-based virtual host	None
ingress.urlPath	URL path for K10 Dashboard (e.g., /k10)	Release.Name
ingress.annotations	Additional Ingress object annotations	{}
ingress.tls.enabled	Configures a TLS use for ingress.host	false
ingress.tls.secretName	Specifies a name of TLS secret	None
global.persistence.enabled	Use PVS to persist data	true
global.persistence.size	Default global size of volumes for K10 persistent services	20Gi
global.persistence.catalog.size	Size of a volume for catalog service	global.persistence.size
global.persistence.jobs.size	Size of a volume for jobs service	global.persistence.size
global.persistence.logging.size	Size of a volume for logging service	global.persistence.size
global.persistence.metering.size	Size of a volume for metering service	global.persistence.size
global.persistence.storageClass	Specified StorageClassName will be used for PVCs	None
global.airgapped.repository	Specify the helm repository for offline (airgapped) installation	''
secrets.awsAccessKeyId	AWS access key ID (required for AWS deployment)	None
secrets.awsSecretAccessKey	AWS access key secret	None
secrets.awsIamRole	ARN of the AWS IAM role assumed by K10 to perform any AWS operation.	None
secrets.googleApiKey	Non-default base64 encoded GCP Service Account key file	None
secrets.azureTenantId	Azure tenant ID (required for Azure deployment)	None
secrets.azureClientId	Azure Service App ID	None
secrets.azureClientSecret	Azure Service APP secret	None
secrets.azureResourceGroup	Resource Group name that was created for the Kubernetes cluster	None
secrets.azureSubscriptionID	Subscription ID in your Azure tenant	None
secrets.azureResourceMgrEndpoint	Resource management endpoint for the Azure Stack instance	None
secrets.azureADEndpoint	Azure Active Directory login endpoint	None
secrets.azureADResourceID	Azure Active Directory resource ID to obtain AD tokens	None
secrets.vsphereEndpoint	vSphere endpoint for login	None
secrets.vsphereUsername	vSphere username for login	None
secrets.vspherePassword	vSphere password for login	None
secrets.dockerConfigPath	Use --set-file secrets.dockerConfigPath=path_to_docker_config.yaml to specify docker config for image pull	None
cacertconfigmap.name	Name of the ConfigMap that contains a certificate for a trusted root certificate authority	None
clusterName	Cluster name for better logs visibility	None
metering.mode	Control license reporting (set to airgap for private-network installs)	None
externalGateway.create	Configures an external gateway for K10 API services	false
externalGateway.annotations	Standard annotations for the services	None
externalGateway.fqdn.name	Domain name for the K10 API services	None
externalGateway.fqdn.type	Supported gateway type: route53-mapper or external-dns	None
externalGateway.awsSSLCertARN	ARN for the AWS ACM SSL certificate used in the K10 API server	None
auth.basicAuth.enabled	Configures basic authentication for the K10 dashboard	false
auth.basicAuth.htpasswd	A username and password pair separated by a colon character	None
auth.basicAuth.secretName	Name of an existing Secret that contains a file generated with htpasswd	None
auth.k10AdminGroups	A list of groups whose members are granted admin level access to K10's dashboard	None
auth.k10AdminUsers	A list of users who are granted admin level access to K10's dashboard	None
auth.tokenAuth.enabled	Configures token based authentication for the K10 dashboard	false
auth.oidcAuth.enabled	Configures Open ID Connect based authentication for the K10 dashboard	false
auth.oidcAuth.providerURL	URL for the OIDC Provider	None
auth.oidcAuth.redirectURL	URL to the K10 gateway service	None
auth.oidcAuth.scopes	Space separated OIDC scopes required for userinfo. Example: "profile email"	None
auth.oidcAuth.prompt	The type of prompt to be used during authentication (none, consent, login or select_account)	select_account
auth.oidcAuth.clientID	Client ID given by the OIDC provider for K10	None
auth.oidcAuth.clientSecret	Client secret given by the OIDC provider for K10	None
auth.oidcAuth.usernameClaim	The claim to be used as the username	sub
auth.oidcAuth.usernamePrefix	Prefix that has to be used with the username obtained from the username claim	None
auth.oidcAuth.groupClaim	Name of a custom OpenID Connect claim for specifying user groups	None
auth.oidcAuth.groupPrefix	All groups will be prefixed with this value to prevent conflicts	None
auth.openshift.enabled	Enables access to the K10 dashboard by authenticating with the OpenShift OAuth server	false
auth.openshift.serviceAccount	Name of the service account that represents an OAuth client	None
auth.openshift.clientSecret	The token corresponding to the service account	None
auth.openshift.dashboardURL	The URL used for accessing K10's dashboard	None
auth.openshift.openshiftURL	The URL for accessing OpenShift's API server	None
auth.openshift.insecureCA	To turn off SSL verification of connections to OpenShift	false
auth.openshift.useServiceAccountCA	Set this to true to use the CA certificate corresponding to the Service Account auth.openshift.serviceAccount usually found at /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt	false
auth.ldap.enabled	Configures Active Directory/LDAP based authentication for the K10 dashboard	false
auth.ldap.restartPod	To force a restart of the authentication service pod (useful when updating authentication config)	false
auth.ldap.dashboardURL	The URL used for accessing K10's dashboard	None
auth.ldap.host	Host and optional port of the AD/LDAP server in the form host:port	None
auth.ldap.insecureNoSSL	Required if the AD/LDAP host is not using TLS	false
auth.ldap.insecureSkipVerifySSL	To turn off SSL verification of connections to the AD/LDAP host	false
auth.ldap.startTLS	When set to true, ldap:// is used to connect to the server followed by creation of a TLS session. When set to false, ldaps:// is used.	false
auth.ldap.bindDN	The Distinguished Name(username) used for connecting to the AD/LDAP host	None
auth.ldap.bindPW	The password corresponding to the bindDN for connecting to the AD/LDAP host	None
auth.ldap.bindPWSecretName	The name of the secret that contains the password corresponding to the bindDN for connecting to the AD/LDAP host	None
auth.ldap.userSearch.baseDN	The base Distinguished Name to start the AD/LDAP search from	None
auth.ldap.userSearch.filter	Optional filter to apply when searching the directory	None
auth.ldap.userSearch.username	Attribute used for comparing user entries when searching the directory	None
auth.ldap.userSearch.idAttr	AD/LDAP attribute in a user's entry that should map to the user ID field in a token	None
auth.ldap.userSearch.emailAttr	AD/LDAP attribute in a user's entry that should map to the email field in a token	None
auth.ldap.userSearch.nameAttr	AD/LDAP attribute in a user's entry that should map to the name field in a token	None
auth.ldap.userSearch.preferredUsernameAttr	AD/LDAP attribute in a user's entry that should map to the preferred_username field in a token	None
auth.ldap.groupSearch.baseDN	The base Distinguished Name to start the AD/LDAP group search from	None
auth.ldap.groupSearch.filter	Optional filter to apply when searching the directory for groups	None
auth.ldap.groupSearch.nameAttr	The AD/LDAP attribute that represents a group's name in the directory	None
auth.ldap.groupSearch.userMatchers	List of field pairs that are used to match a user to a group.	None
auth.ldap.groupSearch.userMatchers.userAttr	Attribute in the user's entry that must match with the groupAttr while searching for groups	None
auth.ldap.groupSearch.userMatchers.groupAttr	Attribute in the group's entry that must match with the userAttr while searching for groups	None
auth.groupAllowList	A list of groups whose members are allowed access to K10's dashboard	None
services.securityContext	Custom security context for K10 service containers	{"runAsUser" : 1000, "fsGroup": 1000}
services.securityContext.runAsUser	User ID K10 service containers run as	1000
services.securityContext.runAsGroup	Group ID K10 service containers run as	1000
services.securityContext.fsGroup	FSGroup that owns K10 service container volumes	1000
injectKanisterSidecar.enabled	Enable Kanister sidecar injection for workload pods	false
injectKanisterSidecar.namespaceSelector.matchLabels	Set of labels to select namespaces in which sidecar injection is enabled for workloads	{}
injectKanisterSidecar.objectSelector.matchLabels	Set of labels to filter workload objects in which the sidecar is injected	{}
injectKanisterSidecar.webhookServer.port	Port number on which the mutating webhook server accepts request	8080
gateway.insecureDisableSSLVerify	Specifies whether to disable SSL verification for gateway pods	false
genericVolumeSnapshot.resources.[requests|limits].[cpu|memory]	Resource requests and limits for Generic Volume Snapshot restore pods	{}
prometheus.server.enabled	If false, K10 Prometheus server will not be created. k10 dashboard will not function properly if this option is set to false	true
prometheus.server.persistentVolume.enabled	If true, K10 Prometheus server will create a Persistent Volume Claim	true
prometheus.server.persistentVolume.size	K10 Prometheus server data Persistent Volume size	30Gi
prometheus.server.persistentVolume.storageClass	StorageClassName used to create Prometheus PVC. Setting this option overwrites global StorageClass value	""
prometheus.server.retention	(optional) K10 Prometheus data retention	"30d"
resources.<podName>.<containerName>.[requests|limits].[cpu|memory]	Overwrite default K10 container resource requests and limits	varies by container
route.enabled	Specifies whether the K10 dashboard should be exposed via route	false
route.host	FQDN (e.g., .k10.example.com) for name-based virtual host	""
route.path	URL path for K10 Dashboard (e.g., /k10)	/
route.annotations	Additional Route object annotations	{}
route.labels	Additional Route object labels	{}
route.tls.enabled	Configures a TLS use for route.host	false
route.tls.insecureEdgeTerminationPolicy	Specifies behavior for insecure scheme traffic	Redirect
route.tls.termination	Specifies the TLS termination of the route	edge
apigateway.serviceResolver	Specifies the resolver used for service discovery in the API gateway (dns or endpoint)	dns
limiter.genericVolumeSnapshots	Limit of concurrent generic volume snapshot create operations	10
limiter.genericVolumeCopies	Limit of concurrent generic volume snapshot copy operations	10
limiter.genericVolumeRestores	Limit of concurrent generic volume snapshot restore operations	10
limiter.csiSnapshots	Limit of concurrent CSI snapshot create operations	10
limiter.providerSnapshots	Limit of concurrent cloud provider create operations	10
cluster.domainName	Specifies the domain name of the cluster	cluster.local