Advanced Install Options

Starter Edition and Licensing

By default, K10 comes with an embedded Starter Edition license. The starter edition license allows you to use the software on a cluster with at most 50 worker nodes for one month and then 10 nodes after the one month period. The starter edition license embedded in each release is valid for 12 months after which an update to the latest released version will be required which will extend the license. You can remove the node restriction by updating to Enterprise Edition and obtaining the appropriate license from the Kasten team.

Using A Custom License During Install

To install a license provided by Kasten that removes the node restriction, please add the following to any of the helm install commands. K10 dynamically retrieves the license key and a pod restart is not required.

...
          --set license=<license-text>

Changing Licenses

To add a new license to K10, a secret needs to be created in the K10 namespace (default is kasten-io) with the requirement that the license text be set in a field named license. To do this from the command line, run:

$ kubectl create secret generic <license-secret-name> \
    --namespace kasten-io \
    --from-literal=license="$(echo '<base64_encoded_license>' | base64 --decode)"

Multiple license secrets can exist simultaneously and K10 will check if any are valid. This license check is done periodically and so, no K10 restarts are required if a different existing license becomes required (e.g., due to a cluster expansion or an old license expiry) or when a new license is added.

The resulting license will look like:

 apiVersion: v1
 data:
   license: Y3Vz...
 kind: Secret
 metadata:
   creationTimestamp: "2020-04-14T23:50:05Z"
   labels:
     app: k10
     app.kubernetes.io/instance: k10
     app.kubernetes.io/managed-by: Helm
     app.kubernetes.io/name: k10
     helm.sh/chart: k10-2.5.23
     heritage: Helm
     release: k10
   name: k10-custom-license
   namespace: kasten-io
 type: Opaque

Similarly, old licenses can be removed by deleting the secret that contains it.

$ kubectl delete secret <license-secret-name> \
    --namespace kasten-io

Manually Creating or Using an Existing Service Account

The following instructions can be used to create a new Service Account that grants K10 the required permissions to Kubernetes resources and the use the given Service Account as a part of the install process. The instructions assume that you will be installing K10 in the kasten-io namespace.

# Create kasten-io namespace if have not done it yet.
$ kubectl create namespace kasten-io

# Create a ServiceAccount for k10 k10-sa
$ kubectl --namespace kasten-io create sa k10-sa

# Create a cluster role binding for k10-sa
$ kubectl create clusterrolebinding k10-sa-rb \
    --clusterrole cluster-admin \
    --serviceaccount=kasten-io:k10-sa

Following the SA creation, you can install K10 using:

$ helm install k10 kasten/k10 --namespace=kasten-io \
    --set rbac.create=false \
    --set serviceAccount.create=false \
    --set serviceAccount.name=k10-sa
$ helm install kasten/k10 --name=k10 \
    --set rbac.create=false \
    --set serviceAccount.create=false \
    --set serviceAccount.name=k10-sa

Pinning K10 to Specific Nodes

While not generally recommended, there might be situations (e.g., test environments, nodes reserved for infrastructure tools, or clusters without autoscaling enabled) where K10 might need to be pinned to a subset of nodes in your cluster. You can do this easily with an existing deployment by using a combination of NodeSelectors and Taints and Tolerations.

The process to modify a deployment to accomplish this is demonstrated in the following example. The example assumes that the nodes you want to restrict K10 to has the label selector-key: selector-value and a taint set to taint-key=taint-value:NoSchedule.

$ cat << EOF > patch.yaml
spec:
  template:
    spec:
      nodeSelector:
        selector-key: selector-value
      tolerations:
      - key: "taint-key"
        operator: "Equal"
        value: "taint-value"
        effect: "NoSchedule"
EOF

$ kubectl get deployment --namespace kasten-k10 | awk 'FNR == 1 {next} {print $1}' \
  | xargs -I DEP kubectl patch deployments DEP --namepsace kasten-k10 --patch "$(cat patch.yaml)"

Using Trusted Root Certificate Authority Certificates for TLS

Note

For temporary testing of object storage systems that are deployed using self-signed certificates signed by a trusted Root CA, it is also possible to disable certificate verification if the Root CA certificate is not easily available.

If the S3-compatible object store configured in a Location Profile was deployed with a self-signed certificate that was signed by a trusted Root Certificate Authority (Root CA), then the certificate for such a certificate authority has to be provided to K10 to enable successful verification of TLS connections to the object store.

Similarly, to authenticate with a private OIDC provider whose self-signed certificate was signed by a trusted Root CA, the certificate for the Root CA has to be provided to K10 to enable successful verification of TLS connections to the OIDC provider.

Multiple Root CAs can be bundled together in the same file.

Install Root CA in K10's namespace

Assuming K10 will be deployed in the kasten-io namespace, the following instructions will make a private Root CA certificate available to K10.

Note

The name of the Root CA certificate must be custom-ca-bundle.pem

# Create a ConfigMap that will contain the certificate.
# Choose any name for the ConfigMap
$ kubectl --namespace kasten-io create configmap custom-ca-bundle-store --from-file=custom-ca-bundle.pem

To provide the Root CA certificate to K10, add the following to the Helm install command.

--set cacertconfigmap.name=<name-of-the-configmap>

Install Root CA in application's namespace when using Kanister sidecar

If you either use K10's Kanister sidecar injection feature for injecting the Kanister sidecar in your application's namespace or if you have manually added the Kanister sidecar, you must create a ConfigMap containing the Root CA in the application's namespace and update the application's specification so that the ConfigMap is mounted as a Volume. This will enable the Kanister sidecar to verify TLS connections successfully using the Root CA in the ConfigMap.

Assuming that the application's namespace is named test-app, use the following command to create a ConfigMap containing the Root CA in the application's namespace:

Note

The name of the Root CA certificate must be custom-ca-bundle.pem

# Create a ConfigMap that will contain the certificate.
# Choose any name for the ConfigMap
$ kubectl --namespace test-app create configmap custom-ca-bundle-store --from-file=custom-ca-bundle.pem

This is an example of a VolumeMount that must be added to the application's specification.

- name: custom-ca-bundle-store
  mountPath: "/etc/ssl/certs/custom-ca-bundle.pem"
  subPath: custom-ca-bundle.pem

This is an example of a Volume that must be added to the application's specification.

- name: custom-ca-bundle-store
  configMap:
    name: custom-ca-bundle-store

Troubleshooting

If K10 is deployed without the cacertconfigmap.name setting, validation failures such as the one shown below will be seen while configuring a Location Profile using the web based user interface.

In the absence of the cacertconfigmap.name setting, authentication with a private OIDC provider will fail. K10's logs will show an error x509: certificate signed by unknown authority.

If you do not install the Root CA in the application namespace when using a Kanister sidecar with the application, the logs will show an error x509: certificate signed by unknown authority when the sidecar tries to connect to any endpoint that requires TLS verification.

Running K10 Containers as a Specific User

K10 service containers run with UID and fsGroup 1000 by default. If the storage class K10 is configured to use for its own services requires the containers to run as a specific user, then the user can be modified.

This is often needed when using shared storage, such as NFS, where permissions on the target storage require a specific user.

To run as a specific user (e.g., root (0), add the following to the Helm install command:

--set services.securityContext.runAsUser=0 \
--set services.securityContext.fsGroup=0 \
--set prometheus.server.securityContext.runAsUser=0 \
--set prometheus.server.securityContext.runAsGroup=0 \
--set prometheus.server.securityContext.runAsNonRoot=false \
--set prometheus.server.securityContext.fsGroup=0

Other SecurityContext settings for the K10 service containers can be specified using the --set service.securityContext.<setting name> and --set prometheus.server.securityContext.<setting name> options.

Using Kubernetes Endpoints for Service Discovery

The K10 API gateway uses Kubernetes DNS to discover and route requests to K10 services.

If Kubernetes DNS is disabled or not working, K10 can be configured to use Kubernetes endpoints for service discovery.

To do this, add the following to the Helm install command:

--set apigateway.serviceResolver=endpoint

Complete List of K10 Helm Options

The following table lists the configurable parameters of the K10 chart and their default values.

Parameter

Description

Default

eula.accept

Whether to enable accept EULA before installation

false

eula.company

Company name. Required field if EULA is accepted

None

eula.email

Contact email. Required field if EULA is accepted

None

license

License string obtained from Kasten

None

rbac.create

Whether to enable RBAC with a specific cluster role and binding for K10

true

scc.create

Whether to create a SecurityContextConstraints for K10 ServiceAccounts

false

serviceAccount.create

Specifies whether a ServiceAccount should be created

true

serviceAccount.name

The name of the ServiceAccount to use. If not set, a name is derived using the release and chart names.

None

ingress.create

Specifies whether the K10 dashboard should be exposed via ingress

false

ingress.class

Cluster ingress controller class: nginx, GCE

None

ingress.host

FQDN (e.g., .k10.example.com) for name-based virtual host

None

ingress.urlPath

URL path for K10 Dashboard (e.g., /k10)

Release.Name

ingress.annotations

Additional Ingress object annotations

{}

ingress.tls.enabled

Configures a TLS use for ingress.host

false

ingress.tls.secretName

Specifies a name of TLS secret

None

global.persistence.enabled

Use PVS to persist data

true

global.persistence.size

Size of an volume for K10 persistent services

20Gi

global.persistence.storageClass

Specified StorageClassName will be used for PVCs

None

global.airgapped.repository

Specify the helm repository for offline (airgapped) installation

''

secrets.awsAccessKeyId

AWS access key ID (required for AWS deployment)

None

secrets.awsSecretAccessKey

AWS access key secret

None

secrets.awsIamRole

ARN of the AWS IAM role assumed by K10 to perform any AWS operation.

None

secrets.googleApiKey

Non-default base64 encoded GCP Service Account key file

None

secrets.azureTenantId

Azure tenant ID (required for Azure deployment)

None

secrets.azureClientId

Azure Service App ID

None

secrets.azureClientSecret

Azure Service APP secret

None

secrets.ibmSoftLayerApiKey

IBM SoftLayer API key

None

secrets.ibmSoftLayerApiUsername

IBM SoftLayer API username

None

secrets.vsphereEndpoint

vSphere endpoint for login

None

secrets.vsphereUsername

vSphere username for login

None

secrets.vspherePassword

vSphere password for login

None

secrets.dockerConfigPath

Use --set-file secrets.dockerConfigPath=path_to_docker_config.yaml to specify docker config for image pull

None

cacertconfigmap.name

Name of the ConfigMap that contains a certificate for a trusted root certificate authority

None

clusterName

Cluster name for better logs visibility

None

metering.mode

Control license reporting (set to airgap for private-network installs)

None

externalGateway.create

Configures an external gateway for K10 API services

false

externalGateway.annotations

Standard annotations for the services

None

externalGateway.fqdn.name

Domain name for the K10 API services

None

externalGateway.fqdn.type

Supported gateway type: route53-mapper or external-dns

None

externalGateway.awsSSLCertARN

ARN for the AWS ACM SSL certificate used in the K10 API server

None

auth.basicAuth.enabled

Configures basic authentication for the K10 dashboard

false

auth.basicAuth.htpasswd

A username and password pair separated by a colon character

None

auth.basicAuth.secretName

Name of an existing Secret that contains a file generated with htpasswd

None

auth.tokenAuth.enabled

Configures token based authentication for the K10 dashboard

false

auth.oidcAuth.enabled

Configures Open ID Connect based authentication for the K10 dashboard

false

auth.oidcAuth.providerURL

URL for the OIDC Provider

None

auth.oidcAuth.redirectURL

URL to the K10 gateway service

None

auth.oidcAuth.scopes

Space separated OIDC scopes required for userinfo. Example: "profile email"

None

auth.oidcAuth.prompt

The type of prompt to be used during authentication (none, consent, login or select_account)

select_account

auth.oidcAuth.clientID

Client ID given by the OIDC provider for K10

None

auth.oidcAuth.clientSecret

Client secret given by the OIDC provider for K10

None

auth.oidcAuth.usernameClaim

The claim to be used as the username

sub

auth.oidcAuth.usernamePrefix

Prefix that has to be used with the username obtained from the username claim

None

auth.oidcAuth.groupClaim

Name of a custom OpenID Connect claim for specifying user groups

None

auth.oidcAuth.groupPrefix

All groups will be prefixed with this value to prevent conflicts

None

auth.openshift.enabled

Enables access to the K10 dashboard by authenticating with the OpenShift OAuth server

false

auth.openshift.serviceAccount

Name of the service account that represents an OAuth client

None

auth.openshift.clientSecret

The token corresponding to the service account

None

auth.openshift.dashboardURL

The URL used for accessing K10's dashboard

None

auth.openshift.openshiftURL

The URL for accessing OpenShift's API server

None

auth.openshift.insecureCA

To enable/disable TLS verification of connections to the OpenShift API Server

false

services.securityContext

Custom security context for K10 service containers

{"runAsUser" : 1000, "fsGroup": 1000}

services.securityContext.runAsUser

User ID K10 service containers run as

1000

services.securityContext.runAsGroup

Group ID K10 service containers run as

1000

services.securityContext.fsGroup

FSGroup that owns K10 service container volumes

1000

injectKanisterSidecar.enabled

Enable Kanister sidecar injection for workload pods

false

injectKanisterSidecar.namespaceSelector.matchLabels

Set of labels to select namespaces in which sidecar injection is enabled for workloads

{}

injectKanisterSidecar.objectSelector.matchLabels

Set of labels to filter workload objects in which the sidecar is injected

{}

injectKanisterSidecar.webhookServer.port

Port number on which the mutating webhook server accepts request

8080

prometheus.enabled

If false, K10 Prometheus server will not be created

true

prometheus.server.persistentVolume.enabled

If true, K10 Prometheus server will create a Persistent Volume Claim

true

prometheus.server.persistentVolume.size

K10 Prometheus server data Persistent Volume size

30Gi

prometheus.server.persistentVolume.storageClass

StorageClassName used to create Prometheus PVC. Setting this option overwrites global StorageClass value

""

prometheus.server.retention

(optional) K10 Prometheus data retention

"30d"

resources.<podName>.<containerName>.[requests|limits].[cpu|memory]

Overwrite default K10 container resource requests and limits

varies by container

route.enabled

Specifies whether the K10 dashboard should be exposed via route

false

route.host

FQDN (e.g., .k10.example.com) for name-based virtual host

""

route.path

URL path for K10 Dashboard (e.g., /k10)

/

route.annotations

Additional Route object annotations

{}

route.labels

Additional Route object labels

{}

route.tls.enabled

Configures a TLS use for route.host

false

route.tls.insecureEdgeTerminationPolicy

Specifies behavior for insecure scheme traffic

Redirect

route.tls.termination

Specifies the TLS termination of the route

edge

apigateway.serviceResolver

Specifies the resolver used for service discovery in the API gateway (dns or endpoint)

dns