Release Notes

2.5.23

Release Date: 2020-10-10

New Features

  • Additional metrics for catalog store and jobs service operations.

  • Introduced helm flag services.executor.hostNetwork that can be used to enable hostNetwork for executor pods.

  • Improved performance and scalability of exports and retirement of applications with large numbers of artifacts.

  • Added --json flag to k10offline list-images command to provide output in JSON format.

Bug Fixes

  • Fixed the logic that displays the dashboard loading animation.

  • Fixed issue with restore of Kanister protected application in different cluster.

  • The Dashboard UI has been updated with the latest storage regions for Azure and Google.

  • Fixed a bug where deleted applications were not being shown on the applications restore page.

Other Notes

  • Update vSphere documentation with CSI driver requirement.

2.5.22

Release Date: 2020-09-28

New Features

  • Exclude filters can now be applied to workloads that are not ready.

  • Added support for authenticating tokens present in Authorization Bearer Token Header.

  • Add policy label to prometheus metrics for K10 actions which can be used to filter by policy name.

  • Application details on Dashboard now include OpenShift Route resources.

Bug Fixes

  • Fixed the image tag of Dex reported by the k10offline tool.

  • Fixed issue with AWS backups, where user with correct role permissions was reported as unauthorized.

  • Fixed bug with cancel action workflow where cancelling resulted in error message.

  • Fix CrashLoopBackoff issues with the Config service when invalid Profiles were missing Secrets.

  • Fixed an issue that caused exports to repositories created before release 2.5.18 to fail.

  • Fixed retirement and restore of exported restore point for Kanister-enabled application.

  • Fixed potential race between snapshot GC and reusing a deleted directory in a snapshot

  • Fixed an issue where restores get stuck and eventually fail if K10 was deployed with the cacertconfigmap.name helm option and if the namespace where the application is being restored does not have a config map containing the root CA in it.

2.5.20

Release Date: 2020-09-14

New Features

  • Add metrics for catalog store operations.

  • Improved performance and scalability for applications with large numbers of artifacts.

  • Added the ability to set runAsUser option for K10Primer.

Bug Fixes

  • Fixed a bug where the Application list was not getting refreshed on the UI dashboard if the underlying resource watcher encountered a timeout.

  • The K10 dashboard in deployments with no authentication configured or Basic authentication mode did not show an option to create a namespace during restore.

  • Fixed K10Primer issue when validating a CSI provisioner.

  • K10Primer uses the CSI snapshots restoreSize when performing a restore.

  • Only backup StorageClass when taking a snapshot of a PersistentVolumeClaim.

  • Airgapped installation issue, where Prometheus pod was not coming into running state, has been fixed.

  • Fixed issue with handling errors when creating backups by exporting snapshots.

  • On the manual snapshot form, we now show "insufficient permissions" instead of "no options" on the profiles dropdown when the user does not have permissions to list profiles.

2.5.19

Release Date: 2020-09-05

New Features

  • Do not display the Data card on the dashboard for "Basic" users.

  • etcd backup for Kubernetes clusters installed via kubeadm is now supported.

  • etcd backup for OpenShift Container Platform clusters is now supported.

  • Improved response times for expensive queries from the dashboard by using an authorization cache.

  • Users will only be able to see the names of applications or namespaces on the dashboard they have access to.

Bug Fixes

  • Added support for additional GKE regions- asia-northeast2, asia-northeast3, asia-southeast2, us-west3 and us-west4.

  • The persistentvolumeclaims resource is now visible in the resource drop down list.

  • Fix an issue with the Helm chart that would break installs initiated through the Rancher dashboard.

  • Fixed potential index compaction issue that would resurrect deleted content entries during full maintenance

  • Restoring into a new namespace now works correctly for RBAC users with and without namespace creation permissions.

Known Issues

  • Checking a basic user's permissions may be slow when they first login to clusters with a large number of namespaces. This may cause up to a 30 second delay in loading some UI elements such as applications, actions, and compliance information.

Other Notes

  • Increase default memory request for disaster recovery sidecar.

2.5.18

Release Date: 2020-08-31

New Features

  • Change the default timeout for restore operations (from 90 minutes to 10 hours) since users can now cancel stuck jobs via the API.

  • K10 now supports cancellation of in-progress actions through Dashboard and API.

  • Added compliance data stats to Prometheus metrics.

  • K10 dashboard can now authenticate against the built-in OAuth server in OpenShift Container Platform environments.

  • Dashboard editor dialog windows now support text search.

  • The Dashboard data page now only shows application data for namespaces the user has permissions to view.

  • Switch service discovery to use Kubernetes DNS by default and provide an optional Helm setting to use Kubernetes endpoints in environments where DNS is disabled or not working.

  • Added the ability to create an infrastructure profile for vSphere.

  • Added a new Dashboard infrastructure profile type for vSphere configuration.

Bug Fixes

  • Adds missing metrics for retire actions, running actions, and pending actions.

  • Fixed restore of application with generic storage backup and Kanister blueprint with hooks.

  • An issue (required volume not mounted) that occurred in case of multi replica workload during GVS is fixed

  • Added validations for vSphere credentials.

  • vSphere persistent volumes are no longer left in 'Failed' state upon claim deletion.

Known Issues

  • vSphere persistent volumes are left in a 'Released' state. Fixed in v2.0.0 of vSphere's external-provisioner.

2.5.17

Release Date: 2020-08-16

New Features

  • On the policy form, values for exported snapshot retention can be "Set to Zero" with a new action link.

  • K10 deployments now have default resource requests for memory and CPU.

  • Added support for OpenShift 4.4 and 4.5, and Kubernetes 1.18

  • Resource requests and limits can be set by Helm values for K10 deployments.

Bug Fixes

  • Fix PodSpecOverride while restoring applications using Generic Volume Snapshot.

  • Correctly display total/retired artifact counts.

  • Fix problem when backup fails with unready workloads despite ignoreExceptions being set.

  • Fixed timeout issue when restoring CSI backups from an object store.

  • Fixes the retirement of restore points that contain both Kanister-protected workloads and Generic Volume Snapshots in the same restore point.

Known Issues

  • When the authentication service is restarted due to upgrades, manual restarts or errors, users might see 403 errors while accessing the dashboard due to scheduling issues in the gateway service. Restarting the gateway service should resolve the 403 errors.

Deprecations

  • Removed support for OpenShift 4.2 and Kubernetes 1.13.

Other Notes

  • Reduce DeleteSnapshot scope for AWS IAM permissions.

  • Support for OpenShift 4.3 and Kubernetes 1.14 will be removed in an upcoming release.

  • Kanister Blueprints that implement the backup action must return at least one output artifact if they want K10 to invoke a delete action upon restore point retirement.

2.5.16

Release Date: 2020-08-02

New Features

  • Use Kubernetes Endpoints for service discovery instead of cluster DNS.

  • Add license compliance information in prometheus metrics. So that compliance, with respect to time, can be seen in the dashboard.

  • Support token authentication mode with OAuth proxy for OpenShift clusters.

  • Added support for Portworx infrastructure profiles.

  • Added direct (non-CSI) support for Portworx storage.

  • Applications for policies can be selected via wildcard selectors.

  • Added support to create an OpenShift Route object to connect to the K10 dashboard.

Bug Fixes

  • Fixed a Dashboard bug where the retry of a backup action omitted profile info and resulted in failed actions.

  • Fixed issue where restore from a generic-volume-snapshot could result in multiple PVC restore processes.

  • Fixed an issue where an application creation via the OpenShift console fails when Kanister sidecar injection feature is enabled.

  • Fixes issue where export action fails when policy selects no applications to snapshot.

Known Issues

  • ManagedFields in API objects are not preserved when taking backups. Introduced in Kubernetes 1.18.x (OpenShift 4.5.x), they track the actor in the system who last modified each field in an API object. They are used by server-side apply report conflicting patches to objects. Since these are omitted by K10, executing server-side apply after an application is restored may result in different behavior than before restore.

  • Sidecar injection for generic volume snapshots is not supported in Kubernetes 1.18+ or OpenShift 4.5+. Do not use the helm value injectKanisterSidecar.enabled=true on these versions.

  • When K10 is deployed with the helm option auth.tokenAuth.enabled set to true, and when OAuth proxy is used for authentication, the OAuth proxy session is not cleared when the user signs out of the dashboard.

  • PersistentVolumes provisioned by K10 on vSphere do not get removed when they are released.

Upgrade Notes

  • The K10 Helm chart options persistence are moved under global.persistence. Setting global.persistence.storageClass now overwrites default StorageClass for Prometheus PVCs.

2.5.15

Release Date: 2020-07-20

New Features

  • K10 automatically adds the k10.kasten.io/forcegenericbackup="true" annotation to selected workloads to enforce generic backups when the Kanister sidecar injection feature is enabled.

  • Dashboard now shows DR restore progress and displays suggested actions with failure messages.

  • When OIDC based authentication is enabled, if K10 is not able to get the user's information from the OIDC token, it will use the provider's userinfo endpoint to get it.

Bug Fixes

  • Fixed an issue where an injected Kanister sidecar was failing on OpenShift due to a root SecurityContext. An injected Kanister sidecar's SecurityContext is copied from the primary container.

  • If configured authentication method is basic, ignore any authentication cookie in requests.

Other Notes

  • Document how to run Prometheus with a specific user and group ID.

2.5.14

Release Date: 2020-07-07

New Features

  • Added ability to add licenses using the Dashboard.

  • Validate OpenStack Cinder profiles upon creation.

  • Added ability to remove a license using the Dashboard.

  • Improve catalog storage utilization and reduce DR resource and time requirements by performing catalog pruning.

  • K10 Disaster Recovery now performs app-consistent backups of the K10 catalog store.

  • K10 now creates an export restore point whenever a snapshot is exported. This includes when a policy is used to copy snapshots to another region.

  • Exported restore points are now visible in the API as RestorePoint resources in the namespace of the snapshot RestorePoint resource as well as being RestorePointContent resources.

  • K10 policies that select multiple applications now copy each application independently and export all successfully copied applications by default. Application copy errors are noted as exceptions in the ExportAction and an export restore point is not created for an application with a copy error.

  • When manually restoring an application, the UI will provide the option of adding transforms previously used to restore that application.

  • Added support for taking generic snapshots of DeploymentConfigs

  • New OIDC-related settings - auth.oidcAuth.groupClaim and auth.oidcAuth.groupPrefix have been added to K10's Helm chart.

  • Add a Helm option to allow modification of the K10 service security context.

  • Additional statistics are collected for backups exported to object storage.

  • Simplify license updates and deletes. No Helm upgrade or patches required anymore.

  • Support pre-populated namespace labels in the policy creation form.

  • Improvement in the user experience on the dashboard when an OIDC provider returns an error.

  • New OIDC-related setting - auth.oidcAuth.prompt has been added to K10's helm chart.

  • Dashboard now supports specifying region for OpenStack infrastructure profiles.

  • Dashboard charts with multiple result sets are now customizable, allowing you to select which results to display.

  • Support page now displays an upgrade button when a newer version of K10 is available.

Bug Fixes

  • Fixed an issue where switching between K10 clusters while using kubectl proxy would result in a token validation error on the dashboard due to invalid cookies in the browser cache.

  • Fixed an issue that caused Kanister operations to fail when the subject of the Blueprint was an OpenShift DeploymentConfig.

  • Fixes bugs leading to early retirement of snapshots when using storage class overrides or independent retention counts.

  • Fixes bug with object storage data metrics when using the option to ignore exceptions for export.

  • Fixes bug with import after exporting snapshot data using the option to ignore exceptions for export.

  • When "View Action YAML" was clicked, the format of the action was not correct. This has been fixed.

  • Fixed a Dashboard bug when editing a transform that replaces a value with JSON.

  • Fixed cosmetic bug where object storage profiles with no region showed 'undefined' in profile dropdowns.

  • Fixed an issue with K10 installation when these options are used together - cacertconfigmap.name and auth.tokenAuth.enabled.

  • The K10 Helm chart now checks if ingress.annotations are set before using them.

  • Fixed issue with profile validation where the original error was being masked.

  • Fixed an issue where the APIServer was failing to call the mutating webhook endpoint on OpenShift clusters.

Security Issues

  • Module upgrades to address CVE-2020-14040.

  • Upgraded Ambassador to incorporate the Envoy 1.14.3 security update.

Upgrade Notes

  • A schema change is required and will reduce storage consumed by the K10 catalog.

Deprecations

  • The default docs location has changed to support documentation versioning.

2.5.13

Release Date: 2020-06-21

New Features

  • The Dashboard data page now displays object storage usage for each application.

  • Simplify K10 Disaster Recovery by not requiring the K10 cluster passphrase on recovery.

  • The Object Storage Data Usage will now include K10 Disaster Recovery statistics.

  • K10Primer tool will use the same node selector and tolerations for all test pods it creates.

  • Added a new settings tab for viewing installed licenses and license details.

  • The compliance and storage services have been merged with the dashboard service to reduce the total number of the pods required by K10.

  • Add hold support for policy-created backups.

  • Object storage usage metrics can now be viewed for individual applications.

  • New OIDC-related settings - auth.oidcAuth.usernameClaim and auth.oidcAuth.usernamePrefix have been added to K10's Helm chart.

  • Added progress bar to indicate when a Dashboard page is still being loaded.

  • Added support to automatically inject Kanister sidecars into pods for Generic Volume Backup. This can be done cluster-wide or, with label filtering, at the namespace or workload level.

  • Include skipped actions in prometheus metrics.

  • During manual snapshot or policy snapshot, added the ability to filter resources by label.

  • Added a support tab under settings on the Dashboard that displays information about the cluster, K10, and how to contact support.

Bug Fixes

  • Fixed an issue where Kanister actions would fail with OpenShift DeploymentConfig workloads.

  • Fixed a case where object storage data usage may not update immediately following a backup.

  • Fixed bug that prevented the test transform operation from displaying its results.

  • Improve error message when no OIDC configuration is discovered from the provider URL specified for OIDC authentication.

  • Fixed temporary metering service report creation errors when the service is restarted.

  • Fixed bug where policy delete did not remove policy from Dashboard.

  • Fixes issue with retiring restore points for policies that selected zero applications.

  • Fixes metering service bug when the Kubernetes API server is unresponsive.

Deprecations

  • The following labels have been removed from the metrics exposed by jobs service to Prometheus - job_id, phase, policy_id, scheduled_time, start_time, status, finish_time and attempt_count. jobs_running metric has also been removed to optimize storage consumption by Prometheus.

2.5.12

Release Date: 2020-06-07

New Features

  • Policies that select multiple applications treat application snapshot failures independently.

  • Allow users with only namespace access to create backups.

  • New CSI checker application that verifies CSI snapshot/restore capabilities.

  • Move EFS support out of preview mode.

  • K10Primer pre-check validates the existence of required CSI feature gate.

  • Failed backup jobs can now be retried from the Job Details panel.

  • YAML for jobs can now be viewed and copied to the clipboard from the Job Details panel.

  • Add support for Kubernetes auditing.

  • Add guidance for K10 resource requirements.

  • Added support for Kubernetes 1.17 and Beta Snapshot CRDs.

  • K10 can be used with an OpenID Connect(OIDC) provider irrespective of whether the Kubernetes cluster is configured with the same OIDC provider, a different OIDC provider, or no authentication system. K10 achieves this by using Kubernetes User Impersonation.

  • All PVCs within a namespace are snapshotted, independent of being linked to a workload.

  • Added the ability to use pre-made example transforms on the Dashboard.

Bug Fixes

  • Fixed an issue where backup restore points were not displaying the volume snapshot as a selectable artifact on the Dashboard.

  • Disable RBAC resource creation for the Prometheus server which would not work in OCP 3.11 clusters.

  • Fixed compliance calculation issue when using policies with advanced frequency options.

  • Fixed a bug where transform JSON field does not retain its value when editing.

Security Issues

  • When K10 is deployed with OIDC, user-initiated actions (via the API, CLI, or the dashboard) will be attributed to the user instead of the K10 service account.

Upgrade Notes

  • The k10-dashboard-view ClusterRole has been updated and renamed to k10-config-view. Check and update bindings for users and service accounts.

Deprecations

  • We only support Helm v2.16.0+ from this release.

2.5.11

Release Date: 2020-05-29

New Features

  • Added the ability to filter policies by name.

  • Improved the display of job errors by surfacing the nested root cause messages.

Bug Fixes

  • Dashboard login page now accepts a variety of authentication tokens versus only JWT tokens.

2.5.10

Release Date: 2020-05-29

New Features

  • No longer require a VolumeSnapshotClass with Retain deletion policy.

  • Workaround EFS's behavior where a restored instance is placed in a child directory by moving child directory's contents to the file system's root after restore.

  • Adjust namespace metadata for cloned Helm 3 applications.

  • Adding a new tool, K10Primer, that validates a Kubernetes cluster prior to installing K10

  • Installation of trusted but private root certificate authorities to be used by K10 for verifying TLS connections to object stores.

Bug Fixes

  • Exclude VolumeSnapshot objects in application backup.

  • Cleanup VolumeSnapshot resources if the driver failed the snapshot operation.

  • Fixed an issue that caused temporary secrets to be left behind after computing object store data usage statistics.

  • Fixes issue where multiple VolumeSnapshotClasses with K10 annotations caused snapshot failures.

  • Recreate provisioner annotations for Ceph-RBD provisioned Persistent Volumes on restore.

  • Updated prometheus's baseURL and prefixURL Helm values to work with K10 routes.

  • Fix backup data charts not populating on dashboard.

  • Fixed problem deleting old actions.

Known Issues

  • Kanister Blueprints used for database-level application backup currently do not work with private root CAs. An available workaround is to disable TLS verification of these object stores for Location profiles in use with Kanister.

Upgrade Notes

  • This release will perform a catalog schema upgrade.

  • We no longer require or recommend a Retain deletionPolicy for VolumeSnapshotClasses.

2.5.9

Release Date: 2020-05-10

New Features

  • The air-gapped installation process was simplified.

  • Improve the display of job errors by showing error details in a modal window with color syntax-highlighting.

  • Generate skipped jobs when policy scheduler offline across scheduling window.

Bug Fixes

  • Added Ceph profile validation on create.

  • The K10 dashboard will not allow the creation of policies or profiles if the K10 install namespace is not known.

  • Fixed a bug where the code editor window sometimes displayed unformatted code.

  • Fixed a bug that prevented K10 disaster recovery from a manual run of the disaster recovery policy.

  • Fixed a bug that caused some restores to fail after K10 Disaster Recovery.

  • Fixed a bug that could cause object store logical data size to be under-reported.

Security Issues

  • Upgraded several JavaScript packages to address recently disclosed CVEs.

Upgrade Notes

  • K10 image comes with Ceph tools enabled.

Other Notes

  • Combined the policy and profile services to reduce the number of pods used by K10.

2.5.8

Release Date: 2020-05-02

New Features

  • Add AWS Africa (Cape Town) and Europe (Milan) regions.

  • Added infrastructure profiles for direct (non-CSI) integration with Ceph and OpenStack Cinder.

  • Added ability to pause scheduled runs of policies.

  • Support specifying a region when an endpoint is used with S3 compatible Profiles.

  • Retention of snapshots and exported backups supports pausing and editing of policies.

Bug Fixes

  • Discover AWS region from node labels when EC2 instance metadata endpoint is not reachable.

  • EBS snapshot jobs fail gracefully if AWS credentials are not provided.

  • Fixed a bug in the k10-ns-admin Role for GET permission on secrets in the K10 namespace.

  • Fixed unlikely case where manual policy run could retire artifacts created by a scheduled policy run.

2.5.7

Release Date: 2020-04-27

New Features

  • Allow specifying which StorageClass should be used when exporting snapshot data.

  • Volume type transforms on restore are now supported for Azure Disks. Supported storage account types of Azure Disks include Standard_LRS, Premium_LRS, StandardSSD_LRS, UltraSSD_LRS.

  • Policy scheduler now waits until the next scheduled time after a policy edit to start a new job.

  • Reduction in space consumed by the metering service (used for cloud market place billing). The service will delete legacy data and will ensure new data is not retained indefinitely.

  • Added finer control of policy frequency, start times, and snapshot retention to the K10 API and Dashboard.

  • Force a file-system level backup if a workload has the k10.kasten.io/forcegenericbackup annotation.

Bug Fixes

  • Improve Ceph snapshot mechanism.

  • Fixed perpetual UI alerts on outdated failed K10 service/pods.

  • Fixed a bug that caused Object Storage Data Usage statistics to be inaccurate.

  • Fixed a bug with S3-compatible Location profiles. K10 used transport layer security by default even if the user specified http:// as the transport protocol in the location profile's endpoint.

Upgrade Notes

  • Kanister profiles are being deprecated. Disable and re-enable any existing DR policy after an upgrade to switch to using a Location profile.

Deprecations

  • For storage providers that are not supported by K10, do not automatically attempt a file-system backup unless the workload has the k10.kasten.io/forcegenericbackup annotation.

Other Notes

  • Enabled zoom for documentation images.

2.5.6

Release Date: 2020-04-19

Bug Fixes

  • Fixed a bug that caused K10 DR backups to fail after a successful retirement of DR snapshots.

2.5.5

Release Date: 2020-04-17

New Features

  • Add a Prometheus metric to indicate if K10 DR is enabled.

  • New pre-flight script to validate CSI Snapshot capability.

  • Ability to transform PersistentVolumeClaim labels on restore.

Bug Fixes

  • Fixed a bug that caused failure in retirement of K10 Disaster Recovery snapshots.

  • Fix issue with Azure profiles incompatibility while creating import policies.

  • Resolves early retirement of artifacts after K10 disaster recovery.

  • Preserve PersistentVolumeClaim labels on restore.

  • Fixed UI bug that prevented import/restore policy creation.

Other Notes

  • Workaround documented for the migration of EFS CSI Volumes in EKS clusters using the K10 dashboard and AWS CLI/Console.

  • Increase timeout for waiting for ready pods to 15 minutes.

2.5.4

Release Date: 2020-04-12

New Features

  • Support specifying destination region (Azure, AWS) and account (AWS) when exporting snapshots.

  • Added the ability to define TLS certificates in the K10 ingress definition. This allows the use an external ingress controller and definition of a custom FQDN to access the K10 platform through the HTTPS protocol.

  • Reduced the number of Kubernetes workloads by combining the jobs and jobs queue services.

  • Consolidate Profiles into a new type: Location Profiles.

  • Reduction in memory consumed by the metering service (used for cloud marketplace billing).

  • New policies now wait until the first scheduled time to run. Use a manual policy run before then if desired.

  • API support for offset policy run times (e.g., choose an hour to run a daily backup).

  • Volume type transforms on restore are now supported for GCE Persistent Disk and AWS EBS. Supported types for GCE Persistent Disk include pd-ssd and pd-standard. Supported types for AWS EBS include standard, io1, gp2, sc1 and st1.

Bug Fixes

  • Fix cryptography service failing to start when the catalog service isn't yet available.

  • Fixed a bug in the DR Restore tool when no skipResource argument was specified.

Known Issues

  • The Object Storage Data Usage statistics may not be completely accurate.

Upgrade Notes

  • Existing import and export profiles will be converted to location profiles automatically.