Transforms
Transforms enable modifications to Kubernetes resources on restore. Restore actions may include transforms to modify resources prior to creation in the target environment.
Transforms are loosely modeled on JSON Patch (RFC-6902) which structures patches into patch documents containing patch operations. Transforms follow a similar structure to JSON Patch, with transform documents containing transform commands. Transforms deviates from JSON Patch in its support for regular expressions and path globbing.
To take advantage of Kubernetes native RBAC or reuse transforms, they can be aggregated into TransformSet custom resources. Follow Transform Sets page to learn more.
Transform Structure
A transform document is composed of transformation commands to be performed
on a subject
. Each transformation command within the document is
performed in sequence. Processing halts if a transform command results in
an error. If the error is the result of a test
failure processing will
continue to the next document, otherwise the restore will fail.
The example below contains two transform documents that contains two commands.
The first command, test
, checks for the presence of the
'metadata/labels/release'
element within a deployment resource. The
copy
command following the test
will execute if the test succeeds.
The second document will be evaluated independent of the first document.
The second document replaces the "/metadata/labels/component"
key value
with "webserver"
only if the value is "nginx"
.
transforms:
# Group, version, resource and/or name of the resource for transformation.
# Every transform should have at least one resource specified.
- subject:
resource: deployments
# The name of the transform. Required.
name: 'copyRelease'
# ``json`` key below indicates JSON patch document object used to perform
# a JSON patch-like transform. Required.
json:
- op: test
# reference for operation. mandatory.
path: '/metadata/labels/release'
- op: copy
# source reference for operation. only valid for the 'move' and 'copy' operations
from: '/metadata/labels/release'
# target reference for operation. mandatory.
path: '/spec/template/metadata/labels/release'
# apply regex to match expression. Valid for 'replace', 'move',
# 'copy' and 'test'.
regex: 'prod-v.*'
# value is any json structure. only valid for 'add', 'replace'
# and 'test' operation
value: 'prod'
#
# Second transform document also applies transform to deployments.
# Transforms on a resource are guaranteed to be applied in the order
# specified in the restore action.
#
# Group, version, resource and/or name of the resource for transformation.
# Every transform should have at least one resource specified.
- subject:
resource: deployments
# The name of the transform. Required.
name: 'replaceLabelComponent'
# ``json`` key below indicates JSON patch document object used to perform
# a JSON patch-like transform. Required.
json:
- op: test
# reference for operation. mandatory.
path: '/metadata/labels/component'
- op: replace
# reference for operation. mandatory.
path: '/metadata/labels/component'
# use regex to test for presence of element value for replacement
regex: 'nginx'
# value is any json structure. only valid for 'add', 'replace'
# and 'test' operation
value: 'webserver'
Transforms supports six command operations test
, add
, remove
,
copy
, move
and replace
:
test
checks that an element exists (and equals the value / matches the regexp if specified).
add
inserts a new element to the resource definition.
remove
deletes an existing element from the resource definition.
copy
duplicates an element, overwriting the value in the new path if it already exists.
move
relocates an element, overwriting the value in the new path if it already exists.
replace
replaces an existing element with a new element.
The subject
property, as used in the example above, is used to transform
a subset of resources on restore. Only those resources matching the
group
, version
, resource
or name
will be selected from the
restore-point. Any empty subject
property will match all values. The
subject
above matches all deployment resources with any name, for any
group or version.
Each command has an operation, op
, and a path
. op
specifies the
command operation to perform, path
references the element within the
resource to operate on.
copy
and move
operations contain two paths - from
for the source
element of the operation and path
for the destination element of the
operation.
value
contains either an element for comparison or a new element for
inclusion. Similarly regex
can be used for comparing string elements or
processing elements into new string elements.
Transform Validation Requirements
To create a valid transform, it is required to match the following requirements:
Transform's name should not be empty
Transform should contain the spec
Transform should contain at least one operation
Transform should have at least one resource specified
Paths
Every command has at least one path that references element(s) to operate
on. path
and from
are the only command properties that contain paths.
Paths consist of keys delineated by slashes (/
). Keys can be object
property names or array indexes. For example,
"/spec/template/spec/containers/0"
references the first container,
nginx
, within the Deployment resource in the
example below.
Note that every path should be absolute, starting with /
.
When adding an element to an array, "-"
can be used in the path
to
add an element at the and of the array.
Per RFC-6901, paths may contain
character escapes that allow inclusion of /
characters within path
keys. For example "/metadata/annotations/k10~1injectKanisterSidecar"
will reference the k10/injectKanisterSidecar
key on the annotations
object within the resource metadata. ~
characters may be included by
using ~0
.
Regex and Value
The function of value
and regex
properties depend on the operation
performed:
test
withvalue
compares the element atpath
tovalue
for an exact match.
test
withregex
compares the string element atpath
to the regular expression inregex
copy
andmove
that include aregex
property operate only onfrom
string elements that match the regular expression inregex
.
copy
andmove
that include aregex
and avalue
property operate only onfrom
string elements that matchregex
and replace destinationpath
elements with expandedvalue
strings. Capturing groups invalue
are replaced to produce new values fromregex
expressions.
replace
that include aregex
property operate only onpath
string elements that match the regular expression inregex
.
replace
that include aregex
and avalue
property operate only onpath
string elements that matchregex
and replace destination elements with expandedvalue
strings. Capturing groups invalue
are replaced to produce new values fromregex
expressions.
value
and regex
do not apply to the remove
and add
operations. A test
operation with both value
and regex
properties will produce and error.
See re2 for the syntax accepted by regex
.
Capturing groups are referenced in value
with identifiers starting with
a dollar-sign, "$", followed by the index of the capturing group.
The following example copies the mountPath
from the first
volumeMounts
entry to the second using regex
and value
with
capturing group replacement. The example uses the
Deployment resource below. The parent
path, /etc/nginx
from mountPath
is preserved and copied into the
second volumeMounts
as /etc/nginx/config
.
op: copy
from: "spec/template/spec/containers/0/volumeMounts/0/mountPath"
path: "spec/template/spec/containers/0/volumeMounts/1/mountPath"
regex: "([a-z/]+)/ssl"
value: "$1/config"
Path Wildcards
Path wildcards allow path
or from
to apply to sets of elements. A
wildcard can be used in place of a path key to refer to all keys of an
object or array. Valid wildcards are *
and **
- *
references
all keys within one element, **
references all keys across multiple
consecutive elements.
Using the deployment example below, all spec
app
labels can be accessed with "spec/**/app"
, expanding to two
concrete paths:
[
"/spec/selector/matchLabels/app"
"/spec/template/metadata/labels/app"
]
In the example below, all metadata labels can
be accessed with the path "**/metadata/labels/*"
expanding to the four
concrete paths below:
[
"/metadata/labels/app"
"/metadata/labels/version"
"/metadata/labels/release"
"/spec/template/metadata/labels/app"
]
Reference groups can be used in the path
property of the copy
and
move
operations to form new groups of destination paths. Reference
groups allow for constructing new destination paths from source paths
with wildcards. Reference groups have keys that start with $ followed by
a numeric index of the wildcard. The index refers the wildcard ordinal,
counting from one. During processing, the reference group will be replaced
with concrete values of the indexed wildcard.
The following copy command example copies all selector matchLabels
to
the spec template metadata.
op: copy
from: "/**/selector/matchLabels/*"
path: "/spec/template/metadata/labels/$2"
Transform Commands
All of the available command forms are listed below. Examples apply to the Deployment resource below.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
version: 1.7.9
release: canary
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: containers.example.com/nginx:1.7.9
ports:
- containerPort: 443
- containerPort: 80
volumeMounts:
- mountPath: /etc/nginx/ssl
name: secret-volume
- mountPath: /etc/nginx/conf.d
name: configmap-volume
Test
op: test
path: "/metadata/name"
Test that element at path
exists. path
may contain wildcards. If a
test fails, the remainder of the current transform document will be skipped
and processing of the next transform document will begin.
op: test
path: "/metadata/labels"
value: { app: nginx, release: "canary" }
Test that element at path
equals element in value
. Order of map
keys is not significant. Otherwise, set of map keys and values must
match exactly. Arrays must be the same size and elements are compared in
order. If path
contains wildcards, all elements referenced by path
must equal value
. If a test fails, the remaining commands in the current
transform document will be skipped and processing of the next transform
document will begin.
op: test
path: "/metadata/name"
regex: "nginx.*"
Test that string element at path
exists and matches regular expression
in regex
. path
may contain wildcards. If a test fails, the remaining
commands in the current transform document will be skipped and processing of
the next transform document will begin.
Add
op: add
path: "/metadata/label/release"
value: canary
Add value at path
. If path
references a map, the key will be
added or replaced. If path
references an array, a new value will
be inserted. value
must be a value JSON/YAML element. path
may
contain wildcards.
The transform will fail if the path
does not exist. The last key of
the path
may contain a new map key or an array index.
Remove
op: remove
path: "/metadata/label/release"
Remove element at path
. path
may contain wildcards.
The transform will fail if path
does not exist.
Copy
op: copy
from: "/metadata/label"
path: "/spec/selector/matchLabel"
Copy element at from
to path
. If path
references a map, element
will be added/replaced as in add
above. If path
references an array,
copy will insert element as in add
above. If from
contains wildcards,
path
may contain wildcard variables. If from
is a concrete path,
path
may contain wildcards.
The transform will fail if from
or path
does not exist. The last key
of path
may contain a new map key or an array index.
op: copy
from: "metadata/labels/app"
path: "metadata/labels/release"
regex: "(.*)"
value: "prod-$1"
Copy string element at from
to path
. If path
references a map,
node will be added/replaced as in add
above. If path
references is
array, copy will insert element as in add
above. If from
element
matches regular expression in regex
, perform copy of element with
replacement of capturing groups in value
. If from
contains wildcards,
path
may contain wildcard variables. If from
is a concrete path,
path
may contain wildcards.
The transform will fail if from
element does not exist or is not a string.
The transform will fail if path
does not exist. The last key of path
may contain a new map key or an array index.
Move
op: move
from: "/spec/template/spec/containers/1/port"
path: "/spec/template/spec/containers/0/port"
Move element at from
to path
. If path
references a map, element
will be added/replaced as in add
above. If path
references an array,
copy will insert element as in add
above.
The transform will fail if from
or path
does not exist. The last key
of path
may contain a new map key or an array index.
op: move
from: "/metadata/labels/version"
path: "/spec/template/metadata/labels/version"
regex: "(.*)"
value: "v$1"
Move string element at from
to path
. If path
references a map,
element will be added/replaced as in add
above. If path
references
an array, copy will insert element as in add
above. If regular
expression in regex
matches element at from
, perform replacement
of capturing groups in value
. If from
contains wildcards, path
may contain wildcard variables. If from
is a concrete path, path
may contain wildcards.
The transform will fail if from
element does not exist or is not a
string. Transform will fail if path
does not exist. The last key of
path
may contain a new map key or an array index.
Replace
op: replace
path: "/spec/replicas"
value: 5
Replace element at path
with the element in value
. path
must
reference an existing element. path
may contain wildcards.
The transform will fail if path
element does not exist.
op: replace
path: "/spec/template/spec/template/containers/0/image"
regex: ".*[/]([a-z/]+):([0-9.]+)"
value: "$2/$1"
Replace element at path
with string element in value
. If path
references a map, node will be added/replaced as in add
above. If
path
references is array, the element referenced will be replaced. If
element at path
matches regular expression in regex
, replace element
at path
with value. Any capturing groups in value
will be expanded
value
. path
may contain wildcard variables.
The transform will fail if path
element does not exist or is not a
string.
Transform Example
Transform to change the storage class on a persistent volume claim (pvc
):
transforms:
- subject:
resource: persistentvolumeclaims
name: replaceStorageClassName
json:
- op: replace
path: /spec/storageClassName
regex: ^ssd$
value: gp2
Resource to operate on:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pic-gallery
namespace: gallery-app
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi
storageClassName: ssd
volumeMode: Filesystem
Transformed resource:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pic-gallery
namespace: gallery-app
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2Gi
storageClassName: gp2
volumeMode: Filesystem