Template Controller
Template Controller documentation.
The Template Controller is a controller originating from the Kluctl project, but not limited to
Kluctl. It allows to define template objects which are rendered and applied into the cluster based on an input matrix.
In its easiest form, an ObjectTemplate takes one input object (e.g. a ConfigMap) and creates another object
(e.g. a Secret) which is then applied into the cluster.
The Template Controller also offers CRDs which allow to query external resources (e.g. GitHub Pull Requests) which can
then be used as inputs into ObjectTemplates.
Use Cases
Template Controller has many use case. Some are for example:
- Dynamic environments for Pull Requests
- Transformation of Secrets/Objects
Documentation
Reference documentation is available here.
The announcement blog post also contains valuable explanations
and examples.
Installation
Installation instructions can be found here
1 - Installation
Installation documentation
The Template Controller can currently be installed via static manifests or via Helm.
Static Manifests
kubectl apply -f "https://raw.githubusercontent.com/kluctl/template-controller/v0.9.2/deploy/manifests/template-controller.yaml"
Helm
A Helm Chart for the controller is available as well.
To install the controller via Helm, run:
$ helm install template-controller -n template-controller --create-namespace oci://ghcr.io/kluctl/charts/template-controller
The Helm Chart is only distributed as an OCI package. The old Helm Repository found at https://github.com/kluctl/charts
is not maintained anymore.
Upgrading from older Helm Charts
In case you were using the Helm Chart found at https://github.com/kluctl/charts, you’ll need to perform a few extra
steps before you can upgrade to the new OCI based Helm Charts.
Run the following commands while the correct Kubectl Context is set. Please replace <release-name> with the release
name and <release-namespace> with the namespace you used when installing the old Chart.
$ rn=<release-name>
$ ns=<release-namespace>
$ for i in $(kubectl get crd -oname | grep templates.kluctl.io); do kubectl label $i app.kubernetes.io/managed-by=Helm; done
$ for i in $(kubectl get crd -oname | grep templates.kluctl.io); do kubectl annotate $i meta.helm.sh/release-name=$rn; done
$ for i in $(kubectl get crd -oname | grep templates.kluctl.io); do kubectl annotate $i meta.helm.sh/release-namespace=$ns; done
After this, you can perform a normal upgrade using the new OCI Chart.
$ helm upgrade -n <release-namespace> <release-name> oci://ghcr.io/kluctl/charts/template-controller
2 - Specs
template-controller specs
2.1 - v1alpha1 specs
templates.kluctl.io/v1alpha1 documentation
templates.kluctl.io/v1alpha1
This is the v1alpha1 API specification for defining templating related resources.
Specification
Implementation
2.1.1 - ObjectTemplate
ObjectTemplate documentation
The ObjectTemplate API defines templates that are rendered based on a matrix of input values.
Example
apiVersion: v1
kind: ConfigMap
metadata:
name: input-configmap
namespace: default
data:
x: someValue
---
apiVersion: templates.kluctl.io/v1alpha1
kind: ObjectTemplate
metadata:
name: example-template
namespace: default
spec:
serviceAccountName: example-template-service-account
prune: true
matrix:
- name: input1
object:
ref:
apiVersion: v1
kind: ConfigMap
name: input-configmap
templates:
- object:
apiVersion: v1
kind: ConfigMap
metadata:
name: "templated-configmap"
data:
y: "{{ matrix.input1.x }}"
- raw: |
apiVersion: v1
kind: ConfigMap
metadata:
name: "templated-configmap-from-raw"
data:
z: "{{ matrix.input1.x }}"
The above manifests show a simple example that will create two ConfigMaps from one input ConfigMap. The individual fields
possible in ObjectTemplate are described further down.
Spec fields
The following fields are supported in spec.
serviceAccountName
ObjectTemplate requires a service account to access cluster objects. This is required when it gathers input objects
for the matrix and when it applies rendered objects. Please see security for some important notes!
For this to work, the referenced service account must have at least GET, CREATE and UPDATE permissions for
the involved objects and kinds. For the above example, the following service account would be enough:
apiVersion: v1
kind: ServiceAccount
metadata:
name: example-template-service-account
namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: example-template-service-account
namespace: default
rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: example-template-service-account
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: example-template-service-account
subjects:
- kind: ServiceAccount
name: example-template-service-account
namespace: default
interval
Specifies the interval at which the ObjectTemplate is reconciled.
suspend
If set to true, reconciliation is suspended.
prune
If true, the Template Controller will delete rendered objects when either the ObjectTemplate gets deleted or when
the rendered object disappears from the rendered objects list.
matrix
The matrix defines a list of matrix entries, which are then used as inputs into the templates. Each entry results in
a list of values associated with the entry name. All lists are then multiplied together to form the actual matrix of
input values.
Each matrix entry has a name, which is later used to identify the value in the template.
As an example, if you have two entries with simple lists with the following values:
matrix:
- name: input1
list:
- a: v1
b: v2
- name: input2
list:
- c: v3
d: v4
It will lead to the following matrix:
- input1:
a: v1
b: v2
input2:
c: v3
d: v4
Now take the following matrix example with an entry with two list items:
matrix:
- name: input1
list:
- a: v1
b: v2
- a: v1_2
b: v2_2
- name: input2
list:
- c: v3
d: v4
It will lead to the following matrix:
- input1:
a: v1
b: v2
input2:
c: v3
d: v4
- input1:
a: v1_2
b: v2_2
input2:
c: v3
d: v4
Each input value is then used as input when rendering the templates. In the above examples, it means that all templates
are rendered twice, once with matrix.input1 set to the first input value and the second time with the second input
value.
The following matrix entry types are supported:
list
This is the simplest form and represents a list of arbitrary objects. See the above examples.
Due to the use of controller-gen and an internal
limitation in regard to validation and CRD generation,
list elements must be objects at the moment. A future version of the Template Controller will support arbitrary values
(e.g. numbers and strings) as elements.
object
This refers an object on the cluster. The object is read by the controller and then used as an input value for the
matrix. Example:
matrix:
- name: input1
object:
ref:
apiVersion: v1
kind: ConfigMap
name: input-configmap
The referenced object can be of any kind, but the used service account must have access to the
referenced object. The read object is then wholly used as matrix input.
To only use a sub-part of the referenced object, set jsonPath to a valid JSON Path
pointing to the subfield(s) that you want to use. Example:
matrix:
- name: input1
object:
ref:
apiVersion: v1
kind: ConfigMap
name: input-configmap
jsonPath: .data
This will make the data field available as input instead of the full object, meaning that values can be used inside the
templates by simply referring {{ matrix.input1.my_key }} (no .data required).
In case you want to interpret a subfield as an input list instead of a single value, set expandLists to true.
Example:
matrix:
- name: input1
object:
ref:
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGithubPullRequests
name: list-gh-prs
jsonPath: status.pullRequests
expandLists: true
This will lead to one matrix input per list element at status.pullRequests instead of a single matrix input that
represents the list.
templates
templates is a list of template objects. Each template object is rendered and applied once per entry from the
multiplied matrix inputs. When rendering, the context contains the global variable matrix representing the current
entry. matrix has one member field per named matrix input.
In the lists example from above, this would for example give matrix.input1 and matrix.input2 for each render
invocation.
In case a template object is missing the namespace, it is set to the namespace of the ObjectTemplate object.
The service account used for the ObjectTemplate must have permissions to get and apply the
resulting objects.
There are currently two forms of template objects supported, object and raw. object is an inline object where
each string field is treated as independent template to render. raw represents one large (multi-line) string that
is rendered in one-go and then unmarshalled as yaml/json.
It is recommended to prefer object over raw and only revert to raw templates when you need to perform advanced
templating (e.g. {% if ... %} or other control structures) or when it is important to treat a field as non-string
(e.g. boolean or number) when unmarshalled into an object. An example for such case would be if you want to use a
template value for replicas of a Deployment, which MUST be a number.
Example for an object:
templates:
- object:
apiVersion: v1
kind: ConfigMap
metadata:
name: "templated-configmap"
data:
y: "{{ matrix.input1.x }}"
Example for a raw template object:
templates:
- raw: |
apiVersion: v1
kind: ConfigMap
metadata:
name: "templated-configmap-from-raw"
data:
z: "{{ matrix.input1.x }}"
See templating for more details on the templating engine.
2.1.2 - GitProjector
GitProjector documentation
The GitProjector API defines projections of Git repositories.
Projection of Git repositories means that the content of selected branches and selected files are loaded into Kubernetes,
accessible through the status of the GitProjector.
The projected branches and files can then be used as matrix inputs for an ObjectTemplate.
Example
apiVersion: templates.kluctl.io/v1alpha1
kind: GitProjector
metadata:
name: preview
namespace: default
spec:
interval: 1m
url: https://github.com/kluctl/kluctl-examples.git
# In case you use a private repository
secretRef:
name: git-credentials
ref:
branch: main
files:
- glob: "preview-envs/preview-*.yaml"
parseYaml: true
The above example creates a GitProjector that will periodically clone the kluctl-examples repo, look for the main
branch and all files matching the given glob. It will then parse all yamls and make them available through the
GitProjector’s status:
apiVersion: templates.kluctl.io/v1alpha1
kind: GitProjector
metadata:
name: preview
namespace: default
spec:
...
status:
allRefsHash: 104d3dc9b5ffabf5ba3c76532fb71da58757c494acdcb7dff3665d256f516612
conditions:
- lastTransitionTime: "2022-12-14T09:09:51Z"
message: Success
observedGeneration: 1
reason: Success
status: "True"
type: Ready
result:
- files:
- parsed:
- envName: preview-env1
replicas: 3
path: preview-envs/preview-env1.yaml
- parsed:
- envName: preview-env2
replicas: 1
path: preview-envs/preview-env2.yaml
ref:
branch: main
Spec fields
The following fields are supported in spec.
interval
Specifies the interval at which the GitProjector is reconciled.
suspend
If set to true, reconciliation is suspended.
url
The git url of the repository to project. Can either be a https or a git/ssh url.
ref
The git reference to project. Either spec.ref.branch or spec.ref.tag must be set.
Both tags and refs can be regular expressions. In case of a regular expression, the controller will include all matching
refs in the status.result field.
secretRef
Same as in the Kluctl Controllers KluctlDeployment
files
List of file to project into the status. Must be of the format:
...
spec:
...
files:
- glob: "my-file.yaml"
parseYaml: true
Each entry must at least contain a glob which is used to match files. The controller uses the https://github.com/gobwas/glob
library for pattern matching.
If parseYaml is set to true, the controller will try to parse matching files as yaml and include the parsed structured
data in the resulting status. Parsing of yaml is done with the assumption that all files possibly contain multiple yaml
documents, meaning that even yaml files with just a single document will result in a parsed list of one document.
Consider the following matching yaml file:
envName: preview-env1
replicas: 3
This will result in the following projection:
...
status:
result:
- files:
- parsed:
- envName: preview-env1
replicas: 3
path: preview-envs/preview-env1.yaml
ref:
branch: main
If parseYaml is false, the result will contain a raw string representation of the matching files:
...
status:
result:
- files:
- path: preview-envs/preview-env1.yaml
raw: |-
envName: preview-env1
replicas: 3
ref:
branch: main
2.1.3 - GithubComment
GithubComment documentation
The GithubComment API allows to post a comment to a GitHub Pull Request.
Example
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
namespace: default
data:
my-key: |
This can by **any** form of [Markdown](https://en.wikipedia.org/wiki/Markdown) supported by Github.
---
apiVersion: templates.kluctl.io/v1alpha1
kind: GithubComment
metadata:
name: comment-gh
namespace: default
spec:
github:
owner: my-org-or-user
repo: my-repo
pullRequestId: 1234
tokenRef:
secretName: git-credentials
key: github-token
comment:
source:
configMap:
name: my-configmap
key: my-key
The above example will post a comment to the specified pull request. The comment’s content is loaded from the ConfigMap
my-configmap. Other sources are also supported, see the source field documentation for details.
The comment will be updated whenever the underlying comment source changes.
Spec fields
suspend
If set to true, reconciliation of this comment is suspended.
github
Specifies which GitHub project and pull request to post the comment to.
github.owner
Specifies the user or organisation name where the repository is localed.
github.repo
Specifies the repository name to query PRs for.
github.tokenRef
In case of private repositories, this field can be used to specify a secret that contains a GitHub API token.
github.pullRequestId
Specifies the ID of the pull request.
This field specifies the necessary information for the comment content.
This optional field specifies the identifier to mark the comment with so that the controller can identify it. It
defaults to a generated id built from the namespace and name of the comment resource.
This specifies the comment source. Multiple source types are supported, specified via a sub-field.
Raw text for the template’s content. Example:
apiVersion: templates.kluctl.io/v1alpha1
kind: GithubComment
metadata:
name: comment-gh
namespace: default
spec:
github:
owner: my-org-or-user
repo: my-repo
pullRequestId: 1234
tokenRef:
secretName: git-credentials
key: github-token
comment:
source:
text: |
This can by **any** form of [Markdown](https://en.wikipedia.org/wiki/Markdown) supported by Github.
Uses a ConfigMap as source for the comment’s content. Example:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
namespace: default
data:
my-key: |
This can by **any** form of [Markdown](https://en.wikipedia.org/wiki/Markdown) supported by Github.
---
apiVersion: templates.kluctl.io/v1alpha1
kind: GithubComment
metadata:
name: comment-gh
namespace: default
spec:
github:
owner: my-org-or-user
repo: my-repo
pullRequestId: 1234
tokenRef:
secretName: git-credentials
key: github-token
comment:
source:
configMap:
name: my-configmap
key: my-key
Uses a TextTemplate as source for the comment’s content. Example:
apiVersion: templates.kluctl.io/v1alpha1
kind: TextTemplate
metadata:
name: my-texttemplate
namespace: default
spec:
inputs:
... # See TextTemplate documentation for details.
template: |
This can by **any** form of [Markdown](https://en.wikipedia.org/wiki/Markdown) supported by Github.
---
apiVersion: templates.kluctl.io/v1alpha1
kind: GithubComment
metadata:
name: comment-gh
namespace: default
spec:
github:
owner: my-org-or-user
repo: my-repo
pullRequestId: 1234
tokenRef:
secretName: git-credentials
key: github-token
comment:
source:
textTemplate:
name: my-texttemplate
2.1.4 - GitlabComment
GitlabComment documentation
The GitlabComment API allows to post a comment to a Gitlab Merge Request.
Example
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
namespace: default
data:
my-key: |
This can by **any** form of [Markdown](https://en.wikipedia.org/wiki/Markdown) supported by Gitlab.
---
apiVersion: templates.kluctl.io/v1alpha1
kind: GitlabComment
metadata:
name: comment-gl
namespace: default
spec:
gitlab:
project: my-group/my-repo
mergeRequestId: 1234
tokenRef:
secretName: git-credentials
key: gitlab-token
comment:
source:
configMap:
name: my-configmap
key: my-key
The above example will post a comment to the specified pull request. The comment’s content is loaded from the ConfigMap
my-configmap. Other sources are also supported, see the source field documentation for details.
The comment will be updated whenever the underlying comment source changes.
Spec fields
suspend
If set to true, reconciliation of this comment is suspended.
gitlab
Specifies which Gitlab project and merge request to post the comment to.
gitlab.project
Specifies the user or organisation name where the repository is localed.
gitlab.repo
Specifies the repository name to query PRs for.
gitlab.tokenRef
In case of private repositories, this field can be used to specify a secret that contains a Gitlab API token.
github.pullRequestId
Specifies the ID of the pull request.
Same as in GithubComment
2.1.5 - ListGithubPullRequests
ListGithubPullRequests documentation
The ListGithubPullRequests API allows to query the GitHub API for a list of pull requests (PRs). These PRs
can be filtered when needed. The resulting list of PRs is written into the status of the
ListGithubPullRequests object.
The resulting PRs list inside the status can for example be used in ObjectTemplate to create objects based on
pull requests.
Example
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGithubPullRequests
metadata:
name: list-gh-prs
namespace: default
spec:
interval: 1m
owner: podtato-head
repo: podtato-head
state: open
base: main
head: podtato-head:.*
tokenRef:
secretName: git-credentials
key: github-token
The above example will regularly (1m interval) query the GitHub API for PRs inside the podtato-head
repository. It will filter for open PRs and for PRs against the main branch.
Spec fields
interval
Specifies the interval in which to query the GitHub API. Defaults to 5m.
owner
Specifies the user or organisation name where the repository is localed.
repo
Specifies the repository name to query PRs for.
tokenRef
In case of private repositories, this field can be used to specify a secret that contains a GitHub API token.
head
Specifies the head to filter PRs for. The format must be user:ref-name / organization:ref-name. The head
field can also contain regular expressions.
base
Specifies the base branch to filter PRs for. The base field can also contain regular expressions.
labels
Specifies a list of labels to filter PRs for.
state
Specifies the PR state to filter for. Can either be open, closed or all. Default to all.
limit
Limits the number of results to accept. This is a safeguard for repositories with hundreds/thousands of PRs. It defaults
to 100.
Resulting status
The query result is written into the status.pullRequests field of the ListGithubPullRequests object. Each entry
represents a reduced version of the GitHub Pulls API
results. The result is reduced in verbosity to avoid overloading the Kubernetes apiserver. Reduction means that all
fields containing user, repo, orga and label fields are reduced to id, name, login, owner and
full_name.
Please note that the resulting PR objects do not follow the typical camel case notion found in CRDs, as these represent
a copy of GitHub API objects.
Example:
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGithubPullRequests
metadata:
name: list-gh-prs
namespace: default
spec:
...
status:
conditions:
- lastTransitionTime: "2022-11-07T14:55:36Z"
message: Success
observedGeneration: 3
reason: Success
status: "True"
type: Ready
pullRequests:
- base:
label: podtato-head:main
ref: main
repo:
full_name: podtato-head/podtato-head
name: podtato-head
sha: de7e66af16d41b0ef83de9a0b3be6f5cf0caf942
body: "..."
created_at: "2022-02-02T23:06:28Z"
head:
label: vivek:issue-79_implement_ms_ketch
ref: issue-79_implement_ms_ketch
repo:
full_name: vivek/podtato-head
name: podtato-head
sha: 6379b4c8f413dae70daa03a5a13de4267486fd59
number: 151
state: open
title: '...'
updated_at: "2022-02-04T03:53:03Z"
2.1.6 - TextTemplate
TextTemplate documentation
The TextTemplate API allows to define text templates that are rendered into the status of the TextTemplate.
The result can for example be used in GitlabComment/GithubComment.
Example
For the below example to work, you will also have to deploy the RBAC resources documented in
ObjectTemplate.
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
namespace: default
data:
mykey: input-value
---
apiVersion: templates.kluctl.io/v1alpha1
kind: TextTemplate
metadata:
name: example
namespace: default
spec:
serviceAccountName: example-template-service-account
inputs:
- name: input1
object:
ref:
apiVersion: v1
kind: ConfigMap
name: my-configmap
template: |
This template text can use variables from the inputs defined above, for example this: {{ inputs.input1.data.mykey }}.
The above example will render the given template text and write it into the result of the object:
apiVersion: templates.kluctl.io/v1alpha1
kind: TextTemplate
...
status:
conditions:
- lastTransitionTime: "2023-01-16T11:24:15Z"
message: Success
observedGeneration: 2
reason: Success
status: "True"
type: Ready
result: 'This template text can use variables from the inputs defined above, for example this: input-value.'
Spec fields
suspend
If set to true, reconciliation of this TextTemplate is suspended.
serviceAccountName
The service account to use while retrieving template inputs. See the ObjectTemplate
documentation for details.
List of template inputs which are then available while rendering the text template. At the moment, only Kubernetes
objects are supported as inputs, but other types of inputs might be supported in the future.
Example:
apiVersion: templates.kluctl.io/v1alpha1
kind: TextTemplate
metadata:
name: example
namespace: default
spec:
serviceAccountName: example-template-service-account
inputs:
- name: input1
object:
ref:
apiVersion: v1
kind: ConfigMap
name: my-configmap
namespace: default
jsonPath: data
template: |
This template text can use variables from the inputs defined above, for example this: {{ inputs.input1.mykey }}.
Specifies the name of the input, which is then used to refer to the input inside the text template.
Specifies the object to load as input. The specified service account must have proper permissions
to access this object.
template
Specifies the raw template text to be rendered in the reconciliation loop. While rendering, each input is available
via the global inputs variable and the specified name of the input, e.g. `{{ inputs.my_input.sub_field }}.
See templating for more details on the templating engine.
templateRef
Specifies another object to load the template text from. Currently only ConfigMaps are supported.
templateRef.configMap:
Specifies a ConfigMap to load the template from.
Example:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
namespace: default
data:
mykey: input-value
---
apiVersion: v1
kind: ConfigMap
metadata:
name: my-template
namespace: default
data:
template: |
This template text can use variables from the inputs defined above, for example this: {{ inputs.input1.data.mykey }}.
---
apiVersion: templates.kluctl.io/v1alpha1
kind: TextTemplate
metadata:
name: example
namespace: default
spec:
serviceAccountName: example-template-service-account
inputs:
- name: input1
object:
ref:
apiVersion: v1
kind: ConfigMap
name: my-configmap
templateRef:
configMap:
name: my-template
key: template
Resulting status
The resulting rendered template is written into the status and can then be used by other objects, e.g. GitlabComment/GithubComment.
Example:
...
status:
conditions:
- lastTransitionTime: "2023-01-16T11:24:15Z"
message: Success
observedGeneration: 3
reason: Success
status: "True"
type: Ready
result: 'This template text can use variables from the inputs defined above,
for example this: input-value.'
2.1.7 - ListGitlabMergeRequests
ListGitlabMergeRequests documentation
The ListGitlabMergeRequests API allows to query the Gitlab API for a list of merge requests (MRs). These MRs
can be filtered when needed. The resulting list of MRs is written into the status of the
ListGitlabMergeRequests object.
The resulting MRs list inside the status can for example be used in ObjectTemplate to create objects based on
pull requests.
Example
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGitlabMergeRequests
metadata:
name: list-gl-mrs
namespace: default
spec:
interval: 1m
project: my-group/my-repo
state: opened
targetBranch: main
sourceBranch: prefix-.*
tokenRef:
secretName: git-credentials
key: gitlab-token
The above example will regularly (1m interval) query the Gitlab API for MRs inside the my-group/my-repo
project. It will filter for open MRs and for MRs against the main branch.
Spec fields
interval
Specifies the interval in which to query the GitHub API. Defaults to 5m.
project
Specifies the Gitlab project to query MRs for. Must be in the format group/project, where group can also contain
subgroups (e.g. group1/group2/project).
tokenRef
In case of private repositories, this field can be used to specify a secret that contains a Gitlab API token.
targetBranch
Specifies the target branch to filter MRs for. The targetBranch field can also contain regular expressions.
sourceBranch
Specifies the source branch to filter MRs for. The sourceBranch field can also contain regular expressions.
labels
Specifies a list of labels to filter MRs for.
state
Specifies the PR state to filter for. Can either be opened, closed, locked, merged or all. Default to all.
limit
Limits the number of results to accept. This is a safeguard for repositories with hundreds/thousands of MRs. It defaults
to 100.
Resulting status
The query result is written into the status.mergeRequests field of the ListGitlabMergeRequests object. The list is
identical to what is documented in the Gitlab Merge requests API.
Please note that the resulting PR objects do not follow the typical camel case notion found in CRDs, as these represent
a copy of Gitlab API objects.
Example:
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGitlabMergeRequests
metadata:
name: list-gl-mrs
namespace: default
spec:
...
status:
conditions:
- lastTransitionTime: "2022-11-07T14:55:36Z"
message: Success
observedGeneration: 3
reason: Success
status: "True"
type: Ready
mergeRequests:
- id: 1
iid: 1
project_id: 3
title: test1
description: fixed login page css paddings
state: merged
merged_by:
id: 87854
name: Douwe Maan
username: DouweM
state: active
avatar_url: 'https://gitlab.example.com/uploads/-/system/user/avatar/87854/avatar.png'
web_url: 'https://gitlab.com/DouweM'
merge_user:
id: 87854
name: Douwe Maan
username: DouweM
state: active
avatar_url: 'https://gitlab.example.com/uploads/-/system/user/avatar/87854/avatar.png'
web_url: 'https://gitlab.com/DouweM'
merged_at: '2018-09-07T11:16:17.520Z'
closed_by: null
closed_at: null
created_at: '2017-04-29T08:46:00Z'
updated_at: '2017-04-29T08:46:00Z'
target_branch: master
source_branch: test1
upvotes: 0
downvotes: 0
author:
id: 1
name: Administrator
username: admin
state: active
avatar_url: null
web_url: 'https://gitlab.example.com/admin'
assignee:
id: 1
name: Administrator
username: admin
state: active
avatar_url: null
web_url: 'https://gitlab.example.com/admin'
assignees:
- name: Miss Monserrate Beier
username: axel.block
id: 12
state: active
avatar_url: >-
http://www.gravatar.com/avatar/46f6f7dc858ada7be1853f7fb96e81da?s=80&d=identicon
web_url: 'https://gitlab.example.com/axel.block'
reviewers:
- id: 2
name: Sam Bauch
username: kenyatta_oconnell
state: active
avatar_url: >-
https://www.gravatar.com/avatar/956c92487c6f6f7616b536927e22c9a0?s=80&d=identicon
web_url: 'http://gitlab.example.com//kenyatta_oconnell'
source_project_id: 2
target_project_id: 3
labels:
- Community contribution
- Manage
draft: false
work_in_progress: false
milestone:
id: 5
iid: 1
project_id: 3
title: v2.0
description: Assumenda aut placeat expedita exercitationem labore sunt enim earum.
state: closed
created_at: '2015-02-02T19:49:26.013Z'
updated_at: '2015-02-02T19:49:26.013Z'
due_date: '2018-09-22'
start_date: '2018-08-08'
web_url: 'https://gitlab.example.com/my-group/my-project/milestones/1'
merge_when_pipeline_succeeds: true
merge_status: can_be_merged
detailed_merge_status: not_open
sha: '8888888888888888888888888888888888888888'
merge_commit_sha: null
squash_commit_sha: null
user_notes_count: 1
discussion_locked: null
should_remove_source_branch: true
force_remove_source_branch: false
allow_collaboration: false
allow_maintainer_to_push: false
web_url: 'http://gitlab.example.com/my-group/my-project/merge_requests/1'
references:
short: '!1'
relative: my-group/my-project!1
full: my-group/my-project!1
time_stats:
time_estimate: 0
total_time_spent: 0
human_time_estimate: null
human_total_time_spent: null
squash: false
task_completion_status:
count: 0
completed_count: 0
3 - Security
Security documentation.
The Template Controller is a powerful controller that is able to create/apply arbitrary objects from templates and an
input matrix. This has some security implications as it requires you to make sure that you don’t open potential
security vulnerabilities inside your cluster.
This means, you must make sure that your ObjectTemplate objects are either not dependent on external inputs (which
might contain malicious input) or tha the used service account
is restricted enough to not allow malicious modifications to the cluster.
cluster-admin role
Especially watch out when using the cluster-admin (or comparable) role. It can easily lead to privilege escalation if
templates and inputs are too dynamic.
4 - Templating
Templating documentation.
The Template Controller reuses the Jinja2 templating engine of Kluctl.
Documentation is available here.
Predefined variables
You can use multiple predefined variables in your templates. These are:
objectTemplate
Available in templates inside ObjectTemplate and represents the whole
ObjectTemplate that was on your target BEFORE the reconciliation started.
textTemplate
Available in templates inside TextTemplate and represents the whole
TextTemplate that was on your target BEFORE the reconciliation started.
5 - Use Case: Dynamic environments for Pull Requests
Use Case: Dynamic environments for Pull Requests
This use case was the initial and first use case why the Template Controller was created. You can use ListGithubPullRequests
to query the GitHub API for a list of pull requests on a GitHub Repo and then use the result inside a ObjectTemplate
to generate GitOps environments for new pull requests.
Flux
This example will create templated Kustomization
objects. The means, that you should first install Flux on your cluster. The
dev install variant should be sufficient.
podtato-head as example
This example uses the podtato-head demo project to demonstrate the
Template Controller. You must fork the repository and replace all occurrences of podtato-head as owner with your
own username. It is not recommended to blindly use the public repository as you this will lead to unverified and
potentially dangerous environments being deployed into your cluster!
GitHub credentials
In case you want to listen for PRs from a private repository (e.g. because you’ve forked podtato-head), you’ll need to store a
GitHub personal access token
inside a Kubernetes Secret.
apiVersion: v1
kind: Secret
metadata:
name: git-credentials
namespace: default
stringData:
github-token: "<your-github-token>"
WARNING: Of course, in a real setup you would NOT store the plain token inside a manifest, but instead use
Sealed Secrets or SOPS.
A dedicated ServiceAccount
The Template Controller uses service accounts to query matrix inputs and apply rendered objects. These service accounts
determine what the template can access and what not. In this example, we’ll create a service account with the
cluster-admin role, which you should NOT do in production. Instead, define your own Role or ClusterRole and
attach it to the service account. This role should have read/write access to all objects references in the matrix and
the rendered objects.
apiVersion: v1
kind: ServiceAccount
metadata:
name: podtato-head-envs-objecttemplate
namespace: default
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: podtato-head-envs-objecttemplate
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
# WARNING, this is only for demo purposes. You should use a more restricted role for the ObjectTemplate
name: cluster-admin
subjects:
- kind: ServiceAccount
name: podtato-head-envs-objecttemplate
namespace: default
The above serviceAccount is then later referenced inside the ObjectTemplate object.
Listing GitHub pull requests
Listing pull requests from a GitHub repository can be done through the
ListGithubPullRequests CRD. It specifies the GitHub repository to use and
some filter options.
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGithubPullRequests
metadata:
name: list-gh-prs
namespace: default
spec:
interval: 1m
# Replace the owner with your username in case you forked podtato-head
owner: podtato-head
repo: podtato-head
# Ignore closed PRs
state: open
# Only PR's that go against the main branch
base: main
# Replace `podtato-head` with your username. This will only allows heads from your own fork!
# Otherwise, you risk deploying unsafe environments into your cluster!
head: podtato-head:.*
tokenRef:
secretName: git-credentials
key: github-token
After applying this resource, the Template Controller will start to query the GitHub API for matching pull requests and
then store the results inside the status of the ListGithubPullRequests CR. Example:
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGithubPullRequests
metadata:
name: list-gh-prs
namespace: default
spec:
...
status:
conditions:
- lastTransitionTime: "2022-11-07T14:55:36Z"
message: Success
observedGeneration: 3
reason: Success
status: "True"
type: Ready
# The pullRequests list contains much more detailed info, but to keep it short I've reduced verbosity here
pullRequests:
- base:
label: podtato-head:main
ref: main
repo:
full_name: podtato-head/podtato-head
name: podtato-head
sha: de7e66af16d41b0ef83de9a0b3be6f5cf0caf942
body: "..."
created_at: "2022-02-02T23:06:28Z"
head:
label: vivek:issue-79_implement_ms_ketch
ref: issue-79_implement_ms_ketch
repo:
full_name: vivek/podtato-head
name: podtato-head
sha: 6379b4c8f413dae70daa03a5a13de4267486fd59
number: 151
state: open
title: '...'
updated_at: "2022-02-04T03:53:03Z"
The ObjectTemplate
The pullRequests field from the above status can then be used as an input into the an
ObjectTemplate.
apiVersion: templates.kluctl.io/v1alpha1
kind: ObjectTemplate
metadata:
name: pr-envs
namespace: default
spec:
serviceAccountName: podtato-head-envs-objecttemplate
# This causes removal of templated objects in case they disappear from the rendered list of objects
prune: true
matrix:
- name: pr
object:
ref:
apiVersion: templates.kluctl.io/v1alpha1
kind: ListGithubPullRequests
name: list-gh-prs
jsonPath: status.pullRequests
expandLists: true
templates:
- object:
apiVersion: v1
kind: Namespace
metadata:
# Give each one its own namespace
name: "podtato-head-{{ matrix.pr.head.label | slugify }}"
- object:
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
# The pullRequests status field from the ListGithubPullRequests is a reduced form of the REST API result
# of https://docs.github.com/en/rest/pulls/pulls#list-pull-requests, meaning that fields like `head` and `base`
# are also available.
name: "podtato-head-{{ matrix.pr.head.label | slugify }}"
namespace: default
spec:
interval: 5m
url: "https://github.com/{{ matrix.pr.head.repo.full_name }}.git"
ref:
branch: "{{ matrix.pr.head.ref }}"
- object:
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: "podtato-head-env-{{ matrix.pr.head.label | slugify }}"
namespace: default
spec:
interval: 10m
targetNamespace: "podtato-head-{{ matrix.pr.head.label | slugify }}"
sourceRef:
kind: GitRepository
# refers to the same GitRepository created above
name: "podtato-head-{{ matrix.pr.head.label | slugify }}"
path: "./delivery/kustomize/base"
prune: true
The above ObjectTemplate will create 3 objects per pull request:
- A namespace with the name
podtato-head-{{ matrix.pr.head.label | slugify }}. Please note the use of
Jinja2 templating. Details about what can be done can be found in the
ObjectTemplate documentation. - A Flux GitRepository that points to repository
and branch of the current pull request.
- A Flux Kustomization that is deployed into
the above namespace.
6 - Use Case: Transformation of Secrets/Objects
Use Case: Transformation of Secrets/Objects
There are cases where an object can not be created before another object is created by some other component inside the
cluster, meaning that you have no control over the input object.
A simple example is the Zalando Postgres Operator, which allows you
to create a Postgres database with a Custom Resource. Inside the CR, you can define databases and users to be
auto-created. When the operator creates these databases and users, it also auto-creates Kubernetes secrets with the
credentials allowing you to access the databases.
These secrets can however not be used directly when connecting to the databases, as you’d usually have to build some
connection urls (e.g. JDBC urls). Usually, one would create some kind of init script or something like that to
build this url and then pass it to the application that wants to use it.
The Template Controller allows an alternative solution.
Let’s assume you have a sample
Postgres database deployed via the Zalando Postgres Operator. The operator has also created the following secret:
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: foo-user.acid-minimal-cluster.credentials.postgresql.acid.zalan.do
namespace: default
data:
password: aHNiSVF6MFJJa0hTd2ZxS1NiTG5YV3dUQUVqcUtTNFpvU2dyOXp4b3pzMmJvTE02WWl0eTE0YjJTZlNFTHExdw==
username: Zm9vX3VzZXI=
Based on that secret, you’d like to create a new secret with the JDBC url generated.
RBAC
The ObjectTemplate requires a service account with proper access rights for the involved secrets:
apiVersion: v1
kind: ServiceAccount
metadata:
name: postgres-secret-transformer
namespace: default
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: postgres-secret-transformer
namespace: default
rules:
- apiGroups: [""]
resources: ["secrets"]
# give the ObjectTemplate access to the two involved secrets
resourceNames: ["zalando.acid-minimal-cluster.credentials.postgresql.acid.zalan.do", "transformed-postgres-secret"]
verbs: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: postgres-secret-transformer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: postgres-secret-transformer
subjects:
- kind: ServiceAccount
name: postgres-secret-transformer
namespace: default
ObjectTemplate
Use the following ObjectTemplate to perform the transformation:
apiVersion: templates.kluctl.io/v1alpha1
kind: ObjectTemplate
metadata:
name: postgres-secret-transformer
namespace: default
spec:
serviceAccountName: postgres-secret-transformer
prune: true
matrix:
- name: secret
object:
ref:
apiVersion: v1
kind: Secret
name: zalando.acid-minimal-cluster.credentials.postgresql.acid.zalan.do
templates:
- object:
apiVersion: v1
kind: Secret
metadata:
name: "transformed-postgres-secret"
stringData:
jdbc_url: "jdbc:postgresql://acid-minimal-cluster/zalando?user={{ matrix.secret.data.username | b64decode }}&password={{ matrix.secret.data.password | b64decode }}"
# sometimes the key names inside a secret are not what another component requires, so we can simply use different names if we want
username_with_different_key: "{{ matrix.secret.data.username | b64decode }}"
password_with_different_key: "{{ matrix.secret.data.password | b64decode }}"
This will lead to the following transformed-postgres-secret
apiVersion: v1
kind: Secret
metadata:
name: transformed-postgres-secret
namespace: default
type: Opaque
data:
jdbc_url: amRiYzpwb3N0Z3Jlc3FsOi8vaG9zdC9kYXRhYmFzZT91c2VyPWZvb191c2VyJnBhc3N3b3JkPWJVUU52Zkd4amduQUdiaEhOWkZkamtwZFFYbnk1aDdXNGlFU1YyWUxVNnVrRHdXWjBPMjdRb0NBdUJTTnF3TVk=
password_with_different_key: YlVRTnZmR3hqZ25BR2JoSE5aRmRqa3BkUVhueTVoN1c0aUVTVjJZTFU2dWtEd1daME8yN1FvQ0F1QlNOcXdNWQ==
username_with_different_key: Zm9vX3VzZXI=
Base64 decoding the secret data will show:
jdbc_url: jdbc:postgresql://host/database?user=foo_user&password=bUQNvfGxjgnAGbhHNZFdjkpdQXny5h7W4iESV2YLU6ukDwWZ0O27QoCAuBSNqwMY │
password_with_different_key: bUQNvfGxjgnAGbhHNZFdjkpdQXny5h7W4iESV2YLU6ukDwWZ0O27QoCAuBSNqwMY │
username_with_different_key: foo_user
7 - Template Controller API reference
Template Controller API reference
Packages:
templates.kluctl.io/v1alpha1
Package v1alpha1 contains API Schema definitions for the templates.kluctl.io v1alpha1 API group.
Resource Types:
AppliedResourceInfo
(Appears on:
ObjectTemplateStatus)
(Appears on:
CommentSpec)
(Appears on:
GithubCommentSpec,
GitlabCommentSpec)
ConfigMapRef
(Appears on:
CommentSourceSpec)
GitFile
(Appears on:
GitProjectorSpec)
GitProjector
GitProjector is the Schema for the gitprojectors API
GitProjectorResult
(Appears on:
GitProjectorStatus)
GitProjectorResultFile
(Appears on:
GitProjectorResult)
GitProjectorSpec
(Appears on:
GitProjector)
GitProjectorSpec defines the desired state of GitProjector
GitProjectorStatus
(Appears on:
GitProjector)
GitProjectorStatus defines the observed state of GitProjector
GitRef
(Appears on:
GitProjectorResult,
GitProjectorSpec)
GithubComment is the Schema for the githubcomments API
(Appears on:
GithubComment)
GithubCommentSpec defines the desired state of GithubComment
(Appears on:
GithubComment)
GithubCommentStatus defines the observed state of GithubComment
GithubProject
(Appears on:
GithubPullRequestRef,
ListGithubPullRequestsSpec)
GithubPullRequestRef
(Appears on:
GithubCommentSpec)
GitlabComment is the Schema for the gitlabcomments API
(Appears on:
GitlabComment)
GitlabCommentSpec defines the desired state of GitlabComment
(Appears on:
GitlabComment)
GitlabCommentStatus defines the observed state of GitlabComment
GitlabMergeRequestRef
(Appears on:
GitlabCommentSpec)
GitlabProject
(Appears on:
GitlabMergeRequestRef,
ListGitlabMergeRequestsSpec)
ListGithubPullRequests
ListGithubPullRequests is the Schema for the listgithubpullrequests API
ListGithubPullRequestsSpec
(Appears on:
ListGithubPullRequests)
ListGithubPullRequestsSpec defines the desired state of ListGithubPullRequests
ListGithubPullRequestsStatus
(Appears on:
ListGithubPullRequests)
ListGithubPullRequestsStatus defines the observed state of ListGithubPullRequests
ListGitlabMergeRequests
ListGitlabMergeRequests is the Schema for the listgitlabmergerequests API
ListGitlabMergeRequestsSpec
(Appears on:
ListGitlabMergeRequests)
ListGitlabMergeRequestsSpec defines the desired state of ListGitlabMergeRequests
ListGitlabMergeRequestsStatus
(Appears on:
ListGitlabMergeRequests)
ListGitlabMergeRequestsStatus defines the observed state of ListGitlabMergeRequests
LocalObjectReference
(Appears on:
CommentSourceSpec,
GitProjectorSpec)
MatrixEntry
(Appears on:
ObjectTemplateSpec)
MatrixEntryObject
(Appears on:
MatrixEntry)
ObjectRef
(Appears on:
AppliedResourceInfo,
MatrixEntryObject,
TextTemplateInputObject)
ObjectTemplate
ObjectTemplate is the Schema for the objecttemplates API
ObjectTemplateSpec
(Appears on:
ObjectTemplate)
ObjectTemplateSpec defines the desired state of ObjectTemplate
ObjectTemplateStatus
(Appears on:
ObjectTemplate)
ObjectTemplateStatus defines the observed state of ObjectTemplate
SecretRef
(Appears on:
GithubProject,
GitlabProject)
Utility struct for a reference to a secret key.
Template
(Appears on:
ObjectTemplateSpec)
TemplateRef
(Appears on:
TextTemplateSpec)
TemplateRefConfigMap
(Appears on:
TemplateRef)
TextTemplate
TextTemplate is the Schema for the texttemplates API
TextTemplateInput
(Appears on:
TextTemplateSpec)
TextTemplateInputObject
(Appears on:
TextTemplateInput)
TextTemplateSpec
(Appears on:
TextTemplate)
TextTemplateSpec defines the desired state of TextTemplate
TextTemplateStatus
(Appears on:
TextTemplate)
TextTemplateStatus defines the observed state of TextTemplate
This page was automatically generated with gen-crd-api-reference-docs