Knot8 (Notate)

Tune K8s manifests with lenses

mkm@influxdata.com

Simplicity

kubectl apply -f https://github.com/ac/me/releases/download/v1/app.yaml

Current "Package managers"

Often conflate how to:

• discover
• obtain
• deploy
• adapt
• update (detect and apply)

Current "Package managers"

Often conflate how to:

• discover
• obtain
• deploy
• adapt
• update (detect and apply)

The future is already here — it's just not very evenly distributed.

- William Gibson

See Project Manibus

Good old templating

#@ load("@ytt:data", "data")

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

data:

foo: #@ data.values.foo

​

Good old templating

​

$ cat values.yaml

foo: meow

$ ytt -f .

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

data:

foo: meow

Templating is viral

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: {{ template "wordpress.image" . }}

selector:

matchLabels:

app: demo

Hidden complexity

{{- define "wordpress.image" -}}

{{- $registryName := .Values.image.registry -}}

{{- $repositoryName := .Values.image.repository -}}

{{- $tag := .Values.image.tag | toString -}}

{{- if .Values.global }}

{{- if .Values.global.imageRegistry }}

{{- printf "%s/%s:%s" .Values.global.imageRegistry $repositoryName $tag -}}

{{- else -}}

{{- printf "%s/%s:%s" $registryName $repositoryName $tag -}}

{{- end -}}

{{- else -}}

{{- printf "%s/%s:%s" $registryName $repositoryName $tag -}}

{{- end -}}

{{- end -}}

Templating woes

• my template engine != your template engine
• you need a tool even if you just use defaults
• ...

Fields

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

data:

foo: meow # See ^^^ field.knot8.io/foo

​

Fields

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

data:

foo: meow # See ^^^ field.knot8.io/foo

​

knot8 set <app.yaml foo=woof

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

data:

foo: woof # See ^^^ field.knot8.io/foo

​

Find the right item

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: debian:10 # See ^^^ field.knot8.io/appImage

selector:

matchLabels:

app: demo

Find the right item

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: debian:10 # See ^^^ field.knot8.io/image

selector:

matchLabels:

app: demo

knot8 set <app.yaml appImage=myrepo/debian:10

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: myrepo/debian:10 # See ^^^ field.knot8.io/Image

selector:

matchLabels:

app: demo

Inner formats

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

data:

foo: |

some:

app:

config:

bar: 42

​

Inner formats

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

data:

foo: |

some:

app:

config:

bar: 42

​

?

?

?

Inner formats

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

data:

foo: |

some:

app:

config:

bar: 42

​

?

?

?

!!

Inner formats: through the looking glass

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo/~(yaml)/some/app/config/bar

data:

foo: |

some:

app:

config:

bar: 42

​

Inner formats: through the looking glass

​

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo/~(yaml)/some/app/config/bar

data:

foo: |

some:

app:

config:

bar: 42

​

Inner formats: Through the looking glass lens

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo/~(yaml)/some/app/config/bar

data:

foo: |

some:

app:

config:

bar: 42

​

Through the lens

​

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo/~(yaml)/some/app/config/bar

data:

foo: |

some:

app:

config:

bar: 42

​

Through the lens: same thing, just a level deeper

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo/~(yaml)/some/app/config/bar

data:

foo: |

some:

app:

config:

bar: 42

​

It's turtles trees all the way down

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo/~(yaml)/some/app/config/bar/~(toml)/section/subsection/key

data:

foo: |

some:

app:

config:

bar: |

foo = bar

[section.subsection]

key = 42

​

Real world example

apiVersion: v1

kind: ConfigMap

metadata:

name: demo

annotations:

field.knot8.io/cluster: /data/cfg/~(yaml)/relabel_configs/~{"target_label":"cluster"}/replacement

data:

cfg: |

relabel_configs:

- source_labels: [

"__meta_kubernetes_namespace",

"__meta_kubernetes_service_name",

"__meta_kubernetes_endpoint_port_name",

]

action: keep

regex: default;kubernetes;https

- target_label: cluster

replacement: main

​

Image references as inner formats

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image

field.knot8.io/appVersion: image/~(oci)/tag

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: debian:10 # See ^^^ field.knot8.io/image

selector:

matchLabels:

app: demo

Image references as inner formats

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image

field.knot8.io/appVersion: appImage/~(oci)/tag

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: debian:10 # See ^^^ field.knot8.io/appImage

selector:

matchLabels:

app: demo

Lenses

Non-goals (When is knot8 not applicable)

• when you want resources to appear or disappear on a condition
• e.g. optional ingress resource
• it doesn't help you craft manifests
• e.g. with repetitive blocks etc

Workflows

• GitOps
• Fire & forget ...

Workflows

• GitOps
• Fire & forget ...
• kubectl apply -f https://acme.com/releases/v1.1.0/app.yaml
• ... & repent
• kubectl get deploy/foo -oyaml | knot8 set -v bar=123 | kubectl apply -f -

GitOps

• Plug&play
• DEV:
• vim prod/values.yaml && git commit ...
• CI:
• cat app.yaml | knot8 set --from prod/values.yaml | othertools | kubectl ...
• In-place
• DEV:
• knot8 set -f app.yaml && git commit
• CI:
• cat app.yaml | othertools | kubectl ...

Wait, what? why would you? that's wrong

How I learned stop worrying and love the files

3-way merge

• knot8 pull -f app.yaml https://acme.com/releases/v1.2.0/app.yaml
• knot8 pull -f app.yaml oci://acme/app:1.2.0

​

3-way merge

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/original: |

foo: meow

bar: "1"

data:

foo: meow

bar: "1"

​

knot8 set -f app.yaml foo=woof

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/original: |

foo: meow

bar: "1"

data:

foo: woof

bar: "1"

knot8 pull -f app.yaml https://…/v2/app.yaml

​

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/original: |

foo: meow

bar: "1"

data:

foo: woof

bar: "1"

apiVersion: v1

kind: ConfigMap

metadata:

name: bettername

annotations:

field.knot8.io/foo: /data/cfg/~(toml)/foo

field.knot8.io/bar: /data/cfg/~(toml)/bar

knot8.io/original: |

foo: miau

bar: "42"

data:

cfg: |

bar = 42

foo = miau

apiVersion: v1

kind: ConfigMap

metadata:

name: bettername

annotations:

field.knot8.io/foo: /data/cfg/~(toml)/foo

field.knot8.io/bar: /data/cfg/~(toml)/bar

knot8.io/original: |

foo: miau

bar: "42"

data:

cfg: |

bar = 42

foo = woof

Current

New upstream

Merged

knot8 set -f app.yaml appImage/repo=myrepo/foapp

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage: fooapp:1.0

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:1.0

​

​

knot8 pull -f app.yaml https://…/v2/app.yaml

​

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImageName: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)/name

field.knot8.io/appImageTag: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)/tag

knot8.io/original: |

appImageName: fooapp

appImageTag: "1.0"

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:1.0

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImageName: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)/name

field.knot8.io/appImageTag: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)/tag

knot8.io/original: |

appImageName: fooapp

appImageTag: "2.0"

spec:

template:

spec:

containers:

- name: app

image: fooapp:2.0

​

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImageName: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)/name

field.knot8.io/appImageTag: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)/tag

knot8.io/original: |

appImageName: fooapp

appImageTag: "2.0"

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:2.0

​

​

Current

New upstream

Merged

knot8 pull -f app.yaml https://…/v2/app.yaml

​

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage: fooapp:1.0

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:1.0

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage: fooapp:2.0

spec:

template:

spec:

containers:

- name: app

image: fooapp:2.0

​

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage: fooapp:2.0

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:2.0

​

​

​

Current

New upstream

Merged

knot8 pull -f app.yaml https://…/v2/app.yaml

​

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage/*: fooapp:1.0

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:1.0

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage: fooapp:2.0

spec:

template:

spec:

containers:

- name: app

image: fooapp:2.0

​

​

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

knot8.io/original: |

appImage/*: fooapp:2.0

spec:

template:

spec:

containers:

- name: app

image: myrepo/fooapp:1.0

​

​

​

Current

New upstream

Merged

Prior art

• Lenses:
• Tons of research in "bidirectional parsing" or "unparsers" in the FP ivory tower
• Boomerang (project harmony)
• Augeas
• In-place editing workflow:
• KPT The Kubernetes Packaging Toolkit

POC?

• https://github.com/mkmik/knot8

Questions?

Fanout

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

data:

foo: meow # See ^^^ field.knot8.io/foo

---

apiVersion: v1

kind: Secret

metadata:

name: demozzz

annotations:

field.knot8.io/foo: /data/bar/~(base64)

data:

bar: bWVvdw==

​

​

​

Outputs

• In-place modification
• Overlays that play well with other tools: (kustomize, ytt, kubecfg, ...)
• Templates to be used as base for other tools (ytt, kubecfg, ...)

Templates output: jsonnet

$ knot8 convert -f app.yaml -ojsonnet

{

fields:: {

foo: "meow",

},

apiVersion: "v1",

kind: "ConfigMap",

metadata: {

name: "demo2",

annotations: {

"field.knot8.io/foo": "/data/foo",

},

},

data: {

foo: $.fields.foo,

},

}

​

​

Templates output: ytt

$ knot8 convert -f app.yaml -oytt=config.yaml,values.yaml

$ cat config.yaml

#@ load("@ytt:data", "data")

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo

annotations:

field.knot8.io/foo: /data/foo

data:

foo: #@ data.values.fields.foo

$ cat values.yaml

#@data/values

---

fields:

foo: bar

$ ytt -f .

…

Templates output: helm

left as an exercise for the reader

Outputs overlays

$ knot8 set -f app.yaml -o overlay appImage=acme:1.2

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

spec:

template:

spec:

containers:

- image: acme:1.2

​

Outputs overlays

$ knot8 set -f app.yaml -o jsonnet appImage=acme:1.2

{

spec: {

template: {

spec: {

containers: [

{

image: 'acme:1.2',

},

],

},

},

},

}

​

​

​

​

​

Why not overlays?

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

spec:

template:

spec:

containers:

- image: debian:10

name: app

​

Documentation: for humans and ...

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/meta:

foo:

doc: Foo controls a key frobnication parameter.

bar:

doc: Bar strongly affects frobnication efficiency.

data:

foo: meow

bar: "1"

... for machines

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/meta:

foo:

doc: Foo controls a key frobnication parameter.

bar:

doc: Bar strongly affects frobnication efficiency.

type: int

data:

foo: meow

bar: "1"

"Typed" fields

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: debian:10 # See ^^^ field.knot8.io/image

selector:

matchLabels:

app: demo

knot8 set -f app.yaml appImage/version=9

apiVersion: apps/v1

kind: Deployment

metadata:

name: demo

annotations:

field.knot8.io/appImage: /spec/template/spec/containers/~{"name":"app"}/image/~(oci)

spec:

template:

metadata:

labels:

app: demo

spec:

containers:

- name: app

image: debian:10 # See ^^^ field.knot8.io/image

selector:

matchLabels:

app: demo

Schema

$ knot8 info -f app.yaml

appImage

foo

$ knot8 info -f app.yaml --targets

appImage

/items/~{"apiVersion":"v1","kind":"Deployment","metadata":{"name":"demo"}}

/spec/template/spec/containers/~{"name":"app"}/image

foo

/items/~{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"demo2"}}

/data/foo/~(yaml)/some/app/config/bar

/items/~{"apiVersion":"v1","kind":"Deployment","metadata":{"name":"demo"}}

/spec/template/spec/containers/~{"name":"app"}/env/~{"name":"FOO"}/value

$ knot8 info -f app.yaml --targets --flat

appImage /items/~{"apiVersion":"v1","kind":"Deployment","metadata":{"name":"demo"}}/spec/template/spec/containers/~{"name":"app"}/…

foo /items/~{"apiVersion":"v1","kind":"ConfigMap","metadata":{"name":"demo2"}}/data/foo/~(yaml)/some/app/config/bar

foo /items/~{"apiVersion":"v1","kind":"Deployment","metadata":{"name":"demo"}}/spec/template/spec/containers/~{"name":"app"}/env/…

$ knot8 info -f app.yaml --lenses

yaml

toml

oci

...

​

​

$ knot8 set -f app.yaml appImage/tag=9

​

Schema cont'd

$ knot8 info -f app.yaml --lenses --openapi

oci

#/definitions/io.k8s.api.core.v1.Container/properties/image

#/definitions/io.k8s.api.core.v1.EphemeralContainer/properties/image

base64

#/definitions/io.k8s.api.core.v1.Secret/properties/data

#/definitions/io.k8s.api.certificates.v1beta1.CertificateSigningRequestSpec/properties/request

​

AppImage points to a field covered by the openapi schema property "image" declared in the io.k8s.api.core.v1.Container def

and thus we know that by default it can be viewed through the oci lens

$ knot8 set -f app.yaml appImage/tag=9

​

$ knot8 set -f secret.yaml /data/foo/~(base64)=bar

Schema cont'd 2

$ knot8 info -f app.yaml --lenses --annotations

nginxConfig

/items/~{}

/metadata/annotations/nginx.ingress.kubernetes.io~1configuration-snippet

​

​

Well-known annotations can be operated on without explicit fields:

$ knot8 set -f app.yaml /metadata/annotations/nginx.ingress.kubernetes.io~1configuration-snippet/error_page/404=my404.html

​

​

Operations on trees

• Objects: set fields, remove fields
• Arrays: insert, remove, replace
• Strings: regexp replace

Lens examples

$ knot8 set -f app.yaml /spec/template/spec/containers/0/args/~(cmdline:optparse)/foo=wow

$ git diff deployment.yaml

args: [

"-v",

"--baz=quz"

- "--foo=bar",

+ "--foo=wow",

"blah.txt"

]

​

$ git diff deployment.yaml

args: [

"-v",

"--baz=quz"

"--foo"

- "bar",

+ "wow",

"blah.txt"

]

​

knot8 pull -f app.yaml https://…/v2/app.yaml

​

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/original: |

foo: meow

bar: "1"

data:

foo: woof

bar: "1"

apiVersion: v1

kind: ConfigMap

metadata:

name: bettername

annotations:

field.knot8.io/foo: /data/cfg/~(toml)/foo

knot8.io/original: |

foo: miau

data:

cfg: |

foo = miau

apiVersion: v1

kind: ConfigMap

metadata:

name: bettername

annotations:

field.knot8.io/foo: /data/cfg/~(toml)/foo

field.knot8.io/bar: /data/cfg/~(toml)/bar

knot8.io/original: |

foo: miau

data:

cfg: |

foo = woof

Current

New upstream

Merged

knot8 pull -f app.yaml https://…/v2/app.yaml

​

​

apiVersion: v1

kind: ConfigMap

metadata:

name: demo2

annotations:

field.knot8.io/foo: /data/foo

field.knot8.io/bar: /data/bar

knot8.io/original: |

foo: meow

bar: "1"

data:

foo: woof

bar: "2"

apiVersion: v1

kind: ConfigMap

metadata:

name: bettername

annotations:

field.knot8.io/foo: /data/cfg/~(toml)/foo

knot8.io/original: |

foo: miau

data:

cfg: |

foo = miau

error (warning?), unknown field bar

Current

New upstream

Merged