1 of 8

Use CRDs Whenever Possible

@erictune This deck is shared publicly.

2 of 8

Users of CRDs

  • Istio – service mesh
  • Tensorflow – ML framework
  • Project Calico – Overlay network
  • Kubeless – Serverless framework
  • CoreOS DEX – OIDC Identity Provider
  • Applatix Argo - Workflow (CI/CD)
  • Dozens of “Operators” – Custom pod controllers

3 of 8

Benefits to using CRD over Aggr.

  • Momentum: More users behind the feature (see prev slide)
  • Fork/Rebase less code
  • Don't run a separate etcd
  • Lower net memory requirements
  • ​

​

​

4 of 8

CRDs can do a lot currently

As of 1.9:

  • Simple validation, using OpenAPIv3 [link]
    • e.g. date format, max, min, maximumItems, required, pattern match, uniqueness
  • Complex validation, using validating webhook [link]
    • e.g. if field A set then B must also be set
  • Defaulting, using a mutating webhook [link]
    • e.g. if field A is not set then set B to “foo”

​

5 of 8

CRDs will do more soon

​

  • Active work on multi-version support
    • Expected in 1.11
    • Will allow “server-side” conversion between versions.
    • Likely will have three options:
      • Promote without changes (trivial)
      • Rename some e fields between versions (no coding).
      • Arbitrary conversions (you write a webhook)
    • Widely requested

​

6 of 8

CRDs will do more soon

​

  • Scale and Status subresources in 1.10 (#55168)
    • Can use metadata.generation/status.observedGeneration pattern
    • Can scale using an HPA or "kubectl scale"
  • Server-side apply being designed now
    • Will make apply automatically work with many existing CRDs.
  • “v1” status for CRDs expected later this year.
  • Likely to be added, IMO:
    • Strategic Merge Patch support expected via server-side /apply
    • Development framework and tools, help with client generation and api linting

​

​

  • ​

​

7 of 8

When to use API Aggregation

  • When you are a façade in front of another kind of storage (e.g. metrics API is a façade in front of influxdb)
  • When you need custom subresources
    • E.g. like how pods have a /logs subresource
    • Suggestion: implement as a separate API, rather than attach to Kubernetes resource.
  • When your resources are as numerous/chatty as Pods and Nodes.

*Aggregated API Servers are also known as Extension API Servers

​

​

8 of 8

When to use Standalone Apiserver

  • When you really need software fault isolation
    • E.g. Clusters API wants to keep running if main apiserver is broken, because they are responsible for creating the main apiserver.
  • When you need to run as a Global API
    • E.g. Storing a list of all known kubernetes clusters, and not tied to any single one.

*Standalone API server is a separate APIServer Binary with its own Aggregated API Servers are also known as Extension API Servers

​

​