CRI: Long Term Evolution

Alexander Kanevskiy, Mike Brown, Mrunal Patel

Discussion topics

• CRI’s legacy: the dockershim and split of responsibilities
• “imperative” vs. “declarative”
• Dynamic Resource Allocation and CRI
• QoS classes and impact on CRI
• VM-based runtimes and specifics
• Extended resources?
• Dynamic native resources?

CRI: the current state

• The embedded dockershim is now finally deprecated and removed from kubelet in favor of external CRI based container runtimes (k8s v1.24+)
• Container metrics are moving to runtimes
• Set of optimizations already implemented for VM based runtimes (overhead, summary of resources, …)
• Polling vs. event-driven container/pod status

CRI: declarative vs. imperative

• “The truth is somewhere out there”
• Reserved and available resources
• Linux Resources vs. Windows resources
• cgroups v1 and v2
• Extended resources?
• Dynamic native resources?

​

• dockershim and split responsibility between kubelet and runtimes
• “Pod” vs. “Container” entities
• HugePages handling
• cgroups driver

CRI: declarative vs. imperative - example

• node vs CRI for container images “imperative” vs. “declarative”
• kubelet receives a declarative podspec for its node -> kubelet imperative process for ensuring the images:
• query the container runtime for image status for each container in the pod
• based on image pull policy and presence status, kubelet may ask the container runtime image service to pull the image into its cache
• Moving the CRI image service to a declarative model
• kubelet would pass the pod’s container image needs as a declarative request to the container runtime
• Of course the example is more complicated in execution because there is also:
• garbage collection contracts on the node for handling storage pressure issues (both kubelet and the container runtime have gc contracts)
• Authentication information for accessing the images (kubelet image pull secrets and provider keyring integration vs container runtime config)
• When using lazy image pulling, what does it mean for the image to be present, is partially present ok?
• ** So where should the declarative information be handled… kubelet or the container runtime?

CRI and Dynamic Resource Allocation

• Dynamic Resource Allocation (DRA)
• KEP-3063
• Expands the way on how devices can be used within containers

​

• Impact on CRI
• Communication from Kubelet towards Runtime is simple
• set of CDI device IDs per container
• no difference for Linux or Windows (in theory)

CRI and QoS class resources

• QoS-class resources
• KEP-3008
• Qualitative, shared resources for pods and containers

​

• Impact on CRI
• Kubelet to Runtimes
• Pod and Containers have set of class attributes: type + value
• Runtime to Kubelet
• Set of configured/supported classes on the system with possible values

VM based runtimes

• Not all runtimeClasses are equal
• some VM runtimes require knowledge of consumed resources at Pod creation step
• some VM runtimes will not be able to perform in-place scaling

CRI Long Term

• So far, most of the changes to API were handled based on short term goals
• With dockershim gone, we can start to think about best ways to evolve it in long term
• Migrations between versions

“We can’t boil the ocean…”�… but let’s find and boil few small lakes!