Runtime Assisted Mounting of Persistent Volumes

Container RUntime STorage [CRUST] APIs

Current PV FS mounts controlled by CSI plugins

Kubernetes �Node �Host OS

Kubelet

Block Device 1

Block Device 2

kubelet/plugins/kubernetes.io/csi/pv/vol1

kubelet/plugins/kubernetes.io/csi/pv/vol2

Kata Guest OS

/dev/sdc

/dev/sde

Container 1

Kata Sandbox

runC Sandbox

Container 2

runC Sandbox [Privileged]

CSI node plugin container

CRI runtime

Kata

runC

ext4

xfs

guest/fs/dst1

virtio-fs�FUSE

kubelet/pods/volumes/uuid/kubernetes.io~csi/vol1

bind mount

kubelet/pods/volumes/uuid/kubernetes.io~csi/vol2

bind mount

bind mount

container/fs/dst2

Plugin discover volume

Plugin stage volume

Plugin stage volume

Plugin publish volume

Plugin publish volume

CSI NodeStageVolume, CSI NodePublishVolume

CRI CreateContainer

Task Create

Task Create

CSI NodeGetVolumeStats, CSI NodeExpandVolume

/run/kata-containers/sbox-id/shared/dst1

bind mount

virtio-fs

Runtime assisted File System mounts with CRUST APIs

Kubernetes �Node �Host OS

Kubelet

Block Device 1

Block Device 2

Kata Guest OS

/dev/sdc

/dev/sde

Container 1

Kata Sandbox

runC Sandbox [Privileged]

CSI node plugin container

CRI runtime

Kata

runC

guest/fs/dst1

/dev/vda

ext4

kubelet/plugins/kubernetes.io/csi/pv/vol2

runC Sandbox

Container 2

xfs

kubelet/pods/volumes/uuid/kubernetes.io~csi/vol2

bind mount

bind mount

container/fs/dst2

virtio-blk

Plugin discover volume

Plugin stage volume

Plugin publish volume

CSI NodeStageVolume, CSI NodePublishVolume

CRI APIs

CSI NodeGetVolumeStats, CSI NodeExpandVolume

Task Create

Task Create

CRUST APIs

RuntimeGetCapabilities

RuntimeGetSupportedFileSystems�RuntimePublishVolume�RuntimeGetVolumeStats�RuntimeExpandVolume

Changes required in Kubelet and Runtimes

1. CRI (API and runtimes): No changes

​

• Kubelet: Invoke CRUST APIs over UDS specified in runtime class (new field)

​

• RuntimeGetCapabilities/RuntimeGetSupportedFileSystems before sandbox creation
• RuntimePublishVolume after sandbox creation
• RuntimeGetVolumeStats/RuntimeExpandVolume while pod runs

​

• Runtime Handler [OCI]: Implement CRUST APIs

Next Steps

• Continue to refine KEP: https://github.com/kubernetes/enhancements/pull/2893

​

• Feedback from sig-node (Feb 8th): consider plumbing CRUST APIs through CRI. Maintaining runtime handler awareness in Kubelet is undesired.

Thank You!!