Kubernetes Major Architectural Efforts Underway

Brian Grant (@bgrant0607)

Last updated: 11/1/2017

Status: not maintained

We need to propose rough timelines/deadlines for each of these. Ideally these could have schedules for deprecating and freezing the Old Way, alpha/MVP for the New Way, GA for the New Way, and elimination of the Old Way.

  • Multiple github orgs
  • github.com/kubernetes and github.com/kubernetes-incubator aren’t sufficient. We’ve started down the path of creating more orgs, such as kubernetes-client, for specific subprojects. If a subproject needs more than 2-3 repos, it probably needs an org.
  • TODO: specific changes proposed
  • One possibility: Start with per-SIG repos or orgs, and create new repos and orgs as needed.
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • 1.10: At least 1 feature branch
  • 1.11: Whitelisted development in k/k/m only
  • 1.12
  • 1.13
  • Cloud provider
  • The Kubernetes API shouldn’t contain cloud-provider-specific concepts, and kubernetes/kubernetes shouldn’t contain cloud-provider-specific code.
  • The cloudprovider API is approximately frozen.
  • New cloudprovider implementations are not being accepted.
  • The external cloudprovider proposal has been approved.
  • cloud-provider=auto-detect
  • ssh tunnels and apiserver cloudprovider flags/options are deprecated
  • Volume sources are a special case, and discussed below.
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13: Remove ssh tunnels and apiserver cloudprovider code.
  • /cluster and kube-up.sh deprecation
  • The /cluster directory is deprecated.
  • New kube-up provider implementations are not being accepted.
  • Replace in e2e tests
  • Remove from release bundle
  • Remove from kubernetes/kubernetes
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • API machinery needed to be made available to other repositories without vendoring all of kubernetes/kubernetes, in order to be able to develop anything in other repositories, since almost all code in and around Kubernetes needs client libraries, API types, and/or other parts of the API machinery for configuration, interacting with Kubernetes, and/or providing new Kubernetes-style APIs. Furthermore, the API machinery is being adopted by ecosystem projects, such as CRD users.
  • We need a better API IDL (than just Go types + comments and field tags)
  • We need a client SDK
  • We need a server SDK
  • There needs to be a way to define CRDs.
  • The generic parts of kubectl need to work for all Kubernetes-style APIs (“apictl”).
  • API machinery extension
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • Kubectl
  • Kubectl dependencies are being disentangled in order to move it to the kubernetes/kubectl repo.
  • We are moving functionality to server in support of API extension (API aggregation and CRDs) and correct behavior with version skew, aligned with API stability/deprecation guarantees.
  • get/describe rendering needs to be generic
  • General-purpose functionality needs to move to reusable libraries
  • TODO: plugins
  • Once extracted, kubectl should move to independent releases
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • We are moving away from command-line flags for component configuration.
  • We are using the Kubernetes API machinery for configuration as well as for APIs.
  • Kube-proxy
  • Kubelet
  • Scheduler
  • Apiserver
  • Controller-manager
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • APIs -> API groups
  • extensions/v1beta1 needs to be deprecated and eliminated
  • core/v1 APIs need to be migrated to appropriate API groups, along with a bounded number of “v2 API” changes
  • Proposed timeline
  • 1.9
  • Workloads v1
  • Events
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • Proposed timeline
  • 1.9
  • Vault
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • CRI
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • CNI
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • CSI
  • New Volume sources are frozen. FlexVolume must be used for new sources currently.
  • CSI is under development as the backend plugin API for block and file devices.
  • Existing ad hoc network-attached volume sources should be deprecated, replaced by a principled extension API, which would appear in the v2 Pod and PersistentVolumeClaim APIs.
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • Device plugins
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • Endpoints, Service, and Ingress
  • Ingress by default - batteries included
  • Deprecate service link environment variables
  • Endpoints v2 (in new API group)
  • Independence from services (Nucleus vs. Application Layer)
  • Non-Pod endpoints
  • Component and extension registration
  • Scalability: not stored as a single key per service
  • Rethink APIs
  • Multi-cluster endpoints and ingress
  • Kube-proxy and service mesh futures
  • Proposed timeline
  • 1.9
  • 1.10: Batteries-included Ingress
  • 1.11
  • 1.12
  • 1.13
  • Map[string]string is not enough
  • Proposed timeline
  • 1.9
  • 1.10
  • 1.11
  • 1.12
  • 1.13
  • Move container identity (service accounts, tokens, and pod injection) out of the masters and make extensible
  • Gets secrets that are powerful out of etcd
  • Paves the way for extensions to replace workload identity mechanism (cloud provider, spiffe, enterprise)
  • Timeline
  • 1.10
  • 1.11
  • Metrics and resource accounting used to make healthier clusters
  • Standardize metrics server, deprecate heapster
  • Enable third party extension for better metrics for HPA and overcommit and scheduling
  • Add default vertical autosizing
  • Better quota and limit to prevent accidental cluster death
  • Timeline
  • 1.10
  • 1.11
  • 1.12