1 of 7

kiosk Multi-Cluster Design

2 of 7

Multi-Cluster problems

Tenant Admin:

  • How can I manage accounts across multiple clusters? (Multi-Cluster account management)
    • How many spaces can they create or do they own in each cluster?
    • How many computing resources can they use in each cluster?
    • How can I enforce custom restrictions / policies for those spaces?

​

Tenants:

  • What clusters, accounts … do I have access to?

3 of 7

Solution 1: Remote Controllers

  • Management Clusters and Managed Clusters
  • Reconciles done in the management clusters instead of managed clusters

​

  • Advantages:
    • No kiosk installation needed in managed cluster
    • Cluster spanning account quota (potentially) possible
  • Disadvantages:
    • Kind of an anti-pattern (especially remote admission controllers)
    • Increased controller complexity (caching, reconciles, cluster added/removed)
    • Latency might be an issue
    • Space / Account creation etc. only possible in management cluster

​

​

4 of 7

Solution 2: Syncing Resources

  • Syncing of relevant resources to target clusters
    • Accounts
    • AccountQuotas
    • Templates
    • (Related ClusterRoles?)

​

  • Advantages:
    • Reduced controller complexity (almost no controller changes are necessary)
    • Hierarchies of clusters possible
    • Reduced pressure on management cluster controller
  • Disadvantages:
    • Duplication of resources
    • How to merge out of sync resources? (three way merge?)

=> Personally, I think syncing resources seems like the better approach

5 of 7

API Resources Changes

  • New Cluster Resource that defines managed cluster access (basically just a secretRef to a kubeConfig)
  • In addition, we need to define a cluster in Space where the Space should be created and in Accounts and AccountQuotas (and Templates? & TemplateInstances?) which clusters should be targeted / the resource be synced to

​

​

6 of 7

Manager Changes

  • Account controller:
    • reconcile syncs the account to the needed clusters
    • sync the default templates defined in account (should there be also a more verbose way to specify in template where it should be synced to?)
    • save in status old value of clusters, to calculate a delta on change (similar to resource quota hard and status.hard)
    • Only sync relevant part of account to target cluster
  • Account quota controller:
    • Create a new account quota controller that essentially does the same thing as the account controller in terms of sync
  • Potential template controller change:
    • Same as account quota controller

​

7 of 7

API Server Extension Changes

  • kubectl get spaces gets all accounts and gets all clusters for each account. Then it essentially does a kubectl get spaces for the user in each target cluster and combines the output
  • New cluster view (similar to account and space) where only the allowed clusters can be viewed for each user
  • If a cluster in a space is defined, create space forwards the call to the target cluster