1 of 11

Kubernetes Multi-Cluster deployments with GKE

Kishore Jagannath,

Strategic Cloud Engineer in Google,

kishorerj.medium.com

2 of 11

Agenda

Introduction to GKE Cluster
Multi cluster motivation and challenges
Multi cluster architectures
Inter cluster communication with Service Export
Load balancing services across clusters with Multicluster Ingress
Demo

3 of 11

GKE Cluster

A GKE cluster consists of Control Plane and Data Plane. Clusters can be either zonal or regional

Control Plane

Managed by GCP and consists of all kubernetes components

Data Plane

Consists of worker nodes, which runs users application specific pods and other kubernetes resources.

4 of 11

Motivation

Availability:

Spreading clusters across regions provides better disaster recovery. Workloads are shielded from cluster failures

Location:

Place location specific workloads in specific regions for compliance, latency and Data gravity etc.

Scalability:

Operate beyond the scalability limits of a single cluster in GCP

Isolation:

Isolate different environments, support cluster per tenant model, workloads sensitive to upgrades

5 of 11

Challenges

Overhead of Managing multiple clusters instead of single cluster
Facilitate east-west communication of services across multiple clusters.
Scale services across multiple clusters.
Facilitate north-south external communication of services across multiple clusters.

6 of 11

Multi Cluster in GKE

gcloud container fleet memberships register MEMBERSHIP_NAME --gke-uri=GKE_URI --enable-workload-identity

Namespace sameness

Identity Sameness

Service Sameness

7 of 11

Multi Cluster architectural Patterns

8 of 11

Multi-cluster architecture(contd)

9 of 11

Multi Cluster Services

Facilitates east west communication between dependant services.

Clusters belonging to same fleet can export services.

Exported services becomes available to all clusters within Fleet.

Single VIP across fleet

Pods can invoke dependant services across cluster boundaries via implicitly created Service Import.

DNS: SERVICE_EXPORT_NAME.NAMESPACE.svc.clusterset.local

After you create a ServiceExport object, the following domain name resolves to your exported Service from any Pod in any fleet cluster:

SERVICE_EXPORT_NAME.NAMESPACE.svc.clusterset.local

When u create a Service Export Object a Virtual IP for the service is created across the entire fleet. You can also create similar services in a different cluster and expose this with same multi cluster service name. In that case the single virtual IP will front end the service across different clusters.. The multi cluster service will be automatically imported into all clustrers in a fleet. So any other cluster wgich wants to access this service can ping the virtual IP or the DNS for clusterset/fleet mentioned above. Also ServiceImport resources get created automatically u can go kubectl get serviceimport and kubectl get serviceexport.

NOTE: A Headless service has Cluster IP none so anyone connecting to service by default connect to individual pods directly and each pod has a default DNS.. This is useful for statetful sets as you can connect to the exact pod instead of a service doing load balancing and connecting to different pods. For MCS each pod is identified by pod-name.cluster-id .. Cluster-id is important in DNS because the MCS could be running in multiple clusters so any consuming service needs to identify the exact pod in a specific cluster in which it is running to invoke this pod.

10 of 11

Multi-cluster Ingress

Multiple clusters added to a fleet.One of the cluster is a config cluster.

Config cluster hosts the MultiClusterServices and MultiClusterIngress objects

MulticlusterService creates a copy of the service across all clusters registered to Fleet, can be restricted with links.

MultiClusterIngress creates a Global GCP based LB.

11 of 11

Demo