Kubernetes On-Prem Comparison
 Share
The version of the browser you are using is no longer supported. Please upgrade to a supported browser.Dismiss

Comment only
 
 
ABCDEFGHIJKLMNOPQRSTUVWXY
1
VMware Enterprise PKSOpenShift Container Platform (v4.1)Anthos On-Prem (GKE On-Prem)Mesosphere Kubernetes (MKE)RancherCharmed KubernetesMicroK8sKubermaticSUSE CaaS Platform v4MetalK8sIBMDaoCloud EnterprisePlatform9Giant SwarmNirmataMirantis Cloud PlatformKublrDocker Enterprise NutanixCisco Container Platform
2
CompanyPivotal/VMwareRed HatGoogleMesosphereRancher LabsCanonicalCanonicalLoodseSUSEScalityIBMDaoCloud Inc.Platform9 Systems Giant SwarmNirmata, IncMirantisKublrDockerNutanix KarbonCisco
3
Kubernetes GA Versions
v1.13.5v1.13v1.14v1.13.5v1.13.5v1.18 - 1.14v1.18 - 1.14v1.13v1.15v1.11v1.11v1.10v1.10v1.12v1.13v1.12v1.14.7v1.8
4
SDNNSX-TOpenShift Multi-Tenant SDN (based on OVS), plus CNI-compliant network plugins (Flannel, Nuage, Essentials (Calico), Open Contrail, Cisco Contiv, BigSwitch, VMware NSX-T, OVN, and Kuryr for OpenStack network integration)Calico + NSX-TCalicoCanal, Flannel, Calico, WeaveworksCanal, Flannel, Calico, Tigera Secure EE, and CNI pluginsHost bridge, CiliumCanal, CalicoCiliumCalicoCalicoCalico, MacVlan, FlannelCalico, Weave, Flannel, OpenContrailCalico, FlannelFlannel, Calico, Cloud ProviderCalico, Tungsten FabricCalico, Canal, Flannel, Weave, generic CNICalico
5
StoragevSAN, VMFS, iSCSI, FCOpenShift Container Storage (OCS), as well as native support for CSI plugins, NFS, iSCSI, Fiber Channel, OpenStack Cinder, FlexVolume, GlusterFS, Ceph RBD, Azure Disk, Azure File, AWS EBS, GCE Persistent Disk, VMware VMDKvSAN, VMFS, any CSI compatible or supported by KubernetesCSI, local, PortworxAll volume types supported by KubernetesCeph, NFS, iSCSI, FTP, hostpath. NetApp Trident (OnTAP, Santricity, etc), StorageOS and Pure. Cloud storage on Openstack (Cinder), AWS (EFS/EBS), GCP Persistent Disk, AzureHost pathCustomSUSE Enterprise Storage (Ceph), NFS, hostpathNative LVM, NFS, custom NFS,Portworxdocker enterprise vs. openshift vs. rancher 2.xNFS, iSCSICloud Provider, Rook, Portworx, Nutanix, NFSCeph, Rook, EMC, NetApp, etc.Rook/Ceph, any CSI compatible or supported by KubernetesNFS
6
Security FocusModerateHighHighHighHighHighHighModerateHighModerateHighHighHigh
7
Node OSUbuntuRHEL CoreOS and RHEL 7/8Ubuntu, COS (Container Optimize OS) coming soonLinux - variousRancherOS, K3OS, Ubuntu, RHEL, Debian, Fedora, OpenSUSE, CoreOSUbuntu - x86, ARM, ZUbuntu, Debian, Fedora, Arch, other Linux (AMD64 and ARM64)Ubuntu, CoreOS, Container LinuxSLES 15 SP1CentOSUbuntuLinuxLinux - CentOS, RHEL, Ubuntu all supportedCoreOSCentOS, RHEL, UbuntuUbuntuRHEL, CentOS, Ubuntu, Debian, SuSECentOS, OEL, RHEL, SLES, Ubuntu, Windows, (incl s390x and ppc64le)
8
Vanilla KubernetesYesNOYesYesYesYesYesYesYesYesYesYesYesYesYesYesYes
9
Installation Complexitymoderatesimplesimple, automated via gkectlsimplesimplesimplesimplesimplesimplesimplesimplesimplesimple - a SaaS managed service SaaS, or consulting deploymentsimple - SaaSsimplesimple
10
Managed OfferingsOpenShift Dedicated (on AWS and GCP), Azure Red Hat OpenShift, Atos Managed OpenShift, T-Systems AppAgile, Telstra OpenShift DedicatedFully managed + Cloud GKE fully managed on GCPNoneYes. Fully Managed Kubernetes on bare metal, VMware, AWS, OpenStack, Azure, Google, Oracle CloudKublr Cloud, Kublr Accelerators
11
Cost as a % of OpenShift50%100%Subscription per vCPU/nodesimple20%20%FreeSubscription per DC + nodeSubscription per yearSubscription~20-50%, Subscription30%
12
Trial Available30 days30 days - 60 daysPoC possible (Ask Sales) - Fully autonomous coming soon.Open Source - yes, forever
DC/OS Enterprise - Eval and PoC available
UnlimitedFree, optional support from CanonicalFree, optional support from Canonicalonline demo system60 days with updatesFree with no supportYesPoC possibleYes (SaaS), PoC (On-prem)Yes; free unlimited trial license; PoC30 days
13
Supported On-Prem IaaS LayerVMwareBare Metal, OpenStack, VMwareVMware, Bare Metal workers coming soonBare metal, penStack, HyperV, RHV, VmwareUnlimitedBare metal, Openstack, KVM, VMware. AnyOpenStack, VMwareBare metal, KVM, Xen, VMware vSphere/ESXi, Microsoft HyperV, SUSE Bare metal, VMWare, OpenStack-Bare metal, VMWare, OpenStackBare metal, VMware, OpenStackBare Metal, VMwareBare Metal, VMwareBare Metal, OpenStackVMware
14
Cost$500 per pod (application)$1200 per virtual core (workers only) - Two pricing models (OCP and OpenShift Container Engine (OCE), as well as Managed Service Offerings. Pricing is dependent on capabilities and usage of the platform. Ask SalesFree and open source. Enterprise subscription (support and other benefits) available but not required.Free and open source.
Support available for:
$1500 per server, $500 per VM
Free and open source.
Support available for:
$1500 per server, $500 per VM
0%20%Starting at $1000/moDocker EE for RHEL - Advanced $2400 per Node
15
ConfigBOSHAll native Kubernetes tools - Operators and machine controllers to automate full stack provisioning and config of your cluster, including underlying cloud infra, OS hosts, kubernetes masters and workers and all integrated services.Automated via gkectl, updated via custom CRD and YAML configuration files. Terraform + Ansible for DC/OS installation. Full Service Automation for Mesosphere Kubernetes EngineRKE is provisioned through a YAML configuration file. Rancher installs into an RKE cluster. Nodes provisioned by Rancher are configured through the Rancher UI.AutomatedSingle-packageCRDSaltAnsibleCRDs + Ignition + PXESaltKublr API, CLI, UI
16
Documentationhttps://docs.pivotal.io/pks/1-4/index.htmlhttps://docs.openshift.com/ tutorials at http://learn.openshift.comhttps://cloud.google.com/gke-on-prem/docs/https://docs.mesosphere.com/services/kubernetes/latesthttps://rancher.com/docs/https://ubuntu.com/kubernetes/docs/https://microk8s.io/CI/CD + Dev-Toolshttps://www.ibm.com/cloud/privatehttp://guide.daocloud.io/dce-v3.0/latest/dce-3-0-lts-20974188.htmlhttps://docs.platform9.com/https://docs.giantswarm.iohttps://docs.nirmata.io/docs.mirantis.comhttps://docs.kublr.com/https://docs.docker.com/ee/https://www.nutanix.com/2018/11/27/nutanix-karbon-enterprise-grade-kubernetes-solution/https://www.cisco.com/c/en/us/products/cloud-systems-management/container-platform/index.html
17
Preferred Load BalancerNSX-THAProxy (default), F5/Netscaler/NGINX (can be plugged in)F5 BIG-IP, Manual LoadBalancing, Citrix, Seesaw, Integrated coming soondklb, Edge-LB, optional external LBNginx, NetScaler (Citrix ADC)F5 / Nginx, IstioBuilt-in LB, IstioF5 / NginxNginxIstioHAProxy,NginxOctavia, AVINGINXHAProxy, Citrix CPX (NetScaler), Cloud ProviderMetalLB, NGINX includedAny supported by k8s; included OOB cloud native LBs, nginxNginx
18
Integrated ServicesNetworking, Storage, Registry, Logging, Monitoring, Service Mesh, Serverless, Self-Service Catalog, Container Content Networking (Istio) fully managed with Anthos Service Mesh, Storage, Registry, Logging and Monitoring, Service Mesh, Serverless, App Catalog/Marketplace, Multicluster managment (Anthos Config Managment), Centralized UINetworking, storage, logging, monitoring, app catalog, multi-cluster management, pipelines, alerts, global policy management, projectsNetworking, Storage, Registry, Logging, Monitoring, Service Mesh, Container Content IstioMulti-cluster, IdM/AAA/RBAC, Kubernetes RBAC integration, infrastructure provisioning, centralized logging/ monitoring/ alerting, networking
19
Container RegistryHarborOpenShift Integrated Registry, Red Hat Quay, or 3rd-party registryGoogle Container RegistryAny external registryIncluded or configurableIncluded or configurableAny external registry
20
Multicluster ManagementNoNoYesNoNoneNoNoNo
21
Container Content (ISV applications)Red Hat Container Catalog, OperatorHub (integrated) or 3rd-party ISV contentGoogle Marketplace catalogApplication catalog
22
Upgrade mode (automated/manual)ManualFully Automated
23
Serverless container supportCloud Run (Knative)
24
Service Mesh integration/supportAnthos Service Mesh (istio custom distribution) Anthos Service Mesh Dashboard Anthos Service Mesh certificate authority
25
Monitoring, LoggingIntegrated with local stackdriver log forwarders and prometheus. Logging servers, node exporters and metrics servers
26
Configuration ManagementAnthos Config Management
27
Management UIFully integrated with Google Cloud kubernetes managment console
28
PKSOpenShift (container platform)Anthos On-Prem (GKE On-Prem)MesosphereRancherCharmed KubernetesMicroK8sKubermaticSUSE CaaS Platform v4MetalK8sIBMDaoCloud EnterprisePlatform9Giant SwarmNirmataKublrDocker Enterprise
29
Why?Integrates well with VMWare. Cheaper than OpenShift with a comparable ecosystem.Largest installed base of Enterprise Kubernetes. Most flexible architecture (either CaaS or PaaS or FaaS) and ecosystem plugins. Managed/Hosted Offerings (OpenShift Dedicated and Azure Red Hat OpenShift). Large stable company, massive ecosystem, security hardened, 9 years of support.Platform that offers "one pane of glass" for GCP cloud like experience. Offers ability to migrate workloads from VM to containers using Anthos Migrate . Offers ability to interact with other clouds (AWS, Azure, for example) through same single pane of glass. Also, as mentioned in Google's Application Modernization and the decoupling of Infrastructure white paper, "Anthos framework enables decoupling across critical components..." Infrastructure from apps, teams from each other, development from operations, security from devops. Successful decoupling of infrastructure, services and teams minimizes the need for manual coordination, reduces cost and complexity, and significantly increases developer velocity, operational efficiency and business productivity. Mesosphere Kubernetes Engine is a pure open source Kubernetes offering with additional benefits such as;
- High Density Multi-Kubernetes - allowing you to run multiple Kubernetes clusters on the same operating systems without the need for virtualization.
- Full Service Automation - push button cloud-like deployment
- Full HA - simply check a box to ensure HA with health monitoring ensuring continual availability and repair in the event of a failure
- Consistent Kubernetes regardless of IaaS - ensure the same developer and operator experience across all your infrastructure providers. Public cloud providers do not all offer the same version of Kubernetes leading to inconsistencies in versions, features, automation and security
 - Templates for Kubernetes cluster deployment - define once and deploy as required
- Single control plane for operators of Kubernetes clusters

Further advantages when;
- deploying across more than one infrastructure provider with stretch and linked DC/OS clusters
- stateful applications with persistent storage requirements
- big data and analytics workloads such as Spark, Kafka, Cassandra, Hadoop, HDFS, HiveMQ, Elastic
- ensure portability across all providers
- HA across providers


KUDO - Kubernetes Operators declarative creation framework without needing code
100% Open Source solution embracing the CNCF model. Free software, inexpensive support contract, simple setup and maintenance, awesome team. Can provision K8s on many platforms. No vendor lock-in! Designed for edge, on-prem, hybrid and multi-cloud environments. The most flexible solution.Supports more clouds, metal, virt. Free with support available. Multi-cluster management is easy. Multi-cloud integration is great. The support pricing is very attractive and a fully-managed service is also available. Robust, secure, single-node k8s for IoT, edge, developer workstation, CI/CDKubernetes in Kubernetes architecture means a smaller footprint and complete cluster isolation plus HA master control plane. This architecture is unique.Entire stack supported by one enterprise vendor with a large hardware and cloud provider ecosystem, at a lower cost than similar competitors.Free, open source, no commercial people pushing it and being annoyingFully managed Kubernetes with 24/7 SLA of 99.9% uptime and availability for Kubernetes clusters promised by contract. Easy setup and upgrades. Auto detection and repair of issues.
Can provision Kubernetes on VMware, Linux/KVM, OpenStack, Bare metal and AWS. Comes with it's own IaaS layer provisioning if the customer does not have one.
Running K8s in production on-premise for large enterprises such as Adidas, AMAG and Vodafone since 2016. CI/CD pipeline into every installation that facilitates daily updates, hot-fixes, and security patches. Provided update on latest K8s security bug for all supported Kubernetes versions within 24h. Proactive Troubleshooting as Giant Swarm fixes any issue in one of their hundreds of managed Kubernetes clusters across all their customers preventing everyone else running into the same issue. Providing upgrades for all managed open source components including K8s within 30 days after latest release. Guaranteed uptime of 99.95% via SLAs. Providing lots of additional managed services within your K8s clusters. Their support via Slack is outstanding. Offered as cloud based service and dedicated on-prem. Also works with managed Kubernetes services as well (EKS, GKE, AKS). Easy to use. Built for teams providing centralized container-as-a-service to multiple teams. Multi-tenant, multi-cloud, multi-clusterAddresses key challenges in running k8s on-prem by pre-integrating Ceph, Calico, CoreDNS, MetalLB, NGINX, as well as lifecycle management and continuous monitoring systems. Offered through build-operate-transfer delivery model with up to 99.99% SLA guarantees and zero lock-in.Comprehensive enterprise-ready Kubernetes management platform for Ops teams to centrally deliver Kubernetes-as-a-Service across an enterprise (self-service capabilities for BUs/Dev/QA...).
Adds governance, security, operations components around vanilla k8s.
Any interaction on top of Kublr is with pure vanilla Kubernetes (no lock-in).
Delivers uniform Kubernetes deployments across multiple infrastructure providers.
Multi-site Kubernetes capability. Source-available license for enterprise customers.
Security: Unified RBAC across Kubernetes and registry. FIPS certified. Platform: Supports Windows containers and other architectures for Linux like Z and P.
30
Why Not?Vendor lock in. Can get complex (see NSX-T documention for an example). BOSH may scare some people.Cost. Vendor lock in. Major and frequent architecture changes. IBM.....Cost.Vendor lock in. If you're a very small team, have only a single Kubernetes cluster, or only running applications in a single provider - other less mature Kubernetes offerings may be easier to start with.
Smaller company. Initial installer does not provision infrastructure as part of installation.Vendor lock in. No PAAS, just upstream K8s.Single-node specialized distributionSmaller company (risk of acquisition), you'd need to setup an IaaS layer or manage bare metal.I'd expect this to get sunset after the Redhat acquisition.All of the docs are in ChineseYou need to setup IaaS layer (or bring your own cloud)Vendor lock in. Current version uses non-native RBAC
31
32
33
There's an opinionated blog about all of this here: https://kubedex.com/redhat-openshift-vs-pivotal-pks-vs-rancher/
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Loading...