Managing Kubernetes Secret using SealedSecret

Husni Alhamdani

Husni Alhamdani

Certified Kubernetes

• Administrator (CKA)
• Security Specialist (CKS)

Agenda

• The Problem
• SealedSecret Overview
• SealedSecret Components
• How it works?
• SealedSecret Scope
• SealedSecret Installation Guide
• Demo
• Backup / Disaster Recovery / Migration strategy

Solutions: Encrypt your Secret into a SealedSecret, which is safe to store - even to a public repository.

Problem: “I can manage all my k8s config in git, Except Secrets.”

Kubernetes Secret?

“Kubernetes Secrets let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. Storing confidential information in a Secret is safer and more flexible than putting it verbatim in a Pod definition or in a container image”

Built-in Kubernetes Secret

apiVersion: v1

kind: Secret

metadata:

name: test-secret

data:

username: bXktYXBw

password: Mzk1MjgkdmRnN0pi

echo -n 'my-app' | base64 → bXktYXBw

echo -n '39528$vdg7Jb' | base64 → Mzk1MjgkdmRnN0pi

Let use SealedSecret to solve this problem

SealedSecret Overview

A Kubernetes controller and tool for one-way encrypted Secrets, developed by Bitnami

It use asymmetric crypto (AES-256) to encrypt secrets that only the controller can decrypt

Without the private key that is managed by the SealedSecret,

there is no way to decrypt the encrypted data within a SealedSecret

SealedSecret Components

Client Side:

• Kubeseal: CLI tools to create SealedSecret CRD

Cluster Side

• Controller: Watch the SealedSecret CRD and Decrypt it into Kubernetes Secret

Kubeseal

To Encrypt the Secret

To Decrypt the Secret (KEEP ME PRIVATE)

How it works?

Secret

SealedSecret

Secret

SealedSecret Controller

Encrypting process (Kubeseal)

Decrypting process (Controller)

Once it deployed..

Image source: https://engineering.bitnami.com/

SealedSecret Scope

Three different ways/behavior that you can create a Sealed Secret :

1. Strict (default): can’t change the name and the namespaces of your SealedSecret once you’ve created it
2. Namespace-wide: freely rename the SealedSecret within the namespace
3. Cluster-wide: freely move the Secret to any namespace and give it any name you wish

$ kubeseal --scope cluster-wide --format yaml < secret.yaml > sealed-secret.yaml

Example:

Installation

$ helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets

$ helm upgrade -i -n kube-system sealed-secrets sealed-secrets/sealed-secrets

1. Controller

2. Kubeseal

GOOS=$(go env GOOS)

GOARCH=$(go env GOARCH)

wget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.13.1/kubeseal-$GOOS-$GOARCH

sudo install -m 755 kubeseal-$GOOS-$GOARCH /usr/local/bin/kubeseal

Example

$ kubectl create secret generic basic-auth \

--from-literal=user=admin \

--from-literal=password=admin \

--dry-run=client \

-o yaml > basic-auth.yaml

$ kubeseal --format=yaml --controller-namespace=kube-system --controller-name=sealed-secrets \

< basic-auth.yaml > basic-auth-sealed.yaml

Create the Secret

Encrypt with SealedSecret:

Option 1 (require Kubernetes access)

$ kubeseal --fetch-cert --controller-namespace=kube-system --controller-name=sealed-secrets > sealedsecret-pub-cert.pem

$ kubeseal --format=yaml --cert=sealedsecret-pub-cert.pem < basic-auth.yaml > basic-auth-sealed.yaml

Option 2 (without Kubernetes access )

Demo time

Prepare for Disaster Recovery / Migration

Old Kubernetes Cluster

New Kubernetes Cluster

$ kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > sealed-secret-backup.yaml

$ kubectl delete secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key

$ kubectl create -f sealed-secret-backup.yaml

$ kubectl delete pods -n kube-system

Old cluster

New cluster

SealedSecret Controller

Public Key

Private Key

SealedSecret Controller

Public Key

Private Key

Learn more

https://github.com/bitnami-labs/sealed-secrets

Thank you!

https://t.me/husnialhamdani