1 of 24

Spire Server runs only on the management cluster

The other option is to run a Spire server in all the clusters, and federation is configured between them. This has a huge overhead and is not acceptable.

There will be one Spire server running in the management cluster and a single trust domain.

The Spire server in the management cluster needs to be horizontally scaled either as a stateful set or a deployment.

TODO: Specify the spire server configuration here - trust domain, etc.

2 of 24

SPIFFE Concepts

Node Attestation

The attributes of the node are verified. The agent gets an ID

Workload Registration

Tell SPIRE how to identify the workload and which SPIFFE ID to give it.

For eg. give ID1 for pods with name UPF

Workload Attestation

The attributes of the Workload are verified.

The workload gets an identity based on the registration entry

3 of 24

Node Attestation Options

  • K8s Projected Service Account (k8s_psat) method
  • Join Token

4 of 24

Node Attestation

k8s Projected Service Account (PSAT)

method

5 of 24

Node Attestation: Across clusters

Each agent authenticates and verifies itself when it first connects to server

Node attestors interrogate a node and its environment for pieces of information that only that node would be in possession of, to prove the node’s identity.

Examples of proof of the node’s identity include: Possession of an Kubernetes Service Account token

The result of a successful node attestation is that the agent receives a unique SPIFFE ID.

The agent’s SPIFFE ID then serves as the “parent” of the workloads it’s in charge of.

6 of 24

Agent Configuration

Server Configuration

NodeAttestor "k8s_psat" {

plugin_data {

cluster = "MyCluster"

}

}

volumeMounts:

- mountPath: /var/run/secrets/tokens

name: spire-agent

volumes:

- name: spire-agent

projected:

sources:

- serviceAccountToken:

path: spire-agent

expirationSeconds: 600

audience: spire-server

NodeAttestor "k8s_psat" {

plugin_data {

clusters = {

"MyCluster" = {

service_account_allow_list = ["production:spire-agent"]

SaTokenForTokenReview = "path/to/saToken"

}

}

}

agent {

server_address 10.128.x.y

server_port 8081

}

7 of 24

Bootstrap Bundle

Spire Server generates the bootstrap bundle to a configmap

Bundle is to be shipped to the workload cluster (cluster API ?)

Using bundle, Spire Agent initiates node attestation

8 of 24

SaTokenForTokenReview

Service account created on workload cluster with Token Review perms

Secret created on workload cluster for the service account

Kubeconfig created on management cluster with service account token

9 of 24

Node Attestation: Spire Agent in same cluster as server

Server Configuration

NodeAttestor "k8s_psat" {

plugin_data {

clusters = {

"MyCluster" = {

service_account_allow_list = ["production:spire-agent"]

// SaTokenforTokenReview = "path/to/saToken" ===> Not needed as the k8s API Server is in-cluster, and spire server has required roles/roleBinding to do token Review

}

}

}

10 of 24

Node Attestation: Join token method

11 of 24

Join Token

Bootstrap Controller needs to push a join token to every node

  • How to push to every node ?

Or

Other option is to use a single join token for the entire cluster

The join token is sent to the Spire Server which authenticates it

The join token can then be deleted from the workload cluster node

12 of 24

Comparison of k8s_psat, join token methods

  • Bootstrap bundles
    • In both methods, the management cluster needs to push bootstrap bundles to workload cluster nodes
  • Token distribution
    • In the join token method, the management cluster needs to send across a single join token for the entire cluster.
    • In k8s_psat, a service account token is injected into each pod by k8s
  • Token Validation
    • In join token method, the token is a shared secret, and if the token sent by agent is present in server data store, the agent validation is successful.
    • In k8s_psat, the token validation is done by using the token review API of the workload cluster. For this another service token is needed to authenticate against the workload cluster API server.
  • Information available as part of token validation
    • In join token method, no additional information about node, agent is available here. Management cluster needs to maintain a mapping from join token to cluster_id.
    • In k8s_psat method, we get additional verified information such as node name, uid, label, pod name, pod uid, pod label, sa, ns. This can be used for selectors for node registration

13 of 24

Workload Registration

We need to register the workloads with the Spire Server.

We need to give out SPIFFE Ids selectively to those pods which need the IDs.

TODO: Another controller like https://github.com/spiffe/spire-controller-manager

14 of 24

Node SPIFFE ids

Each node can have multiple Ids

One id is assigned automatically as part of the node attestation process. Per-node identity

  • spiffe://<trust_domain>/spire/agent/k8s_psat/<cluster>/<node_id>
  • spiffe://<trust_domain>/spire/agent/join_token/<token>

Alternatively, we can use node selectors (join_token does not support selectors)

  • For example SPIFFE ID spiffe://example.com/nephio-edge-cluster-1 can be assigned to any SPIRE Agent running in a given cluster
  • All nodes in the cluster have labels - key1: wcl_1

15 of 24

Workload SPIFFE Ids

Per cluster UPF

spire-server entry create \

-parentID spiffe://example.com/nephio-edge-cluster-1 \

-spiffeID spiffe://example.com/edge-1/upf \

-selector k8s:pod_name:upf

UPF across all clusters

spire-server entry create \

-parentID spiffe://example.com/nephio-edge-cluster \

-spiffeID spiffe://example.com/edge/upf \

-selector k8s:pod_image:upf

16 of 24

Workload registration, attestation

17 of 24

Nephio Considerations: Node Attestation

18 of 24

Node Attestation: Spire Agent on workload cluster

  • The bootstrap secret controller pushes the bootstrap bundle using the cluster-api to the workload cluster.
  • The bootstrap secret controller also creates a WorkloadClusterSaToken on the workload cluster. This token can be used for token review only
  • The spire agent contacts the spire server with TLS using this bootstrap bundle certificate. In this TLS connection, the spire agent sends a projected sa token
  • The spire server does a token review of this projected sa token with with workload cluster.
  • Post review, the spire server hands out an SVID

19 of 24

Work items for Node Attestation

  1. Create the spire server, and the spire agent in nephio
  2. Bootstrap controller reads the bootstrap.crt and write into the config map when the workload cluster is created
  3. Bootstrap controller needs to write the SaTokenForTokenReview kubeconfig file to a config resource
  4. Spire agent reads the config map and initiates the node attestation
    1. The spire agent might be running and the bootstrap.crt may not yet be available
  5. Spire server needs to read the SaTokenForTokenReview kubeconfig and create a k8s client (contingent on using k8s_psat) - another option in the Spire server configuration is needed.

20 of 24

Work items for Node Registration, Workload Registration

  • Each node in a given cluster to be given a label
  • Add node registration
    • For each cluster, there is a single node id since we are not sure which node the workloads are running
  • Add workload registration - only for those workloads which need secrets
    • Gitea workload based on pod_lablel: app.kubernetes.io/instance: gitea
    • Bootstrap controller:
    • Bootstrap secret controller
  • Test cases
    • Required workloads have ids of the requisite teyp.

21 of 24

Appendix: Using SVID to access secrets

22 of 24

23 of 24

Authentication to the Secrets Store

The secrets store under consideration is Hashicorp Vault.

We can authenticate to vault using a JWT. Vault will use an OIDC discovery service to fetch the appropriate keys [1].

[2] gives details of how to setup OIDC federation between Vault, and the Spire Server. This will allow a SPIRE-identified workload to authenticate against a federated Vault server by presenting just its JWT-SVID.

TODO: Can this be extended to other Secrets stores - AWS KMS, etc.

[1 ]https://developer.hashicorp.com/vault/docs/auth/jwt

[2] https://spiffe.io/docs/latest/keyless/vault/readme/

24 of 24

Secret store needs to map SPIFFE identities to certain paths

Once the authentication to the secrets store is done, the SPIFFE id will be associated with a policy. In the policy we can specify the capabilities (read/write) associated with a given path

path "secret/my-super-secret" {

capabilities = ["read"]

}

TODO: Identify the set of policies. Also, we might need to dynamically create policies based on the workload cluster names. This will ensure that a given workload cluster cannot access secrets of other clusters.��path "secret/regional-cluster-1/my-super-secret" {

capabilities = ["read"]

}