Spire Server runs only on the management cluster
The other option is to run a Spire server in all the clusters, and federation is configured between them. This has a huge overhead and is not acceptable.
There will be one Spire server running in the management cluster and a single trust domain.
The Spire server in the management cluster needs to be horizontally scaled either as a stateful set or a deployment.
TODO: Specify the spire server configuration here - trust domain, etc.
SPIFFE Concepts
Node Attestation | The attributes of the node are verified. The agent gets an ID |
Workload Registration | Tell SPIRE how to identify the workload and which SPIFFE ID to give it. For eg. give ID1 for pods with name UPF |
Workload Attestation | The attributes of the Workload are verified. The workload gets an identity based on the registration entry |
Node Attestation Options
Node Attestation
k8s Projected Service Account (PSAT)
method
Node Attestation: Across clusters
Each agent authenticates and verifies itself when it first connects to server
Node attestors interrogate a node and its environment for pieces of information that only that node would be in possession of, to prove the node’s identity.
Examples of proof of the node’s identity include: Possession of an Kubernetes Service Account token
The result of a successful node attestation is that the agent receives a unique SPIFFE ID.
The agent’s SPIFFE ID then serves as the “parent” of the workloads it’s in charge of.
Agent Configuration
Server Configuration
NodeAttestor "k8s_psat" {
plugin_data {
cluster = "MyCluster"
}
}
volumeMounts:
- mountPath: /var/run/secrets/tokens
name: spire-agent
volumes:
- name: spire-agent
projected:
sources:
- serviceAccountToken:
path: spire-agent
expirationSeconds: 600
audience: spire-server
NodeAttestor "k8s_psat" {
plugin_data {
clusters = {
"MyCluster" = {
service_account_allow_list = ["production:spire-agent"]
SaTokenForTokenReview = "path/to/saToken"
}
}
}
agent {
server_address 10.128.x.y
server_port 8081
}
Bootstrap Bundle
Spire Server generates the bootstrap bundle to a configmap
Bundle is to be shipped to the workload cluster (cluster API ?)
Using bundle, Spire Agent initiates node attestation
SaTokenForTokenReview
Service account created on workload cluster with Token Review perms
Secret created on workload cluster for the service account
Kubeconfig created on management cluster with service account token
Node Attestation: Spire Agent in same cluster as server
Server Configuration
NodeAttestor "k8s_psat" {
plugin_data {
clusters = {
"MyCluster" = {
service_account_allow_list = ["production:spire-agent"]
// SaTokenforTokenReview = "path/to/saToken" ===> Not needed as the k8s API Server is in-cluster, and spire server has required roles/roleBinding to do token Review
}
}
}
Node Attestation: Join token method
Join Token
Bootstrap Controller needs to push a join token to every node
Or
Other option is to use a single join token for the entire cluster
The join token is sent to the Spire Server which authenticates it
The join token can then be deleted from the workload cluster node
Comparison of k8s_psat, join token methods
Workload Registration
We need to register the workloads with the Spire Server.
We need to give out SPIFFE Ids selectively to those pods which need the IDs.
TODO: Another controller like https://github.com/spiffe/spire-controller-manager
Node SPIFFE ids
Each node can have multiple Ids
One id is assigned automatically as part of the node attestation process. Per-node identity
Alternatively, we can use node selectors (join_token does not support selectors)
Workload SPIFFE Ids
Per cluster UPF
spire-server entry create \
-parentID spiffe://example.com/nephio-edge-cluster-1 \
-spiffeID spiffe://example.com/edge-1/upf \
-selector k8s:pod_name:upf
UPF across all clusters
spire-server entry create \
-parentID spiffe://example.com/nephio-edge-cluster \
-spiffeID spiffe://example.com/edge/upf \
-selector k8s:pod_image:upf
Workload registration, attestation
Nephio Considerations: Node Attestation
Node Attestation: Spire Agent on workload cluster
Work items for Node Attestation
Work items for Node Registration, Workload Registration
Appendix: Using SVID to access secrets
Authentication to the Secrets Store
The secrets store under consideration is Hashicorp Vault.
We can authenticate to vault using a JWT. Vault will use an OIDC discovery service to fetch the appropriate keys [1].
[2] gives details of how to setup OIDC federation between Vault, and the Spire Server. This will allow a SPIRE-identified workload to authenticate against a federated Vault server by presenting just its JWT-SVID.
TODO: Can this be extended to other Secrets stores - AWS KMS, etc.
[1 ]https://developer.hashicorp.com/vault/docs/auth/jwt
Secret store needs to map SPIFFE identities to certain paths
Once the authentication to the secrets store is done, the SPIFFE id will be associated with a policy. In the policy we can specify the capabilities (read/write) associated with a given path
path "secret/my-super-secret" {
capabilities = ["read"]
}
TODO: Identify the set of policies. Also, we might need to dynamically create policies based on the workload cluster names. This will ensure that a given workload cluster cannot access secrets of other clusters.��path "secret/regional-cluster-1/my-super-secret" {
capabilities = ["read"]
}