Support for SPIRE (SPIFFE Runtime Environment), a production-ready implementation of SPIFFE, was introduced to Istio in 1.14. Thanks to Envoy’s SDS API, SPIRE can be configured as a source for issuing Istio workload identities.
In addition to issuing strongly attested identities through a combination of different attestation mechanisms, SPIRE can also be integrated with existing PKIs, and allow the federation of different trust domains.
These features offer support for diverse workload and node attestation options by using attributes from both the workloads and the nodes to create more granular identities compared to the traditional trust domain, Kubernetes namespace, and service account combination.
RBAC flaws
Kubernetes has, for many years, defaulted to service accounts, or role-based access control (RBAC), to establish workload identity.
While RBAC is a very common form of access control, it has several very well-understood flaws that also manifest in the Kubernetes space:
- RBAC does not establish or attest identity. It establishes and assigns permissions to groups and roles — not individuals (or workloads). And roles are not the same things as identities. Your role is not your identity.
- RBAC only supports static access decisions. RBAC is fundamentally unable to make dynamic access decisions based on environmental and contextual inputs.
- Functionally, RBAC is a form of perimeter-based trust – it relies on non-granular roles (perimeters) that only have meaning and context within a single organization (perimeter) that defines them. It is simply not readily compatible with a zero trust model, which ultimately is about abandoning reliance on perimeters as the primary form of policy enforcement, nor is it in any way compatible with cross-organizational data policy enforcement.
In order to mitigate some or all of the above, the only recourse in RBAC is “role explosion.” To achieve the required granularity in policy enforcement, RBAC systems of any size have a strong tendency to devolve into a mess of lots of roles — too many to keep track of.
As the Istio service mesh expands across clusters and outside of clusters, as personal data continues to flow over perimeters, and as industry zero trust requirements render older and less effective access control models inadequate, being able to establish a unique identity (and cryptographically attesting that identity) for data-handling workloads becomes as important as establishing a unique identity for a person.
SPIRE and SPIFFE
SPIRE and SPIFFE are CNCF incubating projects that neatly fill that gap. SPIRE is an implementation of the SPIFFE standard that effectively acts as an identity provider for workloads in the same way that, for instance, OIDC providers act as identity providers for humans.
SPIRE has the following characteristics:
- Uses in-band and out-of-band mechanisms to prove that a workload asking for a credential is in fact who they say they are.
- Issues, validates, revokes, and rotates identity documents containing a human-parsable workload identifier, common across x.509 and JWT identity document formats.
- Exposes APIs (including an Envoy-compatible SDS endpoint) to facilitate the above.
SPIRE works via a chain of attestation:
- The SPIRE Server for a cluster attests the SPIRE Agent for a cluster
- Is it the expected image?
- Is it running on the expected node?
- Other configurable out-of-band attestations
- If the attestations pass, the SPIRE Server issues a revocable intermediate CA to the SPIRE Agent, from the root CA the SPIRE Server holds.
- The SPIRE Agent is responsible for attesting individual workloads according to the attestation standards defined by the operator.
- If the attestations succeed, the SPIRE Agent uses its intermediate CA to mint a workload certificate, which can be conveyed to the workload and used for workload mTLS.
- If the attestations fail, the workload is unable to obtain a workload cert, and, in a strict mTLS mesh, is unable to send or receive traffic inside the mesh at all.
Additionally, SPIRE handles cert rotation for both Agent intermediate CAs and workload certs, and SPIRE instances can be federated as well as nested, allowing for models where mesh integrity can be externally attested and revoked, and models where workload identities can be brokered across clouds or multiple meshes in a standardized fashion.
The important thing to note about this architecture is that thanks to the out-of-band attestations, workload identity can be uniquely established without having to implicitly trust individual workloads or the Kubernetes API, with no reliance on any workload secrets. The only secret that must be protected is the root CA of the SPIRE Server itself.
Pluggable design
SPIRE enables these out-of-band checks using a plugin infrastructure, which makes it readily extensible.
You can consider these checks analogous to the mobile phone multifactor or hardware key based auth flows that are now so ubiquitous in your own life – before granting you an identity that is cryptographically bound to you, identity issuers may ask for additional proofs, or factors, to help prove your identity, before granting you an identity document (typically a JWT). SPIRE does exactly this for workloads.
Using multiple factors for identity attestation is an essential aspect of a mature security posture when authenticating human entities – and it is no less essential for authenticating non-human identities. Both are critical for zero trust and defense-in-depth models, and both are critical to building stronger, more attestable systems.
Istio and SPIFFE
Istio has used a limited implementation of the SPIFFE identity document spec for many years, and by default ships its own simple workload identity provider that passes through to Kubernetes service account RBAC.
Practically speaking, this means that:
- Istio issues workload identities based on Kubernetes service accounts.
- Every pod within the same Kubernetes service account has the same identity.
- Nothing can be attested about any of the workloads that share that identity, other than the fact that the Kube API server was instructed to map them to a known service account.
- The Kubernetes API is the single source of trust, and no workload identity attestation is performed.
Istio and SPIRE
As of Istio 1.14, thanks to several years of work and research, it is possible to swap out the baseline Istio-provided SDS server and SPIFFE implementation with SPIRE’s SDS server and SPIFFE implementation. This means that SPIRE is now acting as the Certificate Authority (CA) for the mesh.
When a workload in the mesh is spawned, it asks the mesh for a workload identity in x.509 format – a client certificate, which it must have in order to communicate with other services in an mTLS-enabled mesh.
As before, this request is handled via Envoy SDS – but with SPIRE integration. SPIRE can perform out-of-band checks on the workload asking for an identity before choosing to grant one.
For a workload to be granted an x509 identity
- A workload registration, keyed by a SPIFFE ID, must exist on the SPIRE server that defines certain workload properties. These registrations can be manually created, or automatic (rules-based), or created via a K8S admission controller.
- The agent must be able to attest that the workload actually has all the properties defined by the registration using attestation plugins to perform out-of-band checks.
These out-of-band checks can include interrogating multiple endpoints to establish the identity.
For example, using the Kubernetes API, we can establish what node the workload is running on, what containers are in the pod definition, what the image SHAs of the pod containers are, what privileges processes within the container are running with, and more.
SPIRE can also interrogate cloud provider metadata endpoints to establish the identity of the node the workload is running on, down to the provisioned operating system revision or node image and so on.
Out-of-the-box SPIRE supports many attestation plugins, making it readily extensible in ways that Istio’s SDS server is not, and capable of much stronger and more flexible workload identity attestation than the Istio default.
Going a bit further: Attestation and identity
While using SPIRE to perform out-of-band identity checks on Istio-enabled workloads is now an official integration, Istio still has a documented constraint around SPIFFE ID format that means that even if the workloads are uniquely attested, their SPIFFE identifier in the x509 URI SAN must be in the format:
- spiffe://<trust_domain>/ns/<workload_namespace>/sa/<workload_service_account>
This makes it impossible to distinguish between workloads in the same service account by simply looking at their workload cert, even if they were individually attested by the SPIRE server.
However, with SPIRE + Istio, it is perfectly possible to grant every workload in the Istio mesh a unique SPIFFE ID and not simply share one among workloads with the same service account, if being able to distinguish attested workloads by inspecting their workload cert is important.
In current Istio (1.16) this can be achieved by simply creating your own SPIRE registrations with more granular SPIFFE IDs for matching workloads, such as:
- spiffe://<trust_domain>/ns/<workload_namespace>/sa/<workload_service_account>/node/<node>/wl/<workload_name>
When using SPIRE integration, the granularity of the SPIFFE ID is up to you, although if you use an extended custom identifier format like this, current Istio limitations require that you define a custom DestinationRule to ensure SAN matching for workload certs.
Service mesh defense-in-depth
All this taken together enables mesh-enabled clusters to provide defense-in-depth guarantees that go far beyond what was previously possible, while still integrating cleanly with Istio and Kubernetes defaults – allowing workload identity to be cryptographically bound to workload attributes directly attested by the mesh CA.
This is a huge win for auditability, cluster security, and even more complex scenarios like multi-org data handling and externally-attestable secure meshes – providing and managing these guarantees would be extremely difficult without a mesh.
In a world where customer data loss is a serious liability, auditability is king, attackers are more sophisticated, and defense-in-depth is no longer an unobtainable pie-in-the-sky option, meshes and their control planes are new and invaluable tools.
Conclusion
SPIRE is a valuable tool for establishing and attesting workload identity. With its integration into Istio and support of Envoy’s SDS API, SPIRE provides a secure and reliable way of assigning identities to mesh workloads. This ensures only trustworthy workloads can obtain an identity, making it possible to establish unique and as granular as needed identities.
To get a beginner’s overview of SPIRE, its components, and a demo, check out the overview of SPIRE video.
