One of the most popular features of service mesh is the ability to enforce policy on network traffic. Because all traffic passes through a proxy server, you can ensure that no-one is able to access a service without passing through an enforcement point, and those can be in front of every service, instead of at the network edge.
Further, by contrast to traditional firewalls and Kubernetes’ NetworkPolicy features, you have access to application-layer (Layer 7) information, such as whether a request is an HTTP GET or a POST, or the identity of a user as stored in a cookie.
Ambient mesh, as implemented by Istio’s ambient mode and Gloo Mesh, radically simplifies the delivery of service mesh, and substantially lowers the cost. The way it does this is to separate out Layer 4 and Layer 7 processing into two different sets of proxies. However, this requires us to change our assumptions about access control. We no longer have a Layer 7-aware enforcement point in front of every workload. The good news is that we probably didn’t need it, and if we did, we can add it to our infrastructure — scoped exactly to the set of workloads we need to protect.
Authentication and authorization in ambient mesh
Each workload running in a service mesh has an identity, based on its namespace and service account. These identities are transparently validated within the mesh: mutual authentication is a property of mTLS, which is provided by default when ambient mesh is installed.
Controlling traffic is done by way of AuthorizationPolicy. These policies refer to a destination in the mesh, and contain rules which can allow or deny traffic based on traffic source (from), operations (to), and under which conditions (when).
The ztunnel proxy can always enforce policy, but is only aware of Layer 3 and 4 attributes (hosts and ports). A waypoint is additionally aware of Layer 7 attributes (paths and methods) but isn’t always deployed. To gain the benefits of ambient mesh, we have to be more granular on how we create security policy, and create separate policies for the two different sets of infrastructure. The good news is that it is very easy to understand.
The two different types of authorization policy
Consider a workload in an ambient mesh, with no waypoint deployed. The only place where policy can be enforced is the ztunnel at the server end. By design, ztunnels are not L7-aware, so only L4 policy can be enforced.
When you introduce a waypoint for this workload, you now have two places where policy can be enforced:
When a system has more than one policy enforcement point, policies should always be applied to the earliest enforcement point that can handle them.
For instance, moving all policy enforcement entirely to the waypoint would be a bad idea, as traffic that addresses the workload directly would then be able to bypass our policy. Also, for services that are publicly accessible, we may already have enforced our policy at the gateway at the edge of our mesh. (To avoid double handling, the default behavior of a gateway is to send traffic to the server directly, bypassing the waypoint.)
Instead, we should choose the earliest enforcement point to define our “infrastructure policy”— at the ztunnel to enforce the use of the waypoint for general traffic — and then implement “application policy” at the gateway or waypoint, as appropriate.
Targeting and binding policy
Our conceptual diagrams have ignored the fact that there can be multiple instances of a workload, on more than one node, and there can be multiple instances of a waypoint. We must write our policy to describe where we want it to be applied: that is, on calls to which service, or which workload.
Istio will then bind the policies to the correct proxies to enforce them:
For policy which is to be enforced by ztunnel, you select which workloads to address by the placement of the AuthorizationPolicy in a certain namespace, and an optional label selector. This is similar to how Kubernetes NetworkPolicy works, or how authorization policy was scoped using Istio with sidecars.
Like gateways, waypoints are configured with the Kubernetes Gateway API. Policies are attached to a Gateway using the targetRefs field.
Note that we are not saying “L4 policy” and “L7 policy”. This is because a waypoint proxy can also enforce L4 policy, and we will talk more later about why you might want to do that.
The basics of access control
For workloads without authorization policies applied, Istio allows all requests. When at least one policy exists that contains an ALLOW action, any traffic that does not match an ALLOW policy is denied. This is important to note because, for example, if you allow traffic from an external gateway to a workload, and then later introduce a waypoint, traffic from that waypoint will be denied.
Policy enforcement with ztunnel
For many use cases, using ambient mesh without waypoints — sometimes called the secure overlay layer — is the right choice. Istio’s Layer 4 attributes can be used for authorization policy. These include IP addresses, ports, namespaces and identity principals.
This allows us to define the minimum expectations for micro-segmentation: in a microservices environment, only certain services should be able to speak to each other. In the event that a service is compromised, any traffic generated by that service will appear with its source and identity. If an image processing service has no business communicating with a payment backend, you can easily create a policy to describe that.
Disallowed policy attributes and denied traffic
I mentioned above that Istio’s Layer 4 attributes can be used in policy which is bound to ztunnel. This allows you to take a policy which was used in Istio’s sidecar mode and deploy it in ambient mode. Such policies can also act on Layer 7 attributes, such as a request’s method or path. However, ztunnel is not Layer 7 aware. What happens in this case?
In short, ztunnel fails safe:
DENYpolicies with L7 attributes will be enforced without their HTTP components.ALLOWpolicies with L7 attributes will have the matching rules removed.
In both cases, the result will be more restrictive than requested. Recall that a workload with at least one ALLOW policy will deny traffic that does not match any ALLOW policy. In this case, all traffic will match the default DENY policy.
Adding waypoints to the mix
There are two primary differences between a policy bound to a waypoint, and a policy bound to ztunnel:
- The
targetRefsfield is used instead of a selector - Layer 7 attributes are supported
When you enroll a workload to use a waypoint, it becomes the best place to enforce policy for that workload — even for L4 policies. To understand why, we must look at how identity works in ambient mesh. The mTLS connection is between two workloads, managed by the ztunnel of each:.
In order for the waypoint proxy to be able to decrypt the traffic and examine it, it must terminate the mTLS connection, and start a new connection to the destination:
That means that the identity seen by the server ztunnel is that of the waypoint proxy, not of the client. Therefore, if you want to be able to make policy decisions based on the source identity, you should attach your policy to the waypoint.
Only allowing connections from a waypoint
In order for a waypoint to act as a policy enforcement point, it is important to ensure that it cannot be bypassed. By default, ambient mesh will automatically route service traffic through waypoints. However, the ztunnel for a destination workload will continue to accept connections from any client.
There are cases where this is sensible; the most common is scraping Prometheus metrics from a pod, where you connect directly, and not through its waypoint.
If you need to block other traffic, you should create ztunnel policy to only allow connections from the appropriate waypoint. Add exceptions as required.
Learn more about ambient mesh
Our comprehensive guide to ambient mesh, and its implementations in Istio and Gloo Mesh, is available at ambientmesh.io. The security section contains more information and examples about authorization policy. We also have a free 5-minute lab on access control, and labs on many other ambient mesh topics. Check them out!
