What is Service Mesh?
An infrastructure layer that handles service-to-service traffic in a microservices system, adding mutual TLS, retries, timeouts, traffic splitting and telemetry through proxies instead of code in each service.
Definition
A service mesh is a dedicated infrastructure layer for the network traffic between services, usually inside a Kubernetes cluster. Instead of every microservice implementing its own encryption, retries, timeouts, load balancing and metrics in application code, the mesh intercepts each request between services and applies those behaviours consistently, configured centrally by the platform team. Service mesh architecture has two halves. The data plane is the set of proxies that actually carry the traffic: every request from service A to service B passes through them, and they encrypt it, route it, retry it and record metrics and traces. The control plane is the management component that distributes configuration, routing rules, policies and certificates to those proxies. In Istio the control plane is istiod and the classic data plane is Envoy; in Linkerd the data plane is linkerd2-proxy, a small proxy written in Rust specifically for the mesh. The core features are the same across meshes. Mutual TLS (mTLS) encrypts all service-to-service traffic and has both sides present certificates, so each workload has a cryptographic identity (often a SPIFFE ID tied to its Kubernetes service account) and the mesh can enforce policies such as 'only checkout may call payments'. That is a practical foundation for zero trust inside the cluster, with automatic certificate rotation. Traffic management covers retries, timeouts, circuit breaking, locality-aware load balancing and traffic splitting, for example sending 5% of requests to v2 of a service for a canary release, mirroring traffic for testing, or routing by header. Observability comes for free from the proxies: golden metrics (request rate, error rate, latency) per service pair, access logs and distributed-tracing spans, without touching application code. Sidecar vs sidecarless: the classic model injects a proxy container as a sidecar into every pod, so each pod gets its own proxy. This gives strong isolation and full layer 7 features, but adds memory and CPU to every pod, adds latency hops, and means proxies must be restarted along with application pods to upgrade them. Sidecarless designs move the proxy out of the pod. Istio's ambient mode, generally available since Istio 1.24 in late 2024, splits the job in two: a per-node ztunnel (written in Rust) handles layer 4 duties such as mTLS, identity and basic telemetry for every pod on that node, and optional waypoint proxies (Envoy) are deployed only for namespaces or services that need layer 7 features such as HTTP routing, retries or header-based policy. Cilium offers a mesh built on eBPF in the Linux kernel plus per-node Envoy. Sidecarless meshes are cheaper to run and easier to adopt gradually; sidecars remain the most mature and battle-tested option. Popular implementations include Istio (CNCF graduated, the most feature-rich), Linkerd (CNCF graduated, focused on simplicity and low overhead; note that since 2024 the open source project ships edge releases while stable releases come from Buoyant's distribution), Cilium Service Mesh, Consul service mesh from HashiCorp, and Kuma. The Kubernetes Gateway API, through its GAMMA initiative, is standardising how mesh routing is configured so rules are less tied to one vendor.
When to Use Service Mesh
A service mesh earns its keep when you run many services (dozens or more) owned by several teams, need encryption in transit and service identity everywhere for compliance or zero trust, want uniform retries, timeouts and traffic splitting for safe releases, and want per-service metrics without instrumenting each codebase. It is also useful for multi-cluster or multi-language systems where libraries would have to be maintained in several languages. You probably do not need one if you run a monolith or a handful of services, a single team owns everything, or your platform already gives you what you need: a cloud load balancer, TLS at the ingress, an observability SDK such as OpenTelemetry and sensible client libraries for retries. A mesh adds components to operate, upgrade and debug, new failure modes (proxy misconfiguration, certificate expiry), resource overhead and a learning curve. Start with the problem: if it is only north-south traffic from the internet, an API gateway or ingress controller is the right tool; a mesh mainly governs east-west traffic between services. If you adopt one, start small, for example ambient or Linkerd for mTLS and metrics only, then add layer 7 policies where they pay off.
Service Mesh in Practice
- Adding Istio to a namespace with sidecars: label it, then restart workloads so the injector adds an Envoy container to each new pod. kubectl label namespace shop istio-injection=enabled kubectl rollout restart deployment -n shop
- Adding the same namespace to Istio ambient mode instead. No pod restarts are needed; the node's ztunnel starts handling the traffic with mTLS: kubectl label namespace shop istio.io/dataplane-mode=ambient Then, only if you need layer 7 features for this namespace, deploy a waypoint: istioctl waypoint apply -n shop --enroll-namespace
- Meshing workloads with Linkerd by injecting the proxy into existing manifests, then checking live golden metrics: kubectl get deploy -n shop -o yaml | linkerd inject - | kubectl apply -f - linkerd viz stat deploy -n shop
- A canary with traffic splitting: an HTTPRoute (Gateway API) or an Istio VirtualService for the reviews service lists two backends, reviews-v1 with weight 95 and reviews-v2 with weight 5. The mesh sends roughly 5% of requests to v2; if its error rate stays flat in the mesh dashboards you raise the weight to 50, then 100, all without redeploying either version.
- Enforcing mTLS: an Istio PeerAuthentication resource with mtls mode STRICT in a namespace rejects any plain-text connection to its pods, while an AuthorizationPolicy can then allow only the checkout service account to call payments.
Common Mistakes
- Adopting a mesh for three services because it is fashionable, then spending more time operating Istio than building the product.
- Confusing a service mesh with an API gateway. A gateway manages north-south traffic from external clients (auth, rate limits, API keys); a mesh manages east-west traffic between internal services. Many systems use both.
- Leaving mTLS in permissive mode indefinitely after migration, so plain-text traffic is still accepted and the security benefit is mostly theoretical.
- Stacking retries in the mesh on top of retries in application code, which multiplies load during an outage. Configure retries in one place, with timeouts and budgets.
- Ignoring the resource cost of sidecars: a proxy per pod across hundreds of pods adds up in memory and CPU, which is one reason sidecarless options exist.
- Assuming the mesh replaces application-level authorization. mTLS proves which service is calling, not whether the end user behind the request is allowed to do the action.