A Service Mesh Costs About 15 Percent and Half an Engineer

Every service mesh evaluation I have seen starts with a latency benchmark. Someone runs a load test with and without sidecars, reports that p99 went from 12ms to 14ms, and concludes the overhead is acceptable. The benchmark is usually accurate and it is measuring the least significant cost.

The costs that matter are the resources multiplied across your entire fleet, and the permanent expansion of your operational surface. A mesh puts a proxy in the request path of every service, which means it is now a participant in every latency investigation, every connection error, every mTLS certificate expiry, every upgrade, and every incident review, for as long as you run it. That is the bill. The two milliseconds are a rounding error next to it.

Adding up the sidecar tax

Latency. Each hop through an Envoy sidecar adds roughly 0.3 to 0.8ms at p50 and 1 to 3ms at p99, and a request traverses two proxies — outbound from the caller, inbound to the callee. So budget 1 to 2ms p50 and 3 to 6ms p99 per service-to-service call. For a request that fans out across five internal services, that is 15 to 30ms of p99 that did not exist before, which is material for anything with a tight user-facing budget and irrelevant for a batch pipeline.

CPU. A sidecar consumes roughly 0.3 to 0.6 vCPU per 1,000 requests per second, with the variance driven mostly by whether mTLS is on and how many Envoy filters are configured. Even at idle, a sidecar burns 10 to 30 millicores maintaining connections, running health checks, and processing xDS updates from the control plane.

Memory. This is the one that surprises people. Envoy's memory is dominated by its configuration, not by traffic — specifically by the number of endpoints in the mesh it must know about. In a naive installation, every sidecar receives configuration for every service, so memory scales with cluster size rather than with the service's own needs. Typical footprint runs 50 to 200 MB per sidecar in a small mesh; in a large one without scoping, I have seen sidecars above 1 GB.

Multiply out. A 600-pod mesh at 100 MB and 50 millicores idle per sidecar is 60 GB of memory and 30 cores doing nothing but existing. On typical instance pricing that is somewhere in the range of $2,500 to $4,000 a month before a single request is served. Add the control plane — istiod at three replicas, sized for the endpoint count — and the observability volume the mesh generates, which is frequently the larger number and lands on a budget line that is already growing fastest.

The memory problem has a direct fix, and it is the first thing to configure in any Istio installation:

apiVersion: networking.istio.io/v1
kind: Sidecar
metadata:
  name: default
  namespace: payments
spec:
  egress:
  - hosts:
    - "./*"                    # own namespace
    - "istio-system/*"
    - "shared-data/*"          # only what this namespace actually calls
  outboundTrafficPolicy:
    mode: REGISTRY_ONLY

Scoping the sidecar's visibility to the services a namespace actually talks to routinely cuts proxy memory by 60 to 80 percent in a mesh of any size. Very few installations do this, because the default works and the cost is invisible until someone adds it up.

The half-engineer

The recurring cost that never appears in the evaluation. Some of it is scheduled work: control plane upgrades on the project's release cadence, which for Istio means a supported-version treadmill of roughly quarterly minors; sidecar version skew management; certificate rotation, which is automatic until the day it is not.

Most of it is unscheduled. A mesh inserts itself into failure modes that used to be simple. A connection reset is now potentially the application, the local sidecar, the remote sidecar, a destination rule, an outlier detection ejection, a circuit breaker at maxConnections, or an mTLS policy mismatch. Debugging requires reading Envoy config dumps, and the number of people in most organisations who can fluently read an Envoy config dump is one.

Two specific traps worth knowing in advance, because both cost teams entire days:

Startup ordering. An application container that opens a connection before the sidecar is ready fails, and this manifests as flaky pod starts and broken Jobs rather than as an obvious mesh problem. Native sidecar containers (Kubernetes 1.29+, using initContainers with restartPolicy: Always) fix this properly. Before that, teams worked around it with holdApplicationUntilProxyStarts and retry loops.

Job and CronJob termination. A Job's main container exits; the sidecar does not; the pod never reaches Completed; the Job hangs forever. Also fixed by native sidecars, and a source of considerable pain in clusters that predate them.

spec:
  initContainers:
  - name: istio-proxy
    restartPolicy: Always          # native sidecar: starts first, stops last
    image: docker.io/istio/proxyv2:1.24.0
    resources:
      requests: { cpu: 30m, memory: 96Mi }
      limits:   { memory: 256Mi }

Ambient mode changes the arithmetic, not the conclusion

Istio's ambient mode removes the per-pod sidecar. L4 — mTLS, identity, TCP routing, basic telemetry — is handled by ztunnel, a Rust per-node DaemonSet. L7 features are opt-in via a per-namespace waypoint proxy that only sits in the path of traffic that needs them.

The savings are real and they are concentrated where the sidecar model was worst:

For a mesh whose actual requirement is mTLS and identity-based authorisation — which is most of them — ambient is a large improvement and the memory reduction alone can justify the migration. What it does not change is the operational surface argument. You still run a control plane, still debug an extra network layer, still upgrade on the project's cadence. The half-engineer becomes maybe a third of an engineer.

Four cases where it pays

1. Compliance-mandated encryption in transit between services. If an auditor requires mTLS on every internal hop, a mesh does it in a day with automatic certificate rotation and a workload identity model. The alternative is per-application TLS with certificate distribution, which is more work, more fragile, and inconsistently implemented across languages. This is the clearest case and it is the reason most large meshes exist.

2. Polyglot services needing consistent resilience behaviour. Retries, timeouts, circuit breaking, and outlier detection implemented once in the infrastructure rather than five times in five language ecosystems with five different libraries at five different versions. If you are a Go-and-Java shop, the libraries are good and you may not need this. If you have Python, Node, Rust, Java, and a Ruby service nobody wants to touch, the mesh is doing something genuinely hard to replicate.

3. Progressive delivery at scale. Weighted traffic splitting for canaries, driven by an automated analysis controller like Argo Rollouts or Flagger, with the mesh providing both the traffic shifting and the metrics to judge the canary on. Achievable without a mesh, but noticeably clunkier.

4. Zero-trust segmentation with real identity. Network policy operates on IPs and labels; mesh authorisation operates on cryptographic workload identity, which survives IP reuse and is far harder to spoof. If your threat model includes lateral movement after a pod compromise, that distinction matters — a NetworkPolicy allows anything that happens to land on the right IP, while a mesh policy requires a valid SPIFFE identity backed by a certificate.

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: payments-callers
  namespace: payments
spec:
  selector:
    matchLabels: { app: payments-api }
  action: ALLOW
  rules:
  - from:
    - source:
        principals:
        - cluster.local/ns/checkout/sa/checkout-api
        - cluster.local/ns/refunds/sa/refunds-worker
    to:
    - operation:
        methods: ["POST"]
        paths: ["/v1/charges", "/v1/refunds"]

When to say no

Under about 20 services, the mesh's operational cost exceeds what it solves. You can get mTLS from a smaller tool, resilience from libraries, and observability from OpenTelemetry instrumentation you probably want anyway.

If your primary goal is observability, do not install a mesh for it. Mesh-generated telemetry is L7 metrics without application context — it tells you a call to /v1/charges took 340ms, not which query was slow. Proper instrumentation gives you more for less, and for network-level visibility specifically, eBPF collects the same flow data at a fraction of the cost and none of the request-path risk.

And if you cannot name the person who will own the mesh, do not install it. An unowned mesh degrades into an unupgradable dependency that everyone is afraid to touch, which is a considerably worse position than not having one. This is the same failure that makes portability abstractions go bad: a permanent carrying cost adopted without anyone agreeing to carry it.

The honest recommendation: if you need mTLS and identity-based policy across more than 20 services, run Istio in ambient mode, scope aggressively, and budget a third of an engineer indefinitely. If you need less than that, you do not need a mesh, and the feature matrix that says otherwise is not costed.