Service Discovery

In a cloud or Kubernetes environment, instances are ephemeral — they scale up, crash, get rescheduled, and get new IPs constantly. Hardcoding an IP or even a hostname into a config file breaks the moment that instance disappears, so every microservices architecture needs a mechanism for services to find each other dynamically — a favorite interview probe for whether you've actually operated a distributed system.

What Is Service Discovery?

Service discovery is the way one piece of software automatically finds the current network address of another, instead of that address being hardcoded somewhere. Services register themselves with a central directory when they start up, and other services look them up there whenever they need to talk to them — so if an instance moves or restarts and gets a new address, everyone else finds out automatically instead of breaking.

Where This Is Used

• Kubernetes gives every service a stable internal DNS name, even as the actual pods behind it are constantly created and destroyed.
• Netflix's Eureka registry lets its hundreds of microservices find each other without anyone hand-managing IP addresses.
• Consul, used by companies like Uber and Shopify, tracks the health and location of services across their infrastructure.
• Cloud load balancers use service discovery under the hood to know which backend instances are currently healthy and reachable.

A Day-to-Day Example

Think of calling a large company's front desk instead of trying to remember each employee's personal cell number. Employees change desks, offices, and even buildings all the time, but the front desk always knows how to reach whoever you're looking for right now. You just ask for "the billing department," and the desk connects you to whoever currently holds that role.

Deep Dive: Client-Side vs Server-Side Discovery

The two patterns differ in where the "which instance do I call" decision gets made.

Client-side discovery (e.g. Netflix Eureka + Ribbon)

  Payment Service instances register on startup:
    10.0.1.4:8080, 10.0.1.9:8080, 10.0.2.2:8080  →  registered in Eureka

  Order Service (the caller):
    1. queries Eureka: "give me healthy instances of payment-service"
    2. gets back: [10.0.1.4:8080, 10.0.1.9:8080, 10.0.2.2:8080]
    3. picks one itself (round-robin / least-connections) via Ribbon
    4. calls it directly — no extra hop

Server-side discovery (e.g. Kubernetes Service, or any LB-fronted setup)

  Order Service calls a stable name:  http://payment-service/charge
                                              |
                                              v
                                     kube-proxy / load balancer
                                     (does the registry lookup +
                                      picks a healthy backend Pod)
                                              |
                                              v
                                   Payment Pod  10.0.2.2:8080
  — Order Service never sees individual instance IPs at all.

Client-side discovery removes a network hop and lets the caller apply its own load-balancing policy, but it means every service, in every language your org uses, needs a registry-aware client library — real operational cost across a polyglot org. Server-side discovery keeps application code simple (call a name, get a response) at the cost of routing every request through an extra hop.

Deep Dive: How Kubernetes Does It — Services, kube-proxy, and CoreDNS

Kubernetes is server-side discovery built into the platform, and it's worth knowing the actual mechanics, not just "it has DNS":

• A Service object is given a stable virtual IP (the ClusterIP) that never changes, even as the Pods behind it are replaced. It selects backend Pods by label (e.g. app: payment-service), not by identity.
• An Endpoints (or EndpointSlice) object is continuously updated by the control plane to list the current healthy Pod IPs matching that selector — this is the actual, live registry.
• kube-proxy runs on every node and programs the kernel (via iptables rules or IPVS) so that any packet sent to the Service's ClusterIP gets transparently DNAT'd to one of the real Pod IPs from the Endpoints list — this is why calling a Service has effectively zero added application-level latency, the redirect happens in the kernel's networking stack.
• CoreDNS gives every Service a DNS name (payment-service.default.svc.cluster.local) that resolves to its ClusterIP — so from application code, discovery is just a normal DNS lookup plus a normal TCP connect; nothing service-discovery-specific has to be in your app.

The failure mode worth knowing: if a Pod fails its readiness probe, it's removed from Endpoints (not from the Service, the Service itself is static) — so kube-proxy stops routing to it within roughly one control-loop cycle, without any DNS TTL to wait out.

Key Pointers

• The core problem: service instances have dynamic, short-lived network locations, so callers need a live, queryable source of truth instead of static config.
• Client-side discovery: the calling service queries a registry directly (e.g. Netflix Eureka) and picks an instance itself, often with client-side load balancing (e.g. Ribbon) — fewer network hops, but every client needs registry-aware logic.
• Server-side discovery: the caller hits a fixed endpoint (a load balancer or DNS name), and something else — a load balancer, ingress, or kube-proxy — resolves it to a live instance; the client stays simple.
• A service registry (Consul, Eureka, ZooKeeper) tracks which instances are alive via heartbeats or health checks, and de-registers ones that stop responding.
• Kubernetes bakes server-side discovery into the platform: every Service gets a stable virtual IP and DNS name from CoreDNS, and kube-proxy transparently load-balances traffic across the matching healthy Pods — most teams on k8s never touch Consul or Eureka directly.
• DNS-based discovery (including Kubernetes' own) trades fine-grained control for simplicity — it works with unmodified clients since it's just a name lookup.

How It Works

1. A new service instance starts and registers itself (or is registered by an agent/orchestrator) with the registry, along with its address and a health-check endpoint.
2. The registry periodically checks instance health via heartbeats or active probes and removes instances that stop responding.
3. A caller needing to reach the service either queries the registry directly for a list of healthy instances (client-side) or sends the request to a stable proxy/DNS name that does the lookup on its behalf (server-side).
4. The caller (or the proxy) picks one instance — round-robin, least-connections, or another balancing strategy — and sends the request.
5. When an instance shuts down or fails health checks, it's removed from the registry so future lookups stop routing traffic to it.

Try It Yourself

On any cluster you have access to, you can inspect both layers of Kubernetes discovery directly:

# The stable virtual IP and DNS name — this never changes
kubectl get svc payment-service

# The actual live registry — current healthy Pod IPs backing the Service
kubectl get endpoints payment-service

# Watch Endpoints update in real time as Pods are added/removed
kubectl get endpoints payment-service --watch

# Confirm DNS resolution works from inside the cluster
kubectl run -it --rm debug --image=busybox --restart=Never -- \
  nslookup payment-service.default.svc.cluster.local

# See exactly which kube-proxy rules route to which backend (iptables mode)
iptables -t nat -L KUBE-SERVICES -n | grep payment-service

For a Consul-based setup: consul catalog services lists registered services, and consul health service payment-service shows current instance health — the direct equivalent of kubectl get endpoints.

Interview Q&A

Q: Walk me through what happens when a Pod behind a Kubernetes Service crashes.

A: The kubelet detects the failed liveness/readiness probe and marks the Pod not-ready. The control plane removes it from the Service's Endpoints/EndpointSlice object almost immediately. kube-proxy watches Endpoints and updates its iptables/IPVS rules on every node, so new connections to the Service's ClusterIP stop being routed to that Pod within roughly one control-loop cycle — no DNS caching or TTL involved, because clients never resolve the Pod's IP directly, only the Service's stable ClusterIP.

Q: Is service discovery the same thing as load balancing?

A: No, though they're often bundled. Discovery answers "who is alive and where" — producing a set of healthy instances. Load balancing answers "which of those instances should handle this particular request" — round-robin, least-connections, latency-aware, etc. A Kubernetes Service does both (kube-proxy both tracks Endpoints and picks one), but you can have discovery without smart balancing (e.g. random pick) or a load balancer sitting in front of a manually maintained list with no discovery at all.

Q: What's a real failure mode of client-side discovery?

A: A stale registry view — if a client caches the instance list and doesn't refresh often enough, it keeps sending traffic to an instance that already deregistered or crashed, and gets connection errors until its next refresh. This is a direct trade-off against server-side discovery, where the proxy/kube-proxy layer updates centrally and every client benefits immediately.

Q: Why does Kubernetes give a Service a stable IP instead of just using DNS with a low TTL?

A: DNS alone, even with a low TTL, has caching behavior outside Kubernetes' control (OS resolvers, app-level connection pools that don't re-resolve per request) — a client could hold a stale IP well past the TTL. By giving the Service a genuinely stable virtual IP and doing the real instance-level routing at the kernel/iptables layer instead, Kubernetes avoids relying on every client's DNS caching behavior being well-tuned.

System Design Tie-in

"Design a large-scale service mesh for 1,000+ microservices" is really a service-discovery-plus-reliability problem once you get past the surface. The answer needs: a registry (or Kubernetes' built-in one) as the single source of truth for live instances; consistent health checking so the registry never routes to a dead instance; client-side load balancing or a sidecar proxy (Envoy, in an Istio-style mesh) so balancing decisions don't require a central bottleneck; and propagation of that discovery data fast enough that a rolling deploy of one service doesn't cause a spike in errors for its callers. Naming Kubernetes Services/Endpoints or a sidecar mesh by name, and explaining the registration → health-check → routing-update loop concretely, is what separates a real answer from "we'd use a service registry."

Trade-offs

Client-side discovery avoids an extra hop through a load balancer but pushes registry-awareness and balancing logic into every client, in every language your org uses — that's real maintenance cost at scale. Server-side discovery centralizes that complexity in infrastructure (a load balancer or the k8s control plane), which is simpler for application code but adds a hop and a potential bottleneck if the proxy layer isn't scaled well. A common interview gotcha: conflating service discovery with load balancing — discovery answers "who is alive and where," balancing answers "which of them should handle this request"; most real systems need both, and Kubernetes gives you both for free via Services and kube-proxy.