Load Balancing Strategies

What are load balancing strategies?

A load balancer sits between clients and backend servers, distributing requests to prevent any single server from becoming a bottleneck. It is a core building block for horizontal scalability and high availability.

Layer 4 vs Layer 7

Layer 4 (Transport) - Operates at TCP/UDP level. Routes based on IP address and port. Fast, but cannot inspect request content.

• Examples: AWS NLB, HAProxy TCP mode
• Use when: raw throughput matters, non-HTTP protocols, minimal latency overhead needed

Layer 7 (Application) - Operates at HTTP/HTTPS level. Can route based on URL path, headers, cookies, or request body.

• Examples: AWS ALB, Nginx, Traefik
• Use when: routing /api to one service and /static to another, A/B testing, SSL termination

Distribution Algorithms

Round Robin

Requests distributed sequentially across servers. Simple, equal distribution by count.

• ✅ Simple, zero state
• ❌ Ignores server load - a slow server gets the same traffic as a fast one

Weighted Round Robin

Servers assigned weights; higher-capacity servers receive proportionally more requests.

• ✅ Handles heterogeneous server pools
• ❌ Requires manual weight tuning

Least Connections

Route to the server with the fewest active connections.

• ✅ Adapts to variable request durations (ideal for WebSockets, long-lived connections)
• ❌ Requires connection tracking state

IP Hash / Sticky Sessions

Hash the client IP to always route a client to the same server.

• ✅ Required for stateful apps where session data lives on the server
• ❌ Reduces balancing effectiveness; one server can get overloaded if popular IPs cluster

Least Response Time

Route to the server with the lowest average response time.

• ✅ Optimises for latency
• ❌ More complex to implement; subject to measurement noise

Health Checks

Load balancers poll backends periodically:

• Passive: detect failures from live traffic errors
• Active: send periodic synthetic requests (HTTP GET /health) and remove unresponsive backends

Configure: check interval, failure threshold, recovery threshold, timeout.

Sticky Sessions vs Stateless Design

Sticky sessions (session affinity) keep a client pinned to a server. They work but create problems: rebalancing on server failure, hot spots, difficulty auto-scaling. The better solution is stateless application design - move session state to Redis or a database so any server can handle any request.

Global Load Balancing

For multi-region setups, DNS-based load balancing (Route 53 latency routing, GeoDNS) or Anycast routing directs users to the nearest region before even hitting an L4/L7 balancer.

Interview Angle

Common question: "How would you design the load balancing layer for a high-traffic API?" Answer: L7 ALB with least-connections, stateless backend services, health checks every 5s with a 2-failure threshold, auto-scaling group behind the LB. Mention that sticky sessions are a smell - prefer stateless design.