What a load balancer is and why it matters

Load balancers are one of those infrastructure concepts that sound complicated but are actually pretty straightforward once you understand what problem they're solving.

The problem

A single server can only handle so many requests at once. Under light traffic that's fine. Under heavy traffic it becomes a bottleneck — requests pile up, response times increase, eventually things fall over. The obvious solution is more servers. But then you have a new problem: how do you decide which server handles which request? That's what a load balancer does.

What it actually does

A load balancer sits in front of your servers and distributes incoming traffic across them. A request comes in, the load balancer picks a server from its pool and forwards it there, and the response comes back through. From the client's perspective they're just talking to one address. Behind the scenes, dozens of servers might be handling requests in parallel.

Think of it like a receptionist at a busy office. You arrive, the receptionist figures out which desk is free, and sends you there. You don't need to know how many desks there are or which ones are busy — the receptionist handles that.

Load balancing algorithms

The simplest is round-robin — each request goes to the next server in the list, cycling through. Least connections sends requests to whichever server currently has fewest active connections. Weighted round-robin lets you send more traffic to more powerful servers. Most setups work fine with round-robin or least connections.

Health checks

Good load balancers continuously check that the servers in the pool are actually working. If a server goes down, the load balancer stops sending it traffic automatically. When it comes back, it gets added back to the pool. This is how you get high availability — even if one server dies, traffic keeps flowing to the healthy ones.

nginx as a load balancer

nginx does load balancing natively. A basic config looks like this:

upstream backend {
    least_conn;
    server 10.0.0.1:3000;
    server 10.0.0.2:3000;
    server 10.0.0.3:3000;
}

server {
    location / {
        proxy_pass http://backend;
    }
}

That's it. Three servers, least connections algorithm, nginx handles the distribution. For my Wings nodes this kind of setup is what allows traffic to spread across multiple machines cleanly.