Scaling and Statelessness

Part two of the series. Your napkin math says one server isn’t enough — genuinely, not resume-drivenly. You have two moves, and the second one has a prerequisite that trips up more apps than any other single thing in this book.

Up vs out

• Scale up (vertical): buy a bigger box — more CPU, more RAM. Dead simple, no code changes, and it works right up until it doesn’t: there’s a biggest box, it costs exponentially more as you approach the ceiling, and it’s a single point of failure. One machine, one power supply, one thing to reboot and take everything down.
• Scale out (horizontal): add more boxes behind a load balancer. Near-linear headroom, and redundancy for free — one box dies, the others carry on. This is how anything serious scales.

Almost always the right first move is up (it’s free engineering-wise, and a big box goes a long way), and the right eventual move is out. But scaling out has a condition, and here it is.

The prerequisite: statelessness

Scaling out means the load balancer can send any request to any server. For that to work, no server can hold state that another server would need. Each app server must be interchangeable — a pure function of the request plus the shared data stores behind it. If request 1 lands on server A and request 2 from the same user lands on server B, server B must be able to serve it with zero knowledge of what A did.

The moment a server remembers something between requests that isn’t in a shared store, you’ve broken this. The classic culprit is the session stored in the server’s local memory or filesystem:

User logs in → request hits Server A → session saved in A's local memory
Next request → load balancer sends it to Server B → "who are you? please log in"

The user gets bounced to the login screen at random. Every stateful thing on a local disk has this failure mode: locally-stored sessions, uploaded files written to the app server’s disk, an in-process cache that only one box has, a background job scheduled in one server’s memory.

The sticky-session trap

The tempting fix is sticky sessions (session affinity): tell the load balancer to always send a given user back to the same server. It works, and it’s a trap, because it re-introduces exactly the coupling you scaled out to escape:

• When that server dies, everyone stuck to it loses their session — you got redundancy in theory and lost it in practice.
• Load can’t balance evenly — a server with the “heavy” users stays hot while others idle.
• Rolling deploys become disruptive — draining a server means dislocating all its users.

Stickiness treats the symptom (state on the server) instead of the cause. The real fix is to move the state off the server entirely.

Where state actually belongs

Make the app servers disposable by pushing every kind of state into a shared store built for it:

In Laravel this is mostly configuration, not code — the reason the framework has drivers for exactly these:

// The app servers become interchangeable by pointing state at shared stores
SESSION_DRIVER=redis        // not "file" — that's local to one box
CACHE_STORE=redis           // shared, not array/file
FILESYSTEM_DISK=s3          // uploads leave the app server entirely
QUEUE_CONNECTION=redis      // jobs run on shared workers, not in-process

Notice the pattern: the app tier becomes stateless by making something else hold the state. You haven’t eliminated state — that’s impossible — you’ve concentrated it into a few purpose-built stores that you scale and protect deliberately, and left the app servers as a herd you can add to, kill, and redeploy at will.

The database is the stateful exception

One tier stays stubbornly stateful: the database. You can’t make it interchangeable the way you made the app servers, because it is the state. That’s why it gets its own chapters — replication to add read capacity and redundancy, and the consistency tradeoffs that come with it. Scaling the stateless tier is easy: add boxes. Scaling the stateful tier is the hard part of this whole book, and where most of the interesting tradeoffs live.

Caveats and best practices

• Design for statelessness before you need to scale, not after. Retrofitting it under traffic is miserable. Using Redis sessions and object storage from day one costs nothing and means “add a server” stays a one-line change forever.
• Scale up first; it buys more than you think. A bigger box is boring, reliable, and often carries you for years. Reach for horizontal scaling when you need redundancy or hit the vertical ceiling — not because it sounds more impressive.
• Treat app servers as cattle, not pets. If losing any single app server would lose data or log users out, you still have state on it — find it. The test: could you kill any box mid-request and lose nothing but that request?
• Avoid sticky sessions as an architecture. They’re a smell that state lives in the wrong place. Fine as a stopgap; never the plan.
• Don’t forget the “invisible” state: in-memory caches, scheduled tasks pinned to one host, files written locally by a library’s defaults. These break silently the day you add the second server.

Conclusion

Up   → bigger box: simple, has a ceiling, single point of failure
Out  → more boxes + load balancer: near-linear, redundant — needs statelessness
Break → session/files/cache on the local server = user bounced at random
Fix  → push state to shared stores (Redis, object storage, queue); app tier = cattle

Horizontal scaling is easy once the app is stateless and nearly impossible before — so the real work is making every app server an interchangeable, disposable unit and concentrating state into stores built to hold it. Do that early and scaling is a config change. Next we enter the hard part: the CAP theorem, and why the one stateful tier you couldn’t make disposable forces a choice you can’t avoid.