Why do vibe-coded apps break with real users?
Vibe-coded apps usually break with real users not because the code is wrong but because it has only ever run for one person at a time. Real traffic brings concurrent requests, slow dependencies, retries that multiply load and database limits nobody hit in testing. Keep servers stateless so you can add instances, put a timeout and a failure plan on every external call, cache and queue what you can, and load test the whole user journey before your users do it for you.
Systems do not fail during development
While you build with an AI coding tool, the app has one user: you. Inputs are the ones you thought of, traffic is a few requests a minute, and every dependency is fast because nothing else is using it. Chapter 15 of the book calls this a dangerous illusion. Real problems appear after launch, when users arrive at the same time, requests spike, external APIs slow down, unexpected inputs appear and costs grow faster than planned.
The code did not change between the demo and the outage; the conditions did. A system can be correct and still be unreliable. Development asks whether it works. Production asks what happens when it does not: when the model is slow, the database is unavailable, a service is down or traffic suddenly doubles.
What breaks first in an app built with AI tools
These failures are rarely exotic. None is unique to AI-written code, but none shows up while one person tests the app.
| What you see | Usual cause | Fix |
|---|---|---|
| Users lose sessions or see stale data once a second instance starts | Sessions, counters or job lists kept in one server's memory | Keep state in the database, a cache or a queue |
| Requests hang, then everything slows at once | External calls with no timeout of your own | A timeout on every external call, set from measured latency |
| Errors about too many database connections | Every instance or serverless function opens its own connections | A connection pool, or the pooler your database provider offers |
| Pages that were instant now take seconds | Missing indexes, or one query per list item; both hurt only once tables grow | Index the columns you filter and sort on; fetch lists in one query |
| Timeouts on uploads, AI features or emails | Slow work done inside the web request | Accept the request, queue the work, let a worker finish it |
| A short provider blip becomes a long outage | Retries at several layers, without backoff | Retry in one place, with backoff, jitter and a limit |
| The model bill climbs faster than users | Every action calls the model, uncached and unlimited | Cache, cap per user and call the model only when needed |
When one user becomes one thousand
There are two ways to add capacity. Vertical scaling makes the server bigger. It needs no change to the architecture, which makes it the right first move, but one machine has a ceiling and large machines get expensive quickly. Horizontal scaling adds instances behind a load balancer. It goes much further, but only if the app can run as several copies at once.
That condition is statelessness. If any instance can handle any request, you can add instances freely; if a session, an upload in progress or a rate-limit counter lives in one server's memory, the second instance breaks it. Keep the API stateless and push state into the database, a cache or a queue. Serverless hosting already demands this, because the platform starts and stops instances as it sees fit.
Concurrency inside one instance matters too. A process that handles one request at a time makes everyone queue behind a slow one. In Python, run several worker processes; the book's FastAPI example runs four under Uvicorn. In Node.js, never block the event loop with heavy computation. Find the right number of workers under load, not by guessing.
Autoscaling adds instances when CPU, request rate, queue length or latency crosses a limit, but it is not the first fix. An inefficient design stays inefficient with more copies, and each new instance opens more database connections (a default Postgres server typically allows 100) and sends more calls to the same rate-limited providers. Scaling out moves the bottleneck downstream; plan for where it lands.
How one slow dependency takes the whole app down
The most dangerous failure under load is the cascade. The book traces it: an external model API slows down, pending requests pile up in the backend, the server overloads, the database starts timing out and the whole system becomes unstable. Chapter 1 shows the same chain starting from a traffic spike, with retries turning slow responses into spreading timeouts.
Retries are the accelerant, and they often stack up without anyone deciding so. TanStack Query retries a failed query three times by default in the browser, and the official OpenAI and Anthropic TypeScript SDKs retry certain failures twice. Stacked, one user action can become four requests to your API and twelve calls to a struggling provider. Amazon's guidance on timeouts, retries and backoff shows how fast this compounds through deeper stacks. Retry in one layer, usually the one nearest the failing service, and turn it off in the others.
Five defences stop the chain:
- Timeouts on every external call, so a slow dependency costs a bounded wait. The OpenAI and Anthropic TypeScript SDKs default to a ten-minute timeout, far longer than a user will wait.
- Fallbacks: a simpler path, such as a cached or rule-based answer, when the main one cannot answer.
- Retries with exponential backoff and jitter, for temporary failures only, inside an overall deadline.
- Circuit breakers that stop calling a service after repeated failures, go straight to the fallback while it recovers, and test it again after a cooldown.
- Graceful degradation: decide in advance what each feature does without its dependency, so a failed AI feature says so plainly while the rest of the app works.
Do less work: caching and queues
The book's rule for cost applies to capacity too: good architecture often means doing less work, not doing work faster.
Cache repeated work. Serve static pages and assets from a CDN, cache reference data and common queries, and cache answers to repeated questions when the answer does not depend on who asks. Give every cached item an expiry time, because a stale cache returns wrong answers confidently.
Queue slow work. Anything that takes seconds, such as processing an upload, a long model call or a batch of emails, should not hold a web request open. The API accepts the job and returns at once, a worker drains the queue at a rate it can sustain, and the client checks back. A queue turns a spike into a longer wait instead of an outage.
Ration expensive calls. When many users trigger the model at once, limit calls per request and per user, delay low-priority work and switch optional AI features off under overload. Per-user limits also blunt abuse (vibe-coded app security). For model APIs, LLM API reliability covers which errors to retry and provider fallback, and LLM cost per user covers the bill.
Load test before your users do it for you
Load testing asks whether the app keeps working when many people use it at once. Traffic rarely arrives evenly: one post or newsletter can send hundreds of people within minutes. Find the breaking point on your schedule, not theirs, by measuring four things as load rises:
- Latency: how long responses take. Watch the 95th percentile, not the average, and note the load at which it bends upward.
- Throughput: requests completed per second. When it stops rising as you add users, you have found a limit.
- Queue length: whether waiting work drains or grows. A growing queue is the earliest sign you are falling behind.
- Error rate: timeouts, connection failures and provider errors that only appear under stress.
import http from "k6/http";
import { check, sleep } from "k6";
const BASE = __ENV.BASE_URL || "http://localhost:3000";
export const options = {
stages: [
{ duration: "2m", target: 100 },
{ duration: "3m", target: 100 },
{ duration: "2m", target: 1000 },
{ duration: "3m", target: 1000 },
{ duration: "1m", target: 0 },
],
thresholds: {
http_req_failed: ["rate<0.01"],
"http_req_duration{step:page}": ["p(95)<800"],
"http_req_duration{step:ai}": ["p(95)<8000"],
},
};
// Each virtual user follows the main journey and pauses like a person would.
export default function () {
const page = http.get(`${BASE}/`, { tags: { step: "page" } });
check(page, { "page loads": (r) => r.status === 200 });
sleep(3);
const answer = http.post(
`${BASE}/api/ask`,
JSON.stringify({ question: "Can a glass jar go in the recycling?" }),
{ headers: { "Content-Type": "application/json" }, tags: { step: "ai" } },
);
check(answer, { "AI answer arrives": (r) => r.status === 200 });
sleep(5);
}Make the test realistic, then read the results
Testing one endpoint gives misleading results. Script the journey real users take (sign in, load the main page, save something, use the AI feature) with pauses between steps, and seed the test database with production-sized data, because a missing index is invisible on fifty rows. Locust (Python), k6 (JavaScript) and Apache JMeter all ramp simulated users this way.
Test a staging copy, not production with real users on it (deploying a vibe-coded app covers setting one up). Usage-based hosting and model APIs bill test traffic like real traffic. Replace the model with a stub that waits as long as the real one usually takes, so you measure your own capacity; then run a small, budgeted test against the real provider to learn its latency and limits.
Each bottleneck points to a different fix. Fix the first, test again and find the next; the purpose is to discover where the architecture must change before users force it.
| Bottleneck | How it shows | Usual fix |
|---|---|---|
| Heavy computation | Latency rises in step with users while server CPU is pinned | A queue and workers, batching, or a lighter model; a GPU only when a CPU cannot meet the target |
| Database | Query times grow and connection errors appear | Indexes, fewer queries per page, caching, a connection pooler |
| API server | Throughput flat as users rise; CPU or memory saturated | More workers or instances, once the server is stateless |
| External provider | Its latency spikes; 429 and 5xx responses appear | Timeouts, backoff, a circuit breaker, caching and per-user limits |
| Memory | Instances restart while load holds steady | Find what grows per request, such as an unbounded cache |
The reliability checklist
Chapter 15 sets out eight checks before an AI system meets real users, adapted here for an app built with AI tools. Passing them will not stop every failure, but it makes a catastrophic one far less likely.
- Uncertain or failed AI output has a fallback users can live with.
- Every external call has a timeout, and retries use exponential backoff in one layer only.
- Circuit breakers protect the app from services that keep failing.
- Dashboards track system health and the AI feature's behaviour; see how to monitor a vibe-coded app.
- Alerts fire when important metrics change.
- Caching removes repeated work, with expiry times.
- Servers are stateless, and autoscaling or a written plan covers traffic spikes.
- Load tests of the whole user journey confirm the app holds at the traffic you expect, with room to spare.
Many systems do not fail because they are wrong. They fail because they become popular.
Questions
How many users can a vibe-coded app handle?
There is no fixed number; it depends on the slowest step in your main journey. The limit is often not the web server but the database's connection limit, a provider's rate limit or one slow query. A load test that ramps simulated users until latency or errors climb gives you your own number, and tells you which component sets it.
Why does my app slow down when several people use it at once?
Usually because work that was fast for one user now queues behind other users' work: a single worker handling one request at a time, a blocked Node.js event loop, a query without an index, or a slow external call inside the request. Time each step under load and fix the slow one rather than adding servers.
Will moving to a bigger server fix performance problems?
Sometimes, and it is a sensible first step because nothing in the architecture changes. It will not help when the bottleneck is the database, a third-party rate limit or slow work inside each request. If server CPU or memory is the limit, scale up; if not, a bigger server only raises the bill.
Does serverless hosting make an app scale automatically?
It scales the functions, not everything behind them. More instances mean more database connections, more calls to rate-limited providers and a larger bill, and anything kept in memory between requests becomes unreliable. Serverless removes server management; you still need connection pooling, timeouts, caching and a load test.
What is a retry storm?
A retry storm happens when a service slows down, every client retries at once, and the retries add so much load that the service cannot recover. Retries stacked at several layers make it worse, because the attempts multiply. Prevent it with timeouts, exponential backoff with jitter, a retry limit, retries in one layer only and a circuit breaker.
Can I load test my production app?
Prefer a staging copy with production-sized data and the paid model replaced by a stub. If you must test production, keep it small, run it at the quietest hour, read your host's load testing policy first and set a spending limit, because test requests are billed like real ones.