How do you test a system before real users arrive?
A demo that works proves little. Test the things real days do to a system: concurrent use, retries, abuse, and a release that goes out half-finished. Three problems we found in our own build, and what we still haven't fixed.
- A feature that works once proves little. Fire it concurrently and repeatedly.
- Test the safeguards. A limiter, a lock or a permission rule can exist and still do nothing.
- Check the release process as well as the code: order of steps, and what reaches users first.
- Label results honestly: built, tested, deployed, in production, measured.
- Our own pre-launch review found three real problems. One is fixed in tests only, and one is still open.
Quick answer
Test the things a real day does that a demo doesn't: many people acting at once, requests that repeat, traffic that tries to abuse a limit, and a release that goes out in the wrong order. Test the protections themselves, not just the features. Then write down what is still open. A pre-launch test is only honest if its failures are listed.
The short answer
A demo is one person doing the happy path on a good connection. Launch day is forty people, a bad network and one impatient finger. Test for the second.
Why engineers and operations leaders ask this
You don’t want your first real users to be your test. But you also can’t test everything, and a vendor who says “fully tested” is telling you very little. The useful question is: tested against what?
What good looks like: five things a real day does
| What real use does | What to test | What a pass looks like |
|---|---|---|
| Several people act on one record at once | Fire concurrent requests at the same record, more than once | No duplicates, no gaps, no half-finished changes |
| Requests repeat | Send the same request twice | The second one is recognised, not executed again |
| Someone pushes past a limit | Send traffic beyond the limit | Requests are refused, and you see them refused |
| A permission rule exists | Try the forbidden action as each role | It is blocked, not just hidden in the screen |
| A release goes out | Walk through the release steps in order | The web never reaches users before the function it calls exists |
Two habits sit under the table. Test the safeguard, not just the feature. A lock, a limiter or a permission rule can be in the code and still not do its job. And write down what failed and what is still open. A test report with no open items is a sign nobody looked hard.
What we found in our own build
A clinic-queue platform, built and tested on a demo clinic ahead of launch. These findings come from our own audits before any real patient used it. None of it has been proven under real clinic load.
Three problems from reviewing the clinic-queue platform, written at pattern level.
1. A queue update that could tear. Advancing a queue was a series of separate writes. A failure partway could leave it half-updated. We moved it into one transaction under a per-doctor lock. The fix passed 48 assertions and two concurrency runs locally. It isn’t in production yet. We call it tested, not deployed. The pattern is explained here.
2. A rate limiter that blocked nothing. The first limiter kept its count in the memory of the function handling each request. Functions like these don’t reliably share memory between requests, so the count never built up. When we sent traffic past the limit, in sequence and in parallel, nothing was blocked. We moved the counter into the database and a live test then showed requests being refused. We only found this because we tested the limiter, not because we trusted it.
3. A release that ran out of order. Our release is still four manual steps: database, functions, configuration, web. Once, a feature reached users before the backend function it needed existed. A check that catches that drift is on our list. It isn’t built yet.
What these have in common: all three were found by looking for failure, not by confirming success.
Where this doesn’t apply
A brochure site or an internal tool with five users doesn’t need concurrency runs and a limiter test. Test the main flows, check the permissions, and move on. Also, none of this replaces real use: testing on controlled data tells you the system survives the cases you imagined. Real users find the ones you didn’t. We haven’t measured how many defects a pre-launch review catches, so we don’t claim a rate.
Questions to ask any vendor, including us
- What did you test that a single user doing the happy path wouldn’t reveal?
- Show me a test where many requests hit the same record at once.
- How did you check that your rate limit and permission rules actually block things?
- Walk me through a release. Which step comes first, and what happens if one is skipped?
- What did your last review find that is still open?
Next: how to prevent double bookings and what a tech handover should cover. Part of the guide Buying custom software and hiring a tech vendor.
The findings come from our clinic-platform brief and our own audit notes and are tracked in our claim ledger. The five-row table is our method, not an industry standard. We hold no defect-rate or load measurement for this build, and no figure on this page comes from a third party.
More by GarunaCDX
Let's talk about your build.
Tell us what you are building. A founding engineer replies within one working day with how we would approach it.
