Navian Open Source
GitHub Contact
← All posts

1,000× more testing. Not 1,000× more tokens.

Seshu Guddanti · Navian AI · August 25, 2026 · 4 min read
reliabilitymemorytestingAI code

We're Navian AI. We build Navian Pulse, a real-time decision engine for banks. It scores each event in under a millisecond and decides, in that moment, whether to let money move.

We build it the way the fastest teams build now: AI coding agents write much of the code, under human review and automated gates. That gives a small team enormous leverage. It also changes what has to scale next.

If AI lets us write 10× more code, we don't need 10× more tests, and we don't need to burn 1,000× more tokens generating them. We need a different kind of test: one invariant, written once, that a machine explores across 1,000× more executions on its own.

Machine-scaled testing mattered before AI

More testing is not automatically better; a thousand copies of the same shallow test add little. But broader exploration has always found failures that designed examples miss.

SQLite is a useful benchmark. Its roughly 156,000 lines of C are backed by about 590× as much test code and test-script material. Separately, its dbsqlfuzz infrastructure runs on the order of a billion database mutations a day. Those are different measures, but they point to the same engineering choice: reliability comes from sustained, machine-scale exploration, not from a small collection of happy-path examples. (How SQLite Is Tested)

That is why teams building high-consequence infrastructure moved toward simulation years ago. FoundationDB runs deterministic simulations of an entire cluster and injects failures under controlled, replayable conditions. TigerBeetle uses deterministic simulation to compress long periods of distributed-system behavior into fast, reproducible tests. These systems are not chasing a coverage percentage. They are searching for rare combinations of timing, ordering, failure, and recovery that no team could feasibly enumerate case by case.

The 1,000× layer is not new. AI makes it urgent.

How did the rest of us survive without 1,000× testing?

Through a combination of people, process, and accepted risk.

First, organizations accepted that some defects would escape into production. Staged rollouts, monitoring, incident response, and rollback reduced the blast radius; they did not prove the software correct.

Second, difficult concurrent and distributed code was usually concentrated in a few places and owned by senior engineers. Those engineers carried a large set of unstated requirements in their heads: assumptions about ordering, lifecycle, failure, recovery, and resource bounds that often never appeared in a test.

Third, organizations built layers of process around the code — ownership, design reviews, architecture reviews, change controls, and release gates — to compensate for what their tests could not cover.

That system worked, imperfectly, at human development speed. It becomes a bottleneck when AI increases code output by an order of magnitude. Ten times more code does not come with ten times more senior reviewers, design-review capacity, or institutional memory.

AI changes the economics — and adds one strength

Coding agents introduce three problems we have to engineer around.

The first is span of control. Prompts, rules, examples, and context can steer an agent, but they do not completely constrain the scope or consequences of a change. Every unstated assumption becomes a new testing surface.

The second is variability. The same request can produce different implementations on different runs. Once code is committed, the artifact is stable — but confidence can no longer depend on a developer's familiarity with the author's intent or habits. It needs durable specifications, executable invariants, and reproducible evidence.

The third is uneven quality. Agents produce plausible code quickly, including concurrent and distributed code that once required specialized engineers. Plausible is not the same as correct under an unlucky interleaving, a crash halfway through a write, or a workload that runs for weeks.

But agents add one important strength: they can iterate tirelessly against precise feedback. Give an agent a reproducible failure, and it can often diagnose and propose a patch, then re-run the evidence loop without waiting for the next human workday.

That changes the opportunity. The machine can search for failures; the agent can fix them; people can focus on the invariant, the architecture, and the evidence.

1,000× testing is not an artifact of AI. It is an imperative of AI.

We already test extensively. Why add another layer?

Separate assurance work into three economic models.

Three economic models of software assurance — general-purpose techniques and machine-explored testing scale on compute; designed tests must be authored case by case. The machine-explored layer is the missing one.

General-purpose techniques — SAST, dependency and container scanning, DAST, sanitizers, and chaos engineering — apply reusable rules or faults across many applications. More code may require more compute and produce more findings, and tools like DAST and chaos engineering still need real application-specific configuration and triage; but none of this requires someone to invent a new scenario for every execution path.

Designed tests work differently. Unit, functional, regression, and performance tests can all be automated, but each scenario still has to be specified, reviewed, and maintained. Automating execution makes those cases cheaper to run; it does not make their authoring at scale easy or cheap — including in tokens. Penetration testing sits at the edge of this category: it still includes real exploratory human work, not just executing predefined cases.

The missing layer is application-specific and machine-explored. People author the model and the invariants; the machine instantiates and explores millions of executions, searching for a counterexample. Antithesis offers this discipline as a platform: users package and upload container images that run inside Antithesis's own environment. The navian tools take a different path — open-source Rust crates that run the checks locally, inside your own test suite and CI, with no environment to ship anywhere.

To reach 1,000× testing, the unit of human authoring has to change. That unit is written once. Exploring it further costs compute, not tokens.

This week: navian-memcheck

Last week we released navian-dst, deterministic simulation testing for Rust, targeting hard-to-find concurrency and crash-recovery failures. This week's release turns the same lens on memory.

What it checks. Leak detectors answer one question: was this memory released when it should have been? navian-memcheck answers a different one: memory that's still legitimately in use — is it staying bounded, or just growing forever?

Why it matters. An agent writes the insertion path easily. It often skips the eviction path, unless told not to. Every byte stays reachable, so no leak detector ever complains. Tests pass. Then, weeks later in production, memory keeps climbing until something kills the process. We know because it happened to us: one of our engines grew to 52 GB over several weeks, with zero leaked bytes reported anywhere, before the kernel killed it.

There are only two ways this goes wrong: memory that never stops growing over time, and memory that grows too fast as load scales up. Both look completely fine to a leak detector, because nothing is ever lost. It is just never bounded.

How it works. State the bound once, and let the machine check it under load:

assert_plateau(…)    // after warmup, memory must stop climbing under steady load
assert_bounded(…)    // memory must never cross a hard cap as load increases
assert_linear_in(…)  // cost per entity stays linear — no upward bend as the population grows

The only expensive part is writing the invariant, once. After that, the machine explores load, timing, and scale on its own — no extra scenarios to write, no extra tokens spent generating them.

Install it and point it at a long-lived service: cargo add navian-memcheck. Full API details and worked examples are in the documentation.

navian-memcheck is the second tool in the family. More are coming.

Resources

navian-memcheck

The first tool in the family

Further reading

← Back to all posts