Caching your test suite is easy. Caching it without lying is not.

Your suite is slow and the code hasn't changed since the last green run. Caching that result is a textbook optimisation: an afternoon's work, and the pipeline flies.

It's also a promise, and that's the part nobody writes down: you're asserting that the state you measured is the state you're about to ship. When that promise breaks the system doesn't get slow, which would be the kind failure. It certifies a green that doesn't exist.

When I implemented it in my orchestrator, what looked like a trivial change turned out to rest on two conditions, plus a marker that makes it auditable afterwards. Miss any of the three and the cache stops being an optimisation and becomes a risk that is also invisible.

The starting point

The gate ran its checks serially, one after another. The full suite, with a 40-minute timeout. Then each frontend build, at 5 minutes apiece. And finally mutation analysis, at 20.

That 40-minute timeout says a lot: back in May I had raised it from 10 to 40, because the suite no longer fit. The curve pointed in one direction only.

But duration wasn't the worst of it. This was: when a task failed the gate and had to be retried touching only the frontend, the re-verification ran the entire backend suite again, which hadn't been modified at all. Minutes of compute to confirm something we already knew.

Two changes, one obvious and one delicate

The obvious one: run the suite and the frontend builds in parallel instead of serially. There's no reason for them to wait on each other beyond the fact that's how it was written.

The delicate one: cache the green result of the suite by the combination of project, repository and commit hash. If we already ran the tests against exactly this code and they passed, there's no need to run them again.

Put that way it sounds harmless. This is exactly where the cache stops being an optimisation and becomes the promise: from this point on, the green the system reports doesn't come from running the tests, it comes from trusting that the code is the same.

The two conditions holding the promise up

That's why the cache has two locks, and neither is optional.

The working tree has to be clean. If there are uncommitted changes, the commit hash no longer describes what's on disk. The code about to be evaluated isn't the code that was tested. In that case the cache isn't even consulted: the suite runs.

The result is valid for 24 hours. Not because code changes on its own, but because the environment does: dependencies, containers, test databases, external services. A green from a week ago says very little about today's state.

There's a third, smaller detail I like: the suite's exclusion lock — the one preventing two runs from stepping on each other — is only acquired if something is actually going to run. When the cache hits, the function returns before reaching that line. Without it, we'd be serialising runs that will never happen.

The marker that makes auditing possible

This is the part I consider most important in the whole implementation, and the one I see written down least often elsewhere.

Every result coming from the cache travels flagged as such inside the quality gate's data. It's on record that this task didn't run the tests: it leaned on an earlier run.

What's that for? For the day a green shows up that shouldn't have. On that day I'll be able to reconstruct exactly which tasks leaned on the cache and which genuinely ran, and check whether that's where the problem was.

Without that marker, a false green is indistinguishable from a legitimate one. You have a bug you can't even scope, because you don't know which universe of tasks to search.

It's a principle I now apply to any optimisation replacing a real check with an inference: every automated promise needs to leave the marker that lets you audit it when it breaks. Because it will break eventually, and on that day the marker is all you have.

What was deliberately left out

Mutation analysis — which checks whether the tests actually detect changes in the code, not merely whether they pass — was kept out of the cache on purpose.

The reason is that it measures something different. The suite is evaluated against the state of the repository; mutations are evaluated against the specific task's diff. Two tasks on the same base commit can have completely different diffs, so caching by commit hash would make no sense: they're orthogonal.

What I did add was a short-circuit from the other side: if the task's diff contains no mutable code, the analysis simply doesn't run. There's nothing to mutate. That resolved a large share of cases without touching the underlying logic.

The outcome

Between the parallelism and the cache, the quality gate stopped being the obvious bottleneck it had been. It's still the most expensive step in the system — that 29% of the bill didn't move as much as I expected — but the wait per task dropped noticeably.

What stayed with me above all was a criterion I've reused several times since: speeding up a check is fine as long as you can prove afterwards that the check existed. If the speed-up leaves you unable to audit what was verified and what wasn't, you're not optimising — you're loosening the check and calling it something else.

The front left open

That same June review left another item on the table, one that wasn't about speed and was considerably more uncomfortable.

There was a component I'd been using for over a month, running more than a thousand times a month, and I couldn't say whether it improved anything. It never failed. That was exactly the problem.

That's what the next entry is about.