OpenClaw reports removing 400,000 lines of low-value agent-written tests

• OpenClaw removed roughly 400,000 lines of tests with little reported coverage movement, according to steipete's cleanup report.
• The cleanup used a bounded objective rather than a generic request to tidy tests: steipete's proposed prompt targets the least useful 20% while holding coverage within 2%.
• RayFernando1337's separate five-wave inventory cut a suite from 2,713 to 311 tests and from 38 seconds to 9.7 seconds, according to RayFernando1337's inventory.
• The underlying dispute is test location: RayFernando1337's reply favors QA near the customer runtime over unit tests that merely restate freshly generated code.
• CI spend is part of the motivation, as steipete's reply put it: "If you pay for CI you will understand."

The linked test-audit file asks agents to justify a test's behavior, independent contract, coverage gap, and any test-only production seam. Its mirrored skill page identifies the audit targets as tests that reassert source, duplicate stronger proof, couple behavior to implementation, or preserve those seams.

OpenClaw's maintainer described the removal as a large reduction in agent-written test code with little coverage movement, not as a coverage-reduction exercise.

LLMpsycho independently repeated the headline figure and said they had retained similar piles because deletion felt riskier than leaving them in place LLMpsycho's post. Steipete added that the removed tests came from models less than six months old; newer models write more clever tests, he said, but they are still not reliably useful steipete's follow-up.

The audit skill separates authoring, focused audits, and subsystem-wide campaigns. Its four questions turn a vague notion of test quality into a decision record:

1. What observable behavior does the test cover?
2. What contract does it enforce independently?
3. Why would existing coverage miss the failure?
4. Does it require a production seam used only by tests?

Steipete's companion prompt supplies the campaign constraint: an agent told only to clean up stops too early, while a target of deleting 20% of the least useful tests with at most a 2% coverage change creates a measurable stopping condition.

The skill retains tests around public APIs, plugin SDKs, protocols, configuration, migrations, storage, security, platform behavior, and generated cross-language contracts. It calls for rewriting implementation-coupled tests at the owner boundary rather than deleting a meaningful contract.

RayFernando1337 described the failure mode more bluntly: generated unit tests often restate code written moments earlier, always pass, break during refactors, and consume agent time fixing the test rather than the feature.

His full cleanup prompt defines three standing rules:

• Never write a unit test after writing the code.
• Prefer E2E tests as the sole test mechanism for complex features, producing a repeatable artifact at the end.
• Before isolating a system, list its failure modes, then write the code.

That is a different selection criterion from line coverage. The skill similarly asks whether a candidate test protects an observable contract at the boundary that owns the behavior.

RayFernando1337's smaller campaign provides the only detailed before-and-after inventory in the evidence: five merged waves reduced 260 files to 68, 72,302 lines to 11,712, and 2,713 tests to 311. The stated retention rule was whether deletion could allow a real security, money, or data-loss bug to ship unseen.

NickADobos argued that coding benchmarks leave out the cost of a successful-looking patch that adds a hundred tests no engineer would keep. His proposed anti-slop criterion is review and cleanup burden, not whether an agent crosses a task-completion threshold.

_lopopolo attributed the pattern to an optimization gap: engineers judge a code artifact by future consequences, while models are not trained for long-term coherence or multi-turn regret _lopopolo's reply. That claim remains an explanation for the reported cleanups, rather than a measurement of OpenClaw's suite.

Steipete associated the concern directly with paid CI.

He gave only "It doesn't matter" in another reply steipete's reply. Separately, he wrote that Google rejected an update and it was stalling.