test-trust

Purpose

Audits how far a test suite can be trusted, using runnable evidence instead of opinion.

Source Skill purpose

When to use it

Use test-trust to ask how much a test suite can be trusted, either for a named scope or as a review focus for changed tests. It checks for nondeterminism, order dependence, hollow assertions that remain green when guarded code is broken, and slow tests relative to the scope's median. It proves runner capabilities before relying on them and uses repeat runs and targeted mutations in separate worktrees. The result reports confirmed findings without editing tests or production code.

Full mode establishes a baseline and runtime estimate before selecting mutations. Diff mode narrows attention to changed tests and the units they protect. A candidate is challenged again before reporting; slow behavior alone is informational unless another written condition changes the result.

Source Skill purpose

How it works

Steps of the skill, in source order

Source Flow

Set up audit Resume or create the evidence ledger and clean a recorded stale worktree.
Source Flow
Probe toolchain Discover project test commands and prove required runner capabilities.
Source Flow
Run baseline Measure the scoped suite before making any test mutation.
Source Flow
Choose mutations Select production behavior that changed tests claim to protect.
Source Flow
Estimate cost Use baseline timing and target count to bound the audit runtime.
Source Flow
Probe flaky and slow Repeat runs for nondeterminism and compare durations to the scoped median.
Source Flow
Probe hollow tests Break guarded behavior in an isolated worktree and run its test.
Source Flow
Refute and report Challenge each candidate, then validate confirmed findings and clean isolation.
Source Flow

Key rules

Audit and report without fixing production code or tests.
Source Hard rules
Confirm a finding only when its stated evidence condition was observed.
Source Hard rules
Keep mutations out of the user's working directory.
Source Hard rules
Run tests sequentially to avoid false shared-resource failures.
Source Hard rules
Prefer the project's own test commands over ecosystem defaults.
Source Hard rules

Output and gate

Output

The skill definition states no output.

Source Skill definition

Gate

Confirmed flaky or hollow findings make the test review fail.
Source Gate rule (diff mode)
Without such confirmed findings the review passes; slow and unvalidated concerns remain informational.
Source Gate rule (diff mode)
The findings validator recomputes the gate verdict from recorded evidence.
Source Gate rule (diff mode)

Source