Find the commit that made your test suite slow

The idea

A command-line tool that treats test duration as a tracked metric instead of a vibe. It ingests the machine-readable timing output your test runner already produces on every CI run, stores a per-test history, and flags individual tests whose runtime stepped up rather than drifted. For each step change it runs an automated bisect over the commits in that window and names the commit that caused it.

The output is a short report: this test went from 180ms to 4.1s, it happened between these two commits, and here is the one that did it.

Why build this

Suites rot one commit at a time. Nobody notices the afternoon a session-scoped fixture became function-scoped, or the day a helper started hitting a real HTTP client instead of a stub. Six months later CI takes twenty minutes, people stop running tests locally, and someone gets a quarter-long "speed up CI" project that starts from zero information.

Existing tooling does not close this loop. CI dashboards show total wall time, which hides a 4-second regression inside a 900-test suite. Profilers tell you what is slow now, not when it became slow. pytest --durations and Jest's --verbose give you a snapshot with no memory. The missing piece is history plus attribution, and both are cheap to get now: every mainstream runner can emit structured per-test timings, and CI providers keep artifacts long enough to backfill months of history on the first run.

Stack sketch

• Python CLI built with Typer, distributed as a uv tool install single binary-ish entry point.
• Parsers for the formats runners already emit: JUnit XML (pytest, JUnit, PHPUnit), jest --json, go test -json, cargo nextest --message-format libtest-json.
• DuckDB as the local store, one file committed nowhere and cached in CI, plus an optional S3-backed Parquet directory so a team shares one history.
• Regression detection with ruptures for changepoint detection on each test's duration series — a step change is the signal, noise is not.
• Bisect driver shelling out to git bisect run with a generated script that runs only the suspect test, via GitPython for commit-range math.
• Terminal output with Rich; --format json and a GitHub Actions job summary writer for CI.

Scope for v1

In: - slowdog ingest junit.xml --commit $SHA to record a run. - slowdog report listing regressed tests ranked by seconds added, with the commit window. - slowdog blame <test-id> to bisect that one test inside its window and print the culprit commit. - pytest and Jest parsers only. - Local DuckDB file, plus a documented GitHub Actions snippet for cache and ingest.

Out: - No hosted service, no web UI, no auth. - No flake detection — that is a different product and will pull focus. - No fixing anything. The tool names the commit; a human reads the diff.

Where it could go

The obvious next step is prevention: a slowdog check that fails a pull request when it adds more than N seconds to the suite, with a per-test budget file in the repo. Teams that adopt duration budgets stop needing the archaeology, which is the real win.

After that, the same history unlocks scheduling. Per-test durations are exactly what you need to pack a suite into balanced CI shards, and most sharding today is alphabetical or round-robin. Selling "your six runners now finish within 20 seconds of each other" is easier than selling a report. A third path is widening the metric: peak memory per test, database queries per test, and network calls per test all come from the same ingest-and-changepoint machinery, and query count regressions are usually the cause of the duration regressions anyway.

Watch out for

Bisect assumes a test that can be run in isolation and reproduces its slowness deterministically — order-dependent suites and tests that are slow only on a cold cache will send you to the wrong commit, so the report needs to say "likely" and show the timing spread it measured. Also budget for CI minutes: an unbounded bisect across a long window is expensive, so cap it and let the user raise the cap.