AI delivery impact report

The impact report compares two equal windows of a public repository’s pull request history around a date you choose — usually the day a team adopted an AI coding tool. No account, no install, no card. Run as many as you like on any public repository.

It answers one question: what changed in the pull request data between the two windows. It does not tell you what caused the change — see what it does not claim.

1. Open /impact and paste a repository. A bare owner/repo, a browser URL, or a clone URL all work.
2. Pick the cutover date — the day the thing you want to measure started.
3. Submit. The scan is queued, then runs; the page moves through pending → running → ready.
4. You land on a report with its own unguessable share URL. Anyone with the link can read it.

Scans run one at a time against a shared GitHub rate limit, so a report on a busy repository is not instant. If a scan cannot finish it ends as failed with a plain-language reason.

• 90 days on each side of the cutover by default. Both sides always use the same span, so the comparison stays symmetric.
• Ninety days is long enough to smooth over one quiet fortnight, and short enough that a re-org two years ago does not contaminate the baseline.
• Merged pull requests only. Open and closed-unmerged PRs are not counted.
• Bot-authored PRs are excluded throughout. Dependabot alone can swamp throughput and would make every repository look transformed.

Medians, not means. Lead time and PR size are heavily right-skewed: one six-month-old PR merged during a cleanup would drag a mean far enough to invent a trend that is not there.

A percentage change is only shown when the “before” value is non-zero, because a percentage against zero is undefined.

A report is flagged sparse when either window holds fewer than 20 merged pull requests. Below that, ordinary week-to-week variance swamps any real shift and the percentages are noise.

Every report also shows its total PR count across both windows. Treat it as the honesty signal: small numbers mean a weak comparison, no matter how large the percentage looks.

The report is descriptive, never causal. It says what changed in two windows. It never says a tool caused the change, and it never projects a return on investment.

Anything else moving in the same window — a hiring wave, a re-org, a quarter-end push, a change in review policy — lands in the same numbers. Read a report as the start of a conversation with your team, not as a verdict on a tool.

This is also why speed figures are always shown next to quality figures. A lead time that halved while revert rate doubled is not an improvement.

• Public repositories only. Private repositories need the tracked dashboard — see pricing.
• Size metrics are sampled. GitHub’s list endpoint omits line and file counts, so those need a per-PR call. A scan spends at most 120 of them, sampled evenly across the whole span rather than head-first, so a detailed “before” is never compared against an empty “after”.
• One cutover date. There is no multi-phase or rolling comparison.
• Reports are point-in-time. A report is not refreshed as new PRs merge; run a new one.
• Share links are unguessable, not private. Anyone with the URL can read the report. The underlying repository is public either way.