01
Measure the function
Combine cyclomatic complexity with test coverage using the existing CRAP metric. Where coverage is unavailable, show complexity on its own rather than imply that testing was measured.
Open-source developer tool
crapkit
Know where the code needs attention.
crapkit brings complexity, test coverage and change history together to help developers and coding agents decide what to improve next.
- My role
- Creator & maintainer
- License
- MIT
$ crapkit worklist
classifycalc/grade.py:7- Complexity
- 14
- Test coverage
- 50%
- CRAP score
- 38.5
One function. Complexity and test coverage, in view.
The problem
Where should the next
hour of work go?
Code can be easy to generate and hard to change with confidence. A passing test suite or a repository-wide coverage percentage does not tell you which functions combine complicated logic with gaps in testing.
I created crapkit to make that work visible at the function level, and to make the same evidence usable by people, coding agents and automated checks.
02
Choose the next change
Use change history to rank the work. A complicated function that changes often deserves a different level of attention from code that rarely moves.
03
Keep progress from slipping
Record existing debt as a baseline, then check changes against it. The ratchet allows recorded scores to improve but prevents them from rising.
Built for the way teams work.
Use it from the command line, connect a coding agent through MCP, or run it in a GitHub Actions workflow. Analysis and scoring run locally without telemetry; configured test and other commands retain their own behavior.
A reproducible example
Measuring the effect of better tests.
Eight functions. The same implementation. More boundary tests.
This small public Python fixture starts with four passing test cases. I added 25 cases covering grade boundaries, empty input, missing values and report branches. No implementation code changed. The table shows what crapkit measured before and after.
Passing test cases4 → 29
Sum of CRAP scores60.26 → 29
Functions above target 63 → 0
Measured on with Python 3.11 on Windows; source and tool pinned to 8ff0fab87899.
Swipe or scroll sideways to see every measurement.
| Function | Complexity | Coverage before | Coverage after | CRAP before | CRAP after |
|---|---|---|---|---|---|
grade( score )calc/grade.py | 3 | 0% | 100% | 12 | 3 |
summarize( rows )calc/grade.py | 2 | 0% | 100% | 6 | 2 |
coerce( word )calc/parse.py | 5 | 66.67% | 100% | 5.93 | 5 |
read_row( line , cols )calc/parse.py | 6 | 66.67% | 100% | 7.33 | 6 |
parse( text , cols )calc/parse.py | 3 | 100% | 100% | 3 | 3 |
tally( rows , low , high )calc/report.py | 4 | 100% | 100% | 4 | 4 |
render( bins , width )calc/report.py | 2 | 100% | 100% | 2 | 2 |
banner( bins , total )calc/report.py | 4 | 0% | 100% | 20 | 4 |
Reading the measurements
Coverage is measured within each function: branch coverage when branches exist, otherwise line coverage. CRAP combines that coverage fraction with cyclomatic complexity. With full measured coverage, the score equals the complexity; adding tests did not simplify the code.
The reported total is the sum of the eight function scores, which crapkit calls crap_load. It is not an estimate of hours saved or the number of tests needed. The target of 6 is this example’s configured threshold. Values in the table are rounded; the CSV retains the measured precision.
The CRAP metric was introduced by Alberto Savoia and Bob Evans in 2007. My contribution is crapkit’s tooling around measurement and change checks.
Original metric and limitations · Pinned crapkit documentation
Reproduce the result
Download and extract the package, then choose a fresh Python 3.11+ environment with Git available. Install the supplied pinned requirements in that environment and run the reproduction script. The README includes Windows and macOS/Linux commands.
python -m pip install -r requirements-reproduce.txt
python reproduce.py --output resultsThe runner installs nothing and refuses an existing output directory. It runs the bundled public fixture, adds the supplied tests, and compares the measured result with the published values.
The package contains the original fixture, added tests, MIT license, portable runner, pinned requirements and reviewed results.
Inspect the original public fixture ↗Behind the tools
What do we ask
of the systems we build?
My writing explores human agency, intent and accountability. crapkit addresses one practical part of building software: making the evidence for code changes easier to inspect.