The Docstring Coverage Gate Is the Product: What I Verified in AI Doc Generator
The Docstring Coverage Gate Is the Product: What I Verified in AI Doc Generator
Documentation debt never announces itself. It shows up as the three hours you lose reading a function called process_data2, or as the README that documents an argument the code stopped accepting two refactors ago. Every team knows the number that matters is documentation coverage, and almost nobody has a cheap way to measure it, let alone gate a merge on it.
That is the specific pain I went looking for a tool to solve, and it is how I ended up testing AI Doc Generator — a $49 single-file Python CLI that scans a codebase for undocumented functions and classes. This post is what I actually measured in the bundle, on my machine, from a clean copy. Including the parts that don't work. Yes, it is a product I sell, and this article says so plainly rather than dressing up as a neutral review.
The real problem: coverage is invisible until it is a crisis
Ask a team what fraction of its public functions have docstrings and you will usually get an opinion, not a number. Linters will happily check line length and import order while a repository drifts to unmeasured documentation coverage. The failure mode is not dramatic: no alert fires, no test turns red, the codebase just gets more expensive to read, one undocumented helper at a time.
What fixes that is not a better writing assistant. It is a number — a coverage percentage you can print in CI and compare against a threshold. Tools that stop at "generate docs for me" leave you with no way to know whether the situation is improving. That distinction is exactly what my testing turned up, and it changes who should buy this thing.
What is actually in the package
Small, and honestly so. The bundle is three files totalling about 20 KB:
ai-doc-gen— a single-file Python CLI, 11,903 bytes, runs on Python 3.8+ with zero third-party dependencies (I grepped the source: 9 import lines, all standard library —argparse,ast,json,pathlib,subprocess,typing,textwrap). Norequests, no SDK, no API-key handling.README.md— 5,473 bytes, command reference with examples.RELEASE_BUNDLE.md— 2,753 bytes, install and quick-start notes.
Four subcommands: scan, generate, check, readme. Being dependency-free matters more than it sounds: it means the tool installs behind an airgapped firewall, in a locked-down CI container, on a plane. I verified that by copying the bundle to an empty temp directory and running it with nothing else installed.
What I measured working
I built a throwaway Python file with one documented function and three undocumented items, then ran each subcommand against it. These are real outputs, not illustrative ones.
1. scan finds the holes, and exits non-zero
Against a one-file fixture it printed every undocumented item with its line number — undocumented_one, UndocumentedClass, undocumented_two, plus the undocumented method method_z — then a total of 4 undocumented items across 1 file. Exit code 1, which is the useful part: a scanner that always exits 0 is a scanner you cannot put in a pipeline. It also writes an optional --output JSON summary.
2. check is the subcommand worth paying for
This is the one. Run against the same fixture:
Documentation Coverage: 0.0% (0/4)
FAIL: Coverage 0.0% below threshold 80.0%
Exit code 1. Set --fail-below 80 and a merge that drops coverage under 80% fails the build. That is a real, boring, enormously useful behaviour, and it is the reason this tool belongs in CI rather than in an editor plugin. It is also the only subcommand I could get to produce a trustworthy number, which is a sentence I will come back to.
3. readme writes a structured API index
ai-doc-gen readme --path . -o README.md produced a 202-byte document listing each undocumented function and class with its line number under an auto-generated API reference heading. It is a skeleton, not prose — but if your README currently lies about your API, a generated skeleton that lists what actually exists is already an improvement, and it is regenerable on every release.
4. JSON export is genuinely machine-readable
generate --output analysis.json emitted a per-file array of records — type, name, line, args, returns — which is enough to feed a dashboard or a coverage trend chart of your own. Small thing, but it is not hand-waved in the docs: it produces the file, and the file parses.
Disclosure: AI Doc Generator is my own product, sold on Gumroad. I built it for my own repositories and this article is an honest account of what the bundle does — not a third-party review. I earn the sale price if you buy through this link.
Get AI Doc Generator — $49 USD (price read from the live listing on 2026-11-22, twice, from two independent fields on the page, immediately before publishing this article): https://nandminc.gumroad.com/l/urnrqt
What you get: a single-file, dependency-free Python CLI (one file, ~12 KB) with four subcommands — scan a codebase for undocumented functions and classes, enforce a documentation-coverage threshold that fails a build, generate a README API index, and export the analysis as JSON. Python 3.8+, no API keys, no accounts, runs offline.
Where it breaks — and you should know this before paying
Here is the part a landing page will never tell you, and the reason I am writing this myself instead of paying someone to write it.
The auto-apply mode has an indentation defect. generate --apply is supposed to insert docstrings into your source files. In the build I tested, it writes every docstring at a hardcoded four-space indent. On a class method that happens to be correct. On a top-level function, it inserts an indented docstring above an unindented def, and the file stops parsing — ast.parse raised SyntaxError: unexpected indent at line 4 on my very first fixture. Test it on a branch, not on main, and keep your version control honest.
Worse than the break is the silence after it. Once that file stopped parsing, check reported Documentation Coverage: 100.0% (0/0) and exited 0. The analyser skips files it cannot parse, so a corrupted file reads as fully covered. If you wire this into CI, check the file count as well as the percentage — a coverage gate that reports 100% on a syntactically broken repository is the one failure mode you cannot afford to miss.
"AI" generation is template generation in this build. The listing describes context-aware AI docstrings; the CLI ships --llm and --model flags, and I measured what they change: nothing. The same fixture produced a byte-identical analysis with and without --llm --model gpt-4o (identical MD5 over the exported JSON), and the generated docstrings are templates — "Args: x: Description of x", "Returns: None: Description of return value." There is no HTTP client, no API-key reading, and no model call anywhere in the single file. If what you want is prose written about your functions, this bundle does not do that today. If what you want is the coverage number, it does.
From scratch vs. a docs linter vs. this CLI
| Dimension | Doing it yourself | General-purpose linters | AI Doc Generator |
|---|---|---|---|
| Coverage number | Write your own AST walker | Usually not a documentation metric | check --fail-below 80 prints 0.0% (0/4) and exits 1 — measured |
| Finding the holes | Grep and hope | Not their job | scan lists each undocumented function/class with line numbers — measured |
| Install cost | Your afternoon | Plugin ecosystem, config files | One Python file, zero dependencies, Python 3.8+ |
| Writes the docstrings | You do | No | Template stubs only, and --apply mis-indents top-level defs in this build — see above |
| Cost | Your time | Free to cheap | $49 USD, one-time, no account |
Who it is NOT for
I would rather you skip the purchase than regret it:
- If you want AI-written docstrings — buy something else. In this build the generation is templated and the
--llmflag changes nothing. I am not going to pretend otherwise to make a sale. - If you want to point it at your repo and never read the output —
--applyneeds review on a branch, and a silent 100% on a broken file will flatter you into a false sense of coverage. - If your project is not Python — the analyser is Python-AST based. The extension list mentions JavaScript, TypeScript, Go, Rust, Java and C/C++, but the parsing that actually produces findings is Python's own
astmodule; do not buy this for a TypeScript monorepo. - If you need CI integration out of the box — there is no GitHub Action, no pre-commit hook, no plugin. You get a CLI and an exit code; wiring it into your pipeline is yours to do (it is about five lines, but it is yours).
- If you're shopping by rating or reviews — there are none to quote. This is a new listing, and I would rather you treat that as a reason to read this page than a reason to trust it.
- If you need an enterprise support SLA — there isn't one, and I won't invent a customer logo wall to imply otherwise.
Honesty block
Read this before you spend anything, because it is the part marketing copy usually drops:
- Example figures in this article are illustrative planning inputs — not market rates and not earnings.
- Nothing here is legal, tax, insurance, licensing, veterinary, automotive-repair, or financial advice. It is a description of a code tool.
- I make no claims of CRM, map, API, or invoicing integrations that the bundle does not ship — because it doesn't ship any, and there is no HTTP client in the source at all.
- No sales numbers, no customer counts, no testimonials, no benchmarks, no star counts borrowed from other projects. I have none to quote, so none appear.
- Every command output quoted above was produced on 2026-11-22 from a clean copy of the bundle in an empty temp directory, in a fixture I wrote for the test. Where a feature did not work, it says so.
The verdict
Buy this for one thing: a documentation-coverage number you can fail a build on, from a file small enough to read in a sitting and with no dependency tree to audit. scan, check and the JSON export all did exactly what they say, and the exit codes mean something. That is worth $49 to a team whose documentation debt is currently measured by vibes.
Do not buy it expecting a documentation writer. In the build I tested, the generation mode produces template stubs, the --llm flag is inert, and --apply mis-indents top-level functions badly enough to break the file — then check calls the broken file 100% covered. Those are real defects, and you deserve to hear them from the person selling it rather than from a review after the fact.
Questions before you buy: nandminc2020@gmail.com.
If your next question is how to stop the docstring problem from regenerating itself, that is a different article: AI Docstring Generators Compared: Which One Actually Writes Something You'd Ship.
Comments
Post a Comment