Stop Writing Docstrings by Hand: Automate Python Documentation
The short version: Python's standard library can tell you which functions lack docstrings in about forty lines of ast code. It cannot write them. I spent last week wiring both approaches — a stdlib audit script and three dedicated AI doc tools — into my own small project, then handed the output to an AI reviewer to grade. Here is what survived, what broke, and what I would do again.
If you have read our earlier head-to-head, AI Docstring Generators Compared, you know I already had opinions about these tools. This time the test was different: I automated the docstrings for a real codebase I maintain, and I had an AI grade the result with no idea which tool wrote what. My lint bot became the most honest code reviewer I have ever had.
Why hand-written docstrings quietly die
Every Python developer starts with good intentions. Then a deadline lands, the function still works, and the docstring is "one line for now." Two months later the docstring describes the previous signature, which is worse than no docstring at all, because readers trust it.
The honest truth I had to accept: I have never kept docstrings current by willpower alone. Not once across a full project. The only docstrings in my repos that stayed accurate were the ones some tooling forced me to touch.
What the standard library gives you for free
Before paying anyone anything, it is worth knowing how far the stdlib gets you. Two modules do most of the work:
inspect — runtime introspection
import inspect
from mymodule import api
for name, fn in inspect.getmembers(api, inspect.isfunction):
if not inspect.getdoc(fn):
print(f"missing docstring: {name}")
This runs against imported modules, so it sees exactly what a user of your library sees. Ten lines, zero dependencies. The catch: the module must import cleanly, and dynamic functions (decorators that rebuild signatures, __getattr__ tricks) can fool it.
ast — static auditing without importing
import ast, sys
tree = ast.parse(open(sys.argv[1]).read())
for node in ast.walk(tree):
if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
if ast.get_docstring(node) is None:
print(f"{sys.argv[1]}:{node.lineno} {node.name} lacks a docstring")
Forty lines gets you a coverage report across an entire tree: how many public functions are documented, which modules regressed since last week. Feed it to CI and you have a hard gate. Tools like interrogate and docstr-coverage are essentially polished versions of this script, and they are genuinely good.
Where the stdlib stops
Both approaches answer one question — is there a docstring? — and neither can answer the question that actually matters: is the docstring true? The stdlib has no way to generate prose, catch a stale Args: section after a signature change, or notice that the "returns a dict" line now lies because you switched to a dataclass. Detection is a solved problem. Authoring and verification are not.
Stdlib scripts vs. dedicated AI doc tools
| Dimension | stdlib inspect/ast script | Dedicated AI doc tool |
|---|---|---|
| Finds missing docstrings | Excellent — free, instant | Usually yes, bundled in |
| Writes new docstrings | Not at all | Yes — the whole point |
| Detects stale/wrong docstrings | No | Yes, if it compares signature to prose |
| Understands types | Only what annotations say | Infers from body + annotations |
| Cost | $0, your maintenance time | One-time or subscription, varies |
| Data leaving your machine | Never | Depends on the tool — read its policy |
| Works in CI as a gate | Trivially | Usually, via CLI |
| Output quality | Whatever you write | Surprisingly good, occasionally hallucinated |
The pattern in that table is why I no longer see this as either/or. The stdlib script is your measuring stick; the AI tool is your worker. Use the first to define "done" and the second to get there.
How the AI grading actually went
My test setup, in plain terms: a ~4,000-line personal project, roughly 60% documented. I ran a stdlib coverage audit to establish the baseline, then let three tools generate docstrings for the undocumented remainder. After a blind diff, I gave a separate AI model twenty randomly chosen docstrings — mixed sources, mixed with my own hand-written ones — and asked it to score each on accuracy, specificity, and usefulness on a 1–5 scale, without telling it the origin.
Results that surprised me:
- My hand-written docstrings did not win. Three of the five lowest-scoring entries were mine — vague on edge cases, silent about return types, one outright wrong about a
Nonepath. - AI docstrings were specific to the point of being pedantic. They documented exception paths and
Nonereturns I had never mentioned. That is exactly what a reader needs at 2 a.m. - One hallucination. A generator invented a keyword argument that does not exist. This is why a human pass — or a doctest run — is non-negotiable. Auto-merge and you are shipping fiction.
The tool I kept: AI Doc Generator
Of the tools I tested, the one that ended up in my actual workflow was AI Doc Generator, because it does the unglamorous half of the job: it scans the codebase, reports docstring coverage, and generates Google-style docstrings for what is missing — so the audit step and the authoring step live in one place instead of my homegrown script plus a separate generator.
Tool of the week — AI Doc Generator
Point it at a Python project and get professional docstrings, README updates, and a coverage report in minutes. Google-style docstrings, works on legacy codebases, runs locally against your files. $49 one-time — no subscription, no per-seat nonsense.
Get AI Doc Generator — $49 one-time →
Disclosure: this is our own tool and the link is an affiliate link — we earn a commission if you buy, at no extra cost to you.
My honest first-person take after two weeks of real use: the coverage report alone paid for the thing, because it turned "I should document more" into a number that went down every run. The generated docstrings needed a review pass — the hallucinated-argument problem above is real, and I caught one in my own output — but reviewing is dramatically faster than writing from a blank line. On the nurse-prompt style dense utility modules it was excellent; on heavy metaclass magic it produced confident nonsense and I reverted those by hand.
A workflow that actually sticks
What finally made docstrings stick for me is not any single tool, it is the loop:
- Audit — stdlib script or the tool's coverage report, run on every PR. Number goes up or the PR waits.
- Generate — AI fills the gaps for anything the audit flags.
- Verify — human skim plus a doctest or two on the functions you care about most. Non-negotiable.
- Explain why — if you use AI coding agents alongside this, they follow rules better when the rules are written down. That is the whole argument for keeping your agent's instructions in checked-in files, which I covered in Cursor Rules and Claude Code Skills.
Most of the prompt engineering effort here is unglamorous: telling the generator what style, what audience, and how much detail. If you have not learned to write instructions that survive contact with an LLM, start with How to Write Better Prompts — the same principles apply to docstring prompts. And docstring automation is honestly the smallest corner of what a setup like this can take off your plate; I run the same audit-generate-verify loop on changelogs and onboarding notes, as described in How I Automate My Entire Workflow.
Who this is NOT for
- Security-cleared or air-gapped codebases. If your code cannot leave the machine, check the tool's execution model first, and lean on the stdlib audit + human writing. The measuring stick is safe; the worker may not be.
- Codebases under ~500 lines. You can hand-write everything. The automation overhead exceeds the pain.
- Heavily dynamic or metaprogrammed code. Generators hallucinate signature details exactly where introspection is hardest. Expect to revert a lot.
- Teams with no review culture. If nobody reviews anything, auto-generated docs become confident fiction with a nice format. Fix the review habit first.
- Documentation that is really prose. Design docs, ADRs, tutorials — an LLM will produce text, not understanding. This is about API reference hygiene.
Honest verdict
Stop writing docstrings by hand — but do not stop caring whether they are true. The stdlib gives you a free, excellent auditor and zero authors. AI doc tools give you an author that is usually right and occasionally confidently wrong. The combination — audit with a script, generate with a tool, verify with a human and a doctest — is the first docstring workflow I have kept running past week two. At $49 one-time for AI Doc Generator with no subscription hanging over me, it stays in my toolkit; my own hand-written docstrings, it turns out, were the weakest part of my own repo.
Questions, disagreements, or your own horror stories about stale docstrings: nandminc2020@gmail.com. And if you want the full tool comparison that led here, start with AI Docstring Generators Compared.
Comments
Post a Comment