Rationale¶

Why the format is shaped this way. The short version: almost every decision came from measuring the standard library rather than from taste, and several of them reversed a decision made on taste first.

The problem¶

Two places in the standard library render human-written text to a terminal, and both do it badly.

pydoc treats every docstring as preformatted text. The author wrapped it for the source file, at 72 or 79 columns; the reader has 60, or 200. And there is no agreed way to mark a parameter name, so every author invents one: 'param', `param', *param*, PARAM, or a stray :class:`param` copied from module documentation.

argparse colourises what it generates (usage lines, headers, metavars) but user-supplied descriptions and help stay plain. The frame is richer than the picture.

Both need the same thing: free-flowing text, with exceptions for code and lists, plus a consistent way to highlight inline elements. That is the whole requirement.

Why not just use a Markdown library¶

Because pydoc and argparse are in the standard library and cannot depend on PyPI. The right architecture is pluggable formatters, but a plugin system with no batteries-included implementation is a system nobody uses. Exactly one non-plaintext formatter has to ship, and a conformant CommonMark parser would be among the largest modules in the standard library: a hard sell for a feature this narrow.

The question is "what is the smallest thing that solves the actual problem". That is a question with an empirical answer.

Measured against the corpus¶

Every module, class and function docstring in the Python 3.12 standard library, all 7272 of them:

And what the corpus says about each feature a docstring format might have:

The single-line docstring (37% of the corpus) has to be a no-op, and is.

Why one inline rule is narrowed¶

This is the decision that surprised us, and it came from running both reference parsers over real docstrings.

CommonMark mangles Python prose. reStructuredText does not:

So simplemark took reST's rule: emphasis delimiters must be flanked by whitespace or punctuation, so emphasis never occurs inside a word. One sentence replaces CommonMark's seventeen flanking rules, and it needs no delimiter stack.

It is also a strict narrowing (simplemark never finds emphasis where CommonMark finds none), so borrowing it does not break subset-hood. A clean docstring still renders correctly on GitHub.

The second thing taken from reST: no nested inline markup. That removes recursion, the delimiter stack, and the precedence table from the inline layer. It is the single largest reason the implementation is 788 lines rather than several thousand.

Why reStructuredText is not a compatibility target¶

An earlier draft went further and made every canonical spelling valid reST as well: ``code``, :: before verbatim blocks, no [text](url). It was measured and reversed.

93.0% of the standard library is clean under Markdown-only rules; 83.2% was clean under the reST-compatible ones. The ten points were docstrings needing :: added (249), backticks doubled (264), or a definition-list shape rewritten (385).

The argument underneath the number: reST compatibility bought zero-effort Sphinx autodoc support, but the people who get that benefit are those who already write reST docstrings: the population least likely to adopt a new format. It was a migration tax levied on adopters to subsidise non-adopters.

The tradeoff was genuine: "it is also valid reST, so Sphinx keeps working" was a good answer to the fragmentation objection. The replacement is tooling, and it is built rather than promised: tools/sm2rst.py, 129 lines over the same AST, needing nothing from the parser. A project that builds its documentation with autodoc runs its docstrings through that, and Sphinx sees reStructuredText.

Almost all of those 129 lines are escaping, which is the part that can quietly be wrong, so it is fuzzed: every docstring in the standard library (7272 samples of prose nobody wrote for simplemark) is escaped into reST and fed to docutils, which objects to none of the output. That is a robustness claim about the escaper. It is emphatically not a proposal to convert the standard library to reStructuredText; the stdlib is simply the largest pile of adversarial docstring prose available, and it found four escaping bugs.

Deliberately outside the package: pydoc and argparse render to terminals and have no use for reST, so shipping this would add a third of simplemark's size for nothing the core case needs. That is arguably a better answer than syntactic compatibility was, because it constrains the language not at all. Alongside it: __docformat__ tells tooling what it is reading, and compat mode finds reST constructs so a codebase can migrate away from them deliberately.

The two claims¶

Simplemark is a subset of CommonMark's constructs, with one narrowed inline rule.

That is two claims, and they are testable:

1. Constructs are a subset of CommonMark's, and canonical spellings mean the same thing there.
2. The emphasis rule is a narrowing of CommonMark's.

The claims are testable¶

The test suite runs simplemark against the real markdown-it-py and checks both on real Python prose, and against docutils to verify the flanking rule's provenance against its source. This harness overturned three design decisions during development, including the choice of base language.

That is the point of stating the design as claims rather than as prose: a claim can be wrong in a way a test can catch.

What "clean" buys you¶

A docstring with no diagnostics renders identically in simplemark and under a Markdown renderer; the same text works in pydoc, on GitHub, and in an IDE hover.

Where the two disagree, simplemark picks the reading that is right for Python prose and tells you that Markdown would differ. Three severities: an error is simplemark's problem, a CommonMark note is a portability fact, and a legacy note is migration advice you only hear when you ask for it.

A bug in inspect.cleandoc¶

Found while implementing the preprocessing step, and worth knowing about independently of simplemark.

cleandoc takes the smallest indentation in the body as the margin to strip. When a code block is the only indented content, the block's own indentation is that minimum; it gets stripped, and the example is flattened into prose:

python - <<'EOF'
import inspect

def f():
    """Example::

        >>> f()
    """

print(repr(inspect.cleandoc(f.__doc__)))
EOF

'Example::\n\n>>> f()'

The block is gone. This affects 179 standard library docstrings, 3.9% of the multi-line ones, and pydoc has it today.

Simplemark reads the margin off the closing-quote line instead, which is exactly the enclosing indentation. Measuring how docstrings end says when that works:

Half the corpus states its own margin unambiguously. Simplemark uses it.

Things that were decided twice¶

Worth recording, because the reversals are the argument:

• The base language. First CommonMark ∩ djot, which is elegant; the corpus showed that djot has no indented code blocks and 7.7% of docstrings depend on them. Then CommonMark ∩ reST. Then a CommonMark subset borrowing one reST rule. Finally, when the cost of reST compatibility was measured at ten points of the headline number, a plain CommonMark subset that keeps the rule and drops the compatibility.
• Emphasis. First dropped entirely, on the evidence that every Markdown delimiter collides with Python syntax. Reinstated once reST's flanking rule turned out to solve the collision without dropping the feature, and once the corpus showed *emphasis* is the most common intentional markup in the standard library.
• Definition lists. First included, then dropped, when sampling the 264 candidate docstrings showed they are mostly Args: headers and hand-aligned prose that would be misparsed.
• reStructuredText compatibility. Adopted as a design driver, then measured at ten points of the headline number and dropped. See above.

The full design record¶

The reasoning above is a summary. The complete record (prior art, use cases, every alternative considered and its status) lives in the repository under notes/:

The project exists as a contribution to a discussion about rich text in pydoc and argparse, where one participant asked the question nobody had answered: if just a subset will do, what markdown features can be safely omitted, and will it noticeably reduce the complexity?

This is the answer, with the measurements attached.