Engineering notes

Documentation scripts misread nested Markdown code blocks

Post-mortem updated

Four scripts that edit or check sipnab's documentation mistook a code block shown inside another code block for the end of the outer block; this note records what each script did, the fix, and how to confirm it.

In short

sipnab’s documentation uses Markdown. Markdown is a plain-text format. In it, a line of backticks (`) before and after a block of example code marks it off.

Four scripts edit or check those pages. They decided where a code block ends with a rule that was too simple. So a code block shown inside another code block confused them.

  • Three of them could add web links inside example commands.
  • One could stop checking commands partway through a block without saying so.

All four now use one shared, correct routine. A test fails if a script goes back to its own version.

What happened

A fenced code block in Markdown starts with a line of three or more backticks (`) or tildes (~). The rule the scripts missed is in scripts/lib_markdown.py. That is the repository’s shared Markdown module. The rule:

  • A fence is three or more backticks or tildes, indented at most three spaces.
  • It closes only on a run of the same character. The run must be at least as long as the one that opened it.
  • A fence that is never closed runs to the end of the document.

So a page that shows how to write a code block uses two fences:

  • a longer fence outside,
  • a shorter one inside.
How to write a fenced block:
```
see RFC 7989 §5 for Session-ID
```
still inside the outer block

The four-backtick lines open and close the block. The three-backtick lines are part of its content.

Before commit e2045af9 (2026-08-27), four scripts in scripts/ found code blocks this way:

  • They checked whether a line starts with three backticks (or three tildes).
  • They switched between “inside code” and “outside code” each time one did.

That check cannot tell three backticks from four.

  • So it treated the inner three-backtick line as the end of the block.
  • It treated the next one as the start of a new block.
  • The scripts handled the lines between the two inner markers as ordinary text.

What each script did with that text:

  • rfc-links.py turns RFC citations into links.
    • It rewrote see RFC 7989 §5 for Session-ID inside the block as see [RFC 7989 §5](https://www.rfc-editor.org/rfc/rfc7989.html#section-5) for Session-ID.
    • This is the output of the test in the third step below, run against the old script.
    • Text in that form, pasted into a shell, is a syntax error. bash exits with status 2 and reports syntax error near unexpected token.
  • link-repo-paths.py links repository file paths. fix-line-anchors.py repairs links to source lines. Both used the same check. They treated the same lines as ordinary text.
  • check-cookbook.py reads the commands in the bash code blocks of docs/examples.md (the cookbook page).
    • It runs each one against a test capture, or it confirms that every long flag it names exists in sipnab --help.
    • Its old version ended a block at any line starting with three backticks.
    • So commands after an inner marker were never collected and never checked.
    • The script did not report that it had skipped them.

When the fix landed, docs/examples.md contained no line starting with four backticks or three tildes. So the scripts had skipped no cookbook command. The fix commit describes the defect. It does not name a published page that it changed.

Why it happened

Commit 80869f0a added scripts/lib_markdown.py on 2026-07-28. The module implements the full fence rule above.

  • The four scripts each kept a separate copy of the simpler line check.
  • They did not call the shared module.
  • No test fed any of them a code block nested inside another. So the difference was not detected.

What changed

Commit e2045af9 made all four scripts use the shared module:

  • rfc-links.py, link-repo-paths.py and fix-line-anchors.py import fence_mask.
  • check-cookbook.py imports fences.

It also added scripts/tests/test_fenced_content_survives.py. It checks three things:

  • Text inside a four-backtick block that contains a three-backtick block is left byte for byte unchanged by each of the three link-writing scripts.
  • The same scripts still rewrite citations, paths and line links outside any code block.
    • Without this check, a script that rewrote nothing at all would pass the first one.
  • No script under scripts/ other than lib_markdown.py calls startswith with a string beginning with three backticks or three tildes.
    • The test reads each script’s parsed syntax tree rather than its text.
    • So a comment or Python documentation string that mentions such a call does not trigger it.
    • The first version of the test searched the text. It did trigger on a documentation string.

The file holds eight test functions, which run as 16 cases. CI runs every test under scripts/tests on Linux (python3 -m pytest scripts/tests -q in .github/workflows/ci.yml).

What you can check yourself

Run these from the root of a full (not shallow) clone of the sipnab repository, with pytest installed.

  • The second step changes files in your checkout.
  • The fourth step puts them back.

The steps:

  • Run the test against the current scripts:
python3 -m pytest -q scripts/tests/test_fenced_content_survives.py

................                                                         [100%]
16 passed in 1.00s
  • Put back the four scripts as they were before the fix. The command prints nothing.
git checkout e2045af9^ -- scripts/rfc-links.py scripts/link-repo-paths.py scripts/fix-line-anchors.py scripts/check-cookbook.py
  • Run the nested-block case for rfc-links.py against the old script. It fails. The message shows the link written inside the code block:
python3 -m pytest -q scripts/tests/test_fenced_content_survives.py -k 'longer_fence and rfc'

F                                                                        [100%]
=================================== FAILURES ===================================
___ test_a_longer_fence_is_not_closed_by_a_shorter_run_inside_it[rfc-links] ____

stem = 'rfc-links'

    @pytest.mark.parametrize("stem", TRANSFORMERS)
    def test_a_longer_fence_is_not_closed_by_a_shorter_run_inside_it(stem):
        """Nothing between the ```` markers may change."""
        doc = nested_document(stem)
        out = convert(stem, doc)
        body = doc.split("````")[1]
>       assert body in out, (
            f"{stem} rewrote content inside a four-backtick fence -- the inner "
            f"three-backtick line was read as closing it. Fenced text must survive "
            f"byte-for-byte; use lib_markdown.fence_mask rather than a per-line "
            f"toggle.\n\n{out}"
        )
E       AssertionError: rfc-links rewrote content inside a four-backtick fence -- the inner three-backtick line was read as closing it. Fenced text must survive byte-for-byte; use lib_markdown.fence_mask rather than a per-line toggle.
E         
E         ````markdown
E         How to write a fenced block:
E         ```
E         see [RFC 7989 §5](https://www.rfc-editor.org/rfc/rfc7989.html#section-5) for Session-ID
E         ```
E         still inside the outer block: see RFC 7989 §5 for Session-ID
E         ````
E         
E         Prose below.
E         
E       assert 'markdown\nHow to write a fenced block:\n```\nsee RFC 7989 §5 for Session-ID\n```\nstill inside the outer block: see RFC 7989 §5 for Session-ID\n' in '````markdown\nHow to write a fenced block:\n```\nsee [RFC 7989 §5](https://www.rfc-editor.org/rfc/rfc7989.html#section-5) for Session-ID\n```\nstill inside the outer block: see RFC 7989 §5 for Session-ID\n````\n\nProse below.\n'

scripts/tests/test_fenced_content_survives.py:70: AssertionError
=========================== short test summary info ============================
FAILED scripts/tests/test_fenced_content_survives.py::test_a_longer_fence_is_not_closed_by_a_shorter_run_inside_it[rfc-links]
1 failed, 15 deselected in 0.04s
  • Restore the current scripts. The command prints nothing.
git checkout HEAD -- scripts

Terms used here

  • Markdown: a plain-text format for writing web pages. sipnab’s documentation and these notes use it.
  • Fenced code block (fence): a block of example code in Markdown. A line of three or more backticks or tildes before and after it marks it off.
  • Backtick: the character `. Tilde: the character ~.
  • Script: a small program in the repository’s scripts/ directory that edits or checks files. These scripts are Python programs.
  • Cookbook: the documentation page docs/examples.md. It lists commands a reader can copy and run.
  • Test: a program that checks one behavior and fails when it is wrong. pytest is the tool that runs the Python tests.
  • Gate: a check that must pass before the project accepts a change.
  • CI (continuous integration): the automated build that runs the tests on every change pushed to the project.
  • Commit: one recorded change in the project’s history, named by a short hash such as e2045af9.
  • Syntax tree: the structure Python builds when it reads a program.
    • In it, comments and strings are separate from calls.
    • So a check can look at calls only.