Skip to content

Avoid quadratic whitespace stripping in pre blocks - #287

Open
ac1982 wants to merge 1 commit into
matthewwithanm:developfrom
ac1982:fix-pre-whitespace-performance
Open

ac1982 wants to merge 1 commit into
matthewwithanm:developfrom
ac1982:fix-pre-whitespace-performance

Conversation

@ac1982

@ac1982 ac1982 commented Oct 11, 2026

Copy link
Copy Markdown

Fixes #286. Thanks to @BrianWillows for reporting and identifying the replacement; this PR applies that fix and adds regression coverage.

The default <pre> conversion repeatedly searches a long indentation run for trailing whitespace. Replace the trailing regex substitution with rstrip(' \n'), preserving indentation and the exact set of stripped characters. Keep the module-level regex name for compatibility.

On Python 3.8.20/Linux arm64, the same public markdownify() input (spaces followed by x) measured:

Spaces Before After
8,000 0.361 s 0.00067 s
16,000 1.481 s 0.00057 s
32,000 5.886 s 0.00092 s
65,536 24.137 s 0.00167 s

These are local single-run measurements, not a portable speed guarantee. The regression test checks both preserved indentation and a generous five-second bound (the fixed conversion takes milliseconds). Ten additional public API cases cover indentation, empty/blank input and whitespace that must remain. A separate exhaustive comparison of all 29,524 strings of length 0–9 over space/newline/x found no output differences in strip_pre.

Validation:

  • Before: performance regression fails; ten semantic cases pass.
  • After: complete pytest suite, 94 passed.
  • Repository flake8 and README lint pass.
  • mypy . and mypy --strict tests/types.py pass on the clean source tree.
  • Workflow source/wheel build command passes; git diff --check passes.

Build artifacts were moved out before mypy, because scanning build/lib duplicates the source module. Tests use the public conversion API; no stub changes are needed.

AI assistance: implemented, independently reproduced and tested with Codex.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trailing whitespace strip in <pre> is quadratic on a long whitespace run

1 participant