[Elias-Sun]

Related Issues

• fixes DocumentCleaner(remove_regex=...) strips leading/trailing whitespace of every page, even when the regex matches nothing #13357

Proposed Changes:

DocumentCleaner._remove_regex called .strip() on every \f-separated page after the substitution. As a result, remove_regex removed indentation and newlines at the start and end of every page, even on pages where the regex matched nothing. This happened even when remove_extra_whitespaces, remove_empty_lines and strip_whitespaces were all disabled.

This PR switches to re.subn and strips a page only when the regex matched at least once on it:

• Pages without a match are now left unchanged, so Markdown and code keep their whitespace.
• Pages with a match are stripped exactly as before. Existing outputs (for example, footer removal) and the document IDs derived from them don't change.
• Page breaks (\f) are preserved as before (fix: DocumentCleaner: keep the \f in text #8078).

The remove_regex docstring now describes this behavior.

How did you test it?

• test_remove_regex_strips_only_matching_pages covers three cases: no match (whitespace preserved), match (page stripped as before), and multi-page content (only the matching page is stripped).
• test_remove_regex_with_strip_whitespaces covers a matching page and a non-matching page in the same document with strip_whitespaces=True.
• The no-match, multi-page and strip_whitespaces cases fail on main. The match case passes on both, since that behavior is unchanged.
• hatch run test:unit test/components/preprocessors/test_document_cleaner.py test/components/preprocessors/test_document_preprocessor.py: 37 passed.
• hatch run fmt and hatch run test:types pass.
• Ran a local check on 2000 random multi-page inputs. Each one matched the intended behavior, and whenever every page matched, the output was identical to main.

Notes for the reviewer

Following review, this no longer changes behavior on pages where the regex matches, so the release note has only a fixes section.

Checklist

• I have read the contributors guidelines and the code of conduct.
• I have updated the related issue with new insights and changes.
• I have added unit tests and updated the docstrings.
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I have documented my code.
• I have added a release note file, following the contributors guidelines.
• I have run pre-commit hooks and fixed any issue.

🤖 Generated with Claude Code

[vercel]

Someone is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[HaystackBot]

Hi @Elias-Sun, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

[CLAassistant]

All committers have signed the CLA.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

孙意博 seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[HaystackBot]

Thanks for signing the CLA, @Elias-Sun! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

[swethakommuri]

Tested this branch (3732f75) against the repro from #13357 on WSL2 / Python 3.14:

• remove_regex with a non-matching pattern now preserves indentation and trailing newlines: ' def f():\n return 1\n'
• Newlines around \f page breaks are preserved: '\n# Page 1\n\n\x0c\n## Page 2\n indented\n', matching replace_regexes
• strip_whitespaces=True still strips as expected: ' [TODO] keep ' → 'keep'

LGTM from a user's perspective. Thanks for the quick fix!

[davidsbatista]

Thanks @Elias-Sun!

Dropping the .strip() entirely also changes the output when the regex matches (for example, footer removal), which changes content and document IDs for existing pipelines.

Could you strip only the pages where the regex actually matched, using re.subn and stripping only when the count is greater than 0?

That fixes #13357 without changing behaviour for anyone else. Please also update the "match" test case and drop the upgrade section from the release note.

[Elias-Sun]

Thanks @davidsbatista, done in fb8917b:

• _remove_regex now uses re.subn and strips a page only when the regex matched on it. Pages without a match are left unchanged, and matching pages give the same output as before.
• Updated the "match" test case to expect the stripped page. The multi-page case now checks that only the matching page is stripped.
• Dropped the upgrade section from the release note. It's now fixes only.
• Updated the remove_regex docstring and the PR description to match.

🤖 Addressed by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
haystack-docs	Ready	Preview	Oct 26, 2026 4:43pm UTC

[davidsbatista]

Thanks @Elias-Sun, looks good, all points addressed.

[davidsbatista]

looks good!

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
haystack/components/preprocessors
document_cleaner.py
Project Total

This report was generated by python-coverage-comment-action