You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
MarkdownHeaderSplitter keeps the closing # sequence of ATX headers in header metadata #13281
MarkdownHeaderSplitter keeps the optional closing sequence of an ATX header in the header and parent_headers metadata. A header written as ## Install ## is stored as "Install ##", and every child chunk under # Guide # gets parent_headers == ["Guide #"].
In Markdown the closing #s are not part of the heading text (CommonMark, ATX headings: "The optional closing sequence of #s must be preceded by spaces or tabs and may be followed by spaces or tabs only"). Some Markdown writers and doc converters emit this style.
Error message
None. The metadata is silently wrong.
Expected behavior
header == "Install" and parent_headers == ["Guide"]. Text where the #s are part of the heading should stay as it is: ## C# (no whitespace before the #), ## Install #1 (text after the #s) and ## Install \# (escaped). The chunk content should stay byte-exact.
Header text is taken from match.group(2).strip() in _split_text_by_markdown_headers, so nothing removes the closing sequence. This affects anything that filters or displays chunks by header/parent_headers, and it means the same section gets different metadata depending on which heading style the source used.
This is separate from the fenced-code-block cases in #12954. I have a small fix with regression tests ready and will open a PR.
System:
Haystack version (commit or version number): main @ aca9fc25d
Activity