Skip to content

fix: distinguish empty-string content from None in Document auto-id - #13248

Closed
ompatil1906 wants to merge 1 commit into
deepset-ai:mainfrom
ompatil1906:fix/document-empty-content-id
Closed

ompatil1906 wants to merge 1 commit into
deepset-ai:mainfrom
ompatil1906:fix/document-empty-content-id

Conversation

@ompatil1906

@ompatil1906 ompatil1906 commented Oct 14, 2026 •

Copy link
Copy Markdown

Problem

Document._create_id normalized empty content with text = self.content or None. Consequently Document(content="") and Document(content=None) produced identical auto-generated IDs whenever the rest of their fields matched.

This caused:

  • a misleading DuplicateDocumentError when both were written in the same batch, and
  • silent ID collisions (two documents stored under one ID) when written separately.

Empty string is a valid content value (str | None), so the documents are semantically different.

Fixes #13234

Solution

Use text = self.content directly. The stringification then distinguishes "" from None while leaving the stable ID of a completely empty Document() unchanged.

Verification

  • Added regression test test_empty_string_content_has_distinct_id_from_none that asserts the two IDs differ and that the content=None ID remains the previously published value.
  • Ran the new test together with the existing test_init (both passed).
  • Manually confirmed the IDs now differ.

Release note added under releasenotes/notes/.

Empty content was normalized with ``or None``, causing Document(content="")
and Document(content=None) to share the same ID. This led to spurious
DuplicateDocumentError on batch writes and silent ID collisions otherwise.

The hash now uses the content value as-is so the two cases are distinct.
The stable ID for content=None is unchanged.

Fixes deepset-ai#13234
@ompatil1906
ompatil1906 requested a review from a team as a code owner October 14, 2026 01:37
@ompatil1906
ompatil1906 requested review from sjrl and removed request for a team October 14, 2026 01:37
@vercel

vercel Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

@ompatil1906 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Oct 14, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions

Copy link
Copy Markdown
Contributor

Hi @ompatil1906, thanks for your interest in contributing to Haystack! 🙏

⚠️ Issue #13234 is already being addressed by open pull request(s) #13235. Before opening a PR for an issue, please check whether a PR is already linked to it, and consider contributing to the existing PR instead. We may close duplicate PRs to keep the review queue manageable.

This is an automated message to help us keep the review queue healthy.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @ompatil1906, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot
HaystackBot removed the request for review from sjrl October 14, 2026 02:54
@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Oct 14, 2026
@HaystackBot
HaystackBot marked this pull request as draft October 14, 2026 02:54
@HaystackBot
HaystackBot marked this pull request as ready for review October 14, 2026 04:34
@HaystackBot
HaystackBot requested a review from sjrl October 14, 2026 04:34
@HaystackBot HaystackBot removed the cla-pending PR is in draft until the contributor signs the CLA label Oct 14, 2026
@HaystackBot

Copy link
Copy Markdown
Contributor

Thanks for signing the CLA, @ompatil1906! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

@sjrl

sjrl commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Thanks for the PR! We've decided not to change this behavior since it would change the auto-generated IDs of existing documents with empty content. See #13234 for details. Closing.

@sjrl sjrl closed this Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document: content="" and content=None generate the same auto id

4 participants