[astrogilda]

Related Issues

The catalog route is deepset-ai/haystack-integrations#637. This proposal adds a native framework job.

Proposed Changes

When an agent tool writes data and then raises an error, the terminal result alone cannot describe the durable effect. This job retains native Pipeline, Agent, Tool and tracer transitions, then checks those effects with a separately installed reader.

The workflow builds the current Haystack checkout, checks its installed source bytes, and runs seven fixed fault cases. Only the permitted effect reaches publication. Two changed-input controls must refuse, including a changed invocation whose artifact digest was updated. The job retains the original records, selected policy, decisions, refusals, source pins, wheels and process receipts.

Published Observer source is pinned to 5d9feedcfffcf441380e11e9f25a5db44d274068. The runner explicitly derives the current-host reference by changing only the exact SDK_VERSION and SDK_SOURCE declarations. It retains original and derived bytes, hashes and the two-selector diff. The distinct local wheel version and embedded receipt identify this derivation. All 31 original test controls and substantive guards remain unchanged.

The standalone Hatch route needs no model API key. Action references use complete, checked commit hashes. The workflow has read-only repository permission and a 15-minute limit.

How did you test it?

The full native run passed on Linux against Haystack f2074dae6a2251505b443f30f27a38e82bafb96b. Its runner SHA-256 was 8619003f6059f145fff547e7ab66f08e81aab7b33b2263a8ecf5a64dc677310f:

• 31 original native controls passed, with zero errors, failures or skipped tests.
• 13 source, derivation, environment and process tests passed on Python 3.13.3 and 3.10.21.
• Both changed-input controls refused. The original publication stayed unchanged. Two reader runs returned identical bytes.
• All 23 child-command receipts show clear process groups. The timeout test stopped a real child and grandchild.
• The current host pre-commit configuration passed, including its pinned Docker actionlint hook. Additional Ruff syntax/import and format checks passed.
• Measured driver coverage was 91.97%.
• An actual nested Hatch CLI regression reproduced the inherited environment failure and passed with the cleaned environment.

Internal helpers and constants received underscore prefixes to meet host style. Final driver SHA-256: d711f8441f3be65179b8c6e4d45f14aadcea5f22d2a1a8e80bd4f81ca93eccab. Inverse AST checks confirm a name-only change and unchanged string literals. All 13 guards passed again on Python 3.13 and 3.10. Host pre-commit and Ruff passed on those six files before the last workflow change. That last change only replaces the upload action pin with official Node.js 24 release 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a. The final workflow passed actionlint. Native code stayed unchanged; the full suite and coverage were not repeated. The 91.97% figure measures the earlier runner.

Notes for the reviewer

This PR was prepared with AI tools and is available for human review.

The workload uses scripted generator replies and makes zero model inference calls. Its signed records have PEER scope. The producer, initial policy and signing roles share one operator. Host acceptance, policy updates and artifact retention remain repository decisions.

The existing E2E route can collect the new guard file on Python 3.10. The separate native contract job uses Python 3.13.3 and its own Hatch environment.

Checklist

• Read the contribution guidelines and code of conduct.
• Added unit controls and documentation.
• Used a conventional ci: title.
• Ran the current pre-commit hooks and fixed their reported issues.
• Related issue update: no issue comment or external message was sent.
• Release note: this CI proposal follows the existing ci: release-note exception.
• Human review and upstream workflow approval remain pending.

[vercel]

@astrogilda is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

[CLAassistant]

All committers have signed the CLA.

[anakin87]

Thanks for the contribution. This verification workflow based on Probity is outside the scope of Haystack CI, so I am closing this PR. Feel free to maintain and run these checks in your own repository.