Skip to content

Python: information-flow control prompt injection defense#5331

Draft
eavanvalkenburg wants to merge 2 commits intomainfrom
feature/python-fides
Draft

Python: information-flow control prompt injection defense#5331
eavanvalkenburg wants to merge 2 commits intomainfrom
feature/python-fides

Conversation

@eavanvalkenburg
Copy link
Copy Markdown
Member

Motivation and Context

This draft PR brings the Python information-flow control prompt injection defense work from feature/python-fides toward main for review and integration.

Description

  • adds the FIDES information-flow control security model for tracking integrity and confidentiality through agent execution
  • introduces secure-agent middleware, secure tools, and context-provider-based configuration for policy enforcement and hidden-content handling
  • includes supporting samples, docs, and follow-up fixes needed to keep the Python workspace validation suite green

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.

* fides integration

* documentation

* documentation

* documentation

* human-approval on policy violation

* numenous hyena 'works'

* IFC based implementation

* minor edits in documentation

* rebasing the branch and running the email example

* Add security tests for IFC middleware

* Fix Role.TOOL NameError in approval handling

* tiered labelling scheme

* 3 tier labelling scheme in middleware

* Adapt security middleware to list[Content] tool results

* Refactor SecureAgentConfig as context provider and address Copilot review comments

* Update FIDES docs to reflect context provider pattern and update code for ContextProvider rename

* Fix security examples: use OpenAIChatClient instead of non-existent AzureOpenAIChatClient

* Address PR review: consolidate security modules, remove ContentLineage, update docs

* remove unrelated files

* remove comment from _tools.py and rename decision file

* Fix CI failures: Bandit B110, broken md links, hosted approval passthrough

* apply template to decision doc 0024

* minor fixes to decision doc 0024

---------

Co-authored-by: Aashish <[email protected]>
@moonbox3 moonbox3 added documentation Improvements or additions to documentation python labels Apr 17, 2026
@moonbox3
Copy link
Copy Markdown
Contributor

moonbox3 commented Apr 17, 2026

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework
   _feature_stage.py1151289%97, 154, 161, 163–165, 172, 174, 179, 186, 205, 233
   _security.py72617476%431–432, 450, 453–454, 532–533, 535, 583, 594, 610, 627, 694–695, 698, 702, 709–710, 712, 719–724, 726, 728–735, 737, 740–744, 746–748, 750–751, 754–755, 757–758, 761–763, 765–769, 772–776, 778, 1028, 1034–1035, 1040, 1042–1043, 1073–1074, 1102–1110, 1131, 1234–1235, 1266–1267, 1292–1293, 1297–1298, 1332–1335, 1417–1418, 1423–1428, 1433–1435, 1661, 1764, 1769–1770, 1774, 1778–1780, 1829, 1841, 1843, 1860, 1881, 1892, 1918, 1959–1960, 1985, 2080, 2108–2110, 2145–2147, 2155, 2163, 2411–2412, 2414, 2416–2418, 2421, 2433, 2439–2440, 2442, 2473, 2477–2480, 2482, 2619, 2622–2625, 2628–2629, 2631, 2633, 2635, 2638–2642, 2649, 2653, 2664–2665, 2667, 2669–2671, 2732, 2742–2743
   _tools.py9878691%191–192, 365, 367, 380, 405–407, 415, 433, 447, 454, 461, 484, 486, 493, 501, 584, 588, 620–622, 630, 675–677, 679, 702, 728, 732, 770–772, 776, 798, 913–919, 955, 967, 969, 971, 974–977, 998, 1002, 1006, 1020–1022, 1369, 1393, 1455, 1483, 1497, 1501, 1631, 1635, 1681, 1742–1743, 1851, 1857, 1882–1883, 1904, 1924, 1926, 1982, 2045, 2217–2218, 2238, 2294–2295, 2433–2434, 2501, 2506, 2513
   observability.py7558488%377, 379–380, 383, 386, 389–390, 395–396, 402–403, 409–410, 417, 419–421, 424–426, 431–432, 438–439, 445–446, 453, 610–611, 739, 743–745, 747, 751–752, 756, 794, 796, 807–809, 811–813, 817, 825, 949–950, 1112, 1355–1356, 1454–1459, 1466–1469, 1473–1481, 1488, 1553–1557, 1616–1617, 1751, 1843, 2040, 2258, 2260
TOTAL29088347288% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
5765 30 💤 0 ❌ 0 🔥 1m 30s ⏱️

* Python: follow up FIDES security flow

Refine the secure approval path, mark the security classes with the FIDES experimental feature label, and clean up the related docs/tests. Also fix workspace-level validation regressions uncovered while running the full Python check suite.

Co-authored-by: Copilot <[email protected]>

* Python: remove FIDES GitHub MCP sample

Drop the GitHub MCP security sample from the FIDES follow-up branch while keeping the remaining security docs and samples intact.

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR brings the Python FIDES information-flow control (integrity + confidentiality) prompt-injection/data-exfiltration defense closer to main, adding core security primitives/middleware, DevUI support for policy-violation approvals, and end-to-end samples/docs/tests.

Changes:

  • Adds FIDES security samples and documentation (quick start + developer guide + implementation summary + ADR).
  • Extends DevUI mapping/execution to round-trip policy-violation metadata through approval events.
  • Adjusts core function-invocation plumbing and test suites to support the new approval/policy flow and keep validation green.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
python/samples/02-agents/security/repo_confidentiality_example.py New sample demonstrating confidentiality labels + exfiltration prevention.
python/samples/02-agents/security/email_security_example.py New sample demonstrating integrity taint handling + quarantined processing.
python/samples/02-agents/security/README.md New quick-start documentation for SecureAgentConfig/FIDES patterns.
python/samples/02-agents/security/FIDES_DEVELOPER_GUIDE.md New deep-dive developer guide for the FIDES model and APIs.
python/packages/core/tests/test_security.py New comprehensive unit tests for labeling/middleware/policy enforcement.
python/packages/core/agent_framework/_tools.py Updates tool invocation + approval replacement behavior for policy approvals.
python/packages/core/agent_framework/init.py Exports new FIDES/security APIs and ai_function alias.
python/packages/core/agent_framework/_feature_stage.py Adds ExperimentalFeature.FIDES.
python/packages/core/agent_framework/observability.py Hardens finish_reason attribute handling.
python/packages/devui/agent_framework_devui/_mapper.py Emits policy violation details in approval request events when present.
python/packages/devui/agent_framework_devui/_executor.py Preserves policy-violation metadata on approval responses.
python/packages/devui/tests/devui/test_ui_memory_regression.py Skips the test when CDP websocket URL cannot be obtained.
python/packages/foundry/tests/foundry/test_foundry_embedding_client.py Makes env patching deterministic via clear=True.
docs/features/FIDES_IMPLEMENTATION_SUMMARY.md Adds an implementation summary for FIDES components/deliverables.
docs/decisions/0024-prompt-injection-defense.md New ADR documenting the design rationale and tradeoffs.

- Message-level tracking tests (Phase 1)
- Data exfiltration prevention tests

4. **`docs/decisions/0011-prompt-injection-defense.md`**
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This summary references docs/decisions/0011-prompt-injection-defense.md, but that ADR file doesn’t exist in the repo; the prompt-injection defense ADR added in this PR is docs/decisions/0024-prompt-injection-defense.md. Update the reference so readers can navigate to the correct ADR.

Suggested change
4. **`docs/decisions/0011-prompt-injection-defense.md`**
4. **`docs/decisions/0024-prompt-injection-defense.md`**

Copilot uses AI. Check for mistakes.
Comment on lines +463 to +468
# Email security (prompt injection defense)
PYTHONPATH=packages/core python samples/getting_started/security/email_security_example.py

# Repository confidentiality (data exfiltration prevention)
PYTHONPATH=packages/core python samples/getting_started/security/repo_confidentiality_example.py
```
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The run commands reference samples/getting_started/security/..., but the examples in this PR are located under samples/02-agents/security/. As written, these commands will fail for users following the README.

Copilot uses AI. Check for mistakes.
Comment on lines +480 to +483
- Full documentation: `python/packages/core/FIDES_DEVELOPER_GUIDE.md`
- Test suite: `python/packages/core/tests/test_security.py`
- Email example: `python/samples/getting_started/security/email_security_example.py`
- Repo example: `python/samples/getting_started/security/repo_confidentiality_example.py`
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These “More Information” links point to paths that don’t exist (e.g., python/packages/core/FIDES_DEVELOPER_GUIDE.md and python/samples/getting_started/security/...). Update them to the actual locations in this PR (e.g., python/samples/02-agents/security/FIDES_DEVELOPER_GUIDE.md and python/samples/02-agents/security/*.py).

Copilot uses AI. Check for mistakes.
Comment on lines +1804 to +1813
# Build a map of call_id -> actual result for replacing placeholders
result_by_call_id: dict[str, Content] = {}
for resp in fcc_todo.values():
if resp.approved and resp.function_call is not None and resp.function_call.call_id is not None:
# Map the call_id from the function_call to be replaced
call_id = resp.function_call.call_id
if call_id not in result_by_call_id and approved_function_results:
idx = len(result_by_call_id)
if idx < len(approved_function_results):
result_by_call_id[call_id] = approved_function_results[idx]
Copy link

Copilot AI Apr 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

result_by_call_id is populated by assigning approved results based on positional index (idx = len(result_by_call_id)) rather than matching the actual call_id of each approved_function_results entry. If there are multiple approvals/results, this can associate the wrong tool result with a given call_id. Build the mapping by matching on Content.call_id instead of relying on list/dict iteration order.

Suggested change
# Build a map of call_id -> actual result for replacing placeholders
result_by_call_id: dict[str, Content] = {}
for resp in fcc_todo.values():
if resp.approved and resp.function_call is not None and resp.function_call.call_id is not None:
# Map the call_id from the function_call to be replaced
call_id = resp.function_call.call_id
if call_id not in result_by_call_id and approved_function_results:
idx = len(result_by_call_id)
if idx < len(approved_function_results):
result_by_call_id[call_id] = approved_function_results[idx]
# Build a map of call_id -> actual result for replacing placeholders.
# Match results by their actual call_id instead of relying on positional order.
approved_call_ids = {
resp.function_call.call_id
for resp in fcc_todo.values()
if resp.approved and resp.function_call is not None and resp.function_call.call_id is not None
}
result_by_call_id: dict[str, Content] = {}
for result in approved_function_results:
call_id = getattr(result, "call_id", None)
if call_id is not None and call_id in approved_call_ids and call_id not in result_by_call_id:
result_by_call_id[call_id] = result

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants