diff --git a/docs/architecture.md b/docs/architecture.md index 2282cb2..cb2bf2c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -880,3 +880,18 @@ orchestrator owns correlation, non-persisted preview, explicit revalidated apply, deterministic replay identity, and dependency-order calls into the existing Decision-family services. The CLI remains a parsing and rendering surface. + +## Authority-Aware Planner Context + +`PlannerContextService.prepare()` is a read-only application use case for one +planner assessment. It coordinates five narrow reader ports for verified +repository metadata, a fixed current-document inventory, caller-selected Brain +Knowledge UUIDs, a fixed review inventory, and caller-supplied bounded +historical or frozen-release evidence. The returned immutable package always +contains its seven authority-labelled categories and never allows supporting or +historical evidence to override a verified live checkout. + +The use case has no CLI, persistence model, writer port, cache, source +registry, filesystem discovery, prompt execution, semantic retrieval, or Brain +write. Current-source reads verify the supplied repository checkpoint before +and after reading; a mismatch produces no `CURRENT` assertion. diff --git a/docs/product/prd-authority-aware-planner-context-package.md b/docs/product/prd-authority-aware-planner-context-package.md new file mode 100644 index 0000000..54ada39 --- /dev/null +++ b/docs/product/prd-authority-aware-planner-context-package.md @@ -0,0 +1,408 @@ +# PRD: Authority-Aware Planner Context Package + +## 1. Document status + +**Status:** Product requirements draft ready for product review. + +**Scope:** First post-1.1 product slice; documentation only. + +**Repository state:** No remediation, current 1.1.0 defect, or release blocker +is established. + +**Implementation:** Not authorized by this PRD. + +## 2. Product context + +NeuralEngine provides durable knowledge while repository sources establish the +current implementation and operating constraints. The user-defined product +direction assigns local retrieval and context building to RAG, without making +RAG the product's defining purpose. + +## 3. Problem statement + +A planner repeatedly assembles task context manually from live repositories, +local Brain Knowledge, Handbook contracts and ADRs, review artifacts, selected +historical checkpoints, and explicitly supplied local sources. These sources +have different authority and freshness semantics. Manual assembly is necessary +today but is fragmented and repetitive. + +The required product outcome is a bounded local context package that makes the +evidence, its authority, its freshness, and its provenance visible. It is not a +claim that the current release is defective. + +## 4. Target user + +The primary user is a **planner performing a repository or product assessment**. + +Secondary users may validate package output, but this slice does not introduce +new workflows for builders, reviewers, operators, or end users. + +## 5. Current workflow + +The planner currently: + +1. verifies repository path, branch, `HEAD`, authoritative remote reference, + and worktree state; +2. reads current repository instructions and designated current documents; +3. searches Brain only when durable knowledge is materially relevant; +4. reads selected reviews and explicitly supplied historical evidence; +5. labels current, historical, missing, and conflicting information; and +6. prepares a decision package while retaining final judgment. + +`neural knowledge search` is read-only substring search over Knowledge +statements and rationales in repository load order. It does not assemble this +multi-source package, classify authority, or inspect repository sources. + +## 6. Product goal + +Prepare a local, read-only, deterministic context package for one planner task. + +The package must reduce repeated manual source collection while preserving this +invariant: + +> Retrieved context never overrides the verified live repository. + +The package is supplementary evidence. The caller retains final planning and +authorization judgment. + +## 7. Non-goals + +This PRD does not make current non-goals permanent prohibitions. It excludes +them only from the first slice. + +- Brain writes, Memory consolidation, or automatic Knowledge promotion. +- Embeddings, vector databases, semantic ranking, or semantic retrieval. +- Router behavior, runtime/model selection, or quota decisions. +- Agent orchestration, prompt execution, or code generation. +- Repository mutation, persistent retrieval-event storage, or full + multi-project search. +- A universal source-authority order for future source types. + +## 8. First vertical slice + +**Read-only authority-aware context preparation for one planner assessment.** + +The slice receives a task and verified repository checkpoint, reads only a +narrow approved source set, and returns categorized evidence with provenance. +It neither makes a decision nor performs an action based on retrieved content. + +## 9. User stories + +- As a planner, I need current repository facts visibly separated from + supporting Brain Knowledge so I do not treat historical knowledge as live Git + authority. +- As a planner, I need historical and frozen release evidence labeled by its + original context so I can use it without misrepresenting it as current. +- As a planner, I need missing, unreadable, stale, and conflicting sources + reported explicitly so I can resolve gaps rather than receive silent fallback. +- As a planner, I need stable package order and exact provenance so a repeated + assessment is auditable. + +## 10. Functional requirements + +FR-1. The package shall accept one product-level request and return only the +approved first-slice source categories. + +FR-2. The package shall preserve the verified live repository as authority for +current repository state. + +FR-3. The package shall label every result with an authority class and evidence +state. + +FR-4. The package shall expose conflicts, missing evidence, and unreadable +sources rather than silently resolving or replacing them. + +FR-5. The package shall be read-only for repository and Brain state. + +FR-6. Identical accessible inputs shall produce the same categorized ordering. + +FR-7. The package shall not execute prompts, invoke models, or change planner +policy. + +## 11. Input contract + +The following are product-level placeholders, not final implementation fields: + +- `project_key`: caller-supplied project association. +- `task_statement`: bounded description of the assessment request. +- `verified_repository_checkpoint`: evidence captured before retrieval. +- `optional_source_filters`: caller-selected limits within approved source scope. + +The checkpoint must contain repository path or identifier, branch, `HEAD`, +`origin/main` or equivalent authoritative remote reference, worktree state, and +verification timestamp. + +An absent, incomplete, or unverified checkpoint cannot be represented as a +verified live repository fact. + +## 12. Output contract + +The package shall expose these categories, including empty categories where no +items are returned: + +- `current_authoritative_sources` +- `supporting_brain_knowledge` +- `historical_evidence` +- `stale_or_conflicting_sources` +- `missing_evidence` +- `unreadable_or_inaccessible_sources` +- `provenance` + +The presentation shall visibly distinguish product input, live repository fact, +current supporting knowledge, historical checkpoint, frozen release evidence, +stale source, conflict, missing source, and unreadable source. + +## 13. Source scope + +The initial source set includes only: + +1. verified repository metadata; +2. designated current repository documents; +3. selected local Brain Knowledge records; +4. designated review artifacts; and +5. explicitly supplied historical checkpoints or release evidence. + +The initial source set excludes project chats, arbitrary prompts, unrestricted +Handbook crawling, unrestricted filesystem search, all Brain record types, and +remote web sources. + +Source filters may narrow this set. They must not silently broaden it. + +## 14. Authority model + +1. A verified live checkout is authoritative for current repository state. +2. Brain records are supporting durable knowledge, never live Git authority. +3. Historical and frozen evidence remains valid only in its labeled context. +4. Product requirements and repository facts remain separately labeled. +5. Conflicts are surfaced, not silently resolved. +6. Missing or unreadable authority is never silently substituted with a weaker + source. +7. Final planning judgment remains with the caller. + +This is a first-slice rule set, not a universal ordering for all future sources. + +## 15. Freshness and evidence-state taxonomy + +| State | Meaning and minimum evidence | Display and planning use | Override | +| --- | --- | --- | --- | +| `CURRENT` | Verified current source tied to supplied checkpoint. | Label current; supports planning. | Only as live repository fact. | +| `HISTORICAL` | Identified earlier source with checkpoint/time context. | Label historical; supports comparison. | Never overrides current. | +| `FROZEN_RELEASE_EVIDENCE` | Immutable release/tag evidence with version context. | Label frozen release; supports its release claim. | Never overrides current. | +| `STALE` | Source conflicts with or predates verified current evidence. | Label stale and show counterpart when known. | Never overrides current. | +| `CONFLICTING` | Accessible sources assert incompatible claims with no authorized resolution. | Show all relevant claims. | Never silently resolves. | +| `MISSING` | Required designated source was absent or not supplied. | Show missing locator/category. | Cannot support a claim. | +| `UNREADABLE` | Source was selected but inaccessible, corrupt, or unparsable. | Show bounded failure reason. | Cannot support a claim. | +| `AMBIGUOUS` | Identity, association, or classification cannot be determined from evidence. | Show ambiguity and required clarification. | Cannot support an authoritative claim. | + +## 16. Provenance requirements + +Every returned item shall include source type, source locator, source identity, +project association, retrieval time, authority class, evidence state, exact +extraction boundary or record identity, and checkpoint/version context when +applicable. + +The package shall not require persisted retrieval history. Provenance is output +evidence for the current request, not a new durable Brain record. + +## 17. Conflict and missing-evidence behavior + +When a live repository fact and retrieved supporting source conflict, the +package shall preserve both claims, label the live fact as authoritative for +current repository state, and label the other source stale or conflicting based +on available evidence. + +When a required designated source is missing, the package shall return a +`MISSING` item with requested locator/category and no substituted content. + +When conflict classification itself is uncertain, the package shall return +`AMBIGUOUS` rather than invent precedence. + +## 18. Error and partial-result behavior + +The package may return accessible results with source-specific failures only +when each unavailable source is explicitly categorized as `MISSING`, +`UNREADABLE`, or `AMBIGUOUS`. + +An unavailable verified repository checkpoint prevents claims of `CURRENT` +repository state. The package shall return a bounded failure or partial package +that identifies this condition; it must not select an unverified replacement. + +Malformed source content shall not be repaired, ignored, or converted to a +successful result. + +## 19. Privacy and filesystem boundaries + +Only explicitly selected local sources and designated repository paths are in +scope. The slice shall not crawl the filesystem, discover unrelated projects, +access remote web sources, or read arbitrary prompt/chat files. + +The PRD does not select future access-control mechanisms. A later design must +define allowed roots, path traversal handling, symlink policy, output redaction, +and caller authorization before implementation. + +## 20. Determinism and ordering + +The package shall preserve a documented deterministic category order and stable +ordering within each category for identical accessible inputs. + +Ordering must not depend on model output, implicit recency inference, or +unspecified filesystem enumeration. A later design may select a concrete sort +key only if it preserves exact provenance and authority labels. + +## 21. Evaluation scenarios + +The implementation evaluation plan must include deterministic fixtures for: + +1. clean live checkout with current supporting Brain Knowledge; +2. historical release evidence differing from current state; +3. stale Brain Knowledge conflicting with verified Git; +4. missing required repository document; +5. unreadable or corrupt Brain record; +6. empty results with explicit missing-evidence output; +7. optional source filters that do not expand source scope; +8. stable ordering for identical inputs; +9. proof of no Brain or repository write; and +10. planner use where package output remains supplementary to direct verification. + +## 22. Acceptance criteria + +- A valid request returns all seven output categories, including empty ones. +- Every returned item contains all required provenance values. +- A verified live Git fact is never relabeled or replaced by Brain Knowledge. +- Each fixture in Section 21 has objective expected category, state, and + authority labels. +- Missing, unreadable, conflicting, and ambiguous sources remain visible. +- Repeated identical fixture requests have identical category/item order. +- No-write evaluation proves no Brain or repository content changes. +- The delivered scope contains no excluded behavior from Section 27. + +## 23. Metrics + +The first-slice product metrics are authority-classification accuracy, +provenance completeness, missing-evidence visibility, conflict visibility, +deterministic ordering, no-write guarantee, context-package completeness, and +operator time saved during repeated assessments. + +### Operator-time-saved evaluation + +The manual baseline is a planner preparing the context for one bounded planner +assessment using the current workflow in Section 5: verify the supplied +repository checkpoint, read the designated current documents, search selected +Brain Knowledge only when relevant, read selected review and historical +evidence, label the evidence state, and assemble the decision package. A +manual measurement starts when the planner begins gathering those approved +sources for the assigned task and ends when the completed context package is +ready for independent completeness review. Reading unrelated material, +waiting for unrelated work, breaks, and correcting an invalid run are excluded. + +The measurement unit is elapsed operator minutes per completed planner +context-preparation task. The same unit is used for the manual baseline and +the assisted workflow. The initial evaluation uses at least 12 comparable +fixtures. Each fixture is run once manually and once with the first-slice +context package for the same task statement, verified checkpoint, and approved +source set; the manual and assisted runs form a paired comparison. + +The measurement recorder captures a start timestamp, end timestamp, elapsed +minutes, fixture identifier, workflow (manual or assisted), and any excluded +idle interval for every run. Idle time unrelated to the assigned task is +subtracted from elapsed time. Incomplete, invalid, or quality-guard-failing +runs are recorded separately, retained with their reason, and excluded from +the paired reduction calculation. Raw timestamps, exclusions, results, and +run-status records are retained for product review; subjective estimates are +not measurements. + +For every valid fixture pair, absolute reduction is +`manual_minutes - assisted_minutes`; percentage reduction is +`((manual_minutes - assisted_minutes) / manual_minutes) * 100`. The product +evaluation reports each valid pair and the median percentage reduction across +all valid pairs. The first-slice operator-time metric succeeds only when at +least 12 valid paired fixtures have completed and the median percentage +reduction is at least 15 percent. A single run cannot establish success. + +Time reduction counts only for pairs whose assisted output passes independent +review for completeness, provenance, authority labeling, deterministic +ordering, and the no-write requirement. A time result that fails any of those +conditions is not a successful product result. + +The evaluator records the measurements and the product reviewer signs off the +fixture set, exclusions, quality checks, and result. The implementation owner +may supply the assisted workflow but does not provide evaluation sign-off. +This metric validates first-slice product usefulness only; it does not +authorize architecture optimization, implementation-scope expansion, or any +excluded behavior. + +The product shall not claim improved decision quality without comparative, +separately collected evidence. + +## 24. Risks + +- Retrieved historical material could be mistaken for current repository state. +- Broad source discovery could expose unrelated or sensitive local content. +- Unspecified precedence could conceal conflicts. +- A context package could expand prematurely into generic RAG, Memory, Router, + or Agents behavior. +- Incomplete provenance could make output unauditable. + +The authority, source-scope, and no-write requirements mitigate these risks. + +## 25. Dependencies + +Implementation consideration depends on approval of the source inventory, +designated-document set, authority taxonomy, request/response boundary, +filesystem/privacy boundary, error contract, evaluation fixtures, metric +thresholds, and no-write verification method. + +Current Knowledge search remains a supporting capability; this PRD does not +require changing it. + +## 26. Open product questions + +### Blocking implementation questions + +1. Which repository documents are designated current sources per project? +2. How is project association established for local Brain Knowledge? +3. Which local roots, symlink rules, and redaction rules are permitted? +4. What is the approved deterministic ordering key within each category? +5. What thresholds define acceptable classification and provenance completeness? +6. Who owns fixtures and signs off the no-write verification method? + +### Deferrable questions + +1. Which secondary users consume rendered packages? +2. Which additional source types may enter a later slice? +3. Which presentation format best supports planners? + +### Later design questions + +Architecture placement, concrete interfaces, adapters, schemas, CLI surface, +error types, and persistence decisions require a separate design task. + +## 27. Explicit implementation exclusions + +No implementation may add Brain writes, Memory consolidation, Knowledge +promotion, embeddings, vector storage, semantic ranking, Router selection, +runtime/model selection, Agents orchestration, prompt execution, code +generation, repository mutation, retrieval-event persistence, or unrestricted +multi-project search under this PRD. + +## 28. Go / defer decision gate + +Implementation may be recommended only when all conditions hold: + +1. first-slice source scope is approved; +2. authority rules and state taxonomy are approved; +3. request and response boundaries are approved; +4. evaluation fixtures and objective thresholds exist; +5. privacy and filesystem boundaries are approved; +6. no-write behavior is testable; +7. the slice remains limited to planner context preparation; and +8. a separate product authorization explicitly approves implementation. + +Otherwise, defer implementation and retain the manual evidence-first workflow. + +## 29. Authorization boundary + +This PRD authorizes product review only. It does not authorize implementation, +architecture design, source/test/configuration changes, Brain mutation, Agent +Pack modification, runtime-state changes, staging, commit, push, merge, tag, or +release. diff --git a/src/neural_engine/application/container.py b/src/neural_engine/application/container.py index 6b50f2b..13d7f3d 100644 --- a/src/neural_engine/application/container.py +++ b/src/neural_engine/application/container.py @@ -10,6 +10,7 @@ from neural_engine.application.knowledge_service import KnowledgeService from neural_engine.application.neural_doctor_service import NeuralDoctorService from neural_engine.application.observation_service import ObservationService +from neural_engine.application.planner_context_service import PlannerContextService from neural_engine.application.playbook_evaluation_service import ( PlaybookEvaluationService, ) @@ -71,6 +72,7 @@ LocalDevelopmentEvidenceSource, ) from neural_engine.infrastructure.local_neural_doctor_probe import LocalNeuralDoctorProbe +from neural_engine.infrastructure.local_planner_context_readers import LocalPlannerContextReaders class Container: @@ -102,6 +104,11 @@ def development_evidence_service(self) -> DevelopmentEvidenceService: scoped.experience_service(), ) + def planner_context_service(self) -> PlannerContextService: + """Build the bounded read-only planner-context use case (no CLI exposure).""" + readers = LocalPlannerContextReaders(self._resolved_paths()) + return PlannerContextService(readers, readers, readers, readers, readers) + def decision_action_service(self) -> DecisionActionService: paths = self._resolved_paths() return DecisionActionService( diff --git a/src/neural_engine/application/planner_context_service.py b/src/neural_engine/application/planner_context_service.py new file mode 100644 index 0000000..4442013 --- /dev/null +++ b/src/neural_engine/application/planner_context_service.py @@ -0,0 +1,381 @@ +"""Read-only, authority-aware context preparation for planner assessments.""" + +from __future__ import annotations + +import unicodedata +from datetime import UTC, datetime +from enum import StrEnum +from hashlib import sha256 +from uuid import UUID + +from pydantic import BaseModel, ConfigDict, field_validator, model_validator + +from neural_engine.ports.planner_context_readers import ( + BrainKnowledgeReader, + CurrentDocumentReader, + HistoricalEvidenceReader, + RepositoryMetadataReader, + ReviewEvidenceReader, +) + +CURRENT_DOCUMENTS = ( + "AGENTS.md", + "VISION.md", + "CONTEXT.md", + "README.md", + "ABOUT.md", + "docs/architecture.md", + "docs/conventions.md", + "docs/roadmap.md", + "docs/product/prd-authority-aware-planner-context-package.md", +) +REVIEW_ARTIFACTS = ( + ".agent-work/reviews/review-prd-authority-aware-planner-context-package.md", + ".agent-work/reviews/review-revise-prd-operator-time-metric.md", + ".agent-work/reviews/independent-product-rereview-authority-aware-planner-context-package.md", +) + + +class PlannerContextError(Exception): + """Base bounded application error for impossible package assembly.""" + + +class PlannerCheckpointMismatchError(PlannerContextError): + """A reader could not establish the caller-supplied live checkpoint.""" + + +class EvidenceState(StrEnum): + CURRENT = "CURRENT" + HISTORICAL = "HISTORICAL" + FROZEN_RELEASE_EVIDENCE = "FROZEN_RELEASE_EVIDENCE" + STALE = "STALE" + CONFLICTING = "CONFLICTING" + MISSING = "MISSING" + UNREADABLE = "UNREADABLE" + AMBIGUOUS = "AMBIGUOUS" + + +class SourceType(StrEnum): + REPOSITORY_METADATA = "repository metadata" + DESIGNATED_DOCUMENT = "designated document" + DESIGNATED_REVIEW = "designated review" + BRAIN_KNOWLEDGE = "Brain Knowledge" + HISTORICAL_EVIDENCE = "historical/frozen evidence" + + +class PlannerRepositoryCheckpoint(BaseModel): + """Caller-captured repository fact which must be reverified by the adapter.""" + + model_config = ConfigDict(frozen=True) + + repository_root: str + repository_identity: str + branch: str + head: str + authoritative_remote_ref: str + worktree_state: str + verified_at: datetime + + @field_validator( + "repository_root", + "repository_identity", + "branch", + "head", + "authoritative_remote_ref", + "worktree_state", + ) + @classmethod + def _required_normalized(cls, value: str) -> str: + normalized = unicodedata.normalize("NFC", value).strip() + if not normalized: + raise ValueError("Checkpoint fields must be non-blank.") + return normalized + + @field_validator("verified_at") + @classmethod + def _aware_timestamp(cls, value: datetime) -> datetime: + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError("Checkpoint verification timestamp must be timezone-aware.") + return value.astimezone(UTC) + + +class HistoricalEvidenceInput(BaseModel): + """Caller-supplied bounded historical or frozen-release evidence, never a path.""" + + model_config = ConfigDict(frozen=True) + + locator: str + stable_identity: str + content: str + evidence_state: EvidenceState + checkpoint_or_version: str + authority_class: str = "historical supporting evidence" + + @field_validator("locator", "stable_identity", "content", "checkpoint_or_version") + @classmethod + def _non_blank(cls, value: str) -> str: + normalized = unicodedata.normalize("NFC", value).strip() + if not normalized: + raise ValueError("Historical evidence fields must be non-blank.") + return normalized + + @field_validator("evidence_state") + @classmethod + def _historical_state(cls, value: EvidenceState) -> EvidenceState: + if value not in {EvidenceState.HISTORICAL, EvidenceState.FROZEN_RELEASE_EVIDENCE}: + raise ValueError("Historical evidence must be HISTORICAL or FROZEN_RELEASE_EVIDENCE.") + return value + + +class PlannerSourceFilters(BaseModel): + """Optional selectors which may only narrow the fixed approved inventory.""" + + model_config = ConfigDict(frozen=True) + + current_documents: tuple[str, ...] | None = None + review_artifacts: tuple[str, ...] | None = None + knowledge_ids: tuple[UUID, ...] = () + historical_evidence: tuple[HistoricalEvidenceInput, ...] = () + + @field_validator("current_documents", "review_artifacts") + @classmethod + def _approved_subset( + cls, values: tuple[str, ...] | None, info: object + ) -> tuple[str, ...] | None: + if values is None: + return values + allowed = ( + CURRENT_DOCUMENTS + if getattr(info, "field_name", "") == "current_documents" + else REVIEW_ARTIFACTS + ) + normalized = tuple(unicodedata.normalize("NFC", value) for value in values) + if len(set(normalized)) != len(normalized) or any( + value not in allowed for value in normalized + ): + raise ValueError( + "Source filters may only select a unique subset of the approved inventory." + ) + return normalized + + @model_validator(mode="after") + def _bounded_source_count(self) -> PlannerSourceFilters: + selected_documents = len( + CURRENT_DOCUMENTS if self.current_documents is None else self.current_documents + ) + selected_reviews = len( + REVIEW_ARTIFACTS if self.review_artifacts is None else self.review_artifacts + ) + if ( + selected_documents + + selected_reviews + + len(self.knowledge_ids) + + len(self.historical_evidence) + > 24 + ): + raise ValueError("At most 24 non-metadata sources may be selected.") + return self + + +class PlannerContextRequest(BaseModel): + """Validated immutable request; its task statement is data and is never executed.""" + + model_config = ConfigDict(frozen=True) + + project_key: str + task_statement: str + verified_repository_checkpoint: PlannerRepositoryCheckpoint + optional_source_filters: PlannerSourceFilters | None = None + + @field_validator("project_key", "task_statement") + @classmethod + def _normalized_required(cls, value: str) -> str: + normalized = unicodedata.normalize("NFC", value).strip() + if not normalized: + raise ValueError("Project key and task statement must be non-blank.") + return normalized + + +class SourceEvidence(BaseModel): + """One immutable, bounded source result or visible diagnostic.""" + + model_config = ConfigDict(frozen=True) + + source_type: SourceType + normalized_locator: str + stable_identity: str + external_project_context: str + authority_class: str + evidence_state: EvidenceState + retrieved_at: datetime + extraction_start: int = 0 + extraction_boundary: str = "whole selected record" + checkpoint_or_version: str | None = None + excerpt: str | None = None + content_sha256: str | None = None + diagnostic: str | None = None + asserted_subject_key: str | None = None + + @field_validator("normalized_locator", "stable_identity", "external_project_context") + @classmethod + def _evidence_required(cls, value: str) -> str: + normalized = unicodedata.normalize("NFC", value).strip() + if not normalized: + raise ValueError("Evidence identity and locator fields must be non-blank.") + return normalized + + @field_validator("retrieved_at") + @classmethod + def _retrieved_aware(cls, value: datetime) -> datetime: + if value.tzinfo is None or value.utcoffset() is None: + raise ValueError("Evidence retrieval time must be timezone-aware.") + return value.astimezone(UTC) + + @field_validator("extraction_start") + @classmethod + def _non_negative_start(cls, value: int) -> int: + if value < 0: + raise ValueError("Extraction start must not be negative.") + return value + + @model_validator(mode="after") + def _bounded_excerpt(self) -> SourceEvidence: + if self.excerpt is not None and ( + len(self.excerpt.encode("utf-8")) > 4096 or self.excerpt.count("\n") + 1 > 120 + ): + raise ValueError("Evidence excerpts must be bounded to 120 lines and 4096 bytes.") + if self.evidence_state is EvidenceState.UNREADABLE and self.excerpt is not None: + raise ValueError("Unreadable evidence must not contain an excerpt.") + return self + + +class PlannerContextPackage(BaseModel): + """The exact seven-category, non-persisted planner context response.""" + + model_config = ConfigDict(frozen=True) + + current_authoritative_sources: tuple[SourceEvidence, ...] = () + supporting_brain_knowledge: tuple[SourceEvidence, ...] = () + historical_evidence: tuple[SourceEvidence, ...] = () + stale_or_conflicting_sources: tuple[SourceEvidence, ...] = () + missing_evidence: tuple[SourceEvidence, ...] = () + unreadable_or_inaccessible_sources: tuple[SourceEvidence, ...] = () + provenance: tuple[SourceEvidence, ...] = () + warnings: tuple[str, ...] = () + + +class PlannerContextService: + """Coordinate five reader-only ports without deciding or authorizing a plan.""" + + def __init__( + self, + repository_metadata_reader: RepositoryMetadataReader, + current_document_reader: CurrentDocumentReader, + brain_knowledge_reader: BrainKnowledgeReader, + review_evidence_reader: ReviewEvidenceReader, + historical_evidence_reader: HistoricalEvidenceReader, + ) -> None: + self._repository_metadata_reader = repository_metadata_reader + self._current_document_reader = current_document_reader + self._brain_knowledge_reader = brain_knowledge_reader + self._review_evidence_reader = review_evidence_reader + self._historical_evidence_reader = historical_evidence_reader + + def prepare(self, request: PlannerContextRequest) -> PlannerContextPackage: + filters = request.optional_source_filters or PlannerSourceFilters() + checkpoint = request.verified_repository_checkpoint + before = self._repository_metadata_reader.verify(checkpoint) + current_documents = self._current_document_reader.read_current_documents( + checkpoint, + CURRENT_DOCUMENTS if filters.current_documents is None else filters.current_documents, + ) + reviews = self._review_evidence_reader.read_review_evidence( + checkpoint, + REVIEW_ARTIFACTS if filters.review_artifacts is None else filters.review_artifacts, + ) + after = self._repository_metadata_reader.verify(checkpoint) + knowledge = self._brain_knowledge_reader.read_knowledge( + request.project_key, filters.knowledge_ids + ) + historical = self._historical_evidence_reader.read_historical_evidence( + filters.historical_evidence + ) + checkpoint_valid = ( + before.evidence_state is EvidenceState.CURRENT + and after.evidence_state is EvidenceState.CURRENT + ) + all_items = (*knowledge, *historical) + warnings: list[str] = [] + if checkpoint_valid: + all_items = (before, *current_documents, *reviews, after, *all_items) + else: + warnings.append( + "verified repository checkpoint did not match before and after current-source reads" + ) + all_items = (*all_items, before, after) + return self._assemble(all_items, warnings) + + @staticmethod + def _sort_key(item: SourceEvidence) -> tuple[int, str, str, int, str]: + rank = { + SourceType.REPOSITORY_METADATA: 10, + SourceType.DESIGNATED_DOCUMENT: 20, + SourceType.DESIGNATED_REVIEW: 30, + SourceType.BRAIN_KNOWLEDGE: 40, + SourceType.HISTORICAL_EVIDENCE: 50, + }[item.source_type] + return ( + rank, + item.normalized_locator, + item.stable_identity, + item.extraction_start, + item.content_sha256 or item.stable_identity, + ) + + def _assemble( + self, items: tuple[SourceEvidence, ...], warnings: list[str] + ) -> PlannerContextPackage: + unique = { + ( + item.source_type, + item.normalized_locator, + item.stable_identity, + item.extraction_start, + ): item + for item in items + } + ordered = tuple(sorted(unique.values(), key=self._sort_key)) + current: list[SourceEvidence] = [] + knowledge: list[SourceEvidence] = [] + historical: list[SourceEvidence] = [] + stale: list[SourceEvidence] = [] + missing: list[SourceEvidence] = [] + unreadable: list[SourceEvidence] = [] + for item in ordered: + if item.evidence_state in {EvidenceState.STALE, EvidenceState.CONFLICTING}: + stale.append(item) + elif item.evidence_state in {EvidenceState.MISSING, EvidenceState.AMBIGUOUS}: + missing.append(item) + elif item.evidence_state is EvidenceState.UNREADABLE: + unreadable.append(item) + elif item.source_type is SourceType.BRAIN_KNOWLEDGE: + knowledge.append(item) + elif item.source_type is SourceType.HISTORICAL_EVIDENCE: + historical.append(item) + elif item.evidence_state is EvidenceState.CURRENT: + current.append(item) + return PlannerContextPackage( + current_authoritative_sources=tuple(current), + supporting_brain_knowledge=tuple(knowledge), + historical_evidence=tuple(historical), + stale_or_conflicting_sources=tuple(stale), + missing_evidence=tuple(missing), + unreadable_or_inaccessible_sources=tuple(unreadable), + provenance=ordered, + warnings=tuple(sorted(set(warnings))), + ) + + +def content_sha256(content: str) -> str: + """Return the stable content identity used by local reader adapters.""" + return sha256(content.encode("utf-8")).hexdigest() diff --git a/src/neural_engine/cli.py b/src/neural_engine/cli.py index eac2908..7943ffd 100644 --- a/src/neural_engine/cli.py +++ b/src/neural_engine/cli.py @@ -420,9 +420,24 @@ def _render_status_fields( console.print(f"[bold cyan]{APP_NAME}[/bold cyan]") console.print(f"Version : {__version__}") console.print(f"Resolution source : {source}") - console.print(f"Configured Neural home : {configured}") - console.print(f"Resolved Neural home : {resolved_home}") - console.print(f"Resolved Brain path : {resolved_brain}") + console.print( + f"Configured Neural home : {configured}", + no_wrap=True, + overflow="ignore", + crop=False, + ) + console.print( + f"Resolved Neural home : {resolved_home}", + no_wrap=True, + overflow="ignore", + crop=False, + ) + console.print( + f"Resolved Brain path : {resolved_brain}", + no_wrap=True, + overflow="ignore", + crop=False, + ) console.print(f"Home exists : {_yes_no(home_exists)}") console.print(f"Home is directory : {_yes_no(home_is_directory)}") console.print(f"Home accessible : {_yes_no(home_accessible)}") diff --git a/src/neural_engine/infrastructure/local_planner_context_readers.py b/src/neural_engine/infrastructure/local_planner_context_readers.py new file mode 100644 index 0000000..07d1ccf --- /dev/null +++ b/src/neural_engine/infrastructure/local_planner_context_readers.py @@ -0,0 +1,376 @@ +"""Local, fixed-inventory readers for authority-aware planner context.""" + +from __future__ import annotations + +import re +import subprocess +from collections.abc import Callable +from datetime import UTC, datetime +from pathlib import Path +from uuid import UUID + +from neural_engine.application.planner_context_service import ( + EvidenceState, + HistoricalEvidenceInput, + PlannerRepositoryCheckpoint, + SourceEvidence, + SourceType, + content_sha256, +) +from neural_engine.core.paths import NeuralPaths +from neural_engine.infrastructure.json_knowledge_repository import JsonKnowledgeRepository +from neural_engine.ports.knowledge_repository import ( + KnowledgeIdentityMismatchError, + KnowledgeStoredDataError, +) + +_SECRET_PATTERN = re.compile( + r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----|" + r"\b(?:api_key|token|password|secret)\b\s*[:=]\s*\S+", + re.IGNORECASE, +) + + +class LocalPlannerContextReaders: + """Implements the five narrow reader ports with no filesystem mutation.""" + + def __init__( + self, + paths: NeuralPaths | None = None, + *, + clock: Callable[[], datetime] | None = None, + ) -> None: + self._paths = paths + self._clock = clock or (lambda: datetime.now(UTC)) + + def verify(self, checkpoint: PlannerRepositoryCheckpoint) -> SourceEvidence: + retrieved_at = self._clock() + root, error = self._root(checkpoint.repository_root) + if error is not None: + return self._diagnostic( + SourceType.REPOSITORY_METADATA, + "repository", + checkpoint.repository_identity, + EvidenceState.UNREADABLE, + retrieved_at, + error, + ) + values: dict[str, str] = {} + commands = { + "root": ("rev-parse", "--show-toplevel"), + "branch": ("branch", "--show-current"), + "head": ("rev-parse", "HEAD"), + "remote": ("rev-parse", "origin/main"), + "status": ("status", "--short"), + } + try: + for name, arguments in commands.items(): + values[name] = self._git(root, *arguments) + except RuntimeError as error: + return self._diagnostic( + SourceType.REPOSITORY_METADATA, + "repository", + checkpoint.repository_identity, + EvidenceState.UNREADABLE, + retrieved_at, + str(error), + ) + actual = { + "repository_root": str(root), + "branch": values["branch"], + "head": values["head"], + "authoritative_remote_ref": values["remote"], + "worktree_state": values["status"] or "clean", + } + expected = { + "repository_root": str(root), + "branch": checkpoint.branch, + "head": checkpoint.head, + "authoritative_remote_ref": checkpoint.authoritative_remote_ref, + "worktree_state": checkpoint.worktree_state, + } + if actual != expected: + return self._diagnostic( + SourceType.REPOSITORY_METADATA, + "repository", + checkpoint.repository_identity, + EvidenceState.STALE, + retrieved_at, + "verified checkpoint does not match live repository", + content=repr(actual), + checkpoint=checkpoint.head, + ) + content = "\n".join(f"{key}={value}" for key, value in sorted(actual.items())) + return SourceEvidence( + source_type=SourceType.REPOSITORY_METADATA, + normalized_locator="repository", + stable_identity=checkpoint.head, + external_project_context=checkpoint.repository_identity, + authority_class="verified live repository", + evidence_state=EvidenceState.CURRENT, + retrieved_at=retrieved_at, + extraction_boundary="verified repository metadata", + checkpoint_or_version=checkpoint.head, + excerpt=content, + content_sha256=content_sha256(content), + ) + + def read_current_documents( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: + return self._read_files(checkpoint, locators, SourceType.DESIGNATED_DOCUMENT) + + def read_review_evidence( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: + return self._read_files(checkpoint, locators, SourceType.DESIGNATED_REVIEW) + + def read_knowledge( + self, project_key: str, knowledge_ids: tuple[UUID, ...] + ) -> tuple[SourceEvidence, ...]: + retrieved_at = self._clock() + repository = ( + JsonKnowledgeRepository(paths=self._paths) if self._paths else JsonKnowledgeRepository() + ) + items: list[SourceEvidence] = [] + for knowledge_id in knowledge_ids: + locator = f"brain/knowledge/{knowledge_id}.json" + try: + knowledge = repository.get_by_id(knowledge_id) + except KnowledgeStoredDataError, KnowledgeIdentityMismatchError: + items.append( + self._diagnostic( + SourceType.BRAIN_KNOWLEDGE, + locator, + str(knowledge_id), + EvidenceState.UNREADABLE, + retrieved_at, + "selected Knowledge is malformed or unreadable", + ) + ) + continue + if knowledge is None: + items.append( + self._diagnostic( + SourceType.BRAIN_KNOWLEDGE, + locator, + str(knowledge_id), + EvidenceState.MISSING, + retrieved_at, + "selected Knowledge does not exist", + ) + ) + continue + content = knowledge.model_dump_json(indent=2) + items.append( + SourceEvidence( + source_type=SourceType.BRAIN_KNOWLEDGE, + normalized_locator=locator, + stable_identity=str(knowledge.id), + external_project_context=project_key, + authority_class="caller-selected supporting Knowledge", + evidence_state=EvidenceState.HISTORICAL, + retrieved_at=retrieved_at, + extraction_boundary="complete selected Knowledge record", + excerpt=self._excerpt(content), + content_sha256=content_sha256(content), + diagnostic="relevance is caller-selected", + ) + ) + return tuple(items) + + def read_historical_evidence( + self, evidence: tuple[HistoricalEvidenceInput, ...] + ) -> tuple[SourceEvidence, ...]: + retrieved_at = self._clock() + return tuple( + SourceEvidence( + source_type=SourceType.HISTORICAL_EVIDENCE, + normalized_locator=item.locator, + stable_identity=item.stable_identity, + external_project_context="caller-supplied historical context", + authority_class=item.authority_class, + evidence_state=item.evidence_state, + retrieved_at=retrieved_at, + extraction_boundary="caller-supplied bounded historical evidence", + checkpoint_or_version=item.checkpoint_or_version, + excerpt=self._excerpt(item.content), + content_sha256=content_sha256(item.content), + ) + for item in evidence + ) + + def _read_files( + self, + checkpoint: PlannerRepositoryCheckpoint, + locators: tuple[str, ...], + source_type: SourceType, + ) -> tuple[SourceEvidence, ...]: + retrieved_at = self._clock() + root, root_error = self._root(checkpoint.repository_root) + if root_error is not None: + return tuple( + self._diagnostic( + source_type, + locator, + checkpoint.head, + EvidenceState.UNREADABLE, + retrieved_at, + root_error, + ) + for locator in locators + ) + return tuple( + self._read_file(root, checkpoint, locator, source_type, retrieved_at) + for locator in locators + ) + + def _read_file( + self, + root: Path, + checkpoint: PlannerRepositoryCheckpoint, + locator: str, + source_type: SourceType, + retrieved_at: datetime, + ) -> SourceEvidence: + candidate = root.joinpath(*locator.split("/")) + try: + relative = candidate.relative_to(root) + except ValueError: + return self._diagnostic( + source_type, + locator, + checkpoint.head, + EvidenceState.UNREADABLE, + retrieved_at, + "path escapes verified repository root", + ) + if ( + not locator + or locator.startswith("/") + or any(part in {"", ".", ".."} for part in relative.parts) + ): + return self._diagnostic( + source_type, + locator, + checkpoint.head, + EvidenceState.UNREADABLE, + retrieved_at, + "invalid approved source locator", + ) + try: + if candidate.is_symlink() or not candidate.exists(): + state = ( + EvidenceState.UNREADABLE if candidate.is_symlink() else EvidenceState.MISSING + ) + reason = ( + "symlink sources are rejected" + if candidate.is_symlink() + else "designated source is missing" + ) + return self._diagnostic( + source_type, locator, checkpoint.head, state, retrieved_at, reason + ) + if not candidate.is_file() or candidate.stat().st_size > 65536: + return self._diagnostic( + source_type, + locator, + checkpoint.head, + EvidenceState.UNREADABLE, + retrieved_at, + "source is not a readable regular UTF-8 file within 64 KiB", + ) + raw = candidate.read_bytes() + content = raw.decode("utf-8") + except OSError, UnicodeDecodeError: + return self._diagnostic( + source_type, + locator, + checkpoint.head, + EvidenceState.UNREADABLE, + retrieved_at, + "source cannot be read as UTF-8", + ) + normalized = content.replace("\r\n", "\n").replace("\r", "\n") + digest = content_sha256(normalized) + if _SECRET_PATTERN.search(normalized): + return self._diagnostic( + source_type, + locator, + checkpoint.head, + EvidenceState.UNREADABLE, + retrieved_at, + "content withheld by secret guard", + content=normalized, + checkpoint=checkpoint.head, + ) + return SourceEvidence( + source_type=source_type, + normalized_locator=locator, + stable_identity=f"{checkpoint.head}:{locator}", + external_project_context=checkpoint.repository_identity, + authority_class="verified current repository source", + evidence_state=EvidenceState.CURRENT, + retrieved_at=retrieved_at, + extraction_boundary="first 120 normalized-LF lines, maximum 4096 UTF-8 bytes", + checkpoint_or_version=checkpoint.head, + excerpt=self._excerpt(normalized), + content_sha256=digest, + ) + + @staticmethod + def _root(value: str) -> tuple[Path, str | None]: + root = Path(value) + try: + if not root.is_absolute() or root.is_symlink() or not root.is_dir(): + return root, "verified repository root is unavailable or invalid" + resolved = root.resolve(strict=True) + if resolved != root: + return root, "verified repository root must already be strictly resolved" + return root, None + except OSError: + return root, "verified repository root cannot be resolved" + + @staticmethod + def _git(root: Path, *arguments: str) -> str: + result = subprocess.run( + ["git", "-C", str(root), *arguments], + check=False, + capture_output=True, + text=True, + encoding="utf-8", + ) + if result.returncode != 0: + raise RuntimeError("Git verification failed") + return result.stdout.strip() + + @staticmethod + def _excerpt(content: str) -> str: + lines = content.split("\n")[:120] + bounded = "\n".join(lines) + return bounded.encode("utf-8")[:4096].decode("utf-8", errors="ignore") + + @staticmethod + def _diagnostic( + source_type: SourceType, + locator: str, + identity: str, + state: EvidenceState, + retrieved_at: datetime, + reason: str, + *, + content: str | None = None, + checkpoint: str | None = None, + ) -> SourceEvidence: + return SourceEvidence( + source_type=source_type, + normalized_locator=locator, + stable_identity=identity, + external_project_context="unavailable source", + authority_class="no authority established", + evidence_state=state, + retrieved_at=retrieved_at, + checkpoint_or_version=checkpoint, + content_sha256=content_sha256(content) if content is not None else None, + diagnostic=reason, + ) diff --git a/src/neural_engine/ports/planner_context_readers.py b/src/neural_engine/ports/planner_context_readers.py new file mode 100644 index 0000000..81e398d --- /dev/null +++ b/src/neural_engine/ports/planner_context_readers.py @@ -0,0 +1,51 @@ +"""Narrow read-only boundaries for planner context preparation.""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Protocol +from uuid import UUID + +if TYPE_CHECKING: + from neural_engine.application.planner_context_service import ( + HistoricalEvidenceInput, + PlannerRepositoryCheckpoint, + SourceEvidence, + ) + + +class RepositoryMetadataReader(Protocol): + """Verify one caller-captured repository checkpoint without mutation.""" + + def verify(self, checkpoint: PlannerRepositoryCheckpoint) -> SourceEvidence: ... + + +class CurrentDocumentReader(Protocol): + """Read only designated current documents from one verified root.""" + + def read_current_documents( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: ... + + +class BrainKnowledgeReader(Protocol): + """Read explicitly selected Knowledge records only.""" + + def read_knowledge( + self, project_key: str, knowledge_ids: tuple[UUID, ...] + ) -> tuple[SourceEvidence, ...]: ... + + +class ReviewEvidenceReader(Protocol): + """Read only designated review artifacts from one verified root.""" + + def read_review_evidence( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: ... + + +class HistoricalEvidenceReader(Protocol): + """Convert caller-supplied, already bounded historical evidence.""" + + def read_historical_evidence( + self, evidence: tuple[HistoricalEvidenceInput, ...] + ) -> tuple[SourceEvidence, ...]: ... diff --git a/tests/test_planner_context_service.py b/tests/test_planner_context_service.py new file mode 100644 index 0000000..b070ea7 --- /dev/null +++ b/tests/test_planner_context_service.py @@ -0,0 +1,1279 @@ +from __future__ import annotations + +from dataclasses import dataclass, replace +from datetime import UTC, datetime +from pathlib import Path +from uuid import UUID + +import pytest +from pydantic import ValidationError + +from neural_engine.application.planner_context_service import ( + EvidenceState, + HistoricalEvidenceInput, + PlannerContextPackage, + PlannerContextRequest, + PlannerContextService, + PlannerRepositoryCheckpoint, + PlannerSourceFilters, + SourceEvidence, + SourceType, +) +from neural_engine.core.paths import NeuralPaths +from neural_engine.domain.knowledge import Knowledge, KnowledgeConfidence +from neural_engine.infrastructure.local_planner_context_readers import LocalPlannerContextReaders + +NOW = datetime(2026, 8, 6, tzinfo=UTC) +KNOWLEDGE_ID = UUID("00000000-0000-0000-0000-000000000001") + + +@dataclass(frozen=True) +class ExpectedEvidence: + locator: str + source_type: SourceType + evidence_state: EvidenceState + has_diagnostic: bool + + +@dataclass(frozen=True) +class FixtureExpectation: + fixture_id: str + extra_state: EvidenceState | None + expected_categories: tuple[tuple[str, tuple[str, ...]], ...] + expected_provenance: tuple[ExpectedEvidence, ...] + expected_partial_result: bool + + +_BASE_PROVENANCE = ( + ExpectedEvidence("repository", SourceType.REPOSITORY_METADATA, EvidenceState.CURRENT, False), + ExpectedEvidence("README.md", SourceType.DESIGNATED_DOCUMENT, EvidenceState.CURRENT, False), + ExpectedEvidence( + ".agent-work/reviews/review-prd-authority-aware-planner-context-package.md", + SourceType.DESIGNATED_REVIEW, + EvidenceState.CURRENT, + False, + ), + ExpectedEvidence( + f"brain/{KNOWLEDGE_ID}", SourceType.BRAIN_KNOWLEDGE, EvidenceState.HISTORICAL, True + ), + ExpectedEvidence( + "release/v1.0.0", + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.FROZEN_RELEASE_EVIDENCE, + True, + ), +) + + +def _categories( + *, + stale: tuple[str, ...] = (), + missing: tuple[str, ...] = (), + unreadable: tuple[str, ...] = (), +) -> tuple[tuple[str, tuple[str, ...]], ...]: + return ( + ( + "current_authoritative_sources", + ( + "repository", + "README.md", + ".agent-work/reviews/review-prd-authority-aware-planner-context-package.md", + ), + ), + ("supporting_brain_knowledge", (f"brain/{KNOWLEDGE_ID}",)), + ("historical_evidence", ("release/v1.0.0",)), + ("stale_or_conflicting_sources", stale), + ("missing_evidence", missing), + ("unreadable_or_inaccessible_sources", unreadable), + ) + + +FIXTURE_EXPECTATIONS = ( + FixtureExpectation( + "literal-clean-current-plus-knowledge", None, _categories(), _BASE_PROVENANCE, False + ), + FixtureExpectation( + "literal-historical-and-frozen", None, _categories(), _BASE_PROVENANCE, False + ), + FixtureExpectation( + "literal-stale-knowledge", + EvidenceState.STALE, + _categories(stale=("stale-knowledge",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "stale-knowledge", SourceType.BRAIN_KNOWLEDGE, EvidenceState.STALE, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation( + "literal-conflicting-source", + EvidenceState.CONFLICTING, + _categories(stale=("conflicting-source",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "conflicting-source", SourceType.BRAIN_KNOWLEDGE, EvidenceState.CONFLICTING, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation( + "literal-missing-source", + EvidenceState.MISSING, + _categories(missing=("missing-document",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "missing-document", SourceType.BRAIN_KNOWLEDGE, EvidenceState.MISSING, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation( + "literal-unreadable-source", + EvidenceState.UNREADABLE, + _categories(unreadable=("corrupt-knowledge",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "corrupt-knowledge", SourceType.BRAIN_KNOWLEDGE, EvidenceState.UNREADABLE, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation( + "literal-ambiguous-source", + EvidenceState.AMBIGUOUS, + _categories(missing=("explicit-empty-result",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "explicit-empty-result", SourceType.BRAIN_KNOWLEDGE, EvidenceState.AMBIGUOUS, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation("literal-restrictive-filter", None, _categories(), _BASE_PROVENANCE, False), + FixtureExpectation( + "literal-binary-state", + EvidenceState.UNREADABLE, + _categories(unreadable=("binary-source",)), + ( + *_BASE_PROVENANCE[:3], + ExpectedEvidence( + "binary-source", SourceType.BRAIN_KNOWLEDGE, EvidenceState.UNREADABLE, True + ), + *_BASE_PROVENANCE[3:], + ), + True, + ), + FixtureExpectation( + "literal-oversize-state", + EvidenceState.UNREADABLE, + _categories(unreadable=("oversize-source",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "oversize-source", SourceType.BRAIN_KNOWLEDGE, EvidenceState.UNREADABLE, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation( + "literal-checkpoint-state", + EvidenceState.STALE, + _categories(stale=("checkpoint-race",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "checkpoint-race", SourceType.BRAIN_KNOWLEDGE, EvidenceState.STALE, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), + FixtureExpectation( + "literal-secret-state", + EvidenceState.UNREADABLE, + _categories(unreadable=("secret-guard",)), + ( + *_BASE_PROVENANCE[:4], + ExpectedEvidence( + "secret-guard", SourceType.BRAIN_KNOWLEDGE, EvidenceState.UNREADABLE, True + ), + _BASE_PROVENANCE[4], + ), + True, + ), +) + + +def _checkpoint(root: str = "/tmp/repository") -> PlannerRepositoryCheckpoint: + return PlannerRepositoryCheckpoint( + repository_root=root, + repository_identity="NeuralEngine", + branch="main", + head="a" * 40, + authoritative_remote_ref="a" * 40, + worktree_state="clean", + verified_at=NOW, + ) + + +def _item(source_type: SourceType, state: EvidenceState, locator: str) -> SourceEvidence: + return SourceEvidence( + source_type=source_type, + normalized_locator=locator, + stable_identity=f"id:{locator}", + external_project_context="NeuralEngine", + authority_class="fixture authority", + evidence_state=state, + retrieved_at=NOW, + checkpoint_or_version="a" * 40, + excerpt=None if state is EvidenceState.UNREADABLE else "fixture excerpt", + diagnostic="fixture diagnostic" if state is not EvidenceState.CURRENT else None, + ) + + +class FixtureReaders: + def __init__(self, extra: tuple[SourceEvidence, ...] = ()) -> None: + self.extra = extra + + def verify(self, checkpoint: PlannerRepositoryCheckpoint) -> SourceEvidence: + return _item(SourceType.REPOSITORY_METADATA, EvidenceState.CURRENT, "repository") + + def read_current_documents( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: + return tuple( + _item(SourceType.DESIGNATED_DOCUMENT, EvidenceState.CURRENT, item) for item in locators + ) + + def read_review_evidence( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: + return tuple( + _item(SourceType.DESIGNATED_REVIEW, EvidenceState.CURRENT, item) for item in locators + ) + + def read_knowledge( + self, project_key: str, knowledge_ids: tuple[UUID, ...] + ) -> tuple[SourceEvidence, ...]: + return ( + tuple( + _item(SourceType.BRAIN_KNOWLEDGE, EvidenceState.HISTORICAL, f"brain/{item}") + for item in knowledge_ids + ) + + self.extra + ) + + def read_historical_evidence( + self, evidence: tuple[HistoricalEvidenceInput, ...] + ) -> tuple[SourceEvidence, ...]: + return tuple( + _item(SourceType.HISTORICAL_EVIDENCE, item.evidence_state, item.locator) + for item in evidence + ) + + +class SequencedFixtureReaders(FixtureReaders): + def __init__( + self, + metadata: tuple[SourceEvidence, SourceEvidence], + extra: tuple[SourceEvidence, ...] = (), + ) -> None: + super().__init__(extra) + self._metadata = metadata + self._index = 0 + + def verify(self, checkpoint: PlannerRepositoryCheckpoint) -> SourceEvidence: + result = self._metadata[self._index % len(self._metadata)] + self._index += 1 + return result + + +@dataclass(frozen=True) +class RepresentativeEvidenceExpectation: + source_type: SourceType + state: EvidenceState + authority: str + locator: str + stable_identity: str + diagnostic: str | None + + +@dataclass(frozen=True) +class RepresentativeExpectation: + fixture_id: str + category: str + source_type: SourceType + state: EvidenceState + authority: str + locator: str + stable_identity: str + diagnostic: str | None + warnings: tuple[str, ...] = () + partial_result: bool = False + category_locators: tuple[tuple[str, tuple[str, ...]], ...] = () + provenance: tuple[RepresentativeEvidenceExpectation, ...] = () + + +def _literal_categories( + *, + current: tuple[str, ...] = (), + knowledge: tuple[str, ...] = (), + historical: tuple[str, ...] = (), + stale: tuple[str, ...] = (), + missing: tuple[str, ...] = (), + unreadable: tuple[str, ...] = (), +) -> tuple[tuple[str, tuple[str, ...]], ...]: + return ( + ("current_authoritative_sources", current), + ("supporting_brain_knowledge", knowledge), + ("historical_evidence", historical), + ("stale_or_conflicting_sources", stale), + ("missing_evidence", missing), + ("unreadable_or_inaccessible_sources", unreadable), + ) + + +_REPRESENTATIVE_METADATA = RepresentativeEvidenceExpectation( + SourceType.REPOSITORY_METADATA, + EvidenceState.CURRENT, + "fixture authority", + "repository", + "id:repository", + None, +) + + +REPRESENTATIVE_EXPECTATIONS = ( + RepresentativeExpectation( + "clean-current-plus-knowledge", + "supporting_brain_knowledge", + SourceType.BRAIN_KNOWLEDGE, + EvidenceState.HISTORICAL, + "caller-selected supporting Knowledge", + f"brain/knowledge/{KNOWLEDGE_ID}.json", + str(KNOWLEDGE_ID), + "relevance is caller-selected", + category_locators=_literal_categories( + current=("repository",), knowledge=(f"brain/knowledge/{KNOWLEDGE_ID}.json",) + ), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.BRAIN_KNOWLEDGE, + EvidenceState.HISTORICAL, + "caller-selected supporting Knowledge", + f"brain/knowledge/{KNOWLEDGE_ID}.json", + str(KNOWLEDGE_ID), + "relevance is caller-selected", + ), + ), + ), + RepresentativeExpectation( + "historical-and-frozen", + "historical_evidence", + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.FROZEN_RELEASE_EVIDENCE, + "historical supporting evidence", + "release/v1.0.0", + "v1.0.0", + None, + category_locators=_literal_categories( + current=("repository",), historical=("release/v1.0.0",) + ), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.FROZEN_RELEASE_EVIDENCE, + "historical supporting evidence", + "release/v1.0.0", + "v1.0.0", + None, + ), + ), + ), + RepresentativeExpectation( + "stale-knowledge", + "stale_or_conflicting_sources", + SourceType.BRAIN_KNOWLEDGE, + EvidenceState.STALE, + "fixture authority", + "knowledge/stale", + "id:knowledge/stale", + "fixture diagnostic", + partial_result=True, + category_locators=_literal_categories(current=("repository",), stale=("knowledge/stale",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.BRAIN_KNOWLEDGE, + EvidenceState.STALE, + "fixture authority", + "knowledge/stale", + "id:knowledge/stale", + "fixture diagnostic", + ), + ), + ), + RepresentativeExpectation( + "conflicting-evidence", + "stale_or_conflicting_sources", + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.CONFLICTING, + "fixture authority", + "history/conflict", + "id:history/conflict", + "fixture diagnostic", + partial_result=True, + category_locators=_literal_categories(current=("repository",), stale=("history/conflict",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.CONFLICTING, + "fixture authority", + "history/conflict", + "id:history/conflict", + "fixture diagnostic", + ), + ), + ), + RepresentativeExpectation( + "missing-document", + "missing_evidence", + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.MISSING, + "no authority established", + "README.md", + "a" * 40, + "designated source is missing", + partial_result=True, + category_locators=_literal_categories(current=("repository",), missing=("README.md",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.MISSING, + "no authority established", + "README.md", + "a" * 40, + "designated source is missing", + ), + ), + ), + RepresentativeExpectation( + "corrupt-knowledge", + "unreadable_or_inaccessible_sources", + SourceType.BRAIN_KNOWLEDGE, + EvidenceState.UNREADABLE, + "no authority established", + f"brain/knowledge/{KNOWLEDGE_ID}.json", + str(KNOWLEDGE_ID), + "selected Knowledge is malformed or unreadable", + partial_result=True, + category_locators=_literal_categories( + current=("repository",), unreadable=(f"brain/knowledge/{KNOWLEDGE_ID}.json",) + ), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.BRAIN_KNOWLEDGE, + EvidenceState.UNREADABLE, + "no authority established", + f"brain/knowledge/{KNOWLEDGE_ID}.json", + str(KNOWLEDGE_ID), + "selected Knowledge is malformed or unreadable", + ), + ), + ), + RepresentativeExpectation( + "explicit-empty-result", + "missing_evidence", + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.AMBIGUOUS, + "fixture authority", + "history/empty", + "id:history/empty", + "fixture diagnostic", + partial_result=True, + category_locators=_literal_categories(current=("repository",), missing=("history/empty",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.AMBIGUOUS, + "fixture authority", + "history/empty", + "id:history/empty", + "fixture diagnostic", + ), + ), + ), + RepresentativeExpectation( + "restrictive-filter", + "current_authoritative_sources", + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.CURRENT, + "verified current repository source", + "README.md", + "a" * 40 + ":README.md", + None, + category_locators=_literal_categories(current=("repository", "README.md")), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.CURRENT, + "verified current repository source", + "README.md", + "a" * 40 + ":README.md", + None, + ), + ), + ), + RepresentativeExpectation( + "binary-source", + "unreadable_or_inaccessible_sources", + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.UNREADABLE, + "no authority established", + "README.md", + "a" * 40, + "source cannot be read as UTF-8", + partial_result=True, + category_locators=_literal_categories(current=("repository",), unreadable=("README.md",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.UNREADABLE, + "no authority established", + "README.md", + "a" * 40, + "source cannot be read as UTF-8", + ), + ), + ), + RepresentativeExpectation( + "oversize-source", + "unreadable_or_inaccessible_sources", + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.UNREADABLE, + "no authority established", + "README.md", + "a" * 40, + "source is not a readable regular UTF-8 file within 64 KiB", + partial_result=True, + category_locators=_literal_categories(current=("repository",), unreadable=("README.md",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.UNREADABLE, + "no authority established", + "README.md", + "a" * 40, + "source is not a readable regular UTF-8 file within 64 KiB", + ), + ), + ), + RepresentativeExpectation( + "checkpoint-race", + "stale_or_conflicting_sources", + SourceType.REPOSITORY_METADATA, + EvidenceState.STALE, + "fixture authority", + "repository-after", + "id:repository-after", + "fixture diagnostic", + ("verified repository checkpoint did not match before and after current-source reads",), + True, + category_locators=_literal_categories( + current=("repository-before",), stale=("repository-after",) + ), + provenance=( + RepresentativeEvidenceExpectation( + SourceType.REPOSITORY_METADATA, + EvidenceState.STALE, + "fixture authority", + "repository-after", + "id:repository-after", + "fixture diagnostic", + ), + RepresentativeEvidenceExpectation( + SourceType.REPOSITORY_METADATA, + EvidenceState.CURRENT, + "fixture authority", + "repository-before", + "id:repository-before", + None, + ), + ), + ), + RepresentativeExpectation( + "secret-guard", + "unreadable_or_inaccessible_sources", + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.UNREADABLE, + "no authority established", + "README.md", + "a" * 40, + "content withheld by secret guard", + partial_result=True, + category_locators=_literal_categories(current=("repository",), unreadable=("README.md",)), + provenance=( + _REPRESENTATIVE_METADATA, + RepresentativeEvidenceExpectation( + SourceType.DESIGNATED_DOCUMENT, + EvidenceState.UNREADABLE, + "no authority established", + "README.md", + "a" * 40, + "content withheld by secret guard", + ), + ), + ), +) + + +class DelegatingRepresentativeReaders: + def __init__( + self, expectation: RepresentativeExpectation, paths: NeuralPaths | None = None + ) -> None: + self.expectation = expectation + self.local = LocalPlannerContextReaders(paths, clock=lambda: NOW) + self.calls: dict[str, int] = {} + self.metadata_calls = 0 + + def _called(self, name: str) -> None: + self.calls[name] = self.calls.get(name, 0) + 1 + + def verify(self, checkpoint: PlannerRepositoryCheckpoint) -> SourceEvidence: + self._called("verify") + if self.expectation.fixture_id == "checkpoint-race": + self.metadata_calls += 1 + return _item( + SourceType.REPOSITORY_METADATA, + EvidenceState.CURRENT if self.metadata_calls % 2 else EvidenceState.STALE, + "repository-before" if self.metadata_calls % 2 else "repository-after", + ) + return _item(SourceType.REPOSITORY_METADATA, EvidenceState.CURRENT, "repository") + + def read_current_documents( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: + self._called("documents") + if self.expectation.source_type is SourceType.DESIGNATED_DOCUMENT: + return self.local.read_current_documents(checkpoint, locators) + return () + + def read_review_evidence( + self, checkpoint: PlannerRepositoryCheckpoint, locators: tuple[str, ...] + ) -> tuple[SourceEvidence, ...]: + self._called("reviews") + return () + + def read_knowledge( + self, project_key: str, knowledge_ids: tuple[UUID, ...] + ) -> tuple[SourceEvidence, ...]: + self._called("knowledge") + if self.expectation.fixture_id in {"clean-current-plus-knowledge", "corrupt-knowledge"}: + return self.local.read_knowledge(project_key, knowledge_ids) + if self.expectation.source_type is SourceType.BRAIN_KNOWLEDGE: + return ( + _item( + self.expectation.source_type, self.expectation.state, self.expectation.locator + ), + ) + return () + + def read_historical_evidence( + self, evidence: tuple[HistoricalEvidenceInput, ...] + ) -> tuple[SourceEvidence, ...]: + self._called("historical") + if self.expectation.fixture_id == "historical-and-frozen": + return self.local.read_historical_evidence(evidence) + if self.expectation.source_type is SourceType.HISTORICAL_EVIDENCE: + return ( + _item( + self.expectation.source_type, self.expectation.state, self.expectation.locator + ), + ) + return () + + +def _service(extra: tuple[SourceEvidence, ...] = ()) -> PlannerContextService: + readers = FixtureReaders(extra) + return PlannerContextService(readers, readers, readers, readers, readers) + + +def _request(filters: dict[str, object] | None = None) -> PlannerContextRequest: + return PlannerContextRequest.model_validate( + { + "project_key": " NeuralEngine ", + "task_statement": " inspect context only ", + "verified_repository_checkpoint": _checkpoint(), + "optional_source_filters": filters + or { + "current_documents": ("README.md",), + "review_artifacts": ( + ".agent-work/reviews/review-prd-authority-aware-planner-context-package.md", + ), + "knowledge_ids": (KNOWLEDGE_ID,), + "historical_evidence": ( + { + "locator": "release/v1.0.0", + "stable_identity": "v1.0.0", + "content": "frozen release evidence", + "evidence_state": "FROZEN_RELEASE_EVIDENCE", + "checkpoint_or_version": "v1.0.0", + }, + ), + }, + } + ) + + +def _assert_representative_package( + package: PlannerContextPackage, expectation: RepresentativeExpectation +) -> None: + actual_categories = ( + ( + "current_authoritative_sources", + tuple(item.normalized_locator for item in package.current_authoritative_sources), + ), + ( + "supporting_brain_knowledge", + tuple(item.normalized_locator for item in package.supporting_brain_knowledge), + ), + ( + "historical_evidence", + tuple(item.normalized_locator for item in package.historical_evidence), + ), + ( + "stale_or_conflicting_sources", + tuple(item.normalized_locator for item in package.stale_or_conflicting_sources), + ), + ("missing_evidence", tuple(item.normalized_locator for item in package.missing_evidence)), + ( + "unreadable_or_inaccessible_sources", + tuple(item.normalized_locator for item in package.unreadable_or_inaccessible_sources), + ), + ) + assert actual_categories == expectation.category_locators + actual_provenance = tuple( + RepresentativeEvidenceExpectation( + item.source_type, + item.evidence_state, + item.authority_class, + item.normalized_locator, + item.stable_identity, + item.diagnostic, + ) + for item in package.provenance + ) + assert actual_provenance == expectation.provenance + assert package.warnings == expectation.warnings + actual_partial_result = any( + getattr(package, category) + for category in ( + "stale_or_conflicting_sources", + "missing_evidence", + "unreadable_or_inaccessible_sources", + ) + ) or bool(package.warnings) + assert actual_partial_result is expectation.partial_result + + +def _assert_fixture_expectation( + package: PlannerContextPackage, expectation: FixtureExpectation +) -> None: + for category, expected_locators in expectation.expected_categories: + actual_items = getattr(package, category) + assert tuple(item.normalized_locator for item in actual_items) == expected_locators + + actual_provenance = tuple( + ExpectedEvidence( + item.normalized_locator, + item.source_type, + item.evidence_state, + item.diagnostic is not None, + ) + for item in package.provenance + ) + assert actual_provenance == expectation.expected_provenance + assert all(item.authority_class == "fixture authority" for item in package.provenance) + assert all(item.normalized_locator for item in package.provenance) + assert not package.warnings + + has_partial_result = any( + getattr(package, category) + for category in ( + "stale_or_conflicting_sources", + "missing_evidence", + "unreadable_or_inaccessible_sources", + ) + ) + assert has_partial_result is expectation.expected_partial_result + + +@pytest.mark.parametrize("expectation", FIXTURE_EXPECTATIONS, ids=lambda case: case.fixture_id) +def test_twelve_paired_service_fixtures_are_deterministic( + expectation: FixtureExpectation, +) -> None: + failure_category = { + EvidenceState.STALE: "stale_or_conflicting_sources", + EvidenceState.CONFLICTING: "stale_or_conflicting_sources", + EvidenceState.MISSING: "missing_evidence", + EvidenceState.AMBIGUOUS: "missing_evidence", + EvidenceState.UNREADABLE: "unreadable_or_inaccessible_sources", + } + extra = ( + () + if expectation.extra_state is None + else ( + _item( + SourceType.BRAIN_KNOWLEDGE, + expectation.extra_state, + dict(expectation.expected_categories)[failure_category[expectation.extra_state]][0], + ), + ) + ) + service = _service(extra) + request = _request() + actual = tuple(service.prepare(request).model_dump_json() for _ in range(3)) + package = service.prepare(request) + + assert actual[0] == actual[1] == actual[2] + _assert_fixture_expectation(package, expectation) + + +def test_independent_fixture_oracle_rejects_wrong_category_expectation() -> None: + expectation = FIXTURE_EXPECTATIONS[0] + wrong_expectation = FixtureExpectation( + expectation.fixture_id, + expectation.extra_state, + ( + ("current_authoritative_sources", ("README.md", "repository")), + *expectation.expected_categories[1:], + ), + expectation.expected_provenance, + expectation.expected_partial_result, + ) + + with pytest.raises(AssertionError): + _assert_fixture_expectation(_service().prepare(_request()), wrong_expectation) + + +def test_historical_brain_knowledge_has_one_exclusive_category_and_provenance_item() -> None: + package = _service().prepare(_request()) + knowledge_locator = f"brain/{KNOWLEDGE_ID}" + + assert tuple(item.normalized_locator for item in package.supporting_brain_knowledge) == ( + knowledge_locator, + ) + assert knowledge_locator not in tuple( + item.normalized_locator for item in package.historical_evidence + ) + assert sum(item.normalized_locator == knowledge_locator for item in package.provenance) == 1 + assert tuple(item.normalized_locator for item in package.historical_evidence) == ( + "release/v1.0.0", + ) + + +def test_all_source_categories_and_evidence_states_are_visible() -> None: + states = ( + EvidenceState.STALE, + EvidenceState.CONFLICTING, + EvidenceState.MISSING, + EvidenceState.UNREADABLE, + EvidenceState.AMBIGUOUS, + ) + extras = tuple(_item(SourceType.BRAIN_KNOWLEDGE, state, state.value) for state in states) + package = _service(extras).prepare(_request()) + + assert {item.source_type for item in package.provenance} == set(SourceType) + assert {item.evidence_state for item in package.provenance} == { + EvidenceState.CURRENT, + EvidenceState.HISTORICAL, + EvidenceState.FROZEN_RELEASE_EVIDENCE, + *states, + } + assert package.stale_or_conflicting_sources + assert package.missing_evidence + assert package.unreadable_or_inaccessible_sources + + +def test_filters_only_narrow_and_source_count_is_bounded() -> None: + package = _service().prepare(_request({"current_documents": (), "review_artifacts": ()})) + assert not package.current_authoritative_sources[1:] + with pytest.raises(ValidationError, match="At most 24"): + PlannerSourceFilters(knowledge_ids=tuple(UUID(int=index) for index in range(13))) + with pytest.raises(ValidationError, match="approved inventory"): + PlannerSourceFilters(current_documents=("/etc/passwd",)) + + +def test_local_reader_rejects_path_traversal_absolute_symlink_binary_and_oversize( + tmp_path: Path, +) -> None: + reader = LocalPlannerContextReaders(clock=lambda: NOW) + checkpoint = _checkpoint(str(tmp_path)) + (tmp_path / "ok.md").write_text("ok", encoding="utf-8") + (tmp_path / "binary.md").write_bytes(b"\xff") + (tmp_path / "large.md").write_bytes(b"x" * 65537) + (tmp_path / "target.md").write_text("target", encoding="utf-8") + (tmp_path / "link.md").symlink_to(tmp_path / "target.md") + + cases = ("../outside", "/etc/passwd", "binary.md", "large.md", "link.md") + items = reader.read_current_documents(checkpoint, cases) + assert all(item.evidence_state is EvidenceState.UNREADABLE for item in items) + assert all(item.excerpt is None for item in items) + + +def test_local_reader_secret_guard_and_excerpt_limits_are_read_only(tmp_path: Path) -> None: + reader = LocalPlannerContextReaders(clock=lambda: NOW) + checkpoint = _checkpoint(str(tmp_path)) + (tmp_path / "secret.md").write_text("token=not-a-real-secret", encoding="utf-8") + (tmp_path / "long.md").write_text("\n".join("x" * 80 for _ in range(150)), encoding="utf-8") + before = tuple(sorted(path.name for path in tmp_path.iterdir())) + secret, long = reader.read_current_documents(checkpoint, ("secret.md", "long.md")) + after = tuple(sorted(path.name for path in tmp_path.iterdir())) + + assert before == after + assert secret.evidence_state is EvidenceState.UNREADABLE and secret.excerpt is None + assert secret.content_sha256 + assert long.evidence_state is EvidenceState.CURRENT + assert long.excerpt is not None and len(long.excerpt.encode()) <= 4096 + assert long.excerpt.count("\n") + 1 <= 120 + + +@pytest.mark.parametrize( + ("source_type", "state", "category"), + [ + (SourceType.REPOSITORY_METADATA, EvidenceState.CURRENT, "current_authoritative_sources"), + (SourceType.DESIGNATED_DOCUMENT, EvidenceState.CURRENT, "current_authoritative_sources"), + (SourceType.DESIGNATED_REVIEW, EvidenceState.CURRENT, "current_authoritative_sources"), + (SourceType.BRAIN_KNOWLEDGE, EvidenceState.CURRENT, "supporting_brain_knowledge"), + (SourceType.BRAIN_KNOWLEDGE, EvidenceState.HISTORICAL, "supporting_brain_knowledge"), + (SourceType.HISTORICAL_EVIDENCE, EvidenceState.HISTORICAL, "historical_evidence"), + ( + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.FROZEN_RELEASE_EVIDENCE, + "historical_evidence", + ), + (SourceType.BRAIN_KNOWLEDGE, EvidenceState.STALE, "stale_or_conflicting_sources"), + ( + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.CONFLICTING, + "stale_or_conflicting_sources", + ), + (SourceType.DESIGNATED_DOCUMENT, EvidenceState.MISSING, "missing_evidence"), + (SourceType.BRAIN_KNOWLEDGE, EvidenceState.AMBIGUOUS, "missing_evidence"), + ( + SourceType.HISTORICAL_EVIDENCE, + EvidenceState.UNREADABLE, + "unreadable_or_inaccessible_sources", + ), + ], +) +def test_assemble_routes_each_source_state_to_one_category( + source_type: SourceType, state: EvidenceState, category: str +) -> None: + item = _item(source_type, state, f"matrix/{source_type.value}/{state.value}") + package = _service()._assemble((item,), []) + categories = ( + "current_authoritative_sources", + "supporting_brain_knowledge", + "historical_evidence", + "stale_or_conflicting_sources", + "missing_evidence", + "unreadable_or_inaccessible_sources", + ) + + assert tuple(getattr(package, category)) == (item,) + assert sum(item in getattr(package, name) for name in categories) == 1 + assert package.provenance == (item,) + if source_type is SourceType.BRAIN_KNOWLEDGE: + assert item not in package.current_authoritative_sources + + +@pytest.mark.parametrize( + "fixture_id", + ( + "clean-current-plus-knowledge", + "historical-and-frozen", + "stale-knowledge", + "conflicting-evidence", + "missing-document", + "corrupt-knowledge", + "explicit-empty-result", + "restrictive-filter", + "binary-source", + "oversize-source", + "checkpoint-race", + "secret-guard", + ), +) +def test_legacy_boundary_examples_are_deterministic(tmp_path: Path, fixture_id: str) -> None: + """Exercise each named policy boundary before comparing a literal package result.""" + local = LocalPlannerContextReaders(clock=lambda: NOW) + checkpoint = _checkpoint(str(tmp_path)) + source_item: SourceEvidence + expected_category: str + expected_authority: str + filters: dict[str, object] = { + "current_documents": (), + "review_artifacts": (), + "knowledge_ids": (), + "historical_evidence": (), + } + + if fixture_id == "clean-current-plus-knowledge": + home = tmp_path / "home" + knowledge_directory = home / "brain" / "knowledge" + knowledge_directory.mkdir(parents=True) + knowledge = Knowledge( + id=KNOWLEDGE_ID, + timestamp=NOW, + statement="Selected knowledge", + rationale="fixture", + confidence=KnowledgeConfidence.HIGH, + experience_ids=[], + ) + (knowledge_directory / f"{KNOWLEDGE_ID}.json").write_text( + knowledge.model_dump_json(), encoding="utf-8" + ) + reader = LocalPlannerContextReaders( + NeuralPaths("default", None, home, lambda _path, _mode: True), clock=lambda: NOW + ) + source_item = reader.read_knowledge("NeuralEngine", (KNOWLEDGE_ID,))[0] + expected_category, expected_authority = ( + "supporting_brain_knowledge", + "caller-selected supporting Knowledge", + ) + elif fixture_id == "historical-and-frozen": + source_item = local.read_historical_evidence( + ( + HistoricalEvidenceInput( + locator="release/v1.0.0", + stable_identity="v1.0.0", + content="frozen release evidence", + evidence_state=EvidenceState.FROZEN_RELEASE_EVIDENCE, + checkpoint_or_version="v1.0.0", + ), + ) + )[0] + expected_category, expected_authority = ( + "historical_evidence", + "historical supporting evidence", + ) + elif fixture_id == "stale-knowledge": + source_item = _item(SourceType.BRAIN_KNOWLEDGE, EvidenceState.STALE, "knowledge/stale") + expected_category, expected_authority = "stale_or_conflicting_sources", "fixture authority" + elif fixture_id == "conflicting-evidence": + source_item = _item( + SourceType.HISTORICAL_EVIDENCE, EvidenceState.CONFLICTING, "history/conflict" + ) + expected_category, expected_authority = "stale_or_conflicting_sources", "fixture authority" + elif fixture_id == "missing-document": + source_item = local.read_current_documents(checkpoint, ("missing.md",))[0] + expected_category, expected_authority = "missing_evidence", "no authority established" + elif fixture_id == "corrupt-knowledge": + home = tmp_path / "home" + knowledge_directory = home / "brain" / "knowledge" + knowledge_directory.mkdir(parents=True) + (knowledge_directory / f"{KNOWLEDGE_ID}.json").write_text("{broken", encoding="utf-8") + reader = LocalPlannerContextReaders( + NeuralPaths("default", None, home, lambda _path, _mode: True), clock=lambda: NOW + ) + source_item = reader.read_knowledge("NeuralEngine", (KNOWLEDGE_ID,))[0] + expected_category, expected_authority = ( + "unreadable_or_inaccessible_sources", + "no authority established", + ) + elif fixture_id == "explicit-empty-result": + source_item = _item( + SourceType.HISTORICAL_EVIDENCE, EvidenceState.AMBIGUOUS, "history/empty" + ) + expected_category, expected_authority = "missing_evidence", "fixture authority" + elif fixture_id == "restrictive-filter": + filters["current_documents"] = ("README.md",) + source_item = _item( + SourceType.BRAIN_KNOWLEDGE, EvidenceState.HISTORICAL, "knowledge/filter" + ) + expected_category, expected_authority = "supporting_brain_knowledge", "fixture authority" + elif fixture_id == "binary-source": + (tmp_path / "binary.md").write_bytes(b"\xff") + source_item = local.read_current_documents(checkpoint, ("binary.md",))[0] + expected_category, expected_authority = ( + "unreadable_or_inaccessible_sources", + "no authority established", + ) + elif fixture_id == "oversize-source": + (tmp_path / "large.md").write_bytes(b"x" * 65537) + source_item = local.read_current_documents(checkpoint, ("large.md",))[0] + expected_category, expected_authority = ( + "unreadable_or_inaccessible_sources", + "no authority established", + ) + elif fixture_id == "checkpoint-race": + source_item = _item(SourceType.REPOSITORY_METADATA, EvidenceState.STALE, "repository-after") + expected_category, expected_authority = "stale_or_conflicting_sources", "fixture authority" + else: + (tmp_path / "secret.md").write_text("token=not-a-real-secret", encoding="utf-8") + source_item = local.read_current_documents(checkpoint, ("secret.md",))[0] + expected_category, expected_authority = ( + "unreadable_or_inaccessible_sources", + "no authority established", + ) + + if fixture_id == "checkpoint-race": + readers = SequencedFixtureReaders( + ( + _item(SourceType.REPOSITORY_METADATA, EvidenceState.CURRENT, "repository-before"), + source_item, + ) + ) + service = PlannerContextService(readers, readers, readers, readers, readers) + else: + service = _service((source_item,)) + request = _request(filters) + serializations = tuple(service.prepare(request).model_dump_json() for _ in range(3)) + package = service.prepare(request) + + assert serializations[0] == serializations[1] == serializations[2] + assert tuple(item.normalized_locator for item in getattr(package, expected_category)) == ( + source_item.normalized_locator, + ) + assert source_item.authority_class == expected_authority + assert sum(item == source_item for item in package.provenance) == 1 + if fixture_id == "restrictive-filter": + assert tuple(item.normalized_locator for item in package.current_authoritative_sources) == ( + "repository", + "README.md", + ) + + +@pytest.mark.parametrize( + "expectation", REPRESENTATIVE_EXPECTATIONS, ids=lambda item: item.fixture_id +) +def test_twelve_representative_fixtures_use_literal_oracles( + tmp_path: Path, expectation: RepresentativeExpectation +) -> None: + filters: dict[str, object] = { + "current_documents": (), + "review_artifacts": (), + "knowledge_ids": (), + "historical_evidence": (), + } + paths: NeuralPaths | None = None + if expectation.fixture_id in {"clean-current-plus-knowledge", "corrupt-knowledge"}: + home = tmp_path / "home" + directory = home / "brain" / "knowledge" + directory.mkdir(parents=True) + paths = NeuralPaths("default", None, home, lambda _path, _mode: True) + filters["knowledge_ids"] = (KNOWLEDGE_ID,) + content = ( + "{broken" + if expectation.fixture_id == "corrupt-knowledge" + else Knowledge( + id=KNOWLEDGE_ID, + timestamp=NOW, + statement="Selected knowledge", + rationale="fixture", + confidence=KnowledgeConfidence.HIGH, + experience_ids=[], + ).model_dump_json() + ) + (directory / f"{KNOWLEDGE_ID}.json").write_text(content, encoding="utf-8") + elif expectation.fixture_id == "historical-and-frozen": + filters["historical_evidence"] = ( + HistoricalEvidenceInput( + locator="release/v1.0.0", + stable_identity="v1.0.0", + content="frozen release evidence", + evidence_state=EvidenceState.FROZEN_RELEASE_EVIDENCE, + checkpoint_or_version="v1.0.0", + ), + ) + elif expectation.source_type is SourceType.DESIGNATED_DOCUMENT: + filters["current_documents"] = ("README.md",) + if expectation.fixture_id == "binary-source": + (tmp_path / "README.md").write_bytes(b"\xff") + elif expectation.fixture_id == "oversize-source": + (tmp_path / "README.md").write_bytes(b"x" * 65537) + elif expectation.fixture_id == "secret-guard": + (tmp_path / "README.md").write_text("token=not-a-real-secret", encoding="utf-8") + elif expectation.fixture_id == "restrictive-filter": + (tmp_path / "README.md").write_text("selected", encoding="utf-8") + readers = DelegatingRepresentativeReaders(expectation, paths) + service = PlannerContextService(readers, readers, readers, readers, readers) + request = PlannerContextRequest( + project_key="NeuralEngine", + task_statement="fixture", + verified_repository_checkpoint=_checkpoint(str(tmp_path)), + optional_source_filters=PlannerSourceFilters.model_validate(filters), + ) + serializations = tuple(service.prepare(request).model_dump_json() for _ in range(3)) + package = service.prepare(request) + assert serializations[0] == serializations[1] == serializations[2] + _assert_representative_package(package, expectation) + assert readers.calls["verify"] >= 2 + if expectation.fixture_id in { + "missing-document", + "binary-source", + "oversize-source", + "secret-guard", + "restrictive-filter", + }: + assert readers.calls["documents"] == 4 + if expectation.fixture_id in {"clean-current-plus-knowledge", "corrupt-knowledge"}: + assert readers.calls["knowledge"] == 4 + + +@pytest.mark.parametrize( + "wrong_expectation", + ( + replace( + REPRESENTATIVE_EXPECTATIONS[4], + provenance=( + replace(REPRESENTATIVE_EXPECTATIONS[4].provenance[0], stable_identity="wrong-id"), + REPRESENTATIVE_EXPECTATIONS[4].provenance[1], + ), + ), + replace( + REPRESENTATIVE_EXPECTATIONS[4], + provenance=tuple(reversed(REPRESENTATIVE_EXPECTATIONS[4].provenance)), + ), + replace(REPRESENTATIVE_EXPECTATIONS[4], partial_result=False), + ), + ids=("wrong-stable-identity", "wrong-provenance-order", "wrong-partial-result"), +) +def test_representative_literal_oracle_rejects_wrong_expectation( + tmp_path: Path, wrong_expectation: RepresentativeExpectation +) -> None: + expectation = REPRESENTATIVE_EXPECTATIONS[4] + readers = DelegatingRepresentativeReaders(expectation) + request = PlannerContextRequest( + project_key="NeuralEngine", + task_statement="fixture", + verified_repository_checkpoint=_checkpoint(str(tmp_path)), + optional_source_filters=PlannerSourceFilters( + current_documents=("README.md",), review_artifacts=() + ), + ) + package = PlannerContextService(readers, readers, readers, readers, readers).prepare(request) + with pytest.raises(AssertionError): + _assert_representative_package(package, wrong_expectation)