docs: record Pi pilot and separate agent evaluation from regression

GLOSSARY.md

@@ -0,0 +1,26 @@
+# SACP evaluation vocabulary
+
+本页区分 SACP 的验证对象与评测角色。实现选择与执行边界见 [ADR-0001](./docs/adr/0001-real-agent-evaluation-with-pi.md)。
+
+## Language
+
+**场景回归测试（Scenario regression harness）**：以明确构造的声明和证据为输入，检查 SACP 判定是否符合预期的测试集合。
+_Avoid_: 真实 Agent 评测、Agent 能力 benchmark
+
+**Agent 运行框架（Agent harness）**：承载模型调用、工具执行和会话生命周期的运行系统；在评测中是被驱动和观察的对象。
+_Avoid_: 评测器、独立验收器
+
+**真实 Agent 评测（Agent evaluation harness）**：围绕真实 Agent 执行任务，采集记录、组织独立验收并比较判定结果的评测系统。
+_Avoid_: 仅凭模拟事件得到的真实准确率
+
+**独立任务验收（Independent task acceptance）**：依据预先确定的任务条件，对实际交付进行的检查；不以 Agent 自述或 SACP 的判定作为正确性标准。
+_Avoid_: Agent 自证、以被测规则作为标准答案
+
+**任务完成（Task completion）**：实际交付满足该任务的验收条件。
+_Avoid_: 进程正常退出、会话结束、测试命令返回成功
+
+**完成声明（Completion claim）**：Agent 对任务或步骤已经完成的陈述，其支持程度须与对应证据分别判断。
+_Avoid_: 完成证明
+
+**预检通过（Preflight pass）**：记录输入满足指定证据与授权契约，允许调用方进入下一步。
+_Avoid_: 任务必然正确、外部动作已成功、自动获准发布

README.md

@@ -70,7 +70,9 @@ Positive controls passed: 3/3
 ## Protocol validator: 49/49 strict PASS (0 FAIL)
 ```
 
-The harness uses fixed synthetic scenarios without model calls. It prints by default; `--output-dir .local/harness` explicitly writes reports. Continue through the [example guide](./examples/README.md) for deployment, refund, recovery and optional LangGraph demos.
+This is a **scenario regression harness for the SACP engine**: it constructs claims and mock events directly, without running a coding agent or calling a model. It prints by default; `--output-dir .local/harness` explicitly writes reports. Continue through the [example guide](./examples/README.md) for deployment, refund, recovery and optional LangGraph demos.
+
+A separate real-agent evaluation layer is planned, with Pi as the first integration pilot. It is not implemented or benchmarked yet; see [ADR-0001](./docs/adr/0001-real-agent-evaluation-with-pi.md) for the decision, independent acceptance criteria and limits.
 
 <a id="architecture"></a>
 
@@ -89,7 +91,7 @@ flowchart TB
         Verdict --> Gate[Release / deployment / refund gate]
         Verify --> Recovery[Reconciliation route and next owner]
     end
-    subgraph Offline[Offline harness: verify explicit boundaries]
+    subgraph Offline[Scenario regression: verify engine boundaries]
         Cases[Breakers and positive controls]
     end
     Cases -. Exercise .-> Online
@@ -147,6 +149,7 @@ The base engine uses Python's standard library; protocol/CI tooling requires PyY
 ## 📚 Documentation and contributing
 
 - **Understand:** [Architecture](./docs/architecture.md) · [Evidence model](./docs/evidence-model.md) · [State projection](./docs/state-machine.md)
+- **Evaluation decisions:** [Vocabulary](./GLOSSARY.md) · [ADR-0001: real-agent evaluation with Pi](./docs/adr/0001-real-agent-evaluation-with-pi.md)
 - **Run and develop:** [Examples](./examples/README.md) · [Development and suite commands](./docs/development.md) · [Runtime contracts](./docs/runtime-contracts.md)
 - **Review and extend:** [Maintainer workflow](./docs/maintainer-workflow.md) · [Dirty Run cases](./DIRTY_RUN_CASES.md) · [Contributing](./CONTRIBUTING.md)
 

README.zh-CN.md

@@ -70,7 +70,9 @@ Positive controls passed: 3/3
 ## Protocol validator: 49/49 strict PASS (0 FAIL)
 ```
 
-Harness 使用固定合成场景，不调用模型。默认只打印，显式传入 `--output-dir .local/harness` 才写报告。接着按[示例导航](./examples/README.zh-CN.md)体验部署、退款、恢复和可选 LangGraph demo。
+这里是 **SACP 引擎的场景回归测试（Harness）**：直接构造声明与模拟事件，不运行 coding agent，也不调用模型。默认只打印，显式传入 `--output-dir .local/harness` 才写报告。接着按[示例导航](./examples/README.zh-CN.md)体验部署、退款、恢复和可选 LangGraph demo。
+
+另计划新增真实 Agent 评测层，以 Pi 为首个接入试点。目前尚未实现，也没有对应 benchmark 结果；选择理由、独立验收标准与边界见 [ADR-0001](./docs/adr/0001-real-agent-evaluation-with-pi.md)。
 
 <a id="框架"></a>
 
@@ -89,7 +91,7 @@ flowchart TB
         Verdict --> Gate[发布 / 部署 / 退款门禁]
         Verify --> Recovery[对账路线与下一责任人]
     end
-    subgraph Offline[离线 Harness：验证明确的边界]
+    subgraph Offline[场景回归测试：验证引擎边界]
         Cases[反例与正向控制]
     end
     Cases -. 测试 .-> Online
@@ -147,6 +149,7 @@ PR 代码 → 只读 CI → 证据 artifact
 ## 📚 文档与贡献
 
 - **理解项目：**[架构](./docs/architecture.md) · [证据模型](./docs/evidence-model.md) · [状态投影](./docs/state-machine.md)
+- **评测决策：**[术语](./GLOSSARY.md) · [ADR-0001：真实 Agent 评测与 Pi 选择](./docs/adr/0001-real-agent-evaluation-with-pi.md)
 - **运行与开发：**[示例](./examples/README.zh-CN.md) · [开发与 suite 命令](./docs/development.md) · [运行时契约](./docs/runtime-contracts.md)
 - **审查与扩展：**[维护者流程](./docs/maintainer-workflow.md) · [Dirty Run cases](./DIRTY_RUN_CASES.md) · [贡献指南](./CONTRIBUTING.zh-CN.md)
 

ROADMAP.md

@@ -10,6 +10,12 @@ The text-first spec, validator, dirty cases, Receipt Chain profile, local Doctor
 
 CI runs a cross-platform core matrix, explicit LangGraph integration, strict protocol validation and a deterministic harness with positive controls. Trusted default-branch code cross-checks GitHub execution facts and current PR identity before constructing a receipt. Merging remains a manual maintainer action. See [architecture](./docs/architecture.md) and [verification scope](./docs/verification-2026-10-15.md).
 
+## Next: real-agent evaluation pilot
+
+[ADR-0001](./docs/adr/0001-real-agent-evaluation-with-pi.md) retains the existing deterministic engine scenarios and selects Pi as the first real-agent pilot. Implement a Python outer runner using Pi's JSON subprocess interface, isolate and pin its environment, then collect actual coding-task trajectories and independently verify the delivered patches. The adapter, provider/model, budget and real results are not yet available.
+
+Start with a small fixed task set and compare SACP with a predefined test-based baseline on the same records. Report missed false completions, incorrect blocks, indeterminate outcomes and overhead; separate injected faults from naturally occurring failures. Keep model calls out of required PR CI. Acceptance stages and stop conditions are defined in the ADR.
+
 ## Next: trust and independent acceptance
 
 - Complete an actual controlled fork PR smoke run; synthetic fork API tests are not equivalent.

ROADMAP.zh-CN.md

@@ -10,6 +10,12 @@ SACP/0.1 是当前交换契约。下面描述能力成熟度，不意味着每
 
 CI 已运行跨平台 core 矩阵、显式安装依赖的 LangGraph 集成、严格协议校验，以及包含正向控制的确定性 harness。默认分支上的可信实现交叉核对 GitHub 执行事实和当前 PR 身份，再生成 receipt。合并仍由维护者手动发起。参见[架构](./docs/architecture.md)与[验证范围](./docs/verification-2026-10-15.md)。
 
+## 下一步：真实 Agent 评测试点
+
+[ADR-0001](./docs/adr/0001-real-agent-evaluation-with-pi.md)决定保留现有确定性引擎场景测试，以 Pi 为首个真实 Agent 试点。计划使用 Python 外层 runner 与 Pi JSON 子进程接口，固定并隔离环境，采集实际代码任务轨迹并独立验收交付补丁。目前尚无 adapter、选定的 provider/model、运行预算或真实评测结果。
+
+先用小规模固定任务集，在同一批记录上比较 SACP 与预先定义的测试基线，报告漏检、误拦、不可判定和额外开销；注入故障与自然失败分别统计。模型调用不进入 required PR CI。分阶段验收与停止条件见 ADR。
+
 ## 下一步：可信度与独立验收
 
 - 完成真实、受控的 fork PR 烟测；合成 fork API 测试不能代替它。

STATUS.md

@@ -15,7 +15,7 @@ Start with [README](./README.md) / [中文](./README.zh-CN.md), then [architectu
 | Mock idempotency | Same-database refund reservations, conflict rejection, restart and concurrent calls | Simulated effect identity, not end-to-end exactly-once |
 | Adapters | REST-shaped GitHub payload mapping, trusted reviewer allowlist, CLI, real LangGraph wiring/checkpointers | Payload source authentication remains an integration responsibility |
 | HTTP experiments | Real localhost provider transport, simulated health/rollback | No public SACP API or production provider |
-| Harness | Eight synthetic breakers and three positive controls | Fixed small set; no measured population accuracy |
+| Scenario regression harness | Eight synthetic breakers and three positive controls for the SACP engine | Does not run a real agent/model; no measured population accuracy |
 | Governance | Read-only PR CI, trusted independent attestation, formal receipt, manual squash merge, main revalidation | Semantic review required; real fork E2E still outstanding |
 | Corpus and Doctor | 34 translated receipts, YAML examples, local text-audit CLI | Historical/synthetic inputs; Doctor uses heuristics and does not rerun tasks |
 
@@ -33,4 +33,6 @@ The protocol itself does not require a database or HTTP transport. The Python re
 
 ## Next work
 
+The accepted [real-agent evaluation decision](./docs/adr/0001-real-agent-evaluation-with-pi.md) preserves deterministic scenarios and selects Pi as the first coding-agent pilot, using an outer runner and independent task acceptance. This is planned work: no Pi adapter, selected provider/model or real-agent benchmark result exists yet.
+
 Finish controlled real fork end-to-end acceptance, strengthen isolation against arbitrary malicious Python tests, authenticate external collectors where used, and gather independently reviewed real-world false-completion and false-block cases. Keep schema/transport expansion behind concrete dirty cases and compatibility review. See [ROADMAP](./ROADMAP.md).