Skip to content

feat: add architecture semantic coverage checks - #239

Closed
zhouyuanxinand wants to merge 3 commits into
tt-a1i:mainfrom
zhouyuanxinand:codex/fix-architecture-semantic-coverage
Closed

feat: add architecture semantic coverage checks#239
zhouyuanxinand wants to merge 3 commits into
tt-a1i:mainfrom
zhouyuanxinand:codex/fix-architecture-semantic-coverage

Conversation

@zhouyuanxinand

@zhouyuanxinand zhouyuanxinand commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in semanticChecks contract for repository-backed architecture diagrams and incorporates the requested review fixes.

  • preserves every represented, missing, and explicitly omitted requirement in an auditable receipt ledger;
  • requires both endpoints to exist before a required self-path can pass;
  • rejects empty and whitespace-only omission reasons;
  • emits actionable diagnostics with concrete evidence and supported fixes;
  • rebases onto the current main and rebuilds the checked-in archive canonically with Node 22.

Regenerated files

  • archify/renderers/shared/generated-validators.mjs
  • archify.zip

Validation

  • node --test test/architecture-semantic-coverage.test.mjs — 7/7 passed on Node 22;
  • skill quick_validate.py — passed;
  • canonical scripts/build-zip.sh run twice — identical 77-file archive, SHA-256 5a5a6553cf3dae429cbb931e9d05c89b1461544e72a6b2679ce1a80f84360d9b;
  • git diff --check — passed.

The full repository suite was also exercised locally; Windows-only environment limitations around symlink creation, Unix file modes, and Bash resolution prevent a clean local result, so the Linux CI run remains authoritative for the complete suite.

Closes #237

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the semantic coverage contribution. Reviewed exact head a9e1f8b. Changes are needed to make a passing coverage receipt auditable and to preserve the published package contract.

Spec

Satisfied requirements disappear from the receipt; missing self-paths incorrectly pass; whitespace-only omission reasons suppress missing facts. See inline reproductions.

Standards / delivery

The new diagnostic helper returns only severity/code/message/subject; please include the documented concrete evidence and executable supportedFixes rather than making agents scrape prose. Also rebuild archify.zip canonically: after the intended package.json cleanup, 74 of 77 payload files differ from tracked bytes because the archive uses CRLF and the index/source uses LF (e.g. LICENSE 1168 vs1146 bytes). Normalizing those line endings makes the texts equal, but the committed package is not a byte-reproducible build of tracked source.

Verification

New semantic coverage plus geometry tests: 60 passed, 0 failed. Additional schema/coverage reproductions confirm the findings; archive checked member-by-member. No browser/manual-visual or full-suite pass claimed; no source edits or merge.

}
}

return {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Include the represented requirements in the coverage receipt. On components api/worker with api->worker, an empty checklist and a checklist requiring worker, the direct edge, and the path generate the exact same pass/0-warning/empty-diagnostics receipt. #237 requires represented/missing/explicitly-omitted evidence, but satisfied subjects are skipped above and never serialized; a downstream reader cannot tell what was checked. Persist checked subjects and per-requirement outcomes in the artifact and checker receipt.

}

function hasDirectedPath(adjacency, from, to) {
if (from === to) return true;

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Confirm path endpoints exist before accepting a zero-length path. evaluateArchitectureSemanticCoverage({components:[{id:'api'}],connections:[],semanticChecks:{requiredPaths:[{from:'missing',to:'missing'}]}}) currently returns pass with no diagnostics because from===to returns immediately. No such component/path exists. Preserve valid self-reachability only for existing authored components and add this missing-endpoint regression.

"properties": {
"kind": { "const": "component" },
"id": { "$ref": "common.schema.json#/$defs/id" },
"reason": { "type": "string", "minLength": 1 }

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reject whitespace-only omission reasons. minLength:1 accepts reason:' '; for requiredComponents:['scheduler'] with no scheduler, adding that omission passes schema validation and changes semanticCoverage from warn to pass with zero warnings. The same reason schema occurs for edge/path/external-label omissions. Require non-whitespace content in both source/generated validation and add a regression so suppressions satisfy the reasoned-omission contract.

@zhouyuanxinand
zhouyuanxinand force-pushed the codex/fix-architecture-semantic-coverage branch from a9e1f8b to 12f6dae Compare August 31, 2026 15:49
@zhouyuanxinand

Copy link
Copy Markdown
Contributor Author

Addressed the requested changes in 12f6dae:

  • the passing receipt now retains the full represented/missing/omitted requirement ledger;
  • self-path checks fail when the endpoint component is absent;
  • omission reasons must contain a non-whitespace character;
  • warnings now include concrete evidence and supportedFixes;
  • the branch is rebased onto current main, and archify.zip was rebuilt twice with canonical Node 22 bytes (matching SHA-256 both times).

The focused regression suite passes 7/7, the skill validator passes, and the archive entries match the committed sources. The full local suite is constrained by Windows symlink/Unix-mode/Bash environment failures; please use the Linux CI result for the complete gate and re-review when ready.

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head 12f6daedf15d0c20a0fccbd0668c614564049029. The prior receipt-ledger, missing self-path, whitespace omission, structured-diagnostic, and ZIP byte issues are fixed. One core scope gap still prevents this PR from closing #237 as written.

Spec — derive discovered repository facts instead of requiring authors to restate every omission

Issue #237 asks for semanticCoverage over inferred repository relationships and specifically says a discovered lifecycle scheduler or background worker that is absent from the diagram should warn. The implementation explicitly defines coverage as an “explicit authoring contract, not repository inference” (archify/renderers/architecture/semantic-coverage.mjs:97-102) and returns null whenever the author did not already add semanticChecks.

This reproduces through the public CLI. I created a real Git repository containing tracked src/api.js and src/scheduler.js, pinned the diagram's meta.repository to that exact revision, referenced only src/api.js, omitted the scheduler from the diagram, and ran:

archify validate architecture input.json --repo-root <repo> --json

It exits 0 with ok: true, all 9 artifact checks passing, composition.status: "pass", 0 warnings, no issues, and no semanticCoverage field. In other words, the feature can audit only facts an author already knows and duplicates into required*; it cannot catch the issue's primary failure mode—an important repository fact the author did not know they omitted.

Keep semanticChecks as the ledger if useful, but populate or verify it from a repository-discovery result (or require a completeness-checked evidence manifest) so the fixture above emits a non-blocking scheduler warning. Add that public --repo-root regression. If the intended scope is only an author-supplied checklist validator, narrow the claim and do not close #237 without an explicit maintainer decision.

Verification on a synthetic merge with current main 199360cc6687a7857b54dd188d4922b09e466a4b: full suite 998 passed, 0 failed, 31 skipped; focused semantic/CLI/delivery suite 53/53; git diff --check and generated-validator freshness pass. The archive hash matches the PR description and all 77 canonical payload members match source. These green checks confirm the implementation is internally consistent, but do not cover the missing discovery path above. Current remote head has no CI and is behind main; those remain separate merge gates.

Add opt-in, non-blocking coverage diagnostics for required components, edges, paths, and external relationship labels. Include reasoned omissions, artifact receipts, authoring guidance, regression tests, and the rebuilt skill archive.\n\nRefs tt-a1i#237
@zhouyuanxinand
zhouyuanxinand force-pushed the codex/fix-architecture-semantic-coverage branch from 12f6dae to b735a95 Compare September 1, 2026 07:35
@zhouyuanxinand

Copy link
Copy Markdown
Contributor Author

Addressed the current review on the rebased head b735a95.

  • Repository-backed Architecture validation now reads the pinned commit tree and conservatively discovers non-test runtime source paths containing scheduler, cron, worker, or background.
  • A discovered lifecycle source that is not represented by component source evidence or identity now emits the non-blocking architecture/semantic-discovered-lifecycle-component warning, even when the author supplied no semanticChecks ledger.
  • Added the machine-readable repository-component omission kind, keyed by the exact discovered repository path and requiring a reason.
  • Added the requested public archify validate architecture <input> --repo-root <repo> --json regression: a tracked src/scheduler.js omitted from the diagram warns; a reasoned omission clears the warning.

Validation completed: generated-validator freshness; focused semantic-coverage and repository-evidence tests; Node 22 deterministic ZIP rebuild; and extracted-package smoke on Windows. The full runner also exposes pre-existing upstream snapshot/environment failures in unchanged files (examples/web-app.html template freshness and Windows Git \\.\\nul fixture setup), so those were not folded into this focused PR.

@sunsunsun-java

Copy link
Copy Markdown
Collaborator

感谢你为这个 PR 投入的大量时间,也感谢你认真处理了多轮审核意见。其中可审计清单、结构化诊断和显式省略理由等设计很有价值。
不过,目前改动已经扩展到仓库发现、Schema、渲染、验证、文档和发布包等多个层面,而核心边界仍未解决:架构图是否完整,需要结合 Agent 对仓库的理解、用户确认的范围及部署部署上下文,无法由渲染器仅通过文件名可靠判断。
继续完善需要重新设计,而不再是局部修审核修改。因此我们先关闭这个 PR,避免继续投入过大的修改成本。后续可以将“Agent 发现架构事实”和“确定性检查图中缺失项”拆分为更小、更清晰的方案。
再次感谢你的贡献与耐心。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add semantic coverage checks for repository-backed architecture diagrams

3 participants