feat(plugins): capture harness events as producer envelopes (#997)#998
feat(plugins): capture harness events as producer envelopes (#997)#998sourrris wants to merge 1 commit into
Conversation
…velopes (basicmachines-co#997) Signed-off-by: sourrrish <officialsourish0@gmail.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 72c08126f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if _hook_dir: | ||
| _shared_dir = os.path.join(_hook_dir, "..", "..", "shared") | ||
| sys.path.insert(0, os.path.normpath(_shared_dir)) |
There was a problem hiding this comment.
When the hooks run from the installed plugin package (for example the documented Claude sparse install includes .claude-plugin plugins/claude-code, and the Codex manifest points at plugins/codex), there is no sibling plugins/shared directory in that package. This lookup therefore points outside the packaged plugin, the ImportError is swallowed, and _HAS_ENVELOPE stays false, so installed users never get envelope frontmatter/provenance or captureEvents logging. Vendor the helper inside each plugin or include plugins/shared in the packaged/installable source.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
Implements an opt-in “harness WAL” for Claude Code and Codex by normalizing selected harness hook events into a shared producer-envelope shape, stamping checkpoints with provenance/idempotency metadata, and optionally logging raw envelopes locally for later coalescing.
Changes:
- Added a stdlib-only
HarnessEnvelopemodule with idempotency + redaction helpers and a local JSONL event log writer. - Integrated envelope creation into Claude Code and Codex SessionStart / PreCompact hooks (provenance stamping + optional WAL logging).
- Added forward-compat
ToolLedgerpicoschemas and documented/configured the feature in Claude Code.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| plugins/shared/harness_envelope.py | New shared envelope + redaction + idempotency + JSONL WAL append/rotation helper. |
| plugins/codex/hooks/session-start.py | Logs session_started envelopes locally when enabled. |
| plugins/codex/hooks/pre-compact.py | Stamps Codex checkpoints with provenance/idempotency and optionally logs envelopes. |
| plugins/codex/schemas/tool-ledger.md | Adds ToolLedger schema for future PostToolUse support. |
| plugins/claude-code/hooks/session-start.sh | Passes hook dir into heredoc Python and logs session_started envelopes locally when enabled. |
| plugins/claude-code/hooks/pre-compact.sh | Passes hook dir into heredoc Python, stamps checkpoints with envelope metadata, optionally logs envelopes. |
| plugins/claude-code/schemas/tool-ledger.md | Adds ToolLedger schema for future PostToolUse support. |
| plugins/claude-code/settings.example.json | Documents new captureEvents/redaction/retention settings. |
| plugins/claude-code/DESIGN.md | Documents Harness WAL design, envelope shape, config, and behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if isinstance(value, str): | ||
| # Check for environment-secret-like values | ||
| if SECRET_VALUE_RE.match(value): | ||
| result[key] = "[REDACTED]" | ||
| continue | ||
|
|
||
| # Check for denied paths |
| line = envelope_to_json(envelope) + "\n" | ||
| with open(log_path, "a") as fh: | ||
| fh.write(line) | ||
|
|
||
| # --- Rotation check --- | ||
| # Trigger: log exceeds the cap. Why: unbounded growth would fill disk | ||
| # on long-running projects. Outcome: keep the newest half. | ||
| try: | ||
| with open(log_path) as fh: | ||
| lines = fh.readlines() | ||
| if len(lines) > cap: | ||
| keep = lines[len(lines) // 2 :] | ||
| with open(log_path, "w") as fh: | ||
| fh.writelines(keep) |
| # --- Log the event locally for coalescing --- | ||
| # Trigger: captureEvents is enabled. Why: the local event log feeds future | ||
| # memory routines (SPEC-61) without requiring the note to carry every detail. | ||
| if _HAS_ENVELOPE and envelope and capture_events: | ||
| append_to_event_log(envelope, str(cwd)) |
| cwd=str(cwd), | ||
| project_hint=primary_project, | ||
| hook_name="SessionStart", | ||
| ) | ||
| append_to_event_log(envelope, str(cwd)) |
| if _HAS_ENVELOPE and envelope and capture_events: | ||
| append_to_event_log(envelope, cwd) |
| cwd=cwd, | ||
| project_hint=primary_project, | ||
| hook_name="SessionStart", | ||
| ) | ||
| append_to_event_log(envelope, cwd) |
| - Is append-only (no reads during hook execution) | ||
| - Is capped at 1000 lines (configurable via `eventRetention`); oldest half rotates out |
|
Thanks for taking this on. I think the overall direction is right: a small shared producer-envelope layer, opt-in event capture, provenance on checkpoints, and deferring tool-level capture until we have stable hooks all match what we want for #997. I do not think we can merge this version yet, mostly because the implementation needs to be safe in installed plugin packages, not just from the repo checkout. Could you tighten the v0 around these points?
Please also address all Codex automated review comments before re-review. I would prefer v0 stay narrow: |
Summary
This PR implements issue #997 by introducing a shared, opt-in "harness WAL" path for the Claude Code and Codex plugins. It normalizes supported hook events (
session_started,compaction_imminent) into Basic Memory producer envelopes and stamps checkpoints with provenance and idempotency metadata.Proposed Changes
New Files
HarnessEnvelopedataclass, factory, idempotency logic, redaction utils, and event logging logic.Modified Files
plugins/claude-code/hooks/pre-compact.sh&session-start.sh:BM_HOOK_DIRenvironment variable passing to resolve the real hook directory inside Python inline heredocs (__file__resolves to<stdin>otherwise).harness_envelopeto create envelopes, stamp provenance observations, add idempotency metadata to frontmatter, and log events locally ifcaptureEventsis enabled.plugins/codex/hooks/pre-compact.py&session-start.py:harness_envelopein the Python-native hooks.plugins/claude-code/settings.example.json:captureEvents(defaults tofalse),redactKeys,redactPaths, andeventRetention.plugins/claude-code/DESIGN.md:Verification Plan
just package-check(Passed successfully)just package-check-claude-code(Passed successfully, verified Claude Code manifest schema)Closes #997