memory_daemon.py: extraction always targets the wrong transcript (session IDs can never match JSONL filenames)

[jackhamlin-hue]

Summary

extract_memories() in scripts/core/memory_daemon.py can never match a session to its own
transcript, so every extraction falls through to the "most recent JSONL" fallback — which in
practice is the transcript of a session that is still live. Learnings are mined from the wrong
conversation, and several different stale sessions all get extracted against the same file.

Root cause

Session IDs are generated in .claude/hooks/src/session-register.ts as a base36 timestamp:

return `s-${Date.now().toString(36)}`;   // e.g. s-mtxb59p5

Transcripts under ~/.claude/projects/<project>/ are named with UUIDs, e.g.
d18523bd-….jsonl.

extract_memories() tries to pair them with:

for f in all_jsonls:
    if session_id in f.name or f.stem == session_id:

An s-<base36> string can never be a substring of a UUID filename, so this branch never
matches for any session. Control therefore always reaches the fallback:

# Fallback: Use most recent JSONL if no ID match (common with truncated IDs)
recent_threshold = datetime.now() - timedelta(minutes=10)

A JSONL modified within the last 10 minutes is, almost by definition, one that is still being
appended to — i.e. a live session, frequently the very session whose activity triggered the poll.

Observed

Log from a clean run on main @ d07ff4b (PostgreSQL backend, macOS, Python 3.12):

[16:07:14] Found 4 stale sessions
[16:07:14] Extracting memories for session s-mtxb59p5 in <project>
[16:07:14] Using recent JSONL d18523bd-….jsonl for session s-mtxb59p5 (no ID match)
[16:07:14] Started extraction for s-mtxb59p5 (pid=26011, active=1)
[16:07:14] Extracting memories for session s-mtxb5b24 in <project>
[16:07:14] Using recent JSONL d18523bd-….jsonl for session s-mtxb5b24 (no ID match)
[16:07:14] Started extraction for s-mtxb5b24 (pid=26013, active=2)

Note both sessions resolve to the same JSONL, and that JSONL belonged to the session that
was active at the time — not to either s-mtxb59p5 or s-mtxb5b24.

Impact

This is worse than extracting nothing: learnings get attributed to sessions they did not come
from, and the live conversation is repeatedly mined under the identity of various "ended"
sessions. Because archival_memory rows carry a session_id, the provenance recorded is
wrong.

Suggested fix

Record the transcript path at registration instead of guessing it later. Hooks already receive
transcript_path on stdin — five hooks in this repo use it today (session-outcome.ts,
session-end-cleanup.ts, pre-compact-continuity.ts, skill-activation-prompt.ts,
tldr-read-enforcer.ts). So session-register.ts could persist it next to the session row
(e.g. a transcript_path column on sessions), and extract_memories() could read it
directly and skip the glob entirely.

Failing that, the fallback should at least refuse a JSONL that is still being written to, rather
than preferring it.

Related

• memory_daemon.py: Zombie process reaping and timezone bugs #136 (zombie reaping) masks this in practice: the queue wedges at MAX_CONCURRENT_EXTRACTIONS
  after two spawns, so most sessions never reach extraction at all. Fixing memory_daemon.py: Zombie process reaping and timezone bugs #136 alone would make
  this bug fire much more often.
• Interacts with heartbeat cadence: if last_heartbeat is not refreshed while a session is
  merely idle, live sessions cross the 5-minute stale threshold and get queued for extraction —
  which is how the live transcript ends up being the "most recent JSONL" in the first place.