perf(sync): metadata prefilter for startup optimization

[pablontiv]

Implements #47: avoid re-reading and hashing every input file on every invocation.

Change

• Migration v14 adds file_size INTEGER and file_mtime TEXT to indexed_files (new version block in SetupSchema(); no existing block modified, per the repo's schema migration rule).
• Metadata prefilter in maybeAutoSync: collect size+mtime before hashing and reuse the stored hash when both match.
• Racy-clean guard (isRacyCleanFile, cmd/backscroll/sync_helpers.go): the prefilter is bypassed when a file's mtime is not strictly older than its last_indexed, with a 2-second margin for filesystem granularity.
• v14 lineage added to the compat catalog. Mandatory: without it, the new schema signature is unknown and every operational command fails unsupported_lineage. The fixture is dumped from a real migration run and computes sha256:e4973d30..., matching what SetupSchema() produces.

The racy-clean guard, and why it is required

Without it this optimization silently loses data. A file edited in place with identical length, within the same timestamp tick in which it was indexed, keeps both its size and its mtime. It would be classified unchanged and never re-indexed — no later pass recovers it, and search misses the content permanently with no signal to the user.

This is git's "racily clean" problem, and the guard is git's solution: the reference point is the indexing time, not the wall clock. indexed_files.last_indexed has existed since v1, so no additional schema was needed. The guard is self-limiting — a racy file is hashed once; if unchanged, the next pass records a last_indexed comfortably after its mtime and it stops being racy.

TestRacyCleanEditsAreDetected reproduces the race deterministically (same-length overwrite plus os.Chtimes to force the exact recorded mtime) rather than depending on winning a real one.

Freshness guarantee

A file skips re-hashing only when its size and mtime both match what was recorded and its mtime is at least 2 seconds older than its last_indexed. Everything else is hashed. NULL metadata (all pre-v14 rows) always hashes. Correctness never depends on a watcher.

Timestamp format note

isRacyCleanFile accepts both the SQLite CURRENT_TIMESTAMP layout and RFC3339. This is deliberate: the raw stored text is 2006-01-02 15:04:05, but the column is declared DATETIME and modernc.org/sqlite converts it on read, so Go actually receives RFC3339. Inspecting the file with the sqlite3 CLI shows the raw form and misleadingly suggests a SQLite-layout parse is required — a parse restricted to that layout silently disables the guard entirely. This trap misled three separate reviewers before being pinned down; the tolerance and the code comment exist so it does not recur.

Performance — measured vs unmeasured

Measured, observed output:

$ go test -bench=. -benchmem -run '^$' ./cmd/backscroll
BenchmarkMetadataPrefilter-16    91    13115000 ns/op    8607722 B/op    25463 allocs/op

That is the prefilter path over a synthetic 100-file corpus, at ~13.1 ms/op.

Not measured, and previously asserted in this PR without evidence: the ~10s baseline, the ~2-4s post-optimization figure, the 60-80% improvement, and the 95% skip rate. Those referred to a 1,528-file / 1.08 GiB corpus that no benchmark here covers, and there is no before/after comparison in the suite. They have been removed rather than restated. The mechanism — unchanged, non-racy files skip a full SHA-256 read — is sound, but its magnitude on the real corpus is unquantified in this PR.

Issue #47's acceptance criterion asking for per-phase time and bytes reporting is therefore only partially satisfied.

Verification gate

Observed output of just ci:

ok  github.com/pablontiv/backscroll/internal/sync         3.338s  coverage: 88.8% of statements
ok  github.com/pablontiv/backscroll/internal/tagging      3.327s  coverage: 100.0% of statements
ok  github.com/pablontiv/backscroll/internal/templates    3.318s  coverage: 91.8% of statements
ok  github.com/pablontiv/backscroll/scripts/calibration-extract  3.410s  coverage: 66.0% of statements
Coverage: 85.4%

GitHub CI green on ed5ae30, confirmed against the branch head via git ls-remote.

Tests

• TestRacyCleanEditsAreDetected — same-length same-tick edit IS detected.
• TestMetadataPrefilterSkipsHashingForNonRacyUnchangedFiles — the optimization actually engages for unchanged non-racy files. Every other test asserts that changed files are re-hashed; none asserted that unchanged files are not. That asymmetry is why a guard that classified everything as racy — silently disabling the feature while keeping CI green — went unnoticed. This test is what caught it.
• Truncation, replacement, and NULL-metadata cases retained.

Closes #47

https://claude.ai/code/session_019LDXzStaKrArqJKvy4z3eF