feat: Coalesce link extraction - #11611
Conversation
The coalescing queue holds page ids, so the drain re-reads the page from the database (handlePageUpsertById) instead of using the emitted document. The fixture assigned page.revision in memory only, leaving the drain to extract from the previous revision, so no outbound rows were written and the five lifecycle tests timed out. PageService commits the revision via pushRevision before emitting, so persisting the pointer makes the fixture match production. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A soft delete rewrites the page document in place and keeps its _id, so the drain's existing "page is gone" check does not catch it: handlePageUpsertById still found the page and indexed a source now sitting under /trash, leaving rows a reader could surface as phantom backlinks. Coalescing is what opened this window. Before it the upsert ran synchronously inside the create/update callback, so a delete could not race it. Fetch status alongside the body and decline to index a deleted page. Keyed on STATUS_DELETED rather than STATUS_PUBLISHED because a legacy page carries a null status, which GROWI treats as published; a test pins that so the guard is not later tightened into un-indexing those pages. Verified by mutation check: with the guard removed the new test fails with the row it would have written. This does not complete B2.2. Clearing the rows a page already owned when it was trashed is reconciliation (B5.2), which is still interface-only, so the spec's "delete supersedes a pending upsert" criterion remains unmet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Tick the box to add this pull request to the merge queue (same as
|
extract-internal-links.ts was statically reachable from crowi/index.ts (-> PageLinkService -> handlers), so the unified / remark / rehype stack loaded at every server start. Measured in the built artifact, it adds ~16 MiB RSS on top of the rest of the boot graph (31 MiB in isolation; ~15 MiB of that is shared with mongoose / the page model and paid regardless), and every deployment paid it forever -- including processes that never save a page, such as the server:ci boot check. Extraction only runs on the backlinks drain timer, off the request path, so the stack is now loaded with dynamic import() at the point of use: the ~80 ms first load lands inside a detached timer nobody waits on, and Node's module cache makes later extractions ~0.04 ms. The local ~/services/renderer plugins move too -- relative-links and relative-links-by-pukiwiki-like-linker each reach hast-util-select independently, so leaving either static would keep part of the stack eager. Boot chains into the pipeline: 8 -> 0. Guarded by no-eager-markdown-imports.spec.ts, which walks two roots per .claude/rules/server-boot-imports.md. Mutation-checked before commit: a static 'unified' import in extract-internal-links.ts turns both walks RED; a static 'remark-parse' import in the boot-reachable route file turns only the boot walk RED (the route is not reachable from the backlinks entry), which is why both roots are needed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…t types
PageLinkModel declared reResolveByToPath and reconcileDeletedPages ahead
of their B4/B5 implementations. A declaration without an implementation
is worse than no declaration: PageLink.reResolveByToPath('/x') type-checks
and then throws "is not a function" at runtime. Declare each static when
the schema actually implements it.
ILinkTarget and LinkTargetState had no references anywhere in src/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GET /api/v3/page/backlinks had no annotation, so it was absent from the generated spec while every sibling /page/* route documents itself. The annotation alone would not have been enough: generate-spec-apiv3.sh lists feature route globs one by one and features/backlinks was not among them, so swagger-jsdoc would never have read the block. Add the glob too. Uses @Swagger, matching the 53 existing annotations in this codebase (the @openapi form in AGENTS.md is not used anywhere). Verified with `pnpm run lint:openapi:apiv3`: spec validates, and /page/backlinks appears in the output with its ObjectId $ref resolving. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… B3.5 Found while reviewing B2.2. A recursive duplicate bulk-inserts the copied descendants (PageService.duplicateDescendants -> Page.insertMany) with no per-page event, so their outbound links are never extracted. The duplicated root is already covered -- it goes through PageService.create, which emits 'create'. Recorded rather than fixed because it is not fixable inside this feature: 'duplicate' carries the source page and fires before duplicateDescendantsWithStream runs, so the copies do not yet exist and their ids are never published in any event. The Elasticsearch index has the same blind spot, so the fix belongs in PageService and should be decided for search and backlinks together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… endpoint
apiv3Err turns a raw Error into ErrorV3(err.message), and `message` is an
enumerable own property, so `res.apiv3Err(err, 500)` put driver and query
internals in the response body. Verified by mutation: a rejected
findBacklinks answered
{"errors":[{"code":null,"message":"E11000 duplicate key error
collection: growi.pagelinks"}]}
Answer with an ErrorV3 carrying a stable code instead, keep the detail in
the server log, and add the missing `return` so the catch matches the other
exit paths. Same treatment for the 400 type guard.
The pre-existing "answers 500" test stays green under that mutation, so it
could not have caught this; the new assertion checks what must be *absent*
from the body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… the service PageLinkService held two unrelated responsibilities that shared nothing but the Crowi reference: the permission-filtered read query, and the coalescing write-side queue with its timer state. Split by responsibility domain, per .claude/rules/coding-style.md: - find-backlinks.ts — the read query, no Crowi needed - page-link-upsert-queue.ts — the pacing queue; takes getSiteUrl as a callback rather than a Crowi, so it is unit-testable on its own - page-link-service.ts — thin adapter keeping only what needs a Crowi (event subscription, config access) Behaviour-preserving: the public surface (PageLinkService.create, findBacklinks) is unchanged, so the route needs no edit and all 8 existing queue tests pass untouched apart from moving the DRAIN_INTERVAL_MS / MAX_PAGES_PER_DRAIN import. The lifecycle integration test, which drives real events through the queue to the database and back, also passes unchanged. no-eager-markdown-imports.spec.ts was re-mutation-checked after the move: both walks still go RED, now reporting the chain through page-link-upsert-queue.ts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fromPage carried its own index while also being the prefix of the unique
{ fromPage, toPath } compound index, which already serves every
fromPage-only lookup — replaceOutboundLinks' per-row upsert filter and its
`toPath: { $nin }` delete included. Pure write and storage overhead.
toPath's standalone index has no query at all yet. B4's re-resolve-by-path
introduces one and should add the index alongside it; the compound cannot
serve toPath alone, since it is the wrong prefix.
Remaining indexes are exactly the two that are used: the unique compound,
and toPage for findBacklinkSources. The collection is unreleased, so no
migration is needed to drop the old ones (a deployment that ran a
pre-release build keeps a harmless unused fromPage_1 until reindexed).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A share-link request carries no user and no isSharedPage flag, so GET /page/backlinks 403s for it and the panel could only ever render its error state. The tab and its dropdown entry were offered anyway. Gate on isSharedUser only, not isGuestUser: the endpoint deliberately admits guests so backlinks stay readable on a public wiki (asserted in backlinks.spec.ts). Mirrors how the History tab guards itself — isLinkEnabled on the modal mapping, disabled on the dropdown item. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… criterion to B5.2
B2.2 dropped the standalone {fromPage} and {toPath} indexes, but two places
still described four. B2.1 told the benchmark to check indexes "in
particular {toPage}", which would read their absence as a missing-index
finding; it now names the two that ship and why the others are gone.
B2.2's own rationale keeps its argument but stops enumerating a stale set.
B2.2 was checked off while claiming "a delete during a pending upsert
results in reconcile, not a re-created row". What it ships is narrower: the
drain re-checks status and declines a page that is now STATUS_DELETED, which
closes the window coalescing opened but leaves the rows the page already
owned when it was trashed. Its criterion now states that, and the reconcile
half moves to B5.2 (with req 3.5 added to its trace). The queue-side half —
dropping the id from the dirty set — is listed under B5.3 instead, since it
needs the PageLinkService boundary that task owns, not page-link-sync.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Requirement 3.4 asked for evidence that a heavily-linked hub page's backlinks
come back in interactive time. Measured: median 127 ms against 100,001 pages /
205,000 link rows with a 5,000-inbound hub, ~8x under the 1 s target. Raising the
hub to 20,000 inbound costs only ~1.5x (189 ms), so the shape is sub-linear and
the budget is not at risk at any realistic scale.
Both read queries are index-backed with no COLLSCAN: the distinct rides toPage_1,
the viewer filter rides _id_, and the save-path delete filter ({fromPage, toPath:
{$nin}}) rides fromPage_1_toPath_1 — so the two indexes B2.2 kept are sufficient
and no index/query fix falls out of this task. The no-rescan guarantee is checked
directly: one syncOutboundLinks issues exactly one bulkWrite whose every operation
filter is scoped to the edited fromPage, a sibling page's rows are byte-identical
afterwards, and the collection total moves only by that page's delta.
Written as an env-gated integ test rather than a CI test or a standalone script
(the decision B2.1 asked to make first). It is collected by the app-integration
project but skipped unless BACKLINKS_PERF is set, so CI never pays the 5 s seed,
while the measurement stays in-tree and reuses the crowi harness — it calls the
real findBacklinks instead of a hand-rolled copy that could drift. It expects a
real MongoDB via MONGO_URI; the default in-memory server would produce numbers
that say nothing about a deployment. The harness rewrites the db name to
growi_test_<workerId>, so the dev database is never touched, and the seed is
deleted by id in afterAll.
The seed also asserts the result set (4,000 of 5,000 sources visible under a mixed
grant/status distribution), so it doubles as a correctness check at a scale the
other integration tests do not reach.
Recorded in tasks.md with the reproduction command, the environment, and one
observation left deliberately unactioned: the distinct is FETCH <- IXSCAN rather
than a covered DISTINCT_SCAN, because its key (fromPage) is not in the index it
rides. A {toPage, fromPage} compound would cover it, but that is 10% of a read
with 8x headroom and would re-add per-save write cost B2.2 just removed.
Depends on B2.2 (#11611): the index assertions encode the two-index state that
branch introduced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PR #11611 レビュー: feat: Coalesce link extraction
概要タイトルは B2.2(ライブ抽出の coalescing)ですが、実際には backlinks 機能の複数の改善がまとまっています。
良かった点
🔴 Must fix1. 500 のエラーが「クライアントにも出ない、ログにも出ない」apps/app/src/features/backlinks/server/routes/backlinks.ts:37 logger.error('Failed to get backlinks', err); // ← err が捨てられるGROWI のロガーは pino( 実証:
この行自体は既存だが、本 PR がレスポンスからエラーを外したことで、この行が唯一の診断手段になったためセットで直す必要がある。同 feature の page-link-service.ts:55 は logger.error({ err }, 'Failed to get backlinks');2. spec と実装の乖離 — B2.2 の Done-when が未達のまま
|
| 内容 | 結果 |
|---|---|
unit(no-eager-markdown-imports / page-link-service.spec / backlinks.spec / extract-internal-links / backlinks.spec.tsx) |
52 passed |
integ(page-link-service-handlers.integ) |
11 passed |
| drift spec の mutation check | RED を確認 → revert 済み(working tree クリーン) |
| OpenAPI 生成 | 成功、/page/backlinks 出力を確認 |
| pino の引数順(#1) | 実証済み |
| biome check(backlinks 配下) | 新規指摘なし(既存の noDefaultExport 1 件のみ) |
page-link-lifecycle.integ / page-link-service.integ |
ローカル実行不可 — page-bulk-export の bulk-export.generated 未生成のため(本 PR と無関係)。CI での結果確認が必要 |
…eturn `extractInternalLinks` returns resolved page paths, not link nodes, and `resolveToPages` returns a Map of page ids rather than page documents — both names invited the wrong mental model at the call site. Rename to `extractInternalLinkPaths` (file: extract-internal-link-paths.ts) and `resolveToPageIds`, and update the handlers and specs that reference them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Names: the renamed `extractInternalLinkPaths` / `resolveToPageIds` throughout, `getBacklinksHandlerFactory (routes/backlinks.ts)` for the read endpoint, and the batched resolver contract (paths -> Map, unresolved inputs absent from the map) in place of the per-link `toPath -> id | null` signature that was never implemented. Extraction is documented as async, with the reason (the markdown stack is lazy-loaded to keep it out of the boot graph). File Structure Plan: reconciled against the tree — services/ (not service/), the split interfaces/, the three modules the plan never listed (page-link-service-handlers, page-link-upsert-queue, find-backlinks), and story markers on what is not built yet (B3 backfill, B4.2/B5.2 sync ops, B5.5 badge, B5.6 forward-health section). `ILinkTarget` and `LinkTargetState`: B1.1 planned to declare them up front but B1 shipped without them, so they now belong to the tasks that produce them — the union to B5.1, the DTO to B5.4. Target shapes are unchanged; 6.4 stays covered by B1.8 and B5.4-B5.7. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y loading Review #6 of PR #11611: handlePageUpsert was exported but had exactly one production caller, handlePageUpsertById in the same file. Nothing planned wants the document-taking shape either — B3.2's backfill builds its own {path->_id} map and bulkWrite sink, B3.5 goes through the queue by id, and B4.2/B5.2 are reconcile ops — so the export was effectively test-only. Since the sole caller reads the page with a projection and never populates, the revision is always a ref there: loadBody's isPopulated branch was unreachable and is dropped with it. The integ tests move to handlePageUpsertById and its existing Page/Revision fixtures, which also collapses the extraction/row-sync duplication the reviewer noted between the two describes. The "unpopulated event payload" case goes away with the branch it described — a leftover from the pre-coalescing design where the handler took the emitted document. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The drain interval and per-tick page budget were hardcoded constants in page-link-upsert-queue.ts, so the duty cycle that bounds live extraction could not be tuned per deployment. Add them as env-only config, following the app:vaultDrift* precedent: backlinks:drainIntervalMs BACKLINKS_DRAIN_INTERVAL_MS default 1000 backlinks:maxPagesPerDrain BACKLINKS_MAX_PAGES_PER_DRAIN default 3 The queue receives the pair as constructor input rather than reading config itself, so it stays Crowi-free and unit-testable and CONFIG_DEFINITIONS remains the only place the defaults live. PageLinkService reads them once at construction: unlike app:siteUrl, which admins can change at runtime and which is therefore still a per-drain callback, these are env-only. configManager is loaded at crowi/index.ts:275, long before PageLinkService.create. The queue spec now configures 500 ms / 2 pages — deliberately not the shipped defaults, so the pacing assertions fail if the values are ignored (verified: re-hardcoding 1000/3 turns 7 of its 8 tests red). Note: constructing PageLinkService now requires a loaded configManager, so page-link-service.integ.ts can no longer pass a bare mock<Crowi>(). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
B2.2's duty cycle is now env-tunable rather than a pair of constants, so state the keys, env vars and defaults where design.md and tasks.md describe the lever. Also name handlePageUpsertById as the per-page unit, now that handlePageUpsert is private. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review #9 of PR #11611: `[...this.pagesToUpsert].slice(0, max)` allocated an array of the entire dirty set every tick just to read `max` ids from it, so the per-tick cost scaled with queue depth. A deep queue only happens during a burst, which is the one situation this queue exists to bound — take the batch by iterating with an early break instead. Set iteration is insertion-ordered, so the batch and its FIFO order are unchanged. The loop trusts the budget in a way `slice` did not: `batch.length >= NaN` is never true, so it would never break and one tick would parse the whole queue. That is reachable now that the value comes from an env var, since config-loader parses numeric env vars with a bare `parseInt` and never validates the result — BACKLINKS_MAX_PAGES_PER_DRAIN=three would silently disable the pacing outright. `resolveUpsertQueuePacing` therefore validates both values at the config boundary, falling back per value to the default declared in CONFIG_DEFINITIONS (read from there rather than restated, so the defaults stay single-sourced) and logging what it ignored. Zero and negative budgets are rejected the same way — with those the queue would never drain at all. Tested at the contract level rather than only on the resolver: the queue spec configures NaN and asserts a tick still drains the declared default, so an unwired guard fails there (verified: it is the only failure when the resolveUpsertQueuePacing call is removed). The warning has its own assertion — falling back silently would leave an operator with no way to find the typo. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review #11 of PR #11611: buildPipeline re-ran nine `import()`s and their Promise.all on every extraction. Node caches the modules, so the repeat cost is registry lookups rather than parses, but the await plumbing is pure waste — cache the load promise in a module variable instead. A rejected promise is deliberately not cached: one transient load failure would otherwise disable extraction for the lifetime of the process. The processor stays per call. Both relative-link plugins are configured with the page's own path, so a shared processor would resolve every later page's relative links against the first page it was built for. Every existing case in the spec used the same pagePath, so that mistake would have gone unnoticed — a case that extracts from two different pages now covers it (verified: memoizing the whole pipeline fails it and nothing else). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review #10 of PR #11611: an upsert that never settles leaves `draining` true, so scheduleDrain early-returns for the rest of the process's life while enqueue keeps filling the set — the instance silently stops indexing. Accepted rather than guarded, as the reviewer allowed, and the reason is recorded at the guard and in the spec: a timeout cannot cancel the abandoned run, so it would only race it, and a resurrected stale run would overwrite a newer link set (the upsert is last-writer-wins). Recovery is a restart, after which each page's rows are repaired by its next save or by the B3 backfill — the same path as any other dropped work in this best-effort queue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review of PR #11611 (test section): the drain's STATUS_DELETED guard was only covered by calling handlePageUpsertById directly — the path that actually matters, a create event landing in the queue and the page being trashed before the drain reads it, had no test. Drives it through the wired service: emit create, soft-delete the page in place (path + status, as GROWI does, so the id stays resolvable and the drain still finds it), then assert no rows and no phantom backlink on the target. A second page enqueued in the same tick window supplies the positive signal to wait on — polling for its row proves the batch drained, so the absence assertion cannot pass just because the drain had not run yet. Verified by mutation: dropping `if (page.status === Page.STATUS_DELETED) return` fails this test and nothing else. The afterEach cleanup now also matches /trash… paths, since a soft delete rewrites the path out from under the seed prefix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review #4 of PR #11611: B2.2 removed the redundant {fromPage} and {toPath} index declarations, but Mongoose autoIndex only ever creates — fromPage_1 / toPath_1 survive on any collection already built with them, so dev and staging instances that ran an earlier build keep paying their write cost. Harmless while PageLink is unreleased, which is why no migration ships now. Record that in Migration Strategy together with the rule it implies: once the collection has shipped, removing an index needs a migrate-mongo migration with an explicit dropIndex, not just a schema edit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| 'backlinks:maxPagesPerDrain': defineConfig<number>({ | ||
| envVarName: 'BACKLINKS_MAX_PAGES_PER_DRAIN', | ||
| defaultValue: 3, | ||
| }), |
There was a problem hiding this comment.
drain のペースが固定値というのがあまり良い戦略ではない
1万ページのインポートは 10000/3 ≈ 55分 はキューに残る。
例えば以下のような案はどうか?
案1
「N件/tick」をやめて、1件処理するのにかかった時間を測り、その時間に比例して休む。
const started = performance.now();
await handlePageUpsertById(id, siteUrl);
const workMs = performance.now() - started;
// duty = 0.2 なら「働いた時間の4倍休む」= イベントループ占有率の上限が 20%
await sleep(workMs * (1 / duty - 1));
ページサイズに自動追随する。小さいページは事実上バーストする(1秒で数十件流れる)し、巨大ページは1件ずつ・長めに休む
設定が「占有率の上限」1つに減る。
案X
別に他の案でもよい
検討してほしい
|
miya が指摘したものの中で未対応のものがある 直してほしいもの1. 処理に失敗したページが、更新されないまま終わる
MongoDB 側の一時的なエラー(レプリカセットの切り替え、書き込みのタイムアウトなど)で1ページ分が drain が止まってしまう場合(#10)については「そういうものとして受け入れる」と、コードと 最低限、次のどちらかをお願いします。
2. ページを取り出す前に例外が起きると、1件も処理しないまま同じ失敗を繰り返し続ける
いまの起動順序(設定を読み込んだあとに 3. ゴミ箱との競合を確かめる integ テストが、タイミング次第で失敗する
CI の MongoDB が混んでいて 2 が1秒を超えると、 比較用のページを一緒に登録することで、「実際には drain がまだ走っていないのに、行が無いことを
ついでに直せるもの他にも諸々ありそうなのでチェックしてください 今回は直さなくてよいもの
|
https://redmine.weseek.co.jp/issues/185820
https://redmine.weseek.co.jp/issues/187679
Summary
Replaces the inline per-event link extraction with a coalescing, paced queue (task B2.2), plus the review fixes that came out of it.
Coalescing upsert queue (
page-link-upsert-queue.ts)create/updatehandlers now only mark a page id dirty; a paced timer drains a bounded number of ids per tick (BACKLINKS_DRAIN_INTERVAL_MS/BACKLINKS_MAX_PAGES_PER_DRAIN, default 1000 ms × 3 pages) and awaits one extraction at a time, so an editing burst no longer becomes a blocking spree of markdown parses.handlePageUpsertById).unref()ed — dropped work self-heals on the next edit._id, so the drain would index a source now under/trash. The drain now declines a page withSTATUS_DELETED(keyed on deleted rather than published, because a legacy page'snullstatus means published).Perf
import()at the point of use instead of at server boot (~16 MiB RSS previously paid by every deployment, including processes that never save a page). Extraction runs only on the drain timer, off the request path. Guarded byno-eager-markdown-imports.spec.ts.PageLinkindexes no query uses (fromPageis already the compound index's prefix;toPathhas no query until B4). Unreleased collection, so no migration.Fixes / cleanup
Errortoapiv3Err, which was leaking the driver message into the response body; it answers anErrorV3with a stable code instead.PageLinkServiceby responsibility:find-backlinks.ts(read query),page-link-upsert-queue.ts(pacing, takesgetSiteUrlas a callback so it needs no Crowi), and a thin Crowi-facing adapter. Public surface unchanged.PageLinkModelstatics that were declared ahead of their B4/B5 implementations (they type-checked and threw at runtime) and the unused link-target types.GET /api/v3/page/backlinksin the apiv3 OpenAPI spec, and added thefeatures/backlinksglob thatgenerate-spec-apiv3.shwas missing.Known gap recorded, not fixed: a recursive duplicate bulk-inserts its descendants with no per-page event, so their outbound links are never extracted. Not fixable inside this feature (the
duplicateevent fires before the copies exist) and Elasticsearch has the same blind spot, so it is tracked as task B3.5 to be decided inPageServicefor search and backlinks together.Review fixes
Addressing @miya's review of 2026-07-31. Every numbered item is closed; the ones not listed below (#1 pino argument order, #2 the B2.2 Done-when, #3 stale
design.mdclaims, #5 the share-link rationale, #7 the self-thrownErrorinonUpsert) were already fixed by earlier commits on the branch — #5's wording also corrected in this description above.handlePageUpsertexport is test-only92fe389808handlePageUpsertByIdin the same file, and nothing planned reuses the document-taking shape (B3.2's backfill builds its own{path→_id}map +bulkWritesink; B3.5 goes through the queue by id; B4.2/B5.2 are reconcile ops). The integ tests moved ontohandlePageUpsertById, which also removes the duplication the review noted. With the sole caller reading the page via a projection,loadBody'sisPopulatedbranch was unreachable and is gone.af89ee689e,2e6bcf31dabacklinks:drainIntervalMs/backlinks:maxPagesPerDrain(BACKLINKS_DRAIN_INTERVAL_MS/BACKLINKS_MAX_PAGES_PER_DRAIN, defaults 1000 / 3), following theapp:vaultDrift*precedent. The queue takes the budget as constructor input rather than reading config, so it stays Crowi-free andCONFIG_DEFINITIONSremains the only home for the defaults. Read once at construction: unlikeapp:siteUrl(admin-editable, still a per-drain callback) these are env-only.[...set].slice(0, N)copies the whole queue every tick066d548980Setiteration is insertion-ordered, so the batch and its FIFO order are unchanged. This also made a malformed env var dangerous (batch.length >= NaNnever breaks, so one tick would drain everything), soresolveUpsertQueuePacingnow validates both values at the config boundary, falling back per value to theCONFIG_DEFINITIONSdefault and logging what it ignored.701b4debabtasks.mdB2.2, as the review allowed. A timeout cannot cancel the abandoned run — it would only race it, and a resurrected stale run would overwrite a newer link set (the upsert is last-writer-wins). Recovery is a restart; rows are then repaired by each page's next save or by B3.buildPipelinere-imports nine modules per call5c4c8b520bpagePath, so that mistake passed the whole suite — a two-page case now guards it.ccc893f0dcautoIndexonly ever creates, sofromPage_1/toPath_1remain on any collection already built with them. Harmless whilePageLinkis unreleased (hence still no migration), and Migration Strategy now records the rule that follows: once the collection has shipped, an index removal needs a migrate-mongo migration with an explicitdropIndex.a4b13976ddSTATUS_DELETEDguard was only exercised by callinghandlePageUpsertByIddirectly. Now driven through the wired service: emitcreate, soft-delete the page in place, assert no rows and no phantom backlink. A second page enqueued in the same tick window supplies the positive signal to wait on, so the absence assertion cannot pass merely because the drain had not run yet.On the review's note that the pacing constants were imported from the implementation and therefore not pinned: the queue spec no longer imports anything from the implementation (it configures 500 ms / 2 pages, deliberately not the defaults, so the assertions only hold if the configured values are honoured), and the shipped defaults are pinned as literals in
config-definition.spec.ts. That is stricter than the suggested upper bound — a change to 60000 fails exactly, not just out of range.Each new guard was mutation-checked; the mutation and its single resulting failure are recorded in the commit that adds it.
Drift spec mutation evidence
Required by
apps/app/.claude/rules/server-boot-imports.md. Re-adding a top-levelimport { unified } from 'unified'toextract-internal-link-paths.tsfails both walks with the real chain, while the two entrypoint-existence guards stay green (4 tests | 2 failed):Reverted;
pnpm vitest run no-eager-markdown-importsis green (4 passed).