Add transactional semantic domain sense-count collection - #4340
Add transactional semantic domain sense-count collection#4340imnasnainaec wants to merge 5 commits into
Conversation
Introduce a cached per-project semantic domain sense-count collection and a transactional repository. Writes take a client session so they can run inside the same transaction as the Frontier word write that changed them, keeping the counts atomically in sync. A unique (projectId, domainId) index enforces one document per pair and ensures the collection exists before the first upsert. Includes integration tests exercising upsert/increment, decrement, delete, and unique-index enforcement against a single-node replica set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register ISemanticDomainCountRepository in dependency injection. Use a singleton lifetime so the repository's unique-index creation runs once per process rather than on every word edit, since it is a dependency of the per-word-operation WordRepository. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire semantic domain sense-count deltas into WordRepository's transactional *WithSession helpers so the cached counts commit or roll back atomically with the Frontier word write that changed them: increment on create/restore, decrement on delete/update. ReplaceFrontier and RevertReplaceFrontier inherit correct counts by composing these primitives. DeleteAllFrontierWords now runs in a transaction that also clears the project's counts. Adds integration tests covering the increment, decrement, update, replace, restore, and clear paths, plus a rollback test proving an aborted transaction leaves the counts unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Serve the semantic domain statistics and the domain word-count endpoint from the pre-aggregated ProjectSemanticDomainCount collection instead of scanning the Frontier: StatisticsService.GetSemanticDomainCounts and WordController's GetDomainWordCount now query the count repository, and the now-unused WordRepository.CountFrontierWordsWithDomain is removed. Adds a SemanticDomainCountRepositoryMock and updates the statistics and word controller tests to seed cached counts. Introduces a shared MongoDB set-up fixture so the repository integration fixtures use one replica set, and waits for a writable primary in the test runner so index creation on start-up does not race the election window. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a one-shot mongosh script that rebuilds SemanticDomainCountCollection from the current Frontier, so existing projects get correct cached counts. The script is idempotent and creates the unique (projectId, domainId) index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #4340 +/- ##
==========================================
+ Coverage 75.96% 76.14% +0.18%
==========================================
Files 305 307 +2
Lines 11384 11470 +86
Branches 1411 1416 +5
==========================================
+ Hits 8648 8734 +86
Misses 2332 2332
Partials 404 404
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Redo of #4065 that keeps the cached counts atomically in sync with the Frontier by leveraging the MongoDB transactions added in #4198 (which post-dated #4065).
What
Adds a
ProjectSemanticDomainCountcollection caching, per (project, semantic domain), the number of Frontier sense-occurrences of that domain.StatisticsService.GetSemanticDomainCountsand theGetDomainWordCountendpoint now read from this cache instead of scanning the Frontier.How this differs from #4065
#4065 maintained the counts in a separate service invoked from
WordService, with the count write happening outside the word write's transaction (transactions did not exist when it was written). This version pushes count maintenance intoWordRepository's existing*WithSessiontransaction helpers, so each count delta commits or rolls back atomically with the word write that caused it:ReplaceFrontier/RevertReplaceFrontierinherit correct counts by composing those primitives;DeleteAllFrontierWordsnow runs in a transaction that also clears the project's counts.Because the repository owns the counts, this drops most of #4065's churn: there is no
SemanticDomainCountService, and no changes toWordService,LiftService/LiftMerger, orMergeService(imports and merges already flow through the repository transactions).Backfill
database/backfill-semantic-domain-counts.js(idempotent) populates the collection for existing projects from the current Frontier and creates the unique(projectId, domainId)index. This is not a breaking schema change — see the script header for deployment notes.Testing
SemanticDomainCountRepositoryintegration tests and newWordRepositorytests covering every count path, including a rollback test proving an aborted transaction leaves the counts unchanged.GetDomainWordCountsignature, route, and return type are unchanged; only its summary text differs, which the generator does not emit.Coordination with #4328
This branch was cut from
masterbefore #4328 and overlaps it on test infrastructure:Backend.Tests/Repositories/MongoDbSetUpFixture.cs(a shared single-replica-set fixture) and refactorsWordRepositoryTestsonto it. This PR independently adds a byte-identicalMongoDbSetUpFixture.csand the sameWordRepositoryTestsrefactor, so whichever PR merges second will conflict on those two files. When Move edits to their own collection #4328 lands first, I'll rebase, drop the duplicate fixture, and keep only the incrementalWordRepositoryTestschanges (count assertions + the new repository ctor arg).MongoDbTestRunnerchange here (waiting forhello.isWritablePrimary) is not in Move edits to their own collection #4328 and remains necessary: this PR's repository constructor creates its index in test[SetUp], which races the primary-election window; Move edits to their own collection #4328's fixtures don't write in setup.🤖 Generated with Claude Code
This change is