Skip to content

Add transactional semantic domain sense-count collection - #4340

Draft
imnasnainaec wants to merge 5 commits into
masterfrom
add-semdom-count-collection-txn
Draft

Add transactional semantic domain sense-count collection#4340
imnasnainaec wants to merge 5 commits into
masterfrom
add-semdom-count-collection-txn

Conversation

@imnasnainaec

@imnasnainaec imnasnainaec commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Redo of #4065 that keeps the cached counts atomically in sync with the Frontier by leveraging the MongoDB transactions added in #4198 (which post-dated #4065).

What

Adds a ProjectSemanticDomainCount collection caching, per (project, semantic domain), the number of Frontier sense-occurrences of that domain. StatisticsService.GetSemanticDomainCounts and the GetDomainWordCount endpoint now read from this cache instead of scanning the Frontier.

How this differs from #4065

#4065 maintained the counts in a separate service invoked from WordService, with the count write happening outside the word write's transaction (transactions did not exist when it was written). This version pushes count maintenance into WordRepository's existing *WithSession transaction helpers, so each count delta commits or rolls back atomically with the word write that caused it:

  • increment on create/restore, decrement on delete/update;
  • ReplaceFrontier / RevertReplaceFrontier inherit correct counts by composing those primitives;
  • DeleteAllFrontierWords now runs in a transaction that also clears the project's counts.

Because the repository owns the counts, this drops most of #4065's churn: there is no SemanticDomainCountService, and no changes to WordService, LiftService / LiftMerger, or MergeService (imports and merges already flow through the repository transactions).

Backfill

database/backfill-semantic-domain-counts.js (idempotent) populates the collection for existing projects from the current Frontier and creates the unique (projectId, domainId) index. This is not a breaking schema change — see the script header for deployment notes.

Testing

  • New SemanticDomainCountRepository integration tests and new WordRepository tests covering every count path, including a rollback test proving an aborted transaction leaves the counts unchanged.
  • Full backend suite: 1189 passed / 0 failed.
  • OpenAPI: no generated-client change — the GetDomainWordCount signature, route, and return type are unchanged; only its summary text differs, which the generator does not emit.

Coordination with #4328

This branch was cut from master before #4328 and overlaps it on test infrastructure:

  • Move edits to their own collection #4328 introduces Backend.Tests/Repositories/MongoDbSetUpFixture.cs (a shared single-replica-set fixture) and refactors WordRepositoryTests onto it. This PR independently adds a byte-identical MongoDbSetUpFixture.cs and the same WordRepositoryTests refactor, so whichever PR merges second will conflict on those two files. When Move edits to their own collection #4328 lands first, I'll rebase, drop the duplicate fixture, and keep only the incremental WordRepositoryTests changes (count assertions + the new repository ctor arg).
  • The MongoDbTestRunner change here (waiting for hello.isWritablePrimary) is not in Move edits to their own collection #4328 and remains necessary: this PR's repository constructor creates its index in test [SetUp], which races the primary-election window; Move edits to their own collection #4328's fixtures don't write in setup.

🤖 Generated with Claude Code


This change is Reviewable

imnasnainaec and others added 5 commits July 22, 2026 13:47
Introduce a cached per-project semantic domain sense-count collection and a
transactional repository. Writes take a client session so they can run inside
the same transaction as the Frontier word write that changed them, keeping the
counts atomically in sync. A unique (projectId, domainId) index enforces one
document per pair and ensures the collection exists before the first upsert.

Includes integration tests exercising upsert/increment, decrement, delete, and
unique-index enforcement against a single-node replica set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register ISemanticDomainCountRepository in dependency injection. Use a
singleton lifetime so the repository's unique-index creation runs once per
process rather than on every word edit, since it is a dependency of the
per-word-operation WordRepository.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire semantic domain sense-count deltas into WordRepository's transactional
*WithSession helpers so the cached counts commit or roll back atomically with
the Frontier word write that changed them: increment on create/restore,
decrement on delete/update. ReplaceFrontier and RevertReplaceFrontier inherit
correct counts by composing these primitives. DeleteAllFrontierWords now runs
in a transaction that also clears the project's counts.

Adds integration tests covering the increment, decrement, update, replace,
restore, and clear paths, plus a rollback test proving an aborted transaction
leaves the counts unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Serve the semantic domain statistics and the domain word-count endpoint from
the pre-aggregated ProjectSemanticDomainCount collection instead of scanning
the Frontier: StatisticsService.GetSemanticDomainCounts and WordController's
GetDomainWordCount now query the count repository, and the now-unused
WordRepository.CountFrontierWordsWithDomain is removed.

Adds a SemanticDomainCountRepositoryMock and updates the statistics and word
controller tests to seed cached counts. Introduces a shared MongoDB set-up
fixture so the repository integration fixtures use one replica set, and waits
for a writable primary in the test runner so index creation on start-up does
not race the election window.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a one-shot mongosh script that rebuilds SemanticDomainCountCollection from
the current Frontier, so existing projects get correct cached counts. The
script is idempotent and creates the unique (projectId, domainId) index.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4cf0aa70-da1b-4fad-9194-4e4cbb83408e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch add-semdom-count-collection-txn

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.14%. Comparing base (634121d) to head (4dedc57).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4340      +/-   ##
==========================================
+ Coverage   75.96%   76.14%   +0.18%     
==========================================
  Files         305      307       +2     
  Lines       11384    11470      +86     
  Branches     1411     1416       +5     
==========================================
+ Hits         8648     8734      +86     
  Misses       2332     2332              
  Partials      404      404              
Flag Coverage Δ
backend 87.43% <100.00%> (+0.21%) ⬆️
frontend 66.88% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant