Skip to content

Implement synonym map support (closes #69) - #79

Merged
paulirwin merged 1 commit into
mainfrom
issue/69-synonym-maps
Aug 20, 2026
Merged

paulirwin merged 1 commit into
mainfrom
issue/69-synonym-maps

Conversation

@paulirwin

Copy link
Copy Markdown
Member

Closes #69.

SearchField.SynonymMaps was modelled and parsed but never used, and there were no /synonymmaps routes at all — so an index could name a synonym map, be accepted, and then search exactly as though it had not. /servicestats reported the map count as a hardcoded 0.

Approach

Synonym maps are service-level resources, not index sub-objects — unlike analyzers and normalizers, which live inside the index definition. So this follows ISearchIndexRepository rather than the normalizer template: maps get their own routes, their own {name}.synonymmap.json files beside the index definitions, and their own lifetime. A field opts in by naming one. That indirection is the point of the feature — one map is edited once and takes effect across every field of every index that names it.

Expansion happens at query time only, as it does in Azure. The maps are resolved per search and layered onto the per-field search analyzer, so a field's own analyzer still decides how text is split before any synonym is considered; there is deliberately no index-time equivalent. Expanding while indexing would bake the current rules into the stored terms, leaving documents indexed before an edit disagreeing with those after it — repairable only by a full reindex. Doing it at query time is what makes a map safe to edit, and EditedSynonymMap_TakesEffectWithoutReindexing covers exactly that.

Rules are parsed with Lucene's SolrSynonymParser — the format, and the only format, Azure supports. Both rule forms behave as the service documents:

  • Equivalency (usa, united states) keeps the term that was typed and adds the alternatives.
  • Explicit mapping (dog => canine) replaces it, so dog no longer matches documents holding only dog.

Lucene.Net.Analysis.Common was already referenced, so no new package.

Judgement calls worth a look

A missing map is skipped, not thrown on. Azure refuses to delete a map while an index still names it. The emulator's indexes and maps are separate files a user may edit or restore independently, so failing every search against the index seemed a harsh answer to a dangling name whose only effect is to widen results. A map that is missing at index-creation time is still rejected outright — the mistake is reported where it can be acted on.

SynonymFilter is constructed with ignoreCase: false. This looks wrong at first glance and I verified it empirically before settling on it: the rules are lower-cased when parsed, and any lower-casing field analyzer — the default, and near-universal — has already folded the query by the time the filter sees it, so USA and usa both expand correctly. Passing true would instead make synonym matching quietly case-insensitive for a field whose analyzer deliberately is not. Matching_IgnoresCase pins the behaviour.

ISynonymMapRepository on LuceneNetIndexSearcher is optional. Roughly a dozen existing unit tests construct the searcher directly over a bare Lucene directory; defaulting it to null (no maps, which is the correct behaviour) avoided churning those call sites for a feature they do not exercise.

/synonymmaps responses are hand-serialized with System.Text.Json, not OData — the same reasoning as IndexesController.IndexJson (#41). SynonymMap carries a [JsonExtensionData] bag so unmodelled properties such as encryptionKey survive a round-trip, and OData would have emitted them as {} or dropped them. The listing's value wrapper is written by hand as a result.

Tests

36 unit + 10 integration, all green.

  • 14 expansion in isolation (SynonymMapTests) — asserts on the token stream rather than on search results, because that is what distinguishes an equivalency rule from a mapping rule; a query that merely matched could have matched on the original term.
  • 14 validation (SynonymMapValidationTests)
  • 8 end-to-end (SynonymMapEndToEndTests) — through the real indexing and search path. Documents are indexed with no synonym applied, so a match can come from nothing but the query having been widened. WithoutTheMap_TheSameQueryMatchesNothing sits beside the positive case deliberately: together they show the match comes from expansion rather than from some looser matching.
  • 10 integration (SynonymMapIntegrationTests) — through the real Azure SDK, which decides the routes, the value wrapper and the wire property names. This caught that SynonymMap.Format is not public in SDK 11.7.0.

Full unit suite green at 973 (937 pre-existing + 36), zero warnings.

Notes for review

  • The full integration suite fails on my machine — 166 ImageBuildFailedException failures from ~20 Testcontainers fixtures building images concurrently. It is pre-existing Docker contention rather than a regression: unrelated suites fail identically in bulk and pass in isolation. 46 related tests — including the schema-change and round-trip suites most likely to catch a fault here — pass together. Worth confirming CI is happy.
  • Suggest and autocomplete do not expand synonyms. Azure does apply them there. Scoped out to keep this reviewable; happy to open a follow-up issue.
  • No change to IndexSchemaChangeValidator — Azure allows synonymMaps to change on an existing field, since it is query-time only, and IndexSchemaChangeTests already asserted that.

README documents the feature under a new Synonym maps section.

🤖 Generated with Claude Code

Synonym maps are service-level resources with their own /synonymmaps
routes, stored alongside the index definitions as {name}.synonymmap.json.
A field opts in through the synonymMaps property it already carried but
which was, until now, accepted and ignored.

Expansion happens at query time only, as it does in Azure: the maps are
resolved per search and layered onto the per-field search analyzer, so a
field's own analyzer still decides how text is split before any synonym
is considered. Expanding while indexing would bake the current rules into
the stored terms and leave documents indexed before an edit disagreeing
with those after it, repairable only by a full reindex. Doing it at query
time is what makes a map safe to edit.

Rules are parsed with Lucene's SolrSynonymParser, which is the format —
and the only format — Azure supports. Both rule forms behave as the
service documents them: an equivalency rule keeps the term that was typed
and adds the alternatives, while an arrow rule replaces it.

A field naming a map that does not exist, or one no query could reach, is
rejected when the index is created rather than left to search unexpanded.
A map that goes missing later is skipped instead, since the emulator's
indexes and maps are separate files a user may restore independently, and
failing every search would be a harsh answer to a dangling name that only
widens results.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@paulirwin
paulirwin marked this pull request as ready for review August 20, 2026 21:05
@paulirwin
paulirwin merged commit bec034c into main Aug 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Synonym Maps support

1 participant