fix(catalog): rank on where a word appears and how rare it is

[miguel-heygen]

What

Three ranking defects in the catalog's word search, plus one honesty fix for queries it cannot read.

Measured on a 39-query eval set built from real catalog intents, same set before and after:

	before	after
right move ranked first	31/39 (79%)	33/39 (85%)
right move in the top 3	34/39 (87%)	39/39 (100%)

Why

I ran 30 realistic queries against the shipped CLI while preparing a bug bash and 13 came back wrong. Reading the results rather than the code made the causes obvious, and they were all in the same 75-line scorer.

A name match counted the same as a description match. catalog --query "typewriter effect on a title" ranked the item literally called typewriter seventh, behind entries that merely mention typing. An author typing a move's name is giving the strongest signal available and it was being averaged away.

Plurals shared no vocabulary with the singular. "a stat that counts up and then pulses once" matched nothing in a description reading "lands with a restrained scale pulse", because counts is not count and pulses is not pulse. count-up fell to rank 10. Adding detail to a query made the result strictly worse, which is the opposite of what a search should do.

Common words counted as much as rare ones. This is the one that bit only after fixing the first: weighting the name field put every item merely named *-reveal on top of "reveal a headline one line at a time", because one strong hit on the catalog's most common word outscored several weak hits on the words that actually narrowed it down.

A query in another script reported itself as an empty catalog. Tokenising on [a-z]+ leaves nothing of a Japanese query, so it returned zero results, printed "no items match", and then invited the author to file a gap report about a move the catalog may well have. A real user hit exactly this.

How

localSearch.ts keeps its shape: shared tokens over the square root of the entry's token count. Three things sit on top.

1. Field weighting. textOf now returns {strong, weak} instead of one flat string; name and title are worth 3x. Two callers total, both updated.
2. Plural folding, table-driven, applied to both sides. Deliberately not a real stemmer: Porter folds counter to count and values to valu, which merges moves that mean different things. A guard rule protects press, status, axis before the bare s$ rule runs.
3. Inverse document frequency, so rarity scales each term. The +1 inside the log keeps a token present in every item at a small positive weight rather than exactly zero, so it can never flip a tie on its own.

Separately, hasNoSearchableTokens lets the command distinguish "we could not read your query" from "the catalog has nothing like this". On that path the CLI now says what actually happened and withholds the gap-report prompt, since nothing was searched:

$ hyperframes catalog --query "実写写真のみ 9:16 生活ハック 4カット"
No searchable words in query "実写写真のみ 9:16 生活ハック 4カット".

  Word matching indexes the catalog in English, so a query written in another
  script produces no terms to match and returns nothing. This is not a gap in
  the catalog. Try the same intent in English.

--json carries unsearchable_query: true and omits report_gap in that case.

Test plan

• Unit tests added/updated
• Manual testing performed
• Documentation updated (if applicable)

13 new tests, each a query that failed against the real catalog reduced to the smallest fixture that still reproduces it, including the one that only regresses when field weighting is present without IDF. Existing tests migrated to the fields API.

localSearch.test.ts 21 passed. Full CLI suite 2643 passed, with the 2 pre-existing transcribe failures unchanged (confirmed failing on the base commit).

Verified against the real CLI, not just the harness:

"typewriter effect on a title"                    -> typewriter, hw-title, yt-prism-title
"chat conversation between a user and an assistant" -> chat-message, chat-thread, ai-chat-reveal

The second used to return transitions-blur first, with claude-exchange and chatgpt-exchange absent from the top 5 entirely despite both existing.

Not covered

• The eval set lives in my scratch directory, not the repo. It is 39 hand-labelled queries and it is the only reason I can claim a number here. Worth landing as a checked-in fixture so the next ranker change is measurable rather than argued, but that is its own PR.
• Non-Latin search still does not work, it merely fails honestly now. The catalog is written in English and the on-device tier is bge-small-en-v1.5, also English. Real multilingual search needs a different model, not a different tokeniser.
• Six confirmed catalog gaps are untouched and are content work rather than ranking: an N-node architecture flow diagram, a character or figure, a portrait Ken Burns slideshow, a logo-to-liquid morph, a map-route compositor, and a pull quote. There is no pull-quote item anywhere in 375.
• text that types then backspaces is a gap, not a miss. typewriter is a reveal only and nothing in the catalog deletes. It is the CLI's own --search-miss example, and I removed it from the eval set once I checked rather than counting it against the ranker.