Skip to content

fix(site): strip tags to a fixpoint when extracting llms-full.txt section titles - #62

Merged
lsimons merged 1 commit into
mainfrom
fix/site-llms-tag-strip
Jul 31, 2026
Merged

fix(site): strip tags to a fixpoint when extracting llms-full.txt section titles#62
lsimons merged 1 commit into
mainfrom
fix/site-llms-tag-strip

Conversation

@lsimons

@lsimons lsimons commented Jul 31, 2026

Copy link
Copy Markdown
Member

Closes the one open CodeQL alert on this repo — js/incomplete-multi-character-sanitization
in site/src/plugins/exclude-design-record-from-llms-full.mjs.

What the real defect is

Not a vulnerability, and this PR does not claim to be a security fix: the input is
the project's own build output, the code runs at build time in an
astro:build:done hook, and the extracted string is only ever used as a Set key
to match section headings — it is never re-emitted as HTML.

The genuine defect underneath is that replace(/<[^>]+>/g, '') is not
idempotent
: stripping one match can create a new one out of the surrounding
text, so a heading containing markup (inline code, a <span>, an element
wrapping an emoji) can leave a partial tag behind and fail to match its
llms-full.txt section. The site's own spec pages contain exactly that kind of
heading.

Why the title is still read out of HTML

The preferred fix was to stop parsing HTML at all and read the title from Astro.
That was ruled out empirically, not assumed:

  • astro:build:done's hook argument is { pages: { pathname }[], dir, assets, logger }
    no title, no frontmatter (checked against Astro's own type definitions).
  • Importing astro:content inside the hook fails both ways: statically at
    config-load time (Cannot find module 'astro:content'), and dynamically at
    hook-run time (Vite module runner has been closed) — Astro tears the module
    runner down before this hook fires.
  • astro:routes:resolved's route type exposes entrypoint/pattern, not
    frontmatter.

So the built HTML is the only source available at this point, and the fix is to
make the strip idempotent: stripTagsToFixpoint() loops until the string stops
changing. No new dependency added.

The two pre-existing comment blocks recording why this plugin exists (the
upstream exclude option is not forwarded to llms-full.txt) and why
PAGE_SEPARATOR replaced a blank-line split (a guide page quoting file contents
in a fenced block produced a false page boundary) are left intact.

Verification

Real build, not logic in isolation. Built llms-full.txt/llms-small.txt with
the fix, then reverted the plugin and rebuilt with the original single-pass code,
and diffed: byte-for-byte identical output on the real corpus — 54 spec and
research pages excluded in both, 18 guide/article/get-involved pages surviving
with headings intact, PAGE_SEPARATOR fully cleaned from both files in both
cases. astro check: 0 errors. mise run ci green (unaffected — pure JS change).

The alert should auto-close once code scanning re-runs on main; no manual
dismissal, since this is a fix rather than a suppression.

Co-Authored-By: lsimons-bot bot@leosimons.com

Title extraction for the llms-full.txt design-record filter matched the
built page's <h1> and stripped its markup with a single
`.replace(/<[^>]+>/g, '')` pass. A single pass over arbitrary HTML is not
idempotent: removing one match can turn the surrounding text into
something that looks like a tag too, so a heading with inline markup
(a <code> span, an icon wrapper) could leave a partial tag in the
extracted title and silently break the exact-string match this filter
depends on. Loop the strip to a fixpoint instead.

Also clears CodeQL alert js/incomplete-multi-character-sanitization on
this line. The string is build-time output used only as a Set key, never
re-emitted as HTML, so this was a correctness fix for title extraction
rather than a security fix.

Checked whether the title could come from a non-HTML source instead:
`astro:build:done`'s `pages` list carries only `pathname`, and
`astro:content` is not resolvable at that hook (Vite's module runner is
already closed by the time it fires, confirmed by trying both a static
and a dynamic import against a real build). So the built HTML remains
the only source available here.

Verified against a real site build: llms-full.txt is byte-for-byte
identical before and after this change (spec/research sections still
excluded, page separator still cleaned up, no guide/articles page
dropped).

Co-Authored-By: lsimons-bot <bot@leosimons.com>
Assisted-by: Claude:claude-sonnet-5
@lsimons
lsimons merged commit 153eb29 into main Jul 31, 2026
7 checks passed
@lsimons
lsimons deleted the fix/site-llms-tag-strip branch July 31, 2026 11:09
@lsimons lsimons added this to MDD Jul 31, 2026
@github-project-automation github-project-automation Bot moved this to Done in MDD Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant