docs(site): add the documentation site, phases 0-2 - #49
Merged
Conversation
Publishes a Starlight site at https://schubergphilis.github.io/mdd/ on every merge to main, built from checked-in Markdown under docs/. Phase 0 — the site. Astro + Starlight in site/, LSD Warm theme with self-hosted Merriweather, a remark plugin mapping GitHub alert syntax onto Starlight asides, and nine operator pages in docs/guide/: install, quickstart (offline, no tenant needed), concepts, safety, configuration, commands overview, first Confluence sync, first SharePoint sync, and a pointer to the design record. Deploy workflow ports the organisation repository's, with fetch-depth: 0 so lastUpdated reads real git history. Phase 1 — the pipeline. scripts/sync-docs.py copies docs/**.md into the Astro content tree, derives Starlight frontmatter from each file's heading, demotes specs and research notes (pagefind: false, a design-record banner, a collapsed sidebar group, excluded from llms.txt), rewrites repo-relative links, and fails on one that does not resolve. Raw Markdown twins ship at site/public/<slug>.md for agents that fetch a page directly. Phase 2 — the articles. An article-synthesis skill drafts secondary literature from the spec and research corpus under three constraints: citation density, preserved reversals, and never overwriting a promoted article. Four drafts are included, unpromoted. Requirement 7 lands as scripts/check-mdd-commands.py in mise run ci: it introspects the argparse tree and checks every `mdd ...` string in user-facing prose. The design record is advisory rather than blocking — a spec records intent at the time of writing, so pre-rename command names and rejected proposals there are accurate, not defects. mise run ci stays Node-free. Vale, astro check and the site build sit in mise run docs-check and a path-filtered pull-request workflow. Also fixes link rot in six research notes: moved source paths updated, deleted ir_experiment harness paths de-linked rather than invented. Assisted-by: Claude:claude-opus-5
Vale and the sync script both take docs/articles/ as a path, and git does not track an empty directory, so docs-check failed on a clean checkout. Assisted-by: Claude:claude-opus-5
Vale's exit code covers everything at or above MinAlertLevel, so warning level meant 135 passive-voice findings failed the build. Two passes now: one shows every warning, one decides the outcome on errors alone. Assisted-by: Claude:claude-opus-5
Records the vale-ai-tells measurements, and why they cannot settle the adoption question: the evaluation argued from alert counts against this corpus as a reference standard, but almost none of the corpus is unaided human prose, so a comparison against it measures how much a corpus resembles itself. Proposes preference elicitation instead — minimal rewrite to satisfy a rule, human side-by-side judgement, preferences attributed back to the rule that produced the edit. Assisted-by: Claude:claude-opus-5
"Install" as the primary action described the destination of one link rather than the start of the guide. "Start" says what the button does. The design-record action used the `external` icon, whose up-and-out arrow means the link leaves this site. It does not. Assisted-by: Claude:claude-opus-5
…to be readable The wordmark is a dark slate that vanishes against the dark nav bar. The amber in the mark is already the accent colour and works on both backgrounds, so only the slate changes: scripts/derive-dark-logo.py recolours it to the warm near-white the dark theme uses for headings, mapping pixels by nearest source colour so antialiased edges follow. The article skill's citation-density rule, applied literally, produced an article the author of the system it described could not follow. Density is now stated as a floor on coverage rather than a target for frequency, with limits on placement, plus rules to show before explaining, to keep internal notation out of the opening, and to lead with what is true now. Assisted-by: Claude:claude-opus-5
Every article asserted its argument without showing it, opened in the corpus's own notation, and carried a citation on almost every clause. The maintainer could not follow the near-lossless one, and he designed the system it describes. Each now opens on real bytes from a committed file: an unrecognised macro surviving a round trip, the pre-IR converter ballooning an 80-character page into 750, the refusal text a managed page returns, and a resolved open question with its strikethrough intact. Metric codes and phase shorthand are gone from the bodies, superseded measurements are one sentence of history rather than a table, and citations sit at clause end, one per sentence. Every reversal survives, which was the constraint that mattered. Assisted-by: Claude:claude-opus-5
Starlight lowercases the slug it derives from a filename, so a page written as S07-data-protection.md is served at .../s07-data-protection/. The link rewriter kept the filename's case, so every link into the spec and research sections was a 404 on the deployed site. Nothing caught it: the link checker asks whether the target file exists in the repository, never whether the emitted URL matches what gets served. An existing test asserted the broken form. Raw Markdown twins are fetched by URL too, so their paths fold the same way. Destination filenames keep their original case. Assisted-by: Claude:claude-opus-5
Promoting all four articles emptied the directory, and git does not track an empty directory, so Vale failed on a missing path on a fresh checkout. Assisted-by: Claude:claude-opus-5
…d links The sync step resolves each repo-relative link against the working tree, which catches a link to a file that does not exist. It cannot catch a link to a file that does exist but is served at a different URL, which is how the spec-slug casing bug reached the deployed site unnoticed. scripts/check-site-links.py crawls site/dist after a build and resolves every internal href and src against the pages and assets Astro emitted. It runs as mise run docs-links, inside docs-check. Fragments are stripped rather than verified; anchor checking is a larger job and is not attempted. It immediately found seven broken links nothing else could see. All were Markdown links wrapped across a newline, which CommonMark permits in both the text and the destination, and which the line-by-line rewriter passed through untouched — neither rewritten nor reported. The rewriter now groups contiguous non-code lines into blocks and matches across them, keeping fence and indented-code exclusion intact. docs-build now clears dist first: Astro leaves orphaned output when a slug changes, and the link check would otherwise report stale pages. Assisted-by: Claude:claude-opus-5
The design-record page was in the guide, where it did not belong — it describes the corpus a reader is about to enter, not anything mdd does. It is now docs/design-record/index.md, the introduction above Specs and Research, so the sidebar-position sentence, the why-it-is-published section and the contributing pointer all became redundant and are gone. A Get involved group at the bottom publishes CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md and LICENSE. Those files stay at the repository root and are read from there at build time: GitHub finds them by path for its pull-request and advisory flows, so moving them would break it. LICENSE is not Markdown and renders verbatim in a fenced block. sync-docs.py gains two section shapes for this — an index.md publishing at its section root, and a section sourced from an explicit list of root files. Every Edit page link was broken. Starlight appends the path inside the content collection, but no synced page lives there in the repository: a guide page keeps its numeric filename prefix, specs keep their case, and the Get involved pages come from the root. sync-docs.py now emits a per-page editUrl from the real source path, verified against the working tree rather than by eye. README points at the site near the top, and its documentation section points at a new docs/README.md that maps the subdirectories and explains how the site is built. Assisted-by: Claude:claude-opus-5
The design-record and get-involved output is build input like every other synced directory, and was committed because .gitignore only listed the sections that existed when it was written. Assisted-by: Claude:claude-opus-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements phases 0, 1 and 2 of S06.
What lands
Phase 0 — the site. Astro + Starlight in
site/, LSD Warm theme (palette vendored fromlsimons-dotfiles, Apache-2.0) with self-hosted Merriweather via@fontsource, so everything the site ships is redistributable by a fork. Nine operator pages indocs/guide/. The deploy workflow is ported from the organisation repository, withfetch-depth: 0so Starlight'slastUpdatedreads real git history rather than showing the deploy date on every page.Phase 1 — the pipeline.
scripts/sync-docs.pycopiesdocs/**.mdinto the Astro content tree, derives frontmatter from each file's#heading, demotes the design record, rewrites repo-relative links, and fails the build on one that does not resolve. Raw Markdown twins ship atsite/public/<slug>.md.Phase 2 — the articles. An
article-synthesisskill, plus four drafts, unpromoted. Promotion is a human'sgit mvintodocs/articles/and that gate has not been crossed — the Articles sidebar group is empty until someone reviews them.Decisions worth reviewing
Requirement 7 is scoped to user-facing prose. Running the command checker over everything surfaced 95 mismatches, all correct. About half are commands renamed by S35 — a spec predating it says
mdd confluence syncwhere the command is nowsync-space. The rest are rejected or deferred commands named in "Rejected"/"Out of scope" sections. Neither is a defect: rewriting a spec to match today's CLI would falsify the record. The design record gets an advisory run that reports and exits zero._drafts/did not do what the spec said. S06 claimeddocsLoader()ignores the underscore prefix. Its glob is**/[^_]*.{md,mdx}, and[^_]matches the filename, so_drafts/foo.mdwas loaded and broke the build. Drafts are now_drafts/_<slug>.md; the spec is corrected.llms-full.txtignores the plugin'sexcludeoption instarlight-llms-txt@0.11.0— only thellms-small.txtroute honours it. A localastro:build:doneintegration closes the gap, splitting on an explicit page-separator marker rather than guessing at heading boundaries.Bugs found while documenting, not fixed here
Writing the guide against the code surfaced four things that need their own issues:
gate_pushhas zero production call sites; no path undersrc/mdd/confluence/references the blacklist at all. The SharePoint half is enforced. S07 §2.4 says otherwise. Independently confirmed by two agents.mdd search --type allreturns only.qmd. An inclusive--globoverrides the--type mdinclusion, so the widening filter narrows.sharepoint.sync_rootin config does nothing through the CLI.resolve_sync_rootreads it withgetattragainst a plain dict, so it is alwaysNone. The unit test passes an attribute-shaped mock and passes._user_can_updatereturnsTrueon any exception, so the last cascade layer is advisory.Plus stale spec text in S07 §2.6, S09, S39 and S14. All documented against the code, not the spec.
Gates
mise run ciis green and stays Node-free. Vale,astro checkand the site build run inmise run docs-checkand a path-filtered PR workflow. The site builds 64 pages.Co-Authored-By: lsimons-bot bot@leosimons.com
Added after review: a link check over the built site
The sync step resolves each repo-relative link against the working tree, so it
catches a link to a file that does not exist. It cannot catch a link to a file
that does exist but is served at a different URL — which is exactly how a
casing bug reached the deployed site: Starlight lowercases the slug it derives
from a filename, so
S07-data-protection.mdis published at.../s07-data-protection/, and every link into the spec and research sections404'd while satisfying the existing check.
scripts/check-site-links.pycrawlssite/dist/after a build and resolvesevery internal
hrefandsrcagainst the pages and assets Astro emitted. Itruns as
mise run docs-links, insidedocs-check. Fragments are strippedrather than verified; anchor checking is a larger job and is not attempted.
It found seven broken links on its first run, from two distinct bugs in the
link rewriter. The commit message for
check emitted URLs against the built siteattributes all seven to the first cause; that is wrong, and this is theaccurate account.
Two were links wrapped across a newline. CommonMark permits a newline both
inside the link text and before the destination, and the line-by-line rewriter
saw neither — it did not rewrite them and did not report them:
The rewriter now groups contiguous non-code lines into blocks and matches
across them, keeping fence and indented-code exclusion intact.
Five came from a fence-detection bug, which is the more serious of the two.
Line 251 of
S30-markdown-ir-conversion.mduses a four-backtick inline codespan to quote a literal triple-backtick fence:
The fence detector read those leading four backticks as an opening code
fence. No four-backtick closing fence follows, so every line after 251 was
treated as being inside a code block: passed through untouched, and never
checked, so nothing was reported. Twelve links in that one file were silently
left as relative paths.
This predates this PR — it was in the original rewriter too, hidden behind the
same silent-pass-through symptom. The fix follows CommonMark: a backtick
fence's info string may not contain a backtick, so a backtick run only opens a
fence when no further backtick appears on the line. Tilde fences are
unrestricted and unchanged. It is the only instance in the corpus.
docs-buildnow clearsdistfirst. Astro leaves orphaned output when apage's slug changes, and the link check reads whatever is in
dist/, so astale page would otherwise report failures that no longer exist in the source.