Skip to content

docs(site): add the documentation site, phases 0-2 - #49

Merged
lsimons merged 13 commits into
mainfrom
docs/documentation-site
Jul 30, 2026
Merged

docs(site): add the documentation site, phases 0-2#49
lsimons merged 13 commits into
mainfrom
docs/documentation-site

Conversation

@lsimons

@lsimons lsimons commented Jul 30, 2026

Copy link
Copy Markdown
Member

Implements phases 0, 1 and 2 of S06.

What lands

Phase 0 — the site. Astro + Starlight in site/, LSD Warm theme (palette vendored from lsimons-dotfiles, Apache-2.0) with self-hosted Merriweather via @fontsource, so everything the site ships is redistributable by a fork. Nine operator pages in docs/guide/. The deploy workflow is ported from the organisation repository, with fetch-depth: 0 so Starlight's lastUpdated reads real git history rather than showing the deploy date on every page.

Phase 1 — the pipeline. scripts/sync-docs.py copies docs/**.md into the Astro content tree, derives frontmatter from each file's # heading, demotes the design record, rewrites repo-relative links, and fails the build on one that does not resolve. Raw Markdown twins ship at site/public/<slug>.md.

Phase 2 — the articles. An article-synthesis skill, plus four drafts, unpromoted. Promotion is a human's git mv into docs/articles/ and that gate has not been crossed — the Articles sidebar group is empty until someone reviews them.

Decisions worth reviewing

Requirement 7 is scoped to user-facing prose. Running the command checker over everything surfaced 95 mismatches, all correct. About half are commands renamed by S35 — a spec predating it says mdd confluence sync where the command is now sync-space. The rest are rejected or deferred commands named in "Rejected"/"Out of scope" sections. Neither is a defect: rewriting a spec to match today's CLI would falsify the record. The design record gets an advisory run that reports and exits zero.

_drafts/ did not do what the spec said. S06 claimed docsLoader() ignores the underscore prefix. Its glob is **/[^_]*.{md,mdx}, and [^_] matches the filename, so _drafts/foo.md was loaded and broke the build. Drafts are now _drafts/_<slug>.md; the spec is corrected.

llms-full.txt ignores the plugin's exclude option in starlight-llms-txt@0.11.0 — only the llms-small.txt route honours it. A local astro:build:done integration closes the gap, splitting on an explicit page-separator marker rather than guessing at heading boundaries.

Bugs found while documenting, not fixed here

Writing the guide against the code surfaced four things that need their own issues:

  1. The Confluence confidentiality blacklist is not enforced. gate_push has zero production call sites; no path under src/mdd/confluence/ references the blacklist at all. The SharePoint half is enforced. S07 §2.4 says otherwise. Independently confirmed by two agents.
  2. mdd search --type all returns only .qmd. An inclusive --glob overrides the --type md inclusion, so the widening filter narrows.
  3. sharepoint.sync_root in config does nothing through the CLI. resolve_sync_root reads it with getattr against a plain dict, so it is always None. The unit test passes an attribute-shaped mock and passes.
  4. The S26 restrictions check fails open_user_can_update returns True on any exception, so the last cascade layer is advisory.

Plus stale spec text in S07 §2.6, S09, S39 and S14. All documented against the code, not the spec.

Gates

mise run ci is green and stays Node-free. Vale, astro check and the site build run in mise run docs-check and a path-filtered PR workflow. The site builds 64 pages.

Co-Authored-By: lsimons-bot bot@leosimons.com


Added after review: a link check over the built site

The sync step resolves each repo-relative link against the working tree, so it
catches a link to a file that does not exist. It cannot catch a link to a file
that does exist but is served at a different URL — which is exactly how a
casing bug reached the deployed site: Starlight lowercases the slug it derives
from a filename, so S07-data-protection.md is published at
.../s07-data-protection/, and every link into the spec and research sections
404'd while satisfying the existing check.

scripts/check-site-links.py crawls site/dist/ after a build and resolves
every internal href and src against the pages and assets Astro emitted. It
runs as mise run docs-links, inside docs-check. Fragments are stripped
rather than verified; anchor checking is a larger job and is not attempted.

It found seven broken links on its first run, from two distinct bugs in the
link rewriter. The commit message for check emitted URLs against the built site attributes all seven to the first cause; that is wrong, and this is the
accurate account.

Two were links wrapped across a newline. CommonMark permits a newline both
inside the link text and before the destination, and the line-by-line rewriter
saw neither — it did not rewrite them and did not report them:

[Configuration and
secrets](05-configuration.md)          docs/guide/07-confluence-first-sync.md

... per [S20](
S20-litellm-ai-client.md))             docs/spec/S21-ai-rewrite-and-index.md

The rewriter now groups contiguous non-code lines into blocks and matches
across them, keeping fence and indented-code exclusion intact.

Five came from a fence-detection bug, which is the more serious of the two.
Line 251 of S30-markdown-ir-conversion.md uses a four-backtick inline code
span to quote a literal triple-backtick fence:

  ```` ```confluence-xml ```` fence around the macro's serialised

The fence detector read those leading four backticks as an opening code
fence. No four-backtick closing fence follows, so every line after 251 was
treated as being inside a code block: passed through untouched, and never
checked, so nothing was reported. Twelve links in that one file were silently
left as relative paths.

This predates this PR — it was in the original rewriter too, hidden behind the
same silent-pass-through symptom. The fix follows CommonMark: a backtick
fence's info string may not contain a backtick, so a backtick run only opens a
fence when no further backtick appears on the line. Tilde fences are
unrestricted and unchanged. It is the only instance in the corpus.

docs-build now clears dist first. Astro leaves orphaned output when a
page's slug changes, and the link check reads whatever is in dist/, so a
stale page would otherwise report failures that no longer exist in the source.

lsimons added 13 commits July 30, 2026 21:55
Publishes a Starlight site at https://schubergphilis.github.io/mdd/ on
every merge to main, built from checked-in Markdown under docs/.

Phase 0 — the site. Astro + Starlight in site/, LSD Warm theme with
self-hosted Merriweather, a remark plugin mapping GitHub alert syntax onto
Starlight asides, and nine operator pages in docs/guide/: install,
quickstart (offline, no tenant needed), concepts, safety, configuration,
commands overview, first Confluence sync, first SharePoint sync, and a
pointer to the design record. Deploy workflow ports the organisation
repository's, with fetch-depth: 0 so lastUpdated reads real git history.

Phase 1 — the pipeline. scripts/sync-docs.py copies docs/**.md into the
Astro content tree, derives Starlight frontmatter from each file's heading,
demotes specs and research notes (pagefind: false, a design-record banner, a
collapsed sidebar group, excluded from llms.txt), rewrites repo-relative
links, and fails on one that does not resolve. Raw Markdown twins ship at
site/public/<slug>.md for agents that fetch a page directly.

Phase 2 — the articles. An article-synthesis skill drafts secondary
literature from the spec and research corpus under three constraints:
citation density, preserved reversals, and never overwriting a promoted
article. Four drafts are included, unpromoted.

Requirement 7 lands as scripts/check-mdd-commands.py in mise run ci: it
introspects the argparse tree and checks every `mdd ...` string in
user-facing prose. The design record is advisory rather than blocking — a
spec records intent at the time of writing, so pre-rename command names and
rejected proposals there are accurate, not defects.

mise run ci stays Node-free. Vale, astro check and the site build sit in
mise run docs-check and a path-filtered pull-request workflow.

Also fixes link rot in six research notes: moved source paths updated,
deleted ir_experiment harness paths de-linked rather than invented.

Assisted-by: Claude:claude-opus-5
Vale and the sync script both take docs/articles/ as a path, and git does
not track an empty directory, so docs-check failed on a clean checkout.

Assisted-by: Claude:claude-opus-5
Vale's exit code covers everything at or above MinAlertLevel, so warning
level meant 135 passive-voice findings failed the build. Two passes now:
one shows every warning, one decides the outcome on errors alone.

Assisted-by: Claude:claude-opus-5
Records the vale-ai-tells measurements, and why they cannot settle the
adoption question: the evaluation argued from alert counts against this
corpus as a reference standard, but almost none of the corpus is
unaided human prose, so a comparison against it measures how much a
corpus resembles itself.

Proposes preference elicitation instead — minimal rewrite to satisfy a
rule, human side-by-side judgement, preferences attributed back to the
rule that produced the edit.

Assisted-by: Claude:claude-opus-5
"Install" as the primary action described the destination of one link
rather than the start of the guide. "Start" says what the button does.

The design-record action used the `external` icon, whose up-and-out arrow
means the link leaves this site. It does not.

Assisted-by: Claude:claude-opus-5
…to be readable

The wordmark is a dark slate that vanishes against the dark nav bar. The
amber in the mark is already the accent colour and works on both
backgrounds, so only the slate changes: scripts/derive-dark-logo.py
recolours it to the warm near-white the dark theme uses for headings,
mapping pixels by nearest source colour so antialiased edges follow.

The article skill's citation-density rule, applied literally, produced an
article the author of the system it described could not follow. Density is
now stated as a floor on coverage rather than a target for frequency, with
limits on placement, plus rules to show before explaining, to keep internal
notation out of the opening, and to lead with what is true now.

Assisted-by: Claude:claude-opus-5
Every article asserted its argument without showing it, opened in the
corpus's own notation, and carried a citation on almost every clause. The
maintainer could not follow the near-lossless one, and he designed the
system it describes.

Each now opens on real bytes from a committed file: an unrecognised macro
surviving a round trip, the pre-IR converter ballooning an 80-character
page into 750, the refusal text a managed page returns, and a resolved
open question with its strikethrough intact. Metric codes and phase
shorthand are gone from the bodies, superseded measurements are one
sentence of history rather than a table, and citations sit at clause end,
one per sentence.

Every reversal survives, which was the constraint that mattered.

Assisted-by: Claude:claude-opus-5
Starlight lowercases the slug it derives from a filename, so a page written
as S07-data-protection.md is served at .../s07-data-protection/. The link
rewriter kept the filename's case, so every link into the spec and research
sections was a 404 on the deployed site.

Nothing caught it: the link checker asks whether the target file exists in
the repository, never whether the emitted URL matches what gets served. An
existing test asserted the broken form.

Raw Markdown twins are fetched by URL too, so their paths fold the same way.
Destination filenames keep their original case.

Assisted-by: Claude:claude-opus-5
Promoting all four articles emptied the directory, and git does not track an
empty directory, so Vale failed on a missing path on a fresh checkout.

Assisted-by: Claude:claude-opus-5
…d links

The sync step resolves each repo-relative link against the working tree,
which catches a link to a file that does not exist. It cannot catch a link
to a file that does exist but is served at a different URL, which is how the
spec-slug casing bug reached the deployed site unnoticed.

scripts/check-site-links.py crawls site/dist after a build and resolves every
internal href and src against the pages and assets Astro emitted. It runs as
mise run docs-links, inside docs-check. Fragments are stripped rather than
verified; anchor checking is a larger job and is not attempted.

It immediately found seven broken links nothing else could see. All were
Markdown links wrapped across a newline, which CommonMark permits in both the
text and the destination, and which the line-by-line rewriter passed through
untouched — neither rewritten nor reported. The rewriter now groups
contiguous non-code lines into blocks and matches across them, keeping fence
and indented-code exclusion intact.

docs-build now clears dist first: Astro leaves orphaned output when a slug
changes, and the link check would otherwise report stale pages.

Assisted-by: Claude:claude-opus-5
The design-record page was in the guide, where it did not belong — it
describes the corpus a reader is about to enter, not anything mdd does. It is
now docs/design-record/index.md, the introduction above Specs and Research,
so the sidebar-position sentence, the why-it-is-published section and the
contributing pointer all became redundant and are gone.

A Get involved group at the bottom publishes CONTRIBUTING.md,
CODE_OF_CONDUCT.md, SECURITY.md and LICENSE. Those files stay at the
repository root and are read from there at build time: GitHub finds them by
path for its pull-request and advisory flows, so moving them would break it.
LICENSE is not Markdown and renders verbatim in a fenced block.

sync-docs.py gains two section shapes for this — an index.md publishing at
its section root, and a section sourced from an explicit list of root files.

Every Edit page link was broken. Starlight appends the path inside the
content collection, but no synced page lives there in the repository: a guide
page keeps its numeric filename prefix, specs keep their case, and the Get
involved pages come from the root. sync-docs.py now emits a per-page editUrl
from the real source path, verified against the working tree rather than by
eye.

README points at the site near the top, and its documentation section points
at a new docs/README.md that maps the subdirectories and explains how the
site is built.

Assisted-by: Claude:claude-opus-5
The design-record and get-involved output is build input like every other
synced directory, and was committed because .gitignore only listed the
sections that existed when it was written.

Assisted-by: Claude:claude-opus-5
@lsimons
lsimons merged commit 308dcd5 into main Jul 30, 2026
6 checks passed
@lsimons lsimons added this to MDD Jul 31, 2026
@github-project-automation github-project-automation Bot moved this to Backlog in MDD Jul 31, 2026
@github-project-automation github-project-automation Bot moved this from Backlog to Done in MDD Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant