Skip to content

Add claude_code_docs_map.md to tracked pages - #9

Open
marvin-ambrozio wants to merge 2 commits into
mainfrom
claude/issue-8
Open

Add claude_code_docs_map.md to tracked pages#9
marvin-ambrozio wants to merge 2 commits into
mainfrom
claude/issue-8

Conversation

@marvin-ambrozio

Copy link
Copy Markdown
Collaborator

Summary

  • Adds https://code.claude.com/docs/en/claude_code_docs_map.md as a tracked URL via new extra_urls site config field
  • Handles raw .md files by fetching directly with httpx instead of rendering with Playwright
  • Fixes url_to_filepath to strip .md extensions (avoiding .md.md output filenames)

Closes #8

Test plan

  • Run uv run sync_docs.py and verify docs-md/claude-code/claude_code_docs_map.md is created
  • Verify the content matches the source at https://code.claude.com/docs/en/claude_code_docs_map.md
  • Verify existing docs are still crawled correctly

🤖 Generated with Claude Code

Add support for extra static URLs in site config and handle raw
markdown files by fetching them directly with httpx instead of
rendering with Playwright. Also fix url_to_filepath to strip .md
extensions.

Closes #8

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@gpa-agent gpa-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean implementation for handling raw .md files. A couple of minor suggestions below.

Comment thread sync_docs.py
Comment thread sync_docs.py Outdated
if resp.status_code >= 400:
continue
markdown = resp.text.strip()
if len(markdown) < 100:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Minor: The 100-char minimum threshold makes sense for extracted HTML content (where short output likely means extraction failed), but raw .md files could legitimately be short. Might be worth lowering this for the .md path or removing it, since we're already confident about the source URL.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove check

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Removed the 100-char minimum length check for raw .md files.

Raw markdown files can legitimately be short, unlike extracted HTML
content where short output likely means extraction failed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add claude docs map to tracked pages

3 participants