English · 繁體中文

github-radar

A research tool for scanning public GitHub projects. Pure Python standard library, zero external packages, deterministic results, rerunnable, and auditable.

Two things everyone assumes about GitHub data, measured — a 304 response does not necessarily save your rate limit, and a 95% confidence interval on search coverage can cover the truth 0% of the time. Both findings are reproducible on your own machine in under a minute.

Seven modes; choose what to scan each time:

Mode	Question it answers	Command
discover	Which new projects are taking off?	python -m radar discover --days 90 --topic llm
watch	What changed this week in the area I care about?	python -m radar watch agents
inspect	What do these projects actually look like, and what packages do they use?	python -m radar inspect --repo owner/name
inspire	Has someone built what I want to make, and which one can I build from?	python -m radar inspire "scheduler" --keyword scheduler
verify	What independent time, account, and structure signals are visible around one repo?	python -m radar verify --repo owner/name --quick
coverage	How much of a Search population did these paths likely cover?	python -m radar coverage --query "stars:>100"
supplychain	Which observed dependency packages are structurally prominent but have stale or missing maintenance evidence?	python -m radar supplychain

Every run writes a Traditional Chinese Markdown report to reports/ and stores the data in data/radar.sqlite3.

---

Quick start

Requirement: Python 3.11+ (the core uses the standard-library tomllib to parse manifests, which is not available before 3.11). Older versions are stopped by the entry-point guard with a clear message instead of an obscure import error.

cd "$env:USERPROFILE\Desktop\github-radar"
python -m radar status

Nothing to install. status tells you the current quota, database state, and whether you should set a token.

Dashboard (browser interface)

python -m radar serve          # or double-click 開啟儀表板.bat

It opens at http://127.0.0.1:8160, with eight tabs: Overview, Rankings, Projects, Events, Dependencies, Scans, Verify, and Coverage. All six scans can be triggered directly from the interface; they run in a background thread, with progress and results shown live.

At the top of Overview is the oscilloscope — each project is a point of light:

Channel	What it represents
Angle	Primary language (the outer labels show language and count)
Distance from center	Days since the last push; rings mark 1 day / 7 days / 30 days / 1 year
Point size	Star count (area is proportional to star count)
Faded points	Archived or forked projects
Scan line	Pure animation; it does not represent time or progress

The radius uses a logarithmic scale with hours as the base. This was chosen after measuring the distribution: a local measurement of 192 projects found p25=1.8 hours, p50=9 hours, and p75=66 hours; three quarters had been pushed within three days. A linear or square-root scale would compress 90% of the points into a blur at the center. A day-based scale is also not enough, because half the sample was less than a day.

The points themselves are not animated — one keyframe per point would mean 384 parallel animations for 192 projects, and SVG opacity animation is not guaranteed to run on the compositor. Instead, the scanning sector brightens the area it passes over using screen blending. There is only one animation for the whole chart; points remain fully lit, so animation never hides data.

If more than 80% of the points were pushed within the last 30 days, the oscilloscope will actively show a warning that this mostly reflects your search conditions being recent, not that this group of projects is especially active. This keeps the chart from being read as a conclusion it cannot support.

• Binds only to 127.0.0.1, so it is not exposed externally and does not use authentication.
• The page is entirely self-contained — CSS and JS are embedded, with zero external requests (this machine has TLS interception, which breaks any CDN), and the server also sends a CSP header that blocks external connections.
• Dark mode is the default; the upper-right corner switches to light mode, and the choice is saved in the browser.

At the top of Overview there is a number called observable growth rate. It is the most honest health indicator in the whole tool: how many projects have accumulated at least two observations, meaning how many have a real growth rate that can be calculated rather than an estimated value. It starts low and grows as you scan again.

Setting a token (strongly recommended)

Without a token, anonymous mode allows 60 calls per hour and 10 searches per minute; one scan can use it all. With a token, that becomes 5,000 calls per hour and 30 searches per minute.

# Method one: environment variable (valid for the current window)
$env:GITHUB_TOKEN = 'your-token'

# Method two: save it in a file (does not enter shell history, recommended)
python -m radar discover --token-file C:\path\to\my.token

Create a token at https://github.com/settings/tokens. For scanning public projects, no permissions need to be selected — for a classic token, leave all scopes unchecked; for a fine-grained token, select Public repositories, read-only.

The token is never written to the database, reports, or logs; it is always masked when displayed. config.json and *.token are also in .gitignore.

---

The seven modes

1. discover — find new stars

python -m radar discover --days 90 --min-stars 20 --language python --topic llm --top 30

Finds projects created within the specified period that meet the star threshold, then ranks them by momentum.

Momentum score = stars per day × activity multiplier (based on how long since the last push; archived ×0.15, fork ×0.5).

One thing about “stars per day” must be made explicit: the GitHub API does not provide a historical star curve. On the first run, there is no previous observation, so the growth rate can only be estimated as “total stars ÷ project age”; the report labels this proxy_lifetime. Run again several days later under the same conditions, and the program subtracts the two snapshots, switching to the observed number of stars added during the interval (observed_delta). This is the only way this tool can obtain a real growth rate.

Also, if --days is too small (for example, 21 days), all candidate projects are about the same age. Dividing by age then barely changes the ranking, and the momentum ranking collapses into a popularity ranking. When this happens, the report says so directly and explains why, rather than giving you a high-overlap score and making you guess.

2. watch — monitor activity

python -m radar watch --init agents      # create a watchlist/agents.json template
notepad watchlists\agents.json           # change it to the queries and repos you care about
python -m radar watch agents             # first time: establish a baseline
python -m radar watch agents             # after that: report changes only

Outputs only changes relative to the previous run; it does not reprint the full list. There are five event types: new project, star-count change, project revival (pushing again after being inactive for a long time), project going dormant, and new release (add --releases).

Thresholds live in the watchlist JSON, so adjust them yourself.

3. inspect — analyze code contents

python -m radar inspect --repo psf/requests --repo tiangolo/fastapi
python -m radar inspect --from-run 7 --top 10     # analyze the top 10 from run 7

Does not clone or download source code; everything goes through the API. For each project, about 4–8 calls collect: language distribution, the root-level file list, dependency manifests (actually parses requirements.txt / pyproject.toml / package.json / go.mod / Cargo.toml), README size, whether tests and CI exist, and whether the license permits reuse.

This is the most quota-intensive mode, so by default it only works on targets you specify explicitly.

4. inspire — collect competitors / inspiration

python -m radar inspire "local LLM scheduler" --keyword "local llm scheduler" --keyword "task queue"

Unlike discover, it does not ask who is popular; it asks who is usable. The usability score weights are license 30%, maintenance activity 25%, documentation 20%, size 15%, and adoption 10%. A 300-star, MIT-licensed project that is still updated and readable can beat a 30,000-star zombie project here.

GitHub search is almost useless for Chinese keywords. You can describe the need in Chinese, but you must provide English terms with --keyword. If you do not, the program stops immediately instead of giving you an unhelpful list.

5. verify — compare three independent star signatures

python -m radar verify --repo owner/name --quick       # time dimension only
python -m radar verify --repo owner/name --yes         # allow a full, bounded scan

verify keeps arrival-time bursts, targeted low-activity account features, and lockstep common-star structure side by side. It deliberately has no combined probability or suspicion score; a full scan's 0–3 value and a quick scan's 0–1 value are only unweighted signal counts. Coverage, sampling limits, uncertainty, cost estimates, and actual API usage are written to the report before the evidence sections. A full scan above 500 estimated calls stops unless --yes or an explicit --max-calls N is provided. --quick runs only the time dimension and is the inexpensive first pass.

6. coverage — estimate Search coverage

python -m radar coverage --query "stars:>100"
python -m radar coverage --query "language:python" --path stars=stars:desc --path recent=updated:desc

This opt-in mode compares multiple Search paths because every extra path costs at least one API call. It reports each sample, overlap, the Chapman/Lincoln–Petersen population estimate, 95% CI, and a deliberately coarse coverage interval. Unequal capture probabilities can bias the estimate low; the CI may completely miss the true value. Insufficient overlap is reported as unable to estimate, with plausible causes listed.

7. supplychain — supply-chain warning

python -m radar supplychain

This local-only mode cross-checks the saved direct-dependency graph, centrality, observed push age, and the existing hazard result. It keeps those three dimensions side by side and never creates a single danger score. Packages whose original repo was not scanned remain in the list with an explicit missing-data label. The warning list is incomplete: absence from it does not mean safety. The graph covers only scanned projects, root manifests, and direct dependencies; staleness is an operational definition and hazard is an observational association, not causation. The mode does not fetch package repositories or identify accounts.

---

Quota and cost

Every report ends with “cost for this run”: how many calls were used and how much remains in each quota bucket.

The reserve-watermark mechanism stops proactively when the remaining quota falls below the threshold (fail-closed); it does not keep hitting 403. The reserve amount scales proportionally to each bucket's capacity — 50 reserved when core has 5,000, 3 when search has only 30 — so protecting quota does not make the tool completely unusable.

Add --wait-on-limit to wait for the quota reset before continuing (possibly for an hour; it does not wait by default).

Observed finding: 304 does not necessarily save quota

Many sources say that “a conditional request returning 304 does not count against the rate limit.” On 2026-07-20, a local test sent five consecutive anonymous requests that returned 304, and the quota still dropped by five. Therefore this tool makes no such assumption: after every 304 it compares x-ratelimit-remaining before and after, measures whether it was charged, and reports the observed result.

To re-test it yourself (especially after setting a token, to see whether authenticated requests receive an exemption):

python tools\etag_quota_probe.py

ETag caching saves transfer and parsing either way; just do not use “repeated scans are free” as a premise for scheduling frequency.

---

Feeding the mothership (Frontier Atlas)

github-radar is the first fleet module of Frontier Atlas. Add --atlas to any mode and the run's findings are submitted after the scan completes:

python -m radar discover --days 90 --topic llm --atlas

What gets submitted is not this tool's Markdown report but a contract-shaped, hashed run report. The mothership stores it as a source and quotes each finding from it verbatim as evidence, so every downstream conclusion traces back to a module, a run, and a line. Findings are derived from the structured ModeResult, never by parsing our own report layout.

Worth knowing up front:

• Submitted claims are unreviewed. A module's own output can never reach well_supported — a person has to look at it first. That is the mothership's design, not a limitation here.
• Re-submitting the same scan is refused with a 409. The report carries the scan time rather than the push time, so an unchanged run is byte-identical and the mothership recognises it.
• A failed push never fails the scan: the data is already in the local database and reports/, and can be pushed again later.
• If the mothership is not running, nothing breaks. Without --atlas this tool talks to nothing but the GitHub API.

--atlas takes an optional URL (default http://127.0.0.1:8000) and --atlas-workspace selects the workspace (default Fleet).

---

Data retention

Data	Retention	Reason
etag_cache	14 days	Cache only; losing it merely costs one extra call
code_facts	30 days	Intermediate product; can be fetched again at any time
runs / findings	365 days	Formal scan conclusions; must be reviewable for a year
reports/*.md	Matches the run record	Human-readable Traditional Chinese evidence report
repos	Permanent	Primary records
snapshots	Permanent	Longitudinal observation ledger; all growth rates depend on it, and deleting it makes them impossible to calculate again
stargazers	Permanent	Longitudinal star-timestamp ledger; arrival-time analysis depends on it, and deleting it makes it impossible to calculate again
star_users	30 days	Re-fetchable account-feature cache; the 30-day window reflects slower-changing profile features while keeping deleted-account observations refreshable
starred_lists	7 days	Re-fetchable public account star-list cache; it changes faster than profile features, so the shorter window limits stale structure evidence
dependency_edges	30 days	Rebuildable direct-dependency graph derived from code_facts
watch_events	Permanent	Event ledger

python -m radar prune          # print the policy only; delete nothing
python -m radar prune --yes    # actually delete

Cleanup is fail-closed: without an explicit --yes, it does not change a single row.

---

Development

$env:PYTHONPATH="$env:USERPROFILE\Desktop\github-radar"
python -m unittest discover -s tests -t .

The test count is reported by the command, with no network access throughout. All network I/O goes through an injectable transport, and the clock and sleep can also be injected, so test results are deterministic.

Current baseline: 360 tests.

Module responsibilities:

radar/client.py      GitHub API: ETag, pagination, two rate limits, Search 1000-result cap date-slice workaround
radar/store.py       SQLite: primary records, longitudinal snapshots, dependency graph edges, run records, events, tiered retention cleanup
radar/depgraph.py    Deterministic sample-scoped direct dependency graph with ecosystem-aware names and manifest provenance
radar/scoring.py     Momentum and hazard scores, usability score, low-activity heuristic, null-model comparison
radar/profiles.py    Targeted star-account profile fetch, deterministic sampling, and cache reuse
radar/lockstep.py    Simplified single-repository common-star structure, rarity/time weighting, and null calibration
radar/manifests.py   Dependency manifest parsing
radar/report.py      Traditional Chinese Markdown reports
radar/console.py     Console encoding fix (must be called at every entry point)
radar/modes/         Seven modes
radar/web/api.py     Dashboard data layer: pure functions, Store in, dict out, independently testable
radar/web/server.py  Dashboard routes; only HTTP and JSON
radar/web/app.html   Single-file frontend, all CSS/JS embedded
tools/               One-off verification scripts

Changes to a Python file such as radar/web/api.py require restarting the server to take effect (app.html does not; it is reread from disk on every request).

The color palette was not chosen by feel: the dashboard's four category colors went through a CVD check. In dark mode, the worst adjacent color difference is ΔE 13.0 (threshold 8), normal vision is ΔE 19.3 (threshold 15), and all contrasts are ≥3:1.

---

What this tool does not do

• It does not bypass rate limits, use multiple accounts, rotate IPs, or scrape things that require login to view. It reads public data and uses only the official API.
• It does not judge code quality. It has not run a single line of anyone else's code. Reports only narrow the range you need to inspect manually.
• It does not prove that stars are genuine. Star counts can be manipulated. The report only proves that “the number is moving.”
• It does not inspect subdirectories. inspect reads only the default branch's root directory. Monorepos will be underestimated; manually review any “no tests” result.
• Search does not cover 100%. Search API indexes only public repositories and has delays; when a date range really exceeds 1,000 results, the report explicitly says how many were truncated and never truncates silently.

---

License

This project is licensed under the MIT License.