Skip to content

feat(geoip): in-app database refresh — GeoLite2 with DB-IP Lite fallback, console-managed key - #149

Merged
guohai merged 2 commits into
mainfrom
feat/geoip-in-app-refresh
Sep 2, 2026
Merged

feat(geoip): in-app database refresh — GeoLite2 with DB-IP Lite fallback, console-managed key#149
guohai merged 2 commits into
mainfrom
feat/geoip-in-app-refresh

Conversation

@guohai

@guohai guohai commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Replaces the cron-based scripts/geoip-refresh.sh with an in-app GeoIP database manager, so redeploys are hands-off (Coolify containers have ephemeral filesystems — the old cron-managed ./geoip dir vanished on every redeploy).

What

  • server/geoip-refresh.ts: source selection — console-managed MaxMind key > MAXMIND_LICENSE_KEY env > DB-IP Lite fallback (same MMDB format, no account/key, CC-BY 4.0, monthly builds with previous-month 404 fallback). Download → validate (size + maxmind.open) → atomic rename to canonical City.mmdb/ASN.mmdb → hot reader reload. Failures keep the serving databases untouched (proven live against a real bogus-key 401).
  • Auto-refresh: at startup when databases are missing/stale (>7 days) and on a weekly in-process timer — no cron, self-heals after redeploy.
  • Admin console (Regions page → GeoIP card): source + per-DB status, Refresh button (async with polling, 409 single-flight guard), MaxMind key management (PUT/DELETE /api/admin/geoip/maxmind-key — stored encrypted in systemConfig via the existing credential encryption, never echoed, save triggers an immediate GeoLite2 refresh), and the DB-IP attribution line when on the fallback source.
  • Hardening (post-review round): key-redaction guard proven under key-bearing error URLs across all three egress points (recorded error, serialized status, log line); downloads bounded by DOWNLOAD_TIMEOUT_MS = 120s so a hung fetch can never wedge the refresh state.
  • scripts/geoip-refresh.sh deleted; CLAUDE.md env docs updated (MAXMIND_LICENSE_KEY now optional/bootstrap-only).

Verification

  • 30/30 geoip unit tests (redaction + timeout + source order + atomic-swap under injected failures); 5 route authz/shape tests; npm run check + eslint clean.
  • Real end-to-end DB-IP refresh driven through the admin API on the dev server, including a deliberate bad-key failure showing serving files untouched and zero key leakage in the server log.
  • Independent security-focused review: approved, no critical findings; both Important hardening items fixed and re-review-verified.
  • Full pre-merge gate: unit 1801/1804, audio 12/12, E2E 132/148 (13 skipped) — all 6 first-run failures pass in isolation (known flake classes), zero real regressions.

Generated with SMT smt@agora.build

…ack, admin surface

🤖 Built with SMT <smt@agora.build>
…oads with a timeout

🤖 Built with SMT <smt@agora.build>
@guohai
guohai merged commit bfdcb57 into main Sep 2, 2026
1 of 3 checks passed
@guohai
guohai deleted the feat/geoip-in-app-refresh branch September 2, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant