API Contract Guardian

Local-first API compatibility platform for detecting breaking OpenAPI changes, scoring client impact, and generating CI-ready reports.

Overview

API Contract Guardian compares two OpenAPI specifications—a baseline (production or main) and a candidate (PR or branch)—and produces a structured compatibility report.

It answers:

• What changed in the API contract?
• Is the change breaking, risky, or safe?
• Which clients may be affected?
• How severe is the deployment risk?
• Should CI pass or fail?

This project runs locally. The web UI is not hosted by default. Clone the repository, run the setup commands, start the server, and open the local URL in your browser.

Default local URL: http://127.0.0.1:8000

Key Features

• Deterministic breaking-change detection (38+ rules)
• Indexed OpenAPI comparison for efficient analysis
• Client impact analysis from usage contracts
• Traffic-aware explainable risk scoring (0–100)
• JSON, Markdown, HTML, and PR comment reports
• CLI, REST API, and browser-based UI
• Evaluation suite with precision, recall, and F1 metrics
• GitHub Actions workflow example

Architecture

flowchart LR
    Old[Baseline Spec] --> Loader[Loader]
    New[Candidate Spec] --> Loader
    Loader --> Normalizer[Normalizer]
    Normalizer --> Rules[Rule Engine]
    Rules --> Impact[Client Impact]
    Impact --> Risk[Risk Scorer]
    Risk --> Reports[Reports]
    Reports --> UI[Local Web UI]
    Reports --> CLI[CLI]
    CLI --> CI[GitHub Actions]

Loading

Breaking-change classification is rule-based and deterministic. Optional LLM integration (if configured) rewrites explanations only—it does not change severity or invent findings.

Tech Stack

Layer	Technology
Language	Python 3.12
API / UI	FastAPI, Uvicorn, Jinja2
Data	SQLite (SQLModel)
CLI	Typer
OpenAPI	PyYAML, openapi-schema-validator
Testing	pytest, ruff

Project Structure

api-contract-guardian/
├── api_contract_guardian/   # Core package
│   ├── openapi/             # Loader, ref resolver, normalizer
│   ├── engine/              # Rules, risk scoring, client impact
│   ├── explainers/          # Report explanations
│   ├── reports/             # JSON / MD / HTML generators
│   ├── storage/             # SQLite persistence
│   ├── web/                 # Jinja2 UI templates
│   └── cli.py               # api-guardian CLI
├── examples/
│   ├── specs/               # Sample OpenAPI specs
│   ├── clients/             # Client usage contracts
│   ├── traffic/             # Endpoint traffic CSVs
│   ├── evaluation_cases/    # Labeled test cases
│   └── reports/             # Committed sample reports
├── docs/                    # Architecture, CI, user guide
├── tests/
├── scripts/
└── .github/workflows/       # CI workflow

Quick Start

Requirements: Python 3.12, macOS/Linux/WSL

git clone https://github.com/Rusha304/api-contract-guardian.git
cd api-contract-guardian
make setup

This creates a virtual environment, installs dependencies, and generates sample data.

Run Locally

Start the web server:

make run

Or with auto-reload during development:

make dev

Open the Web UI in Your Browser

After make run, open:

http://127.0.0.1:8000

From the dashboard you can:

1. View recent analyses
2. Run a new analysis against sample specs
3. Browse findings with severity badges
4. Download JSON, Markdown, or HTML reports
5. View evaluation metrics and the rule catalog

Port can be changed via .env (APP_PORT).

Demo Workflow

Run a breaking-change analysis without starting the server:

make demo

This compares ecommerce_v1.yaml vs ecommerce_v2_breaking.yaml and writes a report to reports/demo_report.md.

Other useful commands:

make cli-demo      # Safe vs breaking CLI examples
make evaluate      # Run evaluation suite (F1 metrics)
make test          # Run pytest
make benchmark     # Performance benchmark

CLI Usage

Compare two specs:

api-guardian diff \
  --old examples/specs/ecommerce_v1.yaml \
  --new examples/specs/ecommerce_v2_breaking.yaml \
  --clients examples/clients \
  --traffic examples/traffic/ecommerce_usage.csv \
  --format markdown \
  --out reports/my_report.md

Exit codes:

Code	Meaning
0	No breaking/high-risk changes
1	Breaking or high-risk changes detected
2	Invalid input or spec error

Other commands:

api-guardian serve              # Start web UI
api-guardian evaluate           # Run evaluation suite
api-guardian rules              # Print rule catalog
api-guardian generate-examples  # Regenerate sample data

Using Your Own OpenAPI Specs

api-guardian diff \
  --old /path/to/baseline.yaml \
  --new /path/to/candidate.yaml \
  --format markdown \
  --out reports/real_api_report.md

Sample Reports

Committed demo artifacts (viewable on GitHub without running the app):

File	Description
examples/reports/demo_breaking_report.md	Breaking-change Markdown report
examples/reports/demo_breaking_report.html	Browser-viewable HTML report
examples/reports/demo_breaking_report.json	Machine-readable JSON report
examples/reports/pr_comment_example.md	PR comment template
examples/reports/evaluation_report.md	Detection quality metrics

See also docs/SAMPLE_REPORT.md.

CI Integration

GitHub Actions workflow: .github/workflows/api-contract-check.yml

On each push/PR the workflow:

1. Installs the project
2. Runs tests
3. Runs the evaluation suite
4. Runs a safe compatibility check (must pass)
5. Generates a breaking-change demo report as an artifact

For your own repository, compare a PR spec against a baseline:

- name: API compatibility check
  run: |
    api-guardian diff \
      --old openapi/baseline.yaml \
      --new openapi/openapi.yaml \
      --format markdown \
      --out api-compat-report.md

Configure failure thresholds in .env:

RISK_FAIL_THRESHOLD=70
BREAKING_CHANGE_FAIL=true

Details: docs/CI_INTEGRATION.md

Evaluation

The evaluation suite measures detection quality against 10 labeled cases:

make evaluate

Metrics include precision, recall, F1, and per-rule accuracy. Output: reports/evaluation_report.md

Details: docs/EVALUATION.md

Testing

make test
make lint

Troubleshooting

Issue	Fix
Spec validation error	Ensure OpenAPI 3.x; check YAML syntax
Port already in use	Set APP_PORT=8001 in .env
Database errors	Delete data/api_guardian.db and restart
make setup fails	Requires Python 3.12

Documentation

• Architecture
• Rule Catalog
• User Guide
• CI Integration
• Evaluation
• Performance
• Developer Guide
• Roadmap

License

MIT — see LICENSE