A production-grade End-to-End Testing and Observability Framework combining:
- Playwright (Automated E2E Testing)
- Prometheus (Metrics Scraping)
- Grafana (Dashboard Visualization + Alerts)
- Loki + Promtail (Log Aggregation)
Built for scalability, clarity, and real-world CI/CD pipelines.
- Automation First: Robust E2E UI/API testing with Playwright
- Observability Live: Real-time metrics and logs capture with Prometheus, Loki, and Grafana
- CI/CD Integrated: GitHub Actions automates testing and artifact publishing
| Layer | Tools |
|---|---|
| Testing | Playwright (TypeScript) |
| Metrics | Prometheus + Metrics Server (Express.js + prom-client) |
| Visualization | Grafana |
| Logs | Loki + Promtail |
| CI/CD | GitHub Actions |
| Runtime | Node.js 20.x |
/ (root)
├── src/
│ ├── config/
│ ├── constants/
│ ├── pages/
│ ├── tests/
│ ├── utils/
│ └── observability/
│ └── metrics-server.ts
├── reports/
│ ├── json-reports/
│ ├── playwright-reports/
├── prometheus/
│ └── prometheus.yml
├── observability/
│ ├── docker-compose.loki.yml
│ ├── promtail-config.yaml
├── .github/
│ └── workflows/
├── package.json
├── README.md
├── TestStrategy.md
└── .gitignore
- Playwright HTML Report
- Playwright JSON Report
- Total Tests (
playwright_total_tests) - Passed Tests (
playwright_passed_tests) - Failed Tests (
playwright_failed_tests)
- Playwright Test Logs are streamed via Promtail to Loki
- Structured log fields:
status,suite,title,tags[],project,duration - All logs and metrics visualized in Grafana Dashboards
Playwright tests use @tag annotations to organize and filter tests across CI, logs, and dashboards. Tags are extracted from both test.describe() and test() titles via a custom wrapper and included in structured logs.
| Category | Examples | Purpose |
|---|---|---|
| Type | @smoke, @regression |
Run groups for CI jobs |
| Feature Area | @login, @search, @cart |
Feature scope and filtering |
| Behavior | @flaky, @slow, @quarantine |
Stability and execution visibility |
| Platform | @mobile, @firefox |
Environment-specific logic |
| Priority | @critical, @normal |
Risk-based test grouping |
- Every test should have at least one meaningful tag
- Tags must start with
@and use lowercase with dashes or underscores - Suite-level tags are inherited by all child tests
- Do not tag implementation details (e.g.,
@uses-middleware)
test.describe('@smoke @cart Cart Feature', () => {
test('@slow adds item to cart', async () => {
// This test gets: ["@smoke", "@cart", "@slow"]
});
});Tags appear in both logs (for Loki) and annotations (for filtering or metrics).
-
UI Flows:
- Add Todo
- Toggle Todo
- Delete Todo
- Filter Todos
- Navigate via URL hash (All, Active, Completed)
-
API Health Checks:
- Example or mock API routes if applicable (optional)
- Install dependencies
npm install- Run all Playwright tests
npm run test- Run Playwright tests with different options:
- Debug mode:
npm run test:debug- Smoke tests:
npm run test:smoke- Regression tests:
npm run test:regression- API tests:
npm run test:api- Run Playwright tests for a specific browser:
npm run test:chromium
npm run test:firefox
npm run test:webkit- Clean and re-run tests
npm run test:clean- Start Metrics Server
npm run start:metrics-server- Start Observability Stack (Loki + Promtail)
cd observability
docker-compose -f docker-compose.loki.yml up -d- Access Grafana
http://localhost:3000
- Slack/Email Alerts integration in Grafana
- Advanced log filtering for test debugging
- Full Kubernetes deployment support
- Cloud Grafana dashboards for remote observability
MIT License.
Built with ❤️ for real-world scalable test automation and observability setups.