Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 1 addition & 15 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,20 +64,6 @@ reviews:
- Error responses must use structured JSON `{ error: "..." }` or `{ errors: [...] }`, never plain text.
- The `set_sismo` before_action pattern (find_by + manual 404) is the project standard; keep it consistent.

# ── Rake Tasks / Background Data Ingestion ──────────────────────────
- path: "lib/tasks/**/*.rake"
instructions: |
The Rake task `sismo:fetch_data` fetches ~10,000 earthquake records from the USGS monthly GeoJSON feed
via HTTParty and bulk-inserts them into PostgreSQL. This task runs as a one-off cron/manual invocation, not as a queued job.
Review checklist:
- HTTP resilience: any new or modified external HTTP call MUST handle timeouts (HTTParty default is none),
network errors (Net::OpenTimeout, Net::ReadTimeout, SocketError), and unexpected/malformed JSON.
- Deduplication: existing logic uses `Sismo.find_by(title:)` — if the dedup key changes, verify uniqueness at the DB level too.
- Performance: bulk operations on thousands of records should consider `insert_all` / `upsert_all` or batching,
not individual `save` calls inside `.each`.
- Logging: use `Rails.logger` instead of bare `puts` for production-safe output.
- Never hardcode the USGS URL without allowing ENV override for testing.

# ── Migrations ──────────────────────────────────────────────────────
- path: "db/migrate/**/*.rb"
instructions: |
Expand Down Expand Up @@ -127,7 +113,7 @@ reviews:
# ── Gemfile ─────────────────────────────────────────────────────────
- path: "Gemfile"
instructions: |
Key dependencies: rails 7.2, pg, puma, httparty, will_paginate, rack-cors, dotenv-rails.
Key dependencies: rails 7.2, pg, puma, will_paginate, rack-cors, dotenv-rails.
Dev/test: minitest, debug, rubocop, brakeman, bundler-audit.
Review checklist:
- New gems must justify their addition (avoid bloat in an API-only app).
Expand Down
4 changes: 0 additions & 4 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,6 @@ end

gem 'dotenv-rails', '~> 2.1', '>= 2.1.1', groups: %i[development test]

gem 'httparty', '>= 0.24.0'

gem 'csv'

gem 'will_paginate', '~> 4.0'

gem 'rack-cors', '~> 2.0', '>= 2.0.2'
Expand Down
9 changes: 0 additions & 9 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ GEM
concurrent-ruby (1.3.8)
connection_pool (2.5.5)
crass (1.0.7)
csv (3.3.6)
date (3.5.1)
debug (1.11.1)
irb (~> 1.10)
Expand All @@ -104,10 +103,6 @@ GEM
erubi (1.13.1)
globalid (1.4.0)
activesupport (>= 6.1)
httparty (0.24.2)
csv
mini_mime (>= 1.0.0)
multi_xml (>= 0.5.2)
i18n (1.15.2)
concurrent-ruby (~> 1.0)
io-console (0.8.2)
Expand All @@ -134,8 +129,6 @@ GEM
mini_portile2 (2.8.9)
minitest (5.27.0)
msgpack (1.8.4)
multi_xml (0.9.1)
bigdecimal (>= 3.1, < 5)
net-imap (0.6.6)
date
net-protocol
Expand Down Expand Up @@ -281,10 +274,8 @@ DEPENDENCIES
brakeman (~> 6)
bundler-audit (~> 0.9)
connection_pool (~> 2.4)
csv
debug
dotenv-rails (~> 2.1, >= 2.1.1)
httparty (>= 0.24.0)
minitest (~> 5.22)
pg (~> 1.1)
puma (>= 5.0)
Expand Down
26 changes: 8 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Telurify API

The Rails backend API service for **Telurify**, a platform that collects, processes, and exposes worldwide seismic activity data from the [USGS Earthquake Hazards Program](https://earthquake.usgs.gov/). Events are collected via a background rake task, exposed through a JSON API, and allow users to submit structured "Did You Feel It?" intensity reports.
The Rails backend API service for **Telurify**, a platform that exposes worldwide seismic activity data collected from the [USGS Earthquake Hazards Program](https://earthquake.usgs.gov/). Events are exposed through a JSON API and allow users to submit structured "Did You Feel It?" intensity reports.

> **Note:** The frontend application lives in a separate repository (`telurify-web`), built with Astro and React islands.

Expand All @@ -15,7 +15,6 @@ flowchart LR
ReportsController[ReportsController<br/>POST /v1/sismos/:id/reports]
SM[Sismo Model]
RM[Report Model]
RT[Rake Task<br/>sismo:fetch_data]
RA[Rack::Attack<br/>rate limiting]
end

Expand All @@ -24,8 +23,9 @@ flowchart LR
Redis[(Redis / Upstash)]
end

subgraph External["External"]
USGS[USGS GeoJSON Feed<br/>all_month.geojson]
subgraph External["External Services"]
Ingestion[Telurify Ingestion]
USGS[USGS Earthquake Hazards Program]
end

RC --> SM
Expand All @@ -34,16 +34,16 @@ flowchart LR
RA -->|counters| Redis
SM --> PG
RM --> PG
RT -->|fetch & validate| USGS
RT -->|persist| SM
Ingestion -->|fetch & validate| USGS
Ingestion -->|persist| PG
```

**Component responsibilities:**

| Layer | Responsibility |
|---|---|
| **Backend API** | Serves paginated, filterable seismic events in a JSON:API-style format and accepts structured intensity reports for events. |
| **Rake task** | Pulls the USGS "Past 30 days" GeoJSON feed, validates ranges (magnitude, latitude, longitude), skips duplicates, and persists records. |
| **Ingestion service** | The [Telurify Ingestion service](https://github.com/Euler-B/Telurify-Ingestion) collects, validates, and persists seismic events. |
| **Rack::Attack** | Rate-limits all requests by IP (60 req/min) and throttles the reports endpoint specifically (5 req/min) to prevent spam on a public, unauthenticated endpoint. |
| **PostgreSQL** | Stores `sismos` (events) and `reports`. |
| **Redis (Upstash)** | Backs `rack-attack`'s distributed rate-limit counters in production. |
Expand All @@ -55,7 +55,7 @@ flowchart LR
**Backend**
- Ruby 3.4.10 / Rails 7.2.3 (API-only mode)
- PostgreSQL 16
- `httparty` (USGS feed), `will_paginate`, `rack-cors`
- `will_paginate`, `rack-cors`
- `rack-attack` + `redis` (rate limiting, backed by Upstash in production)
- Linting/security: `rubocop`, `brakeman`, `bundler-audit`

Expand Down Expand Up @@ -87,16 +87,6 @@ This single command will:

> **Note on Redis:** no local Redis is required for development. `rack-attack` falls back to an in-memory store automatically when `REDIS_URL` / `RACK_ATTACK_REDIS_URL` are unset.

### Load seismic data

Fetch the latest 30 days of events from USGS into the database:

```bash
docker compose exec backend bin/rails sismo:fetch_data
```

The task reports how many records were created, skipped as duplicates, and rejected by validation.

### Access the apps

| App | URL |
Expand Down
58 changes: 0 additions & 58 deletions lib/tasks/fetch_sismo_data.rake

This file was deleted.

Loading