Integration/psd seeds viewer - #57
Open
NikolaosSokos wants to merge 55 commits into
Open
Conversation
Closes #52. A seed cannot reliably reproduce a run -- the node's live inventory changes over time, so the same seed selects different channels later. The --seed flag is now accepted (for backward compatibility) but ignored, with a deprecation warning; re-verify findings with 'explore' instead. The report filename ({node}_{ts}_{seed}) and summary.seed are kept unchanged for backward compatibility with the Oculus/dmtri pipeline; the value is now an internal random discriminator. OPEN ISSUE comments in runner.py/report.py updated to record the decision.
Replace the colored-bar timeline with the ASCII glyph timeline (█ both / · neither / ▲ dataselect-only / ▼ availability-only / | gap boundary) to match the CLI/Markdown output. Refresh the CSS (sticky toolbar, monospace timeline, card detail, row hover/selection, styled request buttons/links). Swap the misleading hand-made fixture for a real 30-epoch NOA report (7 inconsistent rows, full coverage+mismatch).
…rowser - Summary: SVG score gauge (color by threshold) + direction breakdown bars alongside the consistent/inconsistent/skipped chips. - Results table: click-to-sort column headers (toggle asc/desc), a gap-count badge, and a Max-gap column; sort dropdown removed. - Detail: per-gap duration, copy-to-clipboard for every request URL, and a clearer full-window vs per-gap grouping. - Report browser: landing page renders an index.json manifest of available reports; a URL box loads any report (e.g. an Oculus report.json) without hand-editing the query string. Demo manifest lists NOA + RESIF samples. 31 node:test cases pass (added fmtDuration, gapStats, renderIndex, table badge/sort-header, per-gap duration + copy).
The landing must list every available report, not a hand-picked pair. Add make-index.mjs: scans a report directory and writes index.json with one entry per report (name, url, node, score, timestamp, inconsistent), newest first. The manifest and the reports symlink are generated/local artifacts, so untrack the curated index.json + the redundant resif copy and gitignore them; deployments (Oculus) regenerate the manifest over their own report tree.
Keep the landing a manifest-driven slot: if an index.json is present (e.g. published by Oculus alongside its latest report JSONs) the list is shown; otherwise the loader stands alone. No scraping or hardcoded report lists. Loading now works three ways — paste a report .json URL, choose a local file, or drop a file onto the page — with validation that the file is an EIDA consistency report before rendering.
Add a two-lane SVG coverage chart (Availability lane + Dataselect lane, mismatch regions highlighted, hover tooltips with exact times) shown above the ASCII line when a record has coverage. For older reports without coverage (e.g. those currently on Oculus) fall back to a single-track 'request window with gaps' chart so every inconsistency still gets a graph. Scales to container width; colours via CSS vars for dark mode.
Add make-oculus-index.mjs: crawls the public Oculus consistency tree, picks the latest report per node, reads each summary, and writes index.json with absolute year-level JSON URLs (Oculus serves these with open CORS, so the browser loads any entry cross-origin). The landing renders the manifest as a one-row-per-node dashboard with node, score (colour-coded), timestamp, and inconsistent count; the URL/file loader stays available beneath it. index.json is a generated artifact (gitignored) — re-run the crawler to refresh, or point the viewer at an Oculus-published manifest.
Older reports (e.g. those currently on Oculus) store only the availability url. Derive the dataselect URL from it — swap the service path and the start/end param names, narrow location/channel to the record's stream — so the detail view offers Run/open/copy for dataselect too, both full-window and per-gap. Stored dataselect_url still wins when present.
runRequest now returns hasData (availability spans > 0, or dataselect 200 with bytes). The detail view shows a colour-coded pill next to each Run button — green HAS DATA, red NO DATA, grey FAILED — with the HTTP/size summary beside it. Re-running a request replaces its previous result instead of stacking.
Clicking a row now smooth-scrolls the detail section into view so the timeline and Requests are immediately visible. Add viewer/README.md documenting the architecture, data flow, every module/function, the timeline renderers, request replay, loaders, security, manifest generators, and how Oculus integrates.
The design write-up was meant as personal documentation, not a product artifact served by the viewer. Remove the in-page 'How this viewer works' link and move the doc out of the served viewer/ folder.
A seed cannot reproduce a run -- the node's live inventory drifts, so the
same seed selects different channels later. Reproduce a finding by replaying
its exact window with 'explore'/'check' instead.
- Remove --seed from the consistency command (now an error, not a warning).
- Drop the seed parameter from run_consistency_check and unseed sampling.
- Drop summary.seed from reports; timestamp is the audit anchor.
- Report filename trailing seed slot -> microseconds
({node}_{YYYYMMDD}_{HHMMSS}_{ffffff}); derive the stem from summary.timestamp
so json/md stay a matched pair. 4-part underscore shape preserved.
- Legacy reports with a seed field still load; markdown regenerator shows it
only when present.
- Update tests, README, docs/cli.md; add CHANGELOG note.
Refs #52. Held local until Oculus/dmtri seed dependence is confirmed.
A node operator can re-run one or all of a report's inconsistencies against the live services and get a verdict per row -- without the boundary walk or dmtri output that 'explore' produces. eida-consistency rerun [REPORT] [-i N ...] [--all] [--json] [--verbose] Defaults to the latest report and its inconsistent rows; --all re-checks every row. Verdicts: PERSISTS / RESOLVED / SKIPPED, plus CONSISTENT / REGRESSED under --all. Read-only, with --json for machine output. Extracts the report loader, target selection, and window re-check into a shared reverify.py; explore now reuses them (behavior unchanged, dead json/requests imports dropped). Adds tests for verdict classification, scope selection, orchestration, rendering, and the CLI; repoints explorer URL-fetch tests at reverify. Docs + CHANGELOG updated. Closes #55.
Verdicts already stream per-row from rerun_report as each finding is re-checked; the CLI additionally rendered an aligned table of the same verdicts at the end. Drop the redundant table, keep the live rows + one-line summary. Removes the now-unused render_table helper; keeps _window (still used by the streaming line) and its formatting tests.
The table showed a bare ▲/▼ glyph for direction and the dataselect HTTP
status (often 'OK') for an inconsistent row, so what the inconsistency was
never read in plain language.
- Results table: 'Dir' -> 'Disagreement' column now shows a labelled pill
('▲ Data only' / '▼ Avail only', full phrasing on hover); 'Status' -> 'Result'
column shows the verdict in words (Inconsistent / Consistent / Skipped)
instead of the raw dataselect status.
- Detail view now leads with a plain sentence, e.g. 'Inconsistency: for 10m
(…), dataselect returned data but availability reported none.'
- New core helpers recordVerdict() and explainRecord() (+5 tests, 47 total).
Every row's max gap equals the fixed 10-minute request window, so the column (and sorting by it) carried no information. Remove it; the 'Gaps' column now sorts by gap count.
Filter options now read 'all ▲▼ / ▲ Data only / ▼ Avail only', matching the column pills. Behaviour unchanged (still filters by gap direction).
Replace the terse 'PSD Findings' table with an explanatory PSD Consistency section: describes the triangle model (dataselect = ground truth), the filled/hollow glyphs, and the 2024-01-01 obligation. Separates real violations (data >=2024 without PSD) from informational pre-2024 gaps, with a one-line summary and worst-first layout so consistent vs inconsistent is obvious.
- check: query PSD and print the A/D/P triangle + a 'Data vs PSD' verdict (consistent / VIOLATION >=2024 / pre-2024 gap / unsupported / skipped). - explore: in verbose mode, query PSD per window and show PSD URL + presence alongside availability/dataselect. - compare: report the PSD counter deltas (data-but-no-PSD, required windows) between two reports; gracefully omitted for pre-PSD reports. - test_all_nodes already exercises PSD via the check_psd=True default.
Add PSD to the HTML viewer, driven by the psd_* fields in reports: - psdChecked/psdVerdict/psdTriad/psdCounts core helpers (filled/hollow triangles ▼▲▶; violation >=2024 vs informational pre-2024 gap). - Summary: PSD chips (violations / pre-2024 gaps / consistent). - Results table: a PSD column with the per-row triad (shown only when the report carries PSD data, so pre-PSD reports render unchanged). - Detail: a PSD line with the triad, verdict, obligation, and day record. - sample-report.json seeded with psd_* fields for demo/tests. 10 new core tests; full viewer suite 56 passing.
…-PSD reports Review follow-up: psdCounts dispatches on a stable psdVerdict.kind enum instead of matching verdict text (no silent miscount on wording changes); psdChips is spliced without introducing whitespace so pre-PSD summary HTML is byte-for-byte unchanged.
- Add a legend above the results table explaining the ▼▲▶ triangle (filled=has data, hollow=missing) and the colour verdicts. - Add a PSD filter dropdown (violations / pre-2024 gaps / consistent / n/a); choosing a PSD category overrides 'only inconsistent' so A-D-consistent PSD violations are still shown. Hidden for pre-PSD reports. - 5 new core tests; viewer suite 61 passing.
…ne, consistency triangle) A self-contained dashboard.html rendering the Availability/Dataselect/PSD triangle from a report's psd_* fields: - three score tiles (A/D, PSD compliance >=2024, PSD coverage all-dates), N/A-aware, plus two count lines that each partition the checks - worded Avail-vs-Data and Data-vs-PSD verdicts; red >=2024 / orange pre-2024 - per-row triad (filled=has data) and a click-to-reveal consistency triangle - per-UTC-day PSD timeline lane (handles midnight-crossing + the ~1s post-midnight bleed of daily records) - graceful degradation: pre-PSD reports render as pure A/D (no PSD UI) - issues-only default, PSD/dir/search filters, sort, light/dark, reproduction run/open/copy buttons (rapid-click guarded) Curated one-of-each-case sample-report.json (real records, self-consistent).
# Conflicts: # README.md # docs/cli.md # src/eida_consistency/cli.py # src/eida_consistency/report/report.py # tests/test_cli.py
…eeds-viewer # Conflicts: # tests/report/test_report.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR combines four related pieces of the consistency system.
#51 — PSD (seedpsd) consistency + scoring
Adds the EIDA PSD service (
eidaws/psd/1/coverage) as a third data product, checked alongside availability and dataselect (dataselect is the ground truth). For each tested window we query PSD and compare:--no-psdto skip). PSD also appears incheck,explore, andcompare.#52 / #55 — Seed removal +
rerunRemoves the
seedmechanism — a seed can't reproduce a finding once the node's live inventory drifts, so it selects different channels later. Report filenames now use the timestamp. In its place, a newreruncommand re-verifies a report's inconsistencies by replaying each finding's exact window (verdict per row:PERSISTS/RESOLVED/SKIPPED):#50 — Interactive HTML report viewer
A self-contained, dependency-free viewer that renders a report's JSON in the browser: the Availability/Dataselect/PSD triangle, both scores, per-row verdicts, a per-day coverage timeline, filters/sort, and live buttons to reproduce a finding's request. Pre-PSD reports render as plain availability-vs-dataselect.