Skip to content

feat(overview): distinguish failed from successful surveys - #602

Merged
mandy-chessell merged 3 commits into
odpi:mainfrom
dwolfson:feat/overview-survey-outcome
Oct 4, 2026
Merged

mandy-chessell merged 3 commits into
odpi:mainfrom
dwolfson:feat/overview-survey-outcome

Conversation

@dwolfson

@dwolfson dwolfson commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Problem

A survey that fails still writes a SurveyReport and whatever findings it reached before it throws, so "a report exists" does not mean "this asset was surveyed". On the live platform the PostgreSQL server surveys always threw (OMES-SURVEY-ACTION-0018) yet their reports counted toward Surveyed, quality annotations and schema coverage on the Overview.

Change (quickstart and freshstart)

Each report is classified from its own properties (_survey_outcome):

  • engine-run surveys carry a completionMessage: OMES-SURVEY-ACTION-0019 ("has completed the analysis") → completed; OMES-SURVEY-ACTION-0018 ("… threw …") → failed;
  • reports created by other tools carry error_count in additionalProperties: 0 → completed, more → failed;
  • anything else (e.g. the database surveys that only record table/column counts) → unknown, never counted as a success.

Surveyed assets, surveyed stores / never surveyed, quality-annotated assets, survey-analysed schemas and the findings tally now come from completed surveys only (a failed run's partial findings are excluded). A new Attention Queue row "Surveys that failed" shows failed reports, assets that have only failed surveys, and reports with an unknown outcome, linking to Operations. If the reports cannot be read, the coverage figures are None (not assumed successful).

Live check: of 16 survey reports, 10 completed, 3 failed, 3 had no outcome recorded. Surveyed went to 8 assets, with 2 more having only failed surveys.

Tests

  • 23 backend survey tests (outcome classification, subject with both a completed and a failed survey, failed-run findings excluded, lookup failures, caps, the real treasury case) + the 20 existing helper tests, both handlers.
  • Page harness: 78 (quickstart) / 72 (freshstart) checks incl. grammar, unknown/None states, call order.
  • Real browser (Chrome on the same machine): new row and tooltips render with live data.
  • Freshstart not run live.

Notes

  • The outcome rule is inferred from two conventions seen on this platform; surveys have no standard outcome field. The database surveys created by Resource Explorer record only counts, so they show as "unknown" until they also record error_count (or the platform gains a completion status on SurveyReport).

🤖 Generated with Claude Code

Follow-up commit: Egeria 6.2 annotation reuse (see #604)

Egeria 6.2 reuses survey annotations, so one annotation can be reported by many survey reports. Two Overview consequences, both handled here:

  • Findings are counted per distinct annotation, not per ReportedAnnotation link (a reused annotation would otherwise be counted once per report). An annotation counts if at least one completed survey reports it; one cited only by failed surveys stays excluded. Quality/schema credit is unchanged.
  • Links are read in pages. Each report links every annotation it reports (new or reused), so the link count grows with every survey even when nothing new is created. A single 1,000-link page was already at 913 on the live platform; once capped the quality/schema/findings figures would silently go to "not measured". _relationship_rows now ends on an empty page (a full or short page is not the last page), up to 10 pages; hitting the limit marks the result capped, and a failure on any page discards the partial rows (None, never a half-read count). The annotation read also honours the as-of time now.

Behaviour on the current platform is unchanged (no reuse yet; 772 distinct annotations, not capped). 32 survey tests now (adds reuse, multi-page, the page limit, failure on page 2, the exactly-one-full-page boundary).

…counts completed surveys only

A survey that fails still writes a SurveyReport and whatever findings it reached before it
throws, so "a report exists" does not mean "this asset was surveyed". On the live platform
the PostgreSQL server surveys always threw (OMES-SURVEY-ACTION-0018) yet their reports
counted toward Surveyed, quality annotations and schema coverage.

Each report is now classified from its own properties:
- engine-run surveys carry a completionMessage: OMES-SURVEY-ACTION-0019 "has completed the
  analysis" -> completed, OMES-SURVEY-ACTION-0018 "... threw ..." -> failed;
- reports created by other tools carry error_count in additionalProperties: 0 -> completed,
  more -> failed;
- anything else (e.g. the database surveys that only record table/column counts) is
  "unknown" and is never counted as a success.

Surveyed assets, surveyed stores / never surveyed, quality-annotated assets, survey-analysed
schemas and the findings tally now come from completed surveys only. A new Attention Queue
row "Surveys that failed" shows failed reports, assets that have only failed surveys and
reports with an unknown outcome, linking to Operations. If the reports cannot be read the
coverage figures are None (not assumed successful).

Live check: of 16 survey reports, 10 completed, 3 failed and 3 had no outcome recorded.

Backend: 23 survey tests (outcome classification, mixed completed/failed subjects, partial
findings of a failed run excluded, lookup failures, caps) plus the 20 existing helper tests,
for both handlers; page harness 78 (quickstart) / 72 (freshstart) checks.

Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
@dwolfson dwolfson self-assigned this Oct 4, 2026
…nks in pages (Egeria 6.2 annotation reuse)

Egeria 6.2 reuses survey annotations: a repeat survey of an unchanged resource links the
annotations found earlier instead of creating new ones, so one annotation can be reported by
many survey reports (odpi#604). The Overview read ReportedAnnotation links
and counted one finding per link, which would count a reused annotation once per report, and
read a single page of 1,000 links, which is already 913 on the live platform and grows with
every survey (each report links every annotation it reports, new or reused). Once capped the
quality / schema / findings figures would silently go to "not measured".

- Findings are tallied by DISTINCT annotation (per type), so reuse does not inflate them. An
  annotation counts if at least one completed survey reports it; one cited only by failed
  surveys stays excluded. Quality/schema credit is unchanged: every asset whose completed
  report cites the annotation is credited.
- New _relationship_rows reads links in pages and ends on an EMPTY page (a short or full page is
  not the last page), up to 10 pages. Reaching the limit marks the result capped; a page that
  fails part-way discards the partial rows (None, never a half-read count). The annotation read
  now also honours the as-of time, which it previously ignored.

Behaviour on the current platform is unchanged (no reuse yet; 772 distinct annotations, not
capped). Backend tests: 32 survey tests (adds reuse, multi-page reads, the page limit, a
failure on page 2, the exactly-one-full-page boundary) for both handlers; helper tests unchanged.

Signed-off-by: Dan Wolfson <dan.wolfson@pdr-associates.com>
@mandy-chessell
mandy-chessell merged commit 75251bf into odpi:main Oct 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants