Skip to content

perf: reduce multi-diagram generation latency without weakening quality gates #175

Description

@sunsunsun-java

Problem

The five-diagram Pi benchmark spent 2,637.592s (43m57.592s) of aggregate agent work even though the renderer/checker commands themselves are fast.

Measured evidence:

  • Architecture rounds 1-23: 334.544s wall time; validate CLI only 4.524s (1.35%).
  • Dataflow seven rounds: 133.849s wall time; validate CLI only 0.704s (0.53%).
  • Four of five diagrams first discovered vertical first-screen overflow after deterministic 9/9 validation.
  • Explicit post-overflow rework totalled 284.759s.
  • Manual review/reporting totalled 401.206s; pure report transcription was at least 133.960s.
  • Repeated sandboxed Chrome launch failures wasted about 23.8s.

The controllable cost is diagnosis, late viewport feedback, repeated orchestration, and report transcription—not renderer CPU.

Goal

Reduce the same five-diagram aggregate benchmark by 20-30% without weakening semantic, geometric, visual, evidence, or delivery quality.

Scope

P0 — actionable diagnostics

  • Return all desktop-readability violations in one receipt instead of only the worst item.
  • Include exact constraints such as max viewBox width/height, allowed offsets, overflow side/pixels, and label-mask owner.
  • Replace generic diagram-level layout failures with structured issues containing subject, evidence, and supported fixes.
  • Make resolved layout analysis available for failed candidates for all diagram types.
  • Add relationship label/mask containment checks against the viewBox.

P0 — viewport feedback before delivery

  • Add a deterministic first-screen budget receipt (adaptive-reader eligibility and fixed-height breakdown).
  • Add a four-light-viewport browser preflight before final delivery.
  • Keep the existing full final visual check, four captures, light/dark coverage, and manual review.

P0 — orchestration and measurement

  • Add a suite runner with typed command/path resolution and isolated output directories.
  • Add an append-only RunRecorder and canonical timing schema.
  • Automatically generate timing tables, hashes, validation/visual receipts, and evidence links.
  • Record artifact-ready, review-ready, and handoff-ready milestones separately.
  • Probe Chrome/CDP capability once per suite and fail closed with an actionable environment receipt.

P1 — safe reuse

  • Add a revision-pinned ProjectIndex/EvidenceLedger that shares only mechanical repository facts.
  • Revalidate every selected source range at final delivery; never reuse inferred topology/causality.
  • Add authoring-kit <type> returning the exact type schema, common schema, and one matching example with hashes.
  • Reuse one Chrome process across a bounded batch while resetting page/theme/reader state and verifying artifact hashes.

P2 — only after profiling

  • Consider batch Git verification or process caching only if profiling shows meaningful benefit.

Non-negotiable quality gates

  • Showcase 9/9, zero errors, zero warnings.
  • Existing 4px route-label clearance, 8px micro-segment, 16px interior segment, and 6px projected-text floors remain unchanged.
  • All four desktop viewports remain mandatory.
  • Light/dark screenshots and manual visual review remain mandatory.
  • No removal of semantic nodes, relationships, labels, cards, or source evidence.
  • No font shrinking, overflow hiding/clipping, or internal scrolling as a shortcut.
  • Final deliver remains atomic and emits specification/artifact hashes.
  • Diagnostic-only changes must preserve the HTML SHA of already-valid fixtures.
  • Routing changes must preserve explicit route controls and cannot reduce the ordinary-model first-pass usability floor.

Verification

  • Unit tests at the new layout-analysis, run-recorder, evidence-index, and visual-session interfaces.
  • Existing geometry, repair-receipt, visual-check, delivery, repository-evidence, golden, and ordinary-model-floor tests pass.
  • Re-run the same five Pi diagrams from one pinned revision.
  • Compare repair rounds, aggregate agent work, artifact-ready time, and handoff-ready time.
  • Confirm node/relationship/label/source-evidence counts do not regress.
  • Confirm final visual receipts and manual review pass unchanged.

Benchmark target

Treat this as an A/B target, not a promise: reduce aggregate work from ~44 minutes to approximately 31-35 minutes on the same five Pi inputs, with no quality-gate reductions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions