Reusable figure selections for run-to-run comparison (#121)#123
Merged
Conversation
df2b438 to
f982138
Compare
When re-processing the same scene with different ASP parameters, the diagnostic figures silently changed what they showed between runs: a fresh ICESat-2 request returned a slightly different point set, the best profile track flipped, the best/worst agreement segments moved, and the detailed- hillshade clip boxes were re-selected from the re-processed intersection- error raster. That made before/after comparison impossible. The asp_plot CLI now writes a <report_stem>_figure_selections.yml sidecar recording every non-deterministic selection, and a new --reuse_selections flag replays a prior run's choices so figures are directly comparable. - New asp_plot/selections.py (FigureSelections dataclass + YAML read/write + clip-box <-> pixel-window helpers; no report/fpdf imports, notebook-safe). bbox_to_pixel_offset rounds to nearest (not floor) so replayed clips don't drift a pixel from floating-point error. - StereoPlotter.plot_detailed_hillshade gains clip_windows= and records the drawn boxes on self.detailed_hillshade_clips (DEM-CRS bboxes; out-of-bounds boxes warn and fall back to auto). - Altimetry: load_atl06sr_from_parquet reuses the exact prior points; profile track (rgt/cycle/spot) and best/worst segments (segments=) are pinnable through the plot methods; get_altimetry_selections() reports the choices. A single run resolves the best track once and shares it across the profile and segment figures. Bumps version to 1.16.0.
ad62a1a to
6a11370
Compare
…e range Two fixes from comparing ASTER non-mapproj (run A) vs mapproj reuse (run B): - Best/worst agreement segments were pinned by km-from-track-start, so when 3-sigma outlier filtering against a different DEM dropped a different first point, the track start shifted and the 'same' segment landed on different ground. Pin by absolute along-track distance (x_atc) instead, which is a stable reference; km extents are kept for plot axes/readability, and legacy km-only manifests still work via fallback. - The reuse path (load_atl06sr_from_parquet) didn't set self._t0/_t1, so plot titles lost their '<t0> to <t1>' date-range line. Restore the time range from the parquet's stored sliderule_parameters. Adds tests: time-range restore from parquet, and segment selection robustness to a shifted track start.
6a11370 to
4dd7633
Compare
It was emitting at the bottom because it was the last dataclass field (we dump with sort_keys=False). Declare it first so it serializes at the top, where schema/version metadata is expected.
Add an annotated example of a *_figure_selections.yml to the --reuse_selections docs section so users can see the schema. Gitignore reports/*_figure_selections.yml because the generated sidecars hardcode absolute local paths (dem_filename, parquet_cache) and are regeneration byproducts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #121.
When re-processing the same scene with different ASP parameters (the workflow in stereopipeline-quickstart and across our own mapproj-vs-no-mapproj example notebooks), the diagnostic figures silently changed what they showed between runs — making before/after comparison impossible:
_select_best_track) flipped,_find_best_worst_segments) moved,All of this selection logic is already deterministic given identical inputs — the non-determinism enters only through the re-fetched ICESat data and the re-processed DEM. So the fix is persist + replay, not seeding.
What changed
asp_plot/selections.py(new):FigureSelectionsdataclass + YAML read/write + clip-box ↔ pixel-window helpers. Imports nothing fromreport.py/fpdf, so it's safe from notebooks.bbox_to_pixel_offsetrounds to nearest (not floor) so a replayed clip doesn't drift a pixel from floating-point error.asp_plot: always writes<report_stem>_figure_selections.ymlnext to the report; new--reuse_selections PATHflag replays a prior run's ICESat-2 points, profile track, best/worst segments, and hillshade clips. A single run now also resolves the best track once and shares it across the profile + segment figures.StereoPlotter.plot_detailed_hillshade: newclip_windows=kwarg (DEM-CRS bboxes); records the drawn boxes onself.detailed_hillshade_clips. Out-of-bounds boxes warn and fall back to automatic selection.Altimetry:load_atl06sr_from_parquet()reuses the exact prior points;plot_atl06sr_dem_profile/plot_best_worst_segmentsaccept pinnedrgt/cycle/spot+segments=;get_altimetry_selections()reports the choices._find_best_worst_segmentsgains asegment_override.Per the design discussion, this is one umbrella flag (not granular toggles), reuses the parquet points for exact reproduction, stores clips as projected coordinates, and resolves the intra-run best-track inconsistency. All new kwargs default to auto, so existing behavior is unchanged.
Testing
tests/test_selections.py(12 tests): YAML round-trip, bbox↔pixel helpers, hillshade clip replay + out-of-bounds fallback, segment-override (+ bad-override fallback), parquet reuse, andget_altimetry_selectionsbundling.tests/test_imports.pyupdated.--reuse_selectionsreproduces the detailed-hillshade clips exactly (this is what surfaced and fixed the floor-vs-round off-by-one).test_altimetryfailures in my local env are an unrelated broken PROJ database (DATABASE.LAYOUT.VERSION.MINOR = 5), identical onmain; new tests avoid that path.Version bumped to 1.16.0.
Notes for reviewers
docs/cli/asp_plot.md.