Skip to content

Commit 6f45404

Browse files
Draw the raster data the notebook was already pulling
Two more maps, both from responses the notebook already had and was rendering as prose. The S2 cover. The section explaining why the screens do not land exactly on what was placed was pure argument: a cover is the field's own cell plus a skirt of smaller cells tracing the boundary, and it is the skirt that pulls the figures towards the surroundings. Now it is a picture -- blue for the L15 cell the placer read, orange for the level-20 refinement, white for the registered boundary. AR2 returns the cover itself: registration is idempotent on the geometry, so an already-registered field answers in about 70 ms with its cover attached, and what gets drawn is the registry's covering rather than a second implementation that would agree today and drift later. It also confirms the prose exactly -- 1 cell at L15 and 98 at L20 for the first field. Layer extents. outside_coverage was a word in a refusal; it is an extent, so it can be drawn. The oil palm map is a regional band across the north and three of the four fields sit south of it, which is why the node declines to report on them -- as against the fourth, inside the band, which comes back no_data: the weaker statement that the map covers you and holds nothing here. Grouped by extent rather than by layer, because the four ICF layers share one national bbox and four identical rectangles stack into a single muddy outline claiming to be four things. Layers not drawn are counted in the legend rather than dropped -- three wider than the view, four declaring no extent -- since a coverage map that quietly omits layers is the failure it exists to illustrate. Tested by planting both regressions. No per-cell endpoint exists, so a true value-per-cell choropleth is not available; a grid of probe registrations would cost 1.6s each and is not worth it. These two draw what the node genuinely returns. Five maps, 257 KB. Still 9 live, 3 local, 0 skipped, 0 failed. Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 2d3bcae commit 6f45404

4 files changed

Lines changed: 2537 additions & 245 deletions

File tree

‎dpi-demo/build_openscience_notebook.py‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -449,6 +449,36 @@ def _find_support_module():
449449
underneath the demo rather than that S2 is doing its job.
450450
""")
451451

452+
md("""
453+
Zoomed to one field, this stops being an argument and becomes a picture.
454+
455+
The blue square is the field's own S2 cell, the one the placer read. The orange
456+
skirt around its edge is the refinement AR2 added when covering the polygon —
457+
much smaller cells, at level 20, tracing the boundary. The white outline is the
458+
registered boundary itself.
459+
460+
Those orange cells are the difference. They are real ground just outside the
461+
blue square, the screen reads them because they are part of the cover, and they
462+
are mostly but not entirely coffee — which is the whole of why the figures come
463+
back near what was placed rather than exactly on it.
464+
465+
The cover is asked of AR2 rather than recomputed here. Registration is
466+
idempotent on the geometry, so an already-registered field comes back in about
467+
70 ms with its cover attached, and what gets drawn is the registry's covering
468+
rather than a second implementation of it.
469+
""")
470+
471+
code("""
472+
if od.have_folium() and HUB_TOKEN:
473+
subject = next(f for f in FIELDS if f['properties']['name'] == SUBJECT)
474+
cover, why = od.s2_cover(subject, HUB_TOKEN)
475+
print(f" {why}")
476+
if cover:
477+
display(od.cover_map(subject, cover))
478+
elif not od.have_folium():
479+
print(od.maps_unavailable())
480+
""")
481+
452482
md("""
453483
### What consent bought
454484
@@ -535,6 +565,32 @@ def _find_support_module():
535565
print(" Not one of the absent layers contributed a value to the verdict.")
536566
""")
537567

568+
md("""
569+
`outside_coverage` is the absence with a shape, so it can be drawn.
570+
571+
Each rectangle is a layer's **declared** extent, as the library states it — not
572+
where the data happens to be good, but where the publisher says the map exists.
573+
The four fields are in red.
574+
575+
The oil palm layer is the one to look at. It is a regional band across the
576+
north, and three of the four fields sit south of it in the coffee belt. When the
577+
node says `outside_coverage` for those three it is not failing to find palm; it
578+
is declining to report on ground its palm map never described. The fourth field
579+
falls inside the band and comes back `no_data`, which is the different and
580+
weaker statement: the map covers you and has nothing here.
581+
582+
Layers wider than this view — the JRC tropical belt, and the global ones — are
583+
counted in the legend rather than drawn, since a rectangle around the whole map
584+
says nothing.
585+
""")
586+
587+
code("""
588+
if od.have_folium() and STACK['terrapipe-os']['up']:
589+
display(od.coverage_map(FIELDS, od.get(f"{od.TERRAPIPE_OS_URL}/layers", token=HUB_TOKEN).json()))
590+
elif not od.have_folium():
591+
print(od.maps_unavailable())
592+
""")
593+
538594
# ==========================================================================
539595
# 8. Other layers
540596
# ==========================================================================

‎dpi-demo/openscience_demo.py‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -626,6 +626,176 @@ def consent_map(
626626
return canvas
627627

628628

629+
def s2_cover(feature: dict[str, Any], token: str) -> tuple[list[str], str]:
630+
"""The S2 cover AR2 derived for this geometry, asked of AR2.
631+
632+
Registration is idempotent on the geometry -- an already-registered shape
633+
comes back in about 70 ms with its cover attached -- so this costs a round
634+
trip rather than a new field, and returns the registry's own cover instead
635+
of a second implementation of the covering algorithm that would agree today
636+
and drift later.
637+
"""
638+
try:
639+
response = requests.post(
640+
f"{NODE_URL}/register-field-boundary",
641+
json={
642+
"wkt": wkt_of(feature["geometry"]),
643+
"return_s2_indices": True,
644+
"s2_index": "15,20",
645+
},
646+
headers={"Authorization": f"Bearer {token}"},
647+
timeout=30,
648+
)
649+
except requests.RequestException as exc:
650+
return [], f"AR2 could not be reached: {exc.__class__.__name__}"
651+
if not response.ok:
652+
return [], f"AR2 answered HTTP {response.status_code}"
653+
cover = ((response.json() or {}).get("S2 Cell Tokens") or {}).get("v2_cover") or []
654+
if not cover:
655+
return [], "AR2 returned no v2 cover for this geometry"
656+
return cover, f"AR2's own v2 cover: {len(cover)} cells"
657+
658+
659+
def cover_map(feature: dict[str, Any], cover: list[str]):
660+
"""The field, and the S2 cells the screen actually read for it.
661+
662+
The one picture that explains why the numbers do not land exactly on what
663+
was placed. The placer read a single L15 cell; the screen reads AR2's cover
664+
of the polygon registered from that cell's corners, which is the L15 cell
665+
plus a skirt of much smaller cells hugging the boundary. That skirt is real
666+
ground just outside the original cell, and it pulls the field's figures
667+
towards its surroundings.
668+
"""
669+
import folium # noqa: PLC0415
670+
import s2sphere # noqa: PLC0415
671+
672+
canvas = _basemap(_centre([feature]), 16)
673+
674+
by_level: dict[int, list[str]] = {}
675+
for cell in cover:
676+
by_level.setdefault(s2sphere.CellId.from_token(cell).level(), []).append(cell)
677+
678+
core = min(by_level) if by_level else None
679+
for level, cells in sorted(by_level.items()):
680+
# The coarse cell is the field's own; everything finer is refinement.
681+
colour = "#2874a6" if level == core else "#e67e22"
682+
for cell in cells:
683+
folium.Polygon(
684+
locations=s2_cell_ring(cell),
685+
color=colour,
686+
weight=1,
687+
fill=True,
688+
fill_opacity=0.25 if level == core else 0.55,
689+
tooltip=f"S2 level {level}: {cell}",
690+
).add_to(canvas)
691+
692+
folium.Polygon(
693+
locations=_ring_of(feature),
694+
color="#ffffff",
695+
weight=2,
696+
fill=False,
697+
tooltip="the registered boundary",
698+
).add_to(canvas)
699+
700+
_legend(
701+
canvas,
702+
{
703+
**{
704+
f"L{level}: {len(cells)} cell{'s' if len(cells) != 1 else ''}"
705+
+ (" (the field's own cell)" if level == core else " (boundary refinement)"): (
706+
"#2874a6" if level == core else "#e67e22"
707+
)
708+
for level, cells in sorted(by_level.items())
709+
},
710+
"the registered boundary": "#ffffff",
711+
},
712+
)
713+
folium.LayerControl(collapsed=True).add_to(canvas)
714+
return canvas
715+
716+
717+
# A bbox wider than this is not a statement about where the layer is so much as
718+
# a statement that it is everywhere; drawn on a map of Honduras it would be a
719+
# rectangle around the whole view, which tells the reader nothing.
720+
WIDER_THAN_THE_MAP = 60.0
721+
722+
723+
def coverage_map(features: list[dict[str, Any]], layers: list[dict[str, Any]]):
724+
"""Each layer's declared extent, and the fields that fall inside or outside.
725+
726+
This is what turns ``outside_coverage`` from a word in a refusal into
727+
something you can see: the oil palm map is a regional band across the north,
728+
three of the four fields are in the coffee belt to the south of it, and the
729+
node is right to refuse them rather than report no palm.
730+
"""
731+
import folium # noqa: PLC0415
732+
733+
canvas = _basemap(_centre(features), 7)
734+
palette = ["#8e44ad", "#16a085", "#d35400", "#2c3e50", "#7f8c8d"]
735+
736+
# Grouped by extent, not by layer: the four ICF layers share one national
737+
# bbox, and drawing four identical rectangles stacks them into a single
738+
# muddy outline that claims to be four things.
739+
shared: dict[tuple[float, ...], list[str]] = {}
740+
everywhere: list[str] = []
741+
undeclared: list[str] = []
742+
for layer in layers:
743+
bbox = (layer.get("coverage") or {}).get("bbox")
744+
layer_id = layer.get("layer_id", "?")
745+
if not bbox:
746+
undeclared.append(layer_id)
747+
continue
748+
west, south, east, north = bbox
749+
if east - west > WIDER_THAN_THE_MAP:
750+
everywhere.append(layer_id)
751+
continue
752+
shared.setdefault(tuple(bbox), []).append(layer_id)
753+
754+
drawn: dict[str, str] = {}
755+
for index, (bbox, ids) in enumerate(shared.items()):
756+
west, south, east, north = bbox
757+
colour = palette[index % len(palette)]
758+
label = ids[0] if len(ids) == 1 else f"{len(ids)} layers sharing one extent"
759+
drawn[label] = colour
760+
folium.Rectangle(
761+
bounds=[[south, west], [north, east]],
762+
color=colour,
763+
weight=2,
764+
fill=True,
765+
fill_opacity=0.06,
766+
tooltip="declared extent of: " + ", ".join(ids),
767+
).add_to(canvas)
768+
769+
for feature in features:
770+
folium.Polygon(
771+
locations=_ring_of(feature),
772+
color="#c0392b",
773+
weight=2,
774+
fill=True,
775+
fill_opacity=0.9,
776+
tooltip=feature["properties"]["title"],
777+
).add_to(canvas)
778+
779+
# Named rather than dropped. A layer missing from a coverage map should not
780+
# be missing silently, which is the same argument the readings make.
781+
not_drawn = {}
782+
if everywhere:
783+
not_drawn[f"{len(everywhere)} wider than this map, not drawn"] = "#bdc3c7"
784+
if undeclared:
785+
not_drawn[f"{len(undeclared)} declaring no extent, not drawn"] = "#bdc3c7"
786+
787+
_legend(
788+
canvas,
789+
{
790+
**{f"{label}: declared extent": colour for label, colour in drawn.items()},
791+
"the four demo fields": "#c0392b",
792+
**not_drawn,
793+
},
794+
)
795+
folium.LayerControl(collapsed=True).add_to(canvas)
796+
return canvas
797+
798+
629799
def _legend(canvas, entries: dict[str, str]) -> None:
630800
"""A plain HTML legend. folium has no first-class one."""
631801
import folium # noqa: PLC0415

0 commit comments

Comments
 (0)