[EPIC] Multi-Format Harvest Pipeline - ISO + DCAT-US v1.1 + DCAT-US v3.0 #6030

What: The datagov-harvester must simultaneously ingest three metadata formats: DCAT-US v1.1 (existing data.json files), DCAT-US v3.0 (new standard under M-25-05), and ISO 19115 encoded as ISO 19139 XML (harvested from WAF and document sources). Each format has different field structures, encoding requirements, and downstream expectations. As v3.0 migration accelerates and GeoPlatform coordination grows, the pipeline has several gaps that need investigation and resolution.

Why:

• M-25-05 requires DCAT-US v3.0 compliance by September 30, 2026
• ISO harvest sources (WAF, document) are active and must continue to work
• ISO records currently produce DCAT-US v1.1 output via MDTranslator, not v3.0
• v3.0 introduces structured types for spatial and temporal that differ from v1.1
• GeoPlatform depends on correct spatial, temporal, and ISO-linked records from Data.gov
• Several pipeline behaviors (xlink handling, v3.0 routing, antimeridian edge cases) are undocumented or unconfirmed

Done When:

• All spike tickets have documented findings that close their open questions
• ISO records produce v3.0-compatible output in the pipeline
• v3.0 spatial (Location object) and temporal (PeriodOfTime object) are correctly handled for all source formats
• v3.0 schema_type routing is confirmed and documented
• Known dropped ISO fields are documented in a decision record
• CSW source type is scoped (even if not yet built)
• Identity resolution gap is documented with a recommended approach

Notes:

• Ticket order below reflects recommended sequencing: spikes first, then implementation
• Spike findings feed into implementation ticket scoping
• Decisions marked NEEDS PM INPUT require product decision before implementation begins