Support harvesting DCAT-US3 DataService objects #6178

User Story

In order to harvest DCAT-US3.0 metadata from non-dataset records, OMB wants DataService objects supported end-to-end in the harvest pipeline.

Acceptance Criteria

• GIVEN a valid DCAT-US3 DataService example is present in the source metadata, WHEN a harvest runs, THEN DataService objects are detected and persisted in the harvest database.
• GIVEN a harvested DataService object contains supported fields, WHEN the harvested record is stored, THEN the selected fields are available for downstream processing without being silently dropped.
• GIVEN a source contains both Dataset and DataService objects, WHEN the harvest runs, THEN DataService objects are handled independently of Dataset objects.

Background

This story splits out the DataService portion of #6000. The parent issue notes that DataService is similar to Dataset but is a distinct class and may deserve separate storage or OpenSearch modeling. It also can contain a list of datasets, to make it more confusing (ie a single API may provide access to multiple datasets and models).

Reference material:

• Develop harvest for DCAT-US3.0 classes/objects (other than dataset) #6000
• https://resources.data.gov/standards/catalog/dcat-us-3/quality-governance/#data-service
• https://github.com/GSA/dcat-us/blob/main/jsonschema/examples/Catalog/good/complete_example.json

Security Considerations (required)

None.

Sketch

• Confirm the minimum supported DataService field set needed for harvest. This should fit as-is, with maybe some additional fields like endpointURL
• identifier missing from required data model; make sure to still expect it and error out of that object if missing.
• Loop through sub field of possible dataset records and harvest those.
• Update harvest extraction and storage logic to handle DataService records.
• Add or adjust tests covering mixed Dataset and DataService source payloads.