Repository navigation
Add isPartOf link #5653
Description
Activity
- modified the milestones: Catalog.data.gov UI Strategy Implementation, Catalog Cutover - Beta Launch to Live
on Mar 10, 2026 Should we give the user a search parameter? Do we have a good sense of collections and how are they messing with search results?
Should we give the user a search parameter? Do we have a good sense of collections and how are they messing with search results?
I think the concept of "parent" and "child" isn't DCAT-US1.1, it's CKAN collections that we created. I think that just muddies the waters. I wouldn't do a search parameter. Most of the time when we did that it was related to trying to find the "true" count and get around the collections filtering.
I would provide a link on the dataset details page, because I consider it a dataset detail.
isPartOf is also derivable from ISO documents, so we need to change how we are setting those parent identifiers when those are in WAF collections. If it's in a WAF collection and it has a parent identifier, always use that.
MDTranslator can also find an "identifier" inside the XML files and can put that into the transformed JSON. If that happens in an XML dataset that is the parent of a waf-collection, then we won't have the right "identifier" in that parent to make that connection with its children who are presuming that the parent's URL is the identifier.
for waf-collections every document has the same
isPartOfincluding the parent. the parent is identified whenisPartOf == identifier. children are found from opensearch by looking forisPartOf == [some value].dcatus (at the source) is different because the parent may not know its parenthood but it can if we process it like waf-collections. i think we should do that in the harvester instead of the catalog (it's inefficient by calculating the parent/child relationship per search results page load and per "show more results"). it's not something that needs to be calculated that many times. it can be determined on each harvest of the source.
what i'm proposing is we add a parent/child calculation at the beginning of harvesting for dcatus sources so that by the time the data gets to catalog the relationship is known and works.
this is a graph problem. here's the scenarios which can come up that we need to address and fix per harvest source (changes would occur on the Dataset table not the original harvest record)
- all children no parent (remains a collection but there's simply no parent record)
- no children 1 parent (don't include
isPartOffor parent. it's just a regular document) - chain of parents (a->b->c->d). use the earliest descendant as the parent. everything else is child. so "d" is the parent and everything else is its child.
- multiple collections and single docs (a->b c->d e ). this would be 2 separate collections with 1 non-collection doc.
- circular references ( a->b->a). this would need to be fixed.
It is a headache to build the parent-child relation. I can see it is even a more headache to maintain the relations.
We have worked out the strategy to show the collection in catalog-next with solr. We can do the same for OpenSearch. It does need an extra OpenSearch query each time you do a dataset listing to figure out whether any of the listed dataset is a parent, if we want to label a parent.
Metadata
Metadata
Assignees
Type
Projects
- Status
User Story
In order to show dataset relationships, data researchers want the link to the relevant datasets represented in the
isPartOfmetadata.Acceptance Criteria
[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]
GIVEN a dataset has an
isPartOfmetadata filled out appropriately (or is harvested via WAF collection type)WHEN the dataset is viewed on catalog
THEN a link to the "parent" record is displayed
GIVEN a dataset is referenced by other datasets with
isPartOfWHEN the dataset is viewed on catalog
THEN a link to the child records and/or a link to search all child records is shown.
Background
Came out of user testing.
Security Considerations (required)
None
Sketch
Here is an example:
I would expect this to be done as some type of dynamic API checking on page load, and if the search result for the
isPartOf(searching theisPartOffield to see if it's a parent, and if the dataset has that field, searchingidentifierfor the content of that field) has content, then a special block of HTML should be displayed.Please note that metadata wise, there's no reason a dataset can't be a parent AND a child. We should always check if someone is a parent, and if the
isPartOffield exists then check to make sure it exists via API and then link to it. If there are multiple matches, prioritize the one in the same org and harvest source.