Repository navigation
Pros and Cons for inventory future #6104
Description
Purpose
We want to support inventory.data.gov long term, but we're not sure how to do that.
Given above architectural design decisions, conducting pros/cons list and research is needed to provide factual knowledge on future steps.
2 days of effort has been allocated and once compete, findings will be demonstrated and specific future actions will be decided.
Acceptance Criteria
[ACs should be clearly demo-able/verifiable whenever possible. Try specifying them using BDD.]
- GIVEN a thorough analysis is completed
WHEN 2 days expires
THEN pros and cons list is showcased with time and value estimates
AND architectural decision is made
AND design and develop tickets are created
Background
Stay with CKAN inventory, or move to custom application/service/tool model?
Sketch
We did this analysis with CKAN for catalog, but the features and tools of inventory are much different. Need complete analysis before deciding path forward.
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
jbrown-xentity commentedon Jun 25, 2026
Discussed high level pros and cons today with @SueValente.
CKAN cons: UI not compliant with 508, building DCAT-US3 metadata entry form is difficult.
custom application cons: Still seem to need data storage, so user management, metadata transfer, user transfer, still have to build DCAT-US3 form, etc.
One possible idea: could we get rid of users without building new custom application? One thought was to use GitHub to manage files and serve them that way as a simple GSA solution. There are 1002 total files, and 6 files that would be over the 100M limit for GitHub file storage, see
https://inventory.data.gov/api/action/resource_search?query=name:for complete list. If we could exclude those files (they are all provided by GSA), then this would be a viable option...jbrown-xentity commentedon Sep 9, 2026
Went ahead and documented current proposal in Wiki, for future publishing to inventory users about the upcoming change.
@sanason if you want to take a look and provide feedback on the doc, here, or in slack: https://github.com/GSA/data.gov/wiki/Inventory-Re%E2%80%90design
This doc may be expanded after this first iteration/discussion with further architecture and tool details (design step).
sanason commentedon Sep 15, 2026
@jbrown-xentity I agree that we should retire the CKAN-based inventory app and build a new custom app, for the reasons you list on the wiki page plus many of the reasons that came up during the planning for catalog (hard to find CKAN developers, inflexible UI, CKAN has shown itself slow to change and adapt to security vulnerabilities/concerns). But I think we need to do a requirements-gathering exercise before we start designing the new app. I'm not convinced, for instance, that we should jettison support for authenticated users.
jbrown-xentity commentedon Sep 15, 2026
@sanason I think we can do both. Hopefully if there are very infrequent users they can continue to use the tool without having to get kicked out and re-added every 90 days.
I updated the wiki: https://github.com/GSA/data.gov/wiki/Inventory-Beta-Re%E2%80%90design. Let me know what you think about the feature list, and how much more detail you want to see. For example "metadata entry for DCAT-US1.1" has a lot of detail within it; for example the publishers list that exists that maps organizations to publishers. But since most of the detail will not be overlapped, I don't really want to get into the weeds on some of that.
jbrown-xentity commentedon Sep 16, 2026
Discussion point for consideration from today's review:
Instead of having organizations that we maintain the user lists for; can we allow any gov personnel with a PIV card to login, create their own catalog(s), share those catalogs with other users, and then export the catalog individually, all catalogs together, or multi-select catalogs?
This complicates the idea of having a published endpoint for metadata that is harvestable from harvest.data.gov (how does access work? How do we link catalogs? How do we make publicly available without user, if tied to user access?). But it does remove data.gov from responsibility of direct management of users, and allows collaboration across departments, agencies, bureau's, teams, etc.
sanason commentedon Sep 17, 2026
@jbrown-xentity What problems would we have if we let arbitrary fed personnel expose metadata endpoints through inventory?
jbrown-xentity commentedon Sep 17, 2026
It's worth considering if there should be a formal process or not. However I don't think it should be on the inventory side, I think it's on the harvester side. It's the harvester that becomes publicly available information and searchable and that is where the gate should be I think, not on inventory.
jbrown-xentity commentedon Sep 18, 2026
@sanason documented our discussion at https://github.com/GSA/data.gov/wiki/Inventory-Beta-Re%E2%80%90design. I think this is ready for review and closeout. I'm building a new ticket now (will place in the same epic) related to the design step; planning out the technical needs in more detail. Eventually estimation of completion can be added to that as well.
sanason commentedon Sep 21, 2026
I think this is fine to close. @SueValente ?