Skip to content

support document update on opensearch sync #5617

Description

@rshewitt

User Story

In order to keep datasets and opensearch synchronized, datagov wants to update documents in opensearch when they're updated in harvester.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

  • GIVEN our current approach for syncing on a scheduled basis
    AND that it only supports creating and deleting records
    WHEN a dataset is updated on harvester
    THEN it will be identifiable and updated on opensearch \

Background

  • current scheduled opensearch sync doesn't support updates. see comment this means opensearch is not synced with the database on dataset update.

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

  • we date-time stamp "last_harvested_date" for datasets on document update in harvester. if we added an index date-time stamp for opensearch documents we can find those which are older than the associated "last_harvested_date" to find out which documents are out-of-sync and need to be updated.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions