Skip to content

support document update on opensearch sync #5617

@rshewitt

Description

@rshewitt

User Story

In order to keep datasets and opensearch synchronized, datagov wants to update documents in opensearch when they're updated in harvester.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

  • GIVEN our current approach for syncing on a scheduled basis
    AND that it only supports creating and deleting records
    WHEN a dataset is updated on harvester
    THEN it will be identifiable and updated on opensearch \

Background

  • current scheduled opensearch sync doesn't support updates. see comment this means opensearch is not synced with the database on dataset update.

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

  • we date-time stamp "last_harvested_date" for datasets on document update in harvester. if we added an index date-time stamp for opensearch documents we can find those which are older than the associated "last_harvested_date" to find out which documents are out-of-sync and need to be updated.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

🏗 In Progress [8]

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions