confirming sweeper has processed a product

[plawton-umd]

@jordanpadams @tloubrieu-jpl How to I determine if sweeper has processed an product using output from pds-registry-client please?

[tloubrieu-jpl]

Hi @plawton-umd ,

When a product has been seen by sweeper, you can run the following registry-client command on your product:

pds-registry-client '/sbnumd-registry/_doc/urn:nasa:pds:nh_derived:lorri_cob:2007_2007_10_06t02_42_18_301_fit::1.0' --pretty

You should see a field "ops:Provenance/ops:reindexed_at" with a time value, for example "2024-11-12T16:57:18.538883".

You can also search for not "swept" products with this query, in a file lets say, query.json:

{
  "query": {
    "bool": {
      "must_not": [
        {
          "exists": {
            "field": "ops:Provenance/ops:reindexed_at"
          }
        }
      ]
    }
  }
}

And run the command:

pds-registry-client '/sbnumd-registry/_search' --data @/Users/loubrieu/tmp/query.json --pretty

Sweeper runs every 2 hours which explains why some products are not "swept" yet.

Thanks

[plawton-umd]

@tloubrieu-jpl
Thank you for the information.

For the collections and bundles we are concerned about, the 2 hours should not be an issue. The latest load date among them was 5-Mar-2025.

I will submit a ticket concerning one oddness found.

[tloubrieu-jpl]

Yes, it sounds like sweeper is not running properly. We will investigate that. Thanks for creating the ticket that will help us to track the issue.

[tloubrieu-jpl]

Sorry, @alexdunnjpl told me the reindexed_at field could be missing, although sweeper ran on the product. This happens if some LDD were not available yet. That means the product is not fully fixed/indexed and will need to go through sweeper again.