Multi-tenant registry OpenSearch query equivalents to single tenant registry

[plawton-umd]

Jordan asked for examples to Open Search queries from the single tenant registries that can be translated into
examples for the (coming soon) multi-tenant registry

Environment variables used

• REGUSER - registry user name. For SBN UMD these end in "_sbnumd".
• OPENSEARCH_URL - that long string starting with "https://search" and ending with ".com:443"

Corrections from meeting

• using curl (many in scripts, so forget what is used)
• manually entering password for these

Examples

• check that access is working

  curl -u $REGUSER $OPENSEARCH_URL

• count content of our registry

  curl -u $REGUSER $OPENSEARCH_URL'/registry/_count?pretty=true'

• count related to bundle without giving VID; for example, spitzer

  curl -u $REGUSER $OPENSEARCH_URL/registry/_count -H "content-type: application/json" -d '{"query":{"simple_query_string": {"query": "urn:nasa:pds:spitzer"} } }' | json_pp

• different count (note "*" at end)

  curl -u $REGUSER $OPENSEARCH_URL/registry/_count -H "content-type:application/json" -d '{"query":{"simple_query_string": {"query":"urn:nasa:pds:spitzer*"} } }' | json_pp

• bundle information based on LIDVID

  curl -u $REGUSER $OPENSEARCH_URL'/registry/_search?q={_id:"urn:nasa:pds:spitzer::1.0"}' | json_pp

• count with LIDVID

  curl -u $REGUSER $OPENSEARCH_URL/registry/_count -H "content-type: application/json" -d '{"query":{"simple_query_string": {"query": "urn:nasa:pds:spitzer::1.0"} } }' | json_pp

• collection information

  curl -u $REGUSER $OPENSEARCH_URL'/registry/_search?q={_id:"urn:nasa:pds:spitzer:spitzer-spec-comet::1.0"}' | json_pp

• LID collection count

  curl -u $REGUSER $OPENSEARCH_URL/registry/_count -H "content-type: application/json" -d '{"query":{"simple_query_string": {"query": "urn:nasa:pds:spitzer:spitzer-spec-comet"} } }' | json_pp

• LIDVID collection count

  curl -u $REGUSER $OPENSEARCH_URL/registry/_count -H "content-type: application/json" -d '{"query":{"simple_query_string": {"query": "urn:nasa:pds:spitzer:spitzer-spec-comet::1.0"} } }' | json_pp

• different collection count

  curl -u $REGUSER $OPENSEARCH_URL/registry/_count -H "content-type: application/json" -d '{"query":{"simple_query_string": {"query": "urn:nasa:pds:spitzer:spitzer-spec-comet*"} } }' | json_pp

[alexdunnjpl]

@jordanpadams @tloubrieu-jpl @plawton-umd how important is partial backward-compatibility with curl? If it's valuable for avoiding modification of a bunch of scripts, I can write something that should work with existing scripts given that a couple of env vars are set and the executable is temporarily aliased to curl in the runtime environment.

Tentative design:

• Python CLI tool
• At runtime, attempts to retrieve values REGISTRY_HOST REGISTRY_COGNITO_USER, REGISTRY_COGNITO_PASSWORD, NODE_ID from environment (is including the password asking for trouble?) and queries user for these values via stdin if any are not set
• Additional args are parsed with argparse, and mirror a subset of curl args
  • -d/--data
  • -o/--output
  • -H/--header
  • --url-query
• One positional arg is accepted, of form https://{host}{:port | null}/some/path{/ | null}?{key1}={value2}... (for backwards compatibility - scheme/host/port would be discarded) or /some/path{/ | null}?{key1}={value2}...
• Output to stdout, or path specified by -o

@jordanpadams @tloubrieu-jpl I'll need

• a test opensearch host, cognito endpoint, cognito username and cognito password
• any necessary details, if any, for what environment I should use to access it (presumably anything on JPLNET?)
• an enumeration of the node ids which will be supported (or is it fine to proceed without any checking for valid node ids?)
• the pattern for f(node_id, single_tenant_index_name)-> actual_multitenant_index_name

[plawton-umd]

@alexdunnjpl @jordanpadams @tloubrieu-jpl
I do not consider curl a requirement. Access to the OpenSearch queries is. I have found it very useful since

• entries that have not been made public are not available via the api (Note: expected results are compared with actual, Open Search provided, results)
• when there is a discrepancy between api and Open Search OR Open Search and harvest logs OR harvest logs and api, the Open Search information can be informative

[alexdunnjpl]

Blocked pending resolution of API Gateway interface discussion