Skip to content

Configure drafter client to pass max-query-timeout #103

@Robsteranium

Description

@Robsteranium

Stardog times out before queries can finish and this can go unnoticed (if drafter swallows the error).

Drafter asks Stardog to time-out queries after 30s (although result may continue to be returned for 90s).

Stardog is configured to allow a maximum time-out of 15 mins.

Drafter will allow privileged users to provide a JWS-signed timeout parameter to their requests to lift the 30s timeout.

We need to generate a key, configure drafter's DRAFTER_JWS_SIGNING_KEY env var, and have the ook.etl/query function pass the signed parameter.

This will leave large results in stasher but they're in binary format (so orders of magnitude smaller than the text versions (which are gigabytes)) and they'll only survive until the next publish. Once we're loading changes only #17 this should be less of a concern.

An alternative would be to make requests directly to Stardog. For this we'd need to install ook on the muttnik box or otherwise tunnel to stardog. We'd also need to change the queries to exclude draft graphs.

Metadata

Metadata

Assignees

Labels

etlRelated to the etl/pipelinesopsRelated to devops/deployment of the app

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions