Introduction

This program implements the client part (which is run by each data provider) of the INDICATE data exchange protocol. Concretely, this program collects aggregated quality indicator results from a database run by the data provider. This program presents the aggregated quality indicator for review and, if the upload is confirmed, sends the results to the INDICATE central hub using the data exchange protocol.

Requirements

• Python >= 3.10 and dependencies as specified in requirements.txt

• Access to the aggregated quality indicator result database (not the primary database with clinical data)

• Access to the INDICATE Data Exchange API

Installation

To install the dependencies of the data exchange client, please execute the following from the root directory:

pip3 install -r requirements.txt

Usage

When not running inside a container, create a .env file with options like

DATABASE_HOST=database-host
DATABASE_PORT=1234
DATABASE_NAME=name
DATABASE_USER=user
DATABASE_PASSWORD=password

PROVIDER_ID=47d3832d-1343-11f1-abf6-ac91a15cb2bb
DATA_EXCHANGE_ENDPOINT=http://data-exchange:1234/
OBSERVATION_COUNT_THRESHOLD=5

LISTEN_ADDRESS=localhost
LISTEN_PORT=8080

or set the corresponding environment variables. Then run the program with

PYTHONPATH=. python3 indicate_data_exchange_client/main.py

The DATABASE_* environment variables enable the program to connect to the database which contains the aggregated quality indicator results. In the context of the INDICATE project, that database is a PostgreSQL server with an OMOP CDM-formatted database named ohdsi.

PROVIDER_ID and DATA_EXCHANGE_ENDPOINT are used for submitting aggregated quality indicator results to the INDICATE central hub. PROVIDER_ID and PROVIDER_ID_FILE populate the same configuration option: PROVIDER_ID uses the value of the environment variable as the value of the configuration option while PROVIDER_ID_FILE uses the value of the environment variable as the name of a file from which the value of the configuration option should be read. The provider id should be a unique id, for example like those generated by the uuid program. OBSERVATION_COUNT_THRESHOLD specifies the minimum number of patients on which a quality indicator result has to be based to be transmitted to the hub.

LISTEN_ADDRESS and LISTEN_PORT define a web server endpoint that serves the review interface and accepts trigger requests.

Interfaces

Once the server is running, data collection can be triggered by sending a POST request to http://${LISTEN_ADDRESS}:${LISTEN_PORT}/api/trigger. For example via

curl --data-ascii '' http://localhost:8080/api/trigger

if LISTEN_ADDRESS has been set to localhost and LISTEN_PORT has been set to 8080. In a typical setup for INDICATE data providers, the cql-on-omop component runs in a different container, computes quality indicator results and writes those results into the database mentioned above. After the CQL engine in the cql-on-omop component finishes, it invokes the trigger endpoint to make the results available for review and subsequent transmission to the hub.

The web-based interface for reviewing results before upload can be accessed at http://${LISTEN_ADDRESS}:${LISTEN_PORT}/review using a browser.

Review Workflow

The result collection, review and upload use the following workflow

1. Access the Review Interface: Open the web interface. It may or may not show results queues for upload. If there are no results queued for upload, wait until the next execution of the CQL engine finishes.

2. CQL engine produces results: After the CQL engine has written results into the database, it will invoke the /api/trigger endpoint so that results are collected from the database.

3. Access the Review Interface: Open the web interface to see pending results.

4. Review: Examine the aggregated quality indicator results.

5. Confirm or Reject:

   • Click "Confirm and Upload" to submit to the central hub.
   • Click "Reject" to discard the data. Nothing is deleted from the local database, only the upload is canceled.

6. Verify: Check that the upload (if requested) succeeded without errors and no pending results are shown.

Code Generation

The directory openapi-generation contains the OpenAPI definition of the data exchange protocol and the configuration for the code generator. To regenerate the code of this program for a new version of the protocol

1. Copy the new OpenAPI definition to openapi-generation/api.yaml

2. Run cd openapi-generation ; ./generate.sh

3. Test the generated code

4. Commit all changes

Licenses

Unless stated otherwise for specific files, the code in this repository is made available under the Apache 2.0 license - see COPYING for the license text.

This project uses the following libraries and assets:

• Javascript library simple-datatables

  Files from the simple-datatables library have been copied into this project as static/SimpleDataTables.css and static/SimpleDataTables.js.

  The simple-datatables library is made available under the LGPLv3 license at https://github.com/fiduswriter/simple-datatables.

• "monitoring-health" SVG icon

  The SVG image "monitoring-health" has been copied into this project as static/favicon.svg.

  The "monitoring-health" SVG image by Denali Design is made available under the MIT License at https://www.svgrepo.com/svg/445892/monitoring-health.