Appointment Mapper by AdityaKBhadragond14 · Pull Request #11 · DigitalHealthIntegration/ETL-German-FHIR-Core

AdityaKBhadragond14 · 2024-03-06T06:44:06Z

Description:
I have created a custom schema "Appointment" to convert the FHIR Appointment resources to OMOP. The schema for "Appointment" table looks like these:

appointment_id -> auto generated id
care_site_id -> care site id where the appointment was scheduled
person_id -> person id for whom the appointment was scheduled
visit_occurrence_id -> visit that the appointment is linked to
scheduled_date_time -> Appointment scheduled datetime
created_datetime -> Appointment created datetime
appointment_status -> Scheduled, booked, completed, etc..
fhir_logical_id -> FHIR Resource Id.
questionnaire_response_id -> questionnaire response id for the appointment resource
appointment_description -> description of the appointment from the appointment resource
appointment_reason -> reason for the appointment

I have added the AppointmentMapper and AppointmentStepListener and other necessary files just following the ETL pipeline codebase approach like for other tables.
Initially the appointment table will be filled without the person_id, visit_occurrence_id and care_site_id. In the post process I have wrote a script post_process_provenance_appointment.sql which will update the entries in the appointment table with person_id, visit_occurrence_id and care_site_id.

…com/DigitalHealthIntegration/ETL-German-FHIR-Core into ab/appointment_mapper

SatwikSShanbhag

Plz look at the comments

SatwikSShanbhag · 2024-03-13T06:50:29Z

+                dbMappings.setFindIcdSnomedMapping(repositories.getIcdSnomedRepository().getIcdSnomedMap());
+                dbMappings.setFindOrphaSnomedMapping(
+                        repositories.getOrphaSnomedMappingRepository().getOrphaSnomedMap());
+                dbMappings
+                        .getOmopConceptMapWrapper()
+                        .setFindValidSnomedConcept(
+                                repositories.getConceptRepository().findValidConceptId(VOCABULARY_SNOMED));
+                dbMappings
+                        .getOmopConceptMapWrapper()
+                        .setFindValidIPRDConcept(
+                                repositories.getConceptRepository().findValidConceptId(VOCABULARY_IPRD));
+                dbMappings.getOmopConceptMapWrapper().setFindValidWHOConcept(
+                        repositories.getConceptRepository().findValidConceptId(VOCABULARY_WHO)
+                );
+                dbMappings
+                        .getOmopConceptMapWrapper()
+                        .setFindValidIcd10GmConcept(
+                                repositories.getConceptRepository().findValidConceptId(VOCABULARY_ICD10GM));
+            }
+            dbMappings.setFindHardCodeConcept(
+                    repositories.getSourceToConceptRepository().sourceToConceptMap());
+            }


can you set only those things whose getters are used in mapper. remove unwanted setters.

SatwikSShanbhag · 2024-03-13T06:53:37Z

+        var wrapper = new OmopModelWrapper();
+        var appointmentLogicalId = fhirReferenceUtils.extractId(srcAppointment);
+        if (Strings.isNullOrEmpty(appointmentLogicalId)){
+            return null;


if you are skipping the resource keep that count and show it in summary log with reason

SatwikSShanbhag · 2024-03-13T06:56:34Z

+        }
+        var appointmentStatus = srcAppointment.getStatusElement().getValueAsString();
+        if (Strings.isNullOrEmpty(appointmentStatus)){
+            return null;


same as above and also it will be good if status is missing in the resource we can add custom valid status.

Adding custom status can be dangerous in this case because we can't say that the appointment is booked or done directly from the code it should come from the resource itself. If the status does not exist it is better to skip it.

SatwikSShanbhag · 2024-03-13T07:06:38Z

+        var createdDateTime = srcAppointment.getCreated().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
+        var appointmentDateTime = srcAppointment.getStart().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();


are we sure that srcAppointment.getCreated() and srcAppointment.getStart() always be non-null?
can we check that first and then assign the value to variable even we can skip if those values are not found

SatwikSShanbhag · 2024-03-13T07:07:13Z

+        if (createdDateTime == null || appointmentDateTime == null){
+            return null;
+        }
+        var appointment = org.miracum.etl.fhirtoomop.model.omop.Appointment.builder()


org.miracum.etl.fhirtoomop.model.omop import this above

This cannot be done because these will cause the conflict between the fhir appointment resource import.

SatwikSShanbhag · 2024-03-13T07:08:44Z

+        var provenanceLogicalId = fhirReferenceUtils.extractId(srcProvenance);
+        var questionnaireResponseLogicalId = fhirReferenceUtils.extractId(srcProvenance.getEntityFirstRep().getWhat().getReferenceElement().getResourceType(),srcProvenance.getEntityFirstRep().getWhat().getReferenceElement().getIdPart());
+        if (Strings.isNullOrEmpty(provenanceLogicalId)){
+            return null;


same keep count of skipped resources

SatwikSShanbhag · 2024-03-13T07:09:21Z

+        }
+        if (!srcProvenance.hasTarget()){
+            return null;
+        }


same keep count of skipped resources

SatwikSShanbhag · 2024-03-13T07:10:36Z

+            var resourceType = provenanceTarget.getReferenceElement().getResourceType();
+            var resourceId = provenanceTarget.getReferenceElement().getIdPart();
+            var resourceLogicalId = fhirReferenceUtils.extractId(resourceType, resourceId);
+            var combinedId = resourceLogicalId + ";" + questionnaireResponseLogicalId;


use string concate method here

SatwikSShanbhag · 2024-03-13T07:16:30Z

-import org.miracum.etl.fhirtoomop.model.omop.ConditionOccurrence;
-import org.miracum.etl.fhirtoomop.model.omop.DeviceExposure;
-import org.miracum.etl.fhirtoomop.model.omop.DrugExposure;
-import org.miracum.etl.fhirtoomop.model.omop.Measurement;
-import org.miracum.etl.fhirtoomop.model.omop.OmopObservation;
-import org.miracum.etl.fhirtoomop.model.omop.Person;
-import org.miracum.etl.fhirtoomop.model.omop.ProcedureOccurrence;
-import org.miracum.etl.fhirtoomop.model.omop.Provider;
-import org.miracum.etl.fhirtoomop.model.omop.VisitDetail;
-import org.miracum.etl.fhirtoomop.model.omop.VisitOccurrence;
-import org.miracum.etl.fhirtoomop.model.omop.Specimen;
+import org.miracum.etl.fhirtoomop.model.omop.*;


remove all import only keep the import of model which is used in wrapper

SatwikSShanbhag

Looks good to me have a look at one comment

SatwikSShanbhag · 2024-03-13T11:56:20Z

        if (createdDateTime == null || appointmentDateTime == null){
            return null;
        }


wherever you are skipping the resource add counter

joiskash · 2024-04-15T11:14:38Z

Can we also add

Appointment description
Reason for booking appointment i.e. for what service.

joiskash

Thanks, please address the comments.

joiskash · 2024-04-15T11:16:54Z

+            .next(stepProcessProvenance)
+            .next(stepProcessAppointments)


Suggested change

.next(stepProcessProvenance)

.next(stepProcessAppointments)

.next(stepProcessProvenance)

.next(stepProcessAppointments)

joiskash · 2024-04-15T11:17:57Z

+          IGenericClient client,
+          IParser fhirParser) {
+
+    var resourceType = "Appointment";


Suggested change

var resourceType = "Appointment";

var resourceType = ResourceType.Appointment.getDisplay();

joiskash · 2024-04-15T11:19:02Z

+    if (StringUtils.isBlank(fhirBaseUrl)) {
+      return createResourceReader(resourceType, dataSource);
+    }
+    return fhirServerItemReader(client, fhirParser, ResourceType.APPOINTMENT.getDisplay(), "");


Please reuse the variable resourceType set above

joiskash · 2024-04-15T11:20:02Z

+import org.springframework.util.StopWatch;
+import javax.sql.DataSource;
+
+import static org.miracum.etl.fhirtoomop.Constants.*;


Please avoid * import

These import is not being used anywhere. Removing it.

joiskash · 2024-04-15T11:22:40Z

+                .scheduledDateTime(appointmentDateTime)
+                .createdDateTime(createdDateTime)
+                .appointmentStatus(appointmentStatus)
+                .fhirLogicalId(appointmentLogicalId).build();


Let us also please consider the appointment description

joiskash · 2024-04-15T11:23:54Z

+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+import javax.persistence.*;


Please fix this import statement

joiskash · 2024-04-15T11:30:54Z

+			select * ,
+			split_part(data_two, ';', 2) as questionnaire_response_logical_id,
+			split_part(data_two, ';', 1) as appointment_logical_id
+			from cds_etl_helper.post_process_map ppm where "type" = 'PROVENANCE' and data_one = 'Appointment'


Please check if Appointment should be APPOINTMENT to maintain uniformity in variable naming.

date_one is supposed to be in lowercase in the post_process_map table because it can also contain not only texts but other type of data like date and date time.

SatwikSShanbhag

please address the comment.

SatwikSShanbhag · 2024-05-06T11:15:26Z

            noFhirReferenceCounter.increment();
            return null;
        }
+        var appReasonList = srcQuestionnaireResponse.getItem().stream().filter(questionnaireResponseItemComponent -> questionnaireResponseItemComponent.getLinkId().equals("service-type")).toList();


can we change variable name as appointmentReasonList to avoid confusion.
also define a constant for service-type and check whether we can process appointment reason thing only for schedule appointment questionnaire response
thanks

SatwikSShanbhag

Looks good to me.

SatwikSShanbhag and others added 8 commits February 27, 2024 14:07

added questionnaire response

f31cdce

refactored code

bdee401

refactored code

cc55c2f

Merge branch 'ss/add_questionnaireResponse_mapper' of https://github.…

34e7a00

…com/DigitalHealthIntegration/ETL-German-FHIR-Core into ab/appointment_mapper

Added Provenance to Post Process Map.

8ecb3d5

Added Custom Appointment Schema and mapper.

b589cd0

corrected the sql script.

e7856e6

corrected the sql script.

7ddfbd1

AdityaKBhadragond14 requested review from SatwikSShanbhag and nsabale7 March 13, 2024 06:14

SatwikSShanbhag requested changes Mar 13, 2024

View reviewed changes

changes.

5503bb4

AdityaKBhadragond14 requested a review from SatwikSShanbhag March 13, 2024 10:05

SatwikSShanbhag approved these changes Mar 13, 2024

View reviewed changes

Merge branch 'main' into ab/appointment_mapper

f1d1b3d

joiskash requested changes Apr 15, 2024

View reviewed changes

AdityaKBhadragond14 and others added 4 commits April 16, 2024 18:57

changes.

bb3631a

Added appointment description to the appointment table.

fad90fb

Merge branch 'main' into ab/appointment_mapper

435e524

added appointment reason.

9ee6f22

AdityaKBhadragond14 requested a review from SatwikSShanbhag May 6, 2024 10:29

SatwikSShanbhag requested changes May 6, 2024

View reviewed changes

refactoring.

ae74c08

AdityaKBhadragond14 requested a review from SatwikSShanbhag May 6, 2024 11:41

SatwikSShanbhag approved these changes May 13, 2024

View reviewed changes

		var createdDateTime = srcAppointment.getCreated().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
		var appointmentDateTime = srcAppointment.getStart().toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();

	var resourceType = "Appointment";
	var resourceType = ResourceType.Appointment.getDisplay();

Conversation

AdityaKBhadragond14 commented Mar 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SatwikSShanbhag left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SatwikSShanbhag Mar 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SatwikSShanbhag left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joiskash commented Apr 15, 2024

Uh oh!

joiskash left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SatwikSShanbhag left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SatwikSShanbhag left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AdityaKBhadragond14 commented Mar 6, 2024 •

edited

Loading

SatwikSShanbhag Mar 13, 2024 •

edited

Loading