Skip to content

Doc legacy database migration/data synchronization job.#284

Open
doug-lovett wants to merge 1 commit into
bcgov:mainfrom
doug-lovett:33734-drs-legacy-migration
Open

Doc legacy database migration/data synchronization job.#284
doug-lovett wants to merge 1 commit into
bcgov:mainfrom
doug-lovett:33734-drs-legacy-migration

Conversation

@doug-lovett

Copy link
Copy Markdown
Collaborator

Issue #: /bcgov/entity#33734

Description of changes:
Semi-automate this task as a GCP cloud run like job. Needs to run locally for now as running a BCGOV VPN client is required to access the legacy database.
The job description is:
Synchronize legacy document records in the Oracle globalp db correspondence schema with the DRS.
Legacy database access requires running a VPN.
Synchronization is in one direction: from legacy to DRS.
Synchronization is in 2 steps:
1. New document records created since the last time this job ran.
2. Updates to document records that exist in the DRS and where changes have occurred since the last time
this job ran (scanning a document).

Detail:
    Execute a summary query to get the most recent DRS scanned doc timestamp and legacy document record key for
    each DRS document class.
    For each document class:
    1. Fetch new records from the legacy database. For each record:
        a. If there is a legacy document, save it to cloud storage.
        b. Insert into the tracking table staging_legacy_documents with the cloud storage path if available.
        c. Insert into the DRS documents table.
        d. Insert into the DRS document_scanning table if the legacy record includes scanning information.
    2. Fetch updated records from the legacy database that have documents.
        a. Save the legacy document to cloud storage.
        b. Update into the tracking table staging_legacy_documents with the cloud storage path and other data.
        c. Update the DRS documents table.
        d. Insert into the DRS document_scanning table if the legacy record includes scanning information.

    Optionally save csv formatted summary/status information for the job by document class.

Signed-off-by: Doug Lovett <doug@diamante.ca>
@doug-lovett doug-lovett self-assigned this Jun 29, 2026
@doug-lovett doug-lovett requested review from argush3 and eve-git June 29, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant