Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
35a7681
added schema for cholera - initial file
Segundev Sep 11, 2025
fc8b3bf
cholera schema updated
Segundev Sep 12, 2025
14c30f7
change to yaml file
Segundev Sep 13, 2025
8944a38
klebsiella metadata v1
Segundev Sep 25, 2025
e36e35f
removed duplicate missing
Segundev Sep 25, 2025
6380a54
added description to both pathogen
Segundev Sep 26, 2025
2b112e3
schema for mpox
Segundev Oct 8, 2025
9263f47
removed picklist and change to free text on "prior_antiviral_treatmen…
Segundev Oct 8, 2025
065fda9
cholera v2 - file being worked on
Segundev Oct 16, 2025
b4ad2e0
deleted file
Segundev Oct 16, 2025
baf8803
moved schemas to a schema folder
Segundev Oct 16, 2025
2319e37
schemas in schema folder
Segundev Oct 16, 2025
ce695e8
muse alternative, tsv to json init
Segundev Oct 16, 2025
ac38b39
deleted schema made for testing faulty definitions
Segundev Oct 21, 2025
9024ec3
added schemas for malaria and covid - schemas are still a work in pro…
Segundev Oct 24, 2025
e53dfff
sample data payload
Segundev Oct 24, 2025
f8ffd84
v3 version with array payload - work in progress
Segundev Oct 28, 2025
1ea3bde
moved files to pathogens folders
Segundev Oct 29, 2025
3a14b3d
cholera schemas in versions and its own folder
Segundev Oct 29, 2025
cc7c39d
klebsiella schemas in versions and its own folder
Segundev Oct 29, 2025
5bd005f
malaria human schemas in versions and its own folder
Segundev Oct 29, 2025
cdcb547
malaria vector schemas in versions and its own folder
Segundev Oct 29, 2025
592303a
mpox schemas in versions and its own folder
Segundev Oct 29, 2025
ff4889a
sars-cov schemas in versions and its own folder
Segundev Oct 29, 2025
fab7879
version 5 schema for cholera and klebsiella - user friendly errors an…
Segundev Oct 30, 2025
dc1e612
change South_Africa to South Africa
Segundev Oct 30, 2025
991ad5e
update
Segundev Nov 4, 2025
9862d8b
sample
Segundev Nov 4, 2025
38c1a42
rest http
Segundev Nov 4, 2025
6be8852
version 5 update
Segundev Nov 14, 2025
c55edef
custom schema
Segundev Nov 14, 2025
c3ecb1d
updated v5
Segundev Nov 20, 2025
40690d5
custom v5
Segundev Nov 20, 2025
40e8c1d
rest http
Segundev Nov 20, 2025
4e1e1ae
rest http
Segundev Nov 20, 2025
adb3804
done with schema update - pending string for empty values - need conf…
Segundev Dec 5, 2025
ddbd13e
Extra change to include type and maxlength
Segundev Dec 5, 2025
1fc0474
Done with Kleb
Segundev Dec 5, 2025
292f275
done with malaria human
Segundev Dec 5, 2025
01b1b45
oneOf on type number
Segundev Dec 5, 2025
c78baaf
commit customv5 malaria vector
Segundev Dec 5, 2025
b7fabe2
done with malaria vector
Segundev Dec 5, 2025
0c27149
done with mpox
Segundev Dec 5, 2025
feb0165
done with sars cov
Segundev Dec 5, 2025
e6e85f2
changed sequencing to sequence
Segundev Dec 10, 2025
74c6e00
ISO Code on provinces
Segundev Dec 10, 2025
d703f37
updated schema field names
Segundev Dec 10, 2025
59e30d3
prior_SARS-CoV-2_treatment_agent added antiviral to match template
Segundev Dec 11, 2025
1344bca
added fasta_file_name and fasta_header_name
Segundev Dec 11, 2025
ab95d53
added principe to sao tome
Segundev Dec 11, 2025
3ecb8e8
remove required from host_disease and host_scientific_name hint text
Segundev Dec 11, 2025
3394108
changed date field empty value to type null
Segundev Dec 11, 2025
8d865e5
change empty date to type null - malaria vector : change two field na…
Segundev Dec 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 27 additions & 9 deletions api/rest.http
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,19 @@
@test_file_md5 = cc70b8e811b39c98a938edf87145e34d

# Fixed IDs for Keycloak permissions
@study_id = study1
@analysis_id = analysis001
@study_id = cholera-267
@analysis_id = analysis003

# Authentication config
@username = admin
@username = system.admin@agari.tech
@password = pass123
@realm = agari
@client_id = dms
@client_secret = dms-secret
@client_secret = VDyLEjGR3xDQvoQlrHq5AB6OwbW0Refc

### 1. LOGIN to get access token
# @name login
POST http://keycloak.local/realms/{{realm}}/protocol/openid-connect/token
POST https://keycloak-staging.openup.org.za/realms/{{realm}}/protocol/openid-connect/token
Content-Type: application/x-www-form-urlencoded

username={{username}}
Expand All @@ -31,7 +31,7 @@ username={{username}}
@token = {{login.response.body.access_token}}

### 2. CREATE STUDY
POST http://song.local/studies/{{study_id}}/
POST https://song-staging.openup.org.za/studies/{{study_id}}/
Authorization: Bearer {{token}}
Content-Type: application/json

Expand All @@ -46,21 +46,39 @@ Content-Type: application/json
}
}


### 2.b GET STUDIES

GET https://song-ilifu.openup.org.za/studies/all


### 3a. SUBMIT ANALYSIS
# @name submitAnalysis
POST http://song.local/submit/{{study_id}}/
POST https://song-staging.openup.org.za/submit/{{study_id}}/
Authorization: Bearer {{token}}
Content-Type: application/json

< ./data/analysis_submission.json
< ./metadata/sample/test_data.json

### 3b. GET ANALYSIS DETAILS
# Extract analysis_id from step 4 response and get full analysis details
@submitted_analysis_id = {{submitAnalysis.response.body.analysisId}}
# @name getAnalysis
GET http://song.local/studies/{{study_id}}/analysis/{{submitted_analysis_id}}
GET https://song-staging.openup.org.za/studies/{{study_id}}/analysis/{{submitted_analysis_id}}
Authorization: Bearer {{token}}


### 3c. GET SCHEMAS REGISTRATION
GET https://song-staging.openup.org.za/schemas/

### 3d. POST SCHEMAS
POST https://song-staging.openup.org.za/schemas
Authorization: Bearer {{token}}
Content-Type: application/json

< ./metadata/sample/test-schema.json


### 4a. INITIALIZE SCORE UPLOAD
# Extract object_id from step 5 response
@object_id = {{getAnalysis.response.body.files[0].objectId}}
Expand Down
101 changes: 101 additions & 0 deletions api/schemas.http
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# @folio = http://localhost:8000
@folio = https://folio-staging.openup.org.za
# @folio = https://folio-ilifu.openup.org.za

# @keycloak = http://keycloak.local
@keycloak = https://keycloak-staging.openup.org.za
# @keycloak = https://keycloak-ilifu.openup.org.za

@realm = agari
@client_id = dms
@client_secret = VDyLEjGR3xDQvoQlrHq5AB6OwbW0Refc

@username = system.admin@agari.tech

@password = pass123


### 1. LOGIN to get access token
# @name login
POST {{keycloak}}/realms/{{realm}}/protocol/openid-connect/token
Content-Type: application/x-www-form-urlencoded

username={{username}}
&password={{password}}
&grant_type=password
&client_id={{client_id}}
&client_secret={{client_secret}}

###
@token = {{login.response.body.access_token}}


##########################
### PATHOGENS
##########################

# @name list_pathogens
GET {{folio}}/pathogens

###

@pathogen_id = 88938f26-94a6-4ea9-8d16-18eb21e9772d

# @name get_pathogen
GET {{folio}}/pathogens/{{pathogen_id}}

###

# @update_pathogen
PUT {{folio}}/pathogens/{{pathogen_id}}
Authorization: Bearer {{token}}
Content-Type: application/json

{
"schema_id": "8f3fc228-df4f-4994-a20f-38ebfa4d61f2"
}


##########################
### SCHEMAS
##########################

# @name list_schemas
GET {{folio}}/schemas
Authorization: Bearer {{token}}

###

# @name register_schema
POST {{folio}}/schemas/
Authorization: Bearer {{token}}
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary456

------WebKitFormBoundary456
Content-Disposition: form-data; name="metadata"

{
"pathogen_id": "{{pathogen_id}}",
"name": "mpox_schema",
"description": "Schema for mpox pathogen"
}

------WebKitFormBoundary456
Content-Disposition: form-data; name="file"; filename="cholera-v5.json"
Content-Type: application/json

< ./metadata/schemas/mpox/custom-mpox-v5.json

------WebKitFormBoundary456--



###

@schema_id = 108f50fa-7082-4525-a721-7cdef52d78bf

# @name get_schema
GET {{folio}}/schemas/{{schema_id}}
Authorization: Bearer {{token}}

###
44 changes: 0 additions & 44 deletions data/analysis_submission.json

This file was deleted.

6 changes: 0 additions & 6 deletions data/test_data.fasta

This file was deleted.

17 changes: 13 additions & 4 deletions helm/keycloak/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ ingress:
paths:
- path: /
pathType: Prefix
service:
name: keycloak
port:
number: 8080 # <— explicitly tell ingress to target 8080
tls: []

extraEnv:
Expand All @@ -38,6 +42,12 @@ extraEnv:
value: "false"
- name: JAVA_OPTS_APPEND
value: "-Djava.awt.headless=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.net.preferIPv4Stack=true -server -Xms512m -Xmx768m"
- name: KC_HOSTNAME
value: keycloak-ilifu.openup.org.za
- name: KC_PROXY
value: "edge"
- name: KC_CORS_ORIGINS
value: "*"

probes:
liveness:
Expand All @@ -46,7 +56,7 @@ probes:
enabled: true
tcpSocket:
port: 8080
initialDelaySeconds: 120
initialDelaySeconds: 120
periodSeconds: 30
timeoutSeconds: 15
failureThreshold: 12
Expand All @@ -59,14 +69,13 @@ keycloak:
database: keycloakDb
username: admin
password: keycloak-db-pass-123

# Admin user
admin:
username: admin
password: admin123

# Keycloak settings
hostname: keycloak.local
healthEnabled: true
startupMode: start-dev

Expand Down
113 changes: 113 additions & 0 deletions metadata/museAlternative/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>TSV to JSON Converter</title>
<style>
body {
font-family: Arial, sans-serif;
max-width: 800px;
margin: 20px auto;
padding: 20px;
}

.file-section {
margin: 20px 0;
padding: 15px;
border: 1px solid #ccc;
border-radius: 5px;
}

button {
background-color: #4CAF50;
color: white;
padding: 10px 20px;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 16px;
}

button:hover {
background-color: #45a049;
}

button:disabled {
background-color: #cccccc;
cursor: not-allowed;
}

pre {
background-color: #f4f4f4;
padding: 15px;
border-radius: 5px;
overflow-x: auto;
}

.input-group {
margin: 10px 0;
}

label {
display: block;
margin-bottom: 5px;
font-weight: bold;
}

input[type="text"] {
width: 100%;
padding: 8px;
border: 1px solid #ccc;
border-radius: 4px;
box-sizing: border-box;
}
</style>
</head>

<body>
<h1>TSV to JSON Converter with File Metadata</h1>

<div class="file-section">
<h2>1. Upload TSV Metadata File</h2>
<input type="file" id="tsvInput" accept=".tsv" />
</div>

<div class="file-section">
<h2>2. Upload Genome Files (FASTA, GenBank, etc.)</h2>
<input type="file" id="genomeFiles" multiple accept=".fasta,.fa,.gb,.genbank,.fastq" />
<p><small>You can select multiple files</small></p>
</div>

<div class="file-section">
<h2>3. Study Information</h2>
<div class="input-group">
<label for="studyId">Study ID:</label>
<input type="text" id="studyId" placeholder="e.g., STUDY001" />
</div>
<div class="input-group">
<label for="analysisName">Analysis Type Name:</label>
<input type="text" id="analysisName" placeholder="e.g., Genome Assembly" />
</div>
<div class="input-group">
<label for="analysisVersion">Analysis Version:</label>
<input type="text" id="analysisVersion" placeholder="e.g., 1.0" />
</div>
</div>

<button id="processBtn" disabled>Process Files</button>

<div class="file-section">
<h2>Output:</h2>
<pre id="output">Upload files and click "Process Files" to see the result...</pre>
</div>

<!-- Include crypto-js library for MD5 hashing -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.1.1/crypto-js.min.js"></script>

<!-- Link your JS file -->
<script src="tsvtojson.js"></script>
</body>

</html>
Loading