feat(chembl_molecule): labelled synonyms/tradeNames + AACT clinical-trial synonym feed

[d0choa]

Tracking issue: opentargets/issues#4414

Summary

Adds source provenance to drug synonyms and a second synonym source mined from clinical trials.

• Schema change — chembl_molecule's synonyms and tradeNames change from array<string> to array<struct<label, source>> (mirroring output/target). Every existing ChEMBL name is tagged source: "ChEMBL". The struct propagates through drug_molecule (pure passthrough) to the final output/drug_molecule.
• AACT clinical-trial synonym feed — a PySpark port (inline in chembl_molecule.py) of the prior work/clinical_pairs/ experiment, integrated as a real pipeline source:
  • parse the OpenAI batch JSONL (aact_extraction_batch_results) → normalized drug member sets per trial
  • anchor members against an in-step ChEMBL name index (with a ≤10 ambiguity cap)
  • apply the experiment's 11 cleanup rules (control noise, drug-class/cell-therapy via word-boundary matching so real drugs like nystatin/cellcept survive, strength/%, single-char, regimen suppression, descriptor-code extraction, plural suppression) and NOVEL/PARENT_CHILD/CONFLICT status
  • keep candidates seen in ≥ 2 distinct trials, append as {label, source: "AACT"}, deduped case-insensitively against existing ChEMBL labels
  • the name fallback only ever picks a ChEMBL-source synonym, so an AACT label never becomes a molecule's display name
• Downstream — search.py flattens synonyms.label/tradeNames.label; openfda.py updated to extract .label before flattening (it consumes drug_molecule and would otherwise break on the struct schema; its stale array<string> test fixtures were also updated).
• Config — aact_extraction_batch_results: input/clinical_report/aact_extraction_batch_results wired into the chembl_molecule step (the same PIS-provided input clinical_report already uses; upstream of everything, no dependency cycle).

End state: many molecules carry synonyms from two sources (ChEMBL + AACT).

Design & plan live in the Obsidian vault: Areas/Work/Projects/Project-PTS/specs/2026-06-10-chembl-molecule-labelled-synonyms-aact-design.md (+ -plan.md).

Test Plan

• make test — full suite green (396 passed)
• uv run ruff check — clean
• Unit coverage: struct schema, batch parse (incl. malformed-line drop), anchor indexes, anchoring + status, all cleanup rules (incl. word-boundary guard), n_trials gate, case-insensitive merge dedup, end-to-end two-source molecule, openfda struct fix
• End-to-end Dataproc run (run-001, il/26.06.0-dev0, ~5 min): 2,348 molecules with both ChEMBL + AACT synonyms, 8,038 (id, label) pairs at n_trials ≥ 2, 0 non-ChEMBL tradeNames — see the validation comment below for the full table
• Scale check: anchoring ran fine on a small 2-worker cluster; embedded-newline AACT object names were a non-issue for Spark's GCS connector

Notes

• This integrates the clinical-trial synonym feed only. The earlier Probes&Drugs synonym-expansion effort is intentionally out of scope.
• Stored AACT labels are the normalized form in v1; a "most-frequent surface form" refinement is a noted follow-up.

[d0choa]

✅ Validated end-to-end on Dataproc (run-001)

Ran the chembl_molecule step on real il/26.06.0-dev0 inputs (drug JSONL + drugbank + the AACT clinical-trial batch extraction). Single step, 1× n1-standard-4 master + 2× n1-highmem-8 workers, ~5 min job. Output: gs://ot-team/ochoa/chembl_molecule_runs/run-001/intermediate/chembl_molecule.

Schema — both columns now array<struct<label:string, source:string>>.

metric	value
total molecules	2,878,135
molecules with an AACT synonym	2,393
molecules with BOTH ChEMBL + AACT	2,348
distinct (id, AACT label) pairs (n_trials ≥ 2)	8,038
tradeNames with a non-ChEMBL source	0 ✓

The "many molecules with synonyms from two sources" goal is met (2,348). The 8,038 pairs are the same order of magnitude as the experiment's design target (~few thousand at n_trials ≥ 2), slightly higher — consistent with CONFLICT cross-pollination (an abbreviation anchoring to more than one molecule) and a different release than the original experiment.

Spot-check of mined synonyms (all correct):

• ATAZANAVIR → atv, atz, boosted atazanavir, atazanavir/ritonavir
• TANDUTINIB → mln518 · SABARUBICIN → bms-195615, men-10755 (R&D codes — descriptor-code extraction working)
• FOLINIC ACID → leucovorin, lv, calcium folinate, folfiri
• CARBIDOPA → carbidopa/levodopa, sinemet cr, lcig · AVIBACTAM → ceftazidime-avibactam, caz-avi

Operational notes:

• The embedded-newline AACT batch object names were a non-issue for Spark's GCS connector.
• The deterministic sha2 entry key (replacing monotonically_increasing_id) ran in production without issue.

[d0choa]

Codex review (gpt-5.5, high effort) — outcome

Ran a Codex review of the branch. No Critical findings; it confirmed no remaining array<string> consumer of these fields. Two substantive items, both addressed:

Fixed — descriptor-code reclassification

Codex correctly flagged that rule #8 rewrote a descriptor phrase to its bare R&D code (akt inhibitor mk2206 → mk2206) after anchoring/status classification, leaving a stale status — so a code that's actually a parent/child of the anchor (canonical vs salt) or already on the anchor could slip through as NOVEL.

Fixed in fix(aact): re-resolve descriptor-extracted codes before the trial gate: code extraction now lives in a dedicated _rewrite_and_reclassify_codes stage between anchoring and cleanup that re-resolves the rewritten code, drops it if it's now on the anchor or newly over-ambiguous, and recomputes NOVEL/PARENT_CHILD/CONFLICT. New unit tests cover the redundant / parent-child / conflict paths. On the real data (run-002) this changed no output counts — it's a correctness safeguard that costs no coverage.

Investigated — keeping CONFLICT (no change)

Codex questioned keeping CONFLICT candidates (cross-molecule labels). The spec deliberately keeps them; I quantified it against run-002:

	count	share
total AACT (id, label) pairs	8,038
NOVEL-like (label nowhere else in ChEMBL)	7,040	87.6%
CONFLICT-like (label is a ChEMBL name elsewhere)	998	12.4%
…combination-looking (/, "with", "plus")	3	0.04%

The CONFLICT-like cases are overwhelmingly same-drug-family variants ChEMBL models as separate molecules without a parentId link — abiraterone ↔ abiraterone acetate, adefovir ↔ adefovir dipivoxil, adalimumab across biosimilar entries, 5-fu under a 5-FU prodrug, stereoisomers. These are real synonyms; genuine cross-drug pollution is rare (3 obvious co-administration cases out of 8,038).

Decision: keep CONFLICT as-is. A higher MIN_CONFLICT_TRIALS would drop legitimate salt/prodrug/biosimilar synonyms (the bulk of the 998) to remove a handful of junk — net-negative. The data validates the spec's original "considered-and-rejected" reasoning.

[DSuveges]

Everything seems sensible to me.

[ireneisdoomed]

Impact on latest clinical mining data: + 3048 drug/disease pairs fully mapped (240 new drugs, 943 diseases)

Great addition, thanks :)