Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
282 lines (231 loc) · 12.6 KB

ARCHITECTURE.md

File metadata and controls

282 lines (231 loc) · 12.6 KB
Raw

Architecture

Purpose

This document describes the OpenAMP Foundry architecture as scientific infrastructure, not only software architecture.

The system is designed to make antimicrobial peptide candidate selection reproducible, auditable, safety-aware, and reviewable.

Prime rule

Architecture should make unsupported claims structurally difficult.

A good architecture does not merely compute scores. It preserves provenance, uncertainty, baselines, release status, and review boundaries.

Current production architecture

candidate records
  -> loaders
  -> validation
  -> feature extraction
  -> independent scorers
  -> ensemble or configured ranker
  -> novelty checker
  -> safety-risk and feasibility checks
  -> diversity selector
  -> evidence certificates
  -> run manifest
  -> report

This is the current dry-lab candidate foundry.

It can support computational nomination and expert-review packaging.

It does not prove biological activity, safety, or clinical value.

Trust architecture

The technical architecture is surrounded by a trust architecture:

safe scope
  -> deterministic commands
  -> schemas
  -> run manifests
  -> evidence certificates
  -> benchmark cards
  -> proof ladder
  -> release status
  -> external review packets
  -> decision records

The trust layer is as important as the scoring layer.

Top-level evidence gates are review controls, not scientific verdicts. The Phase AA AARG- gate is available through the CLI and Make surface so a normal loop can fail closed when RMC, DCR, CFP, or SBW provenance artifacts are missing. Presence of those artifacts does not validate their contents or any biological claim. The Phase R SRG- gate is also available through the CLI and Make surface; only a fully ready verdict passes, while conditional, incomplete, safety-blocked, or malformed records fail closed. Its checked-in example is intentionally blocked until qualified evidence exists. The Phase Z ZAG- gate is available through the same surface and fails closed when the per-family challenge, explanation, adapter-registry, or cheap-baseline artifacts are missing. Presence of those artifacts does not establish benchmark superiority or adapter ranking authority. The Phase Y YAG- gate is available through the same surface and fails closed when the baseline comparison, feature-importance, selection-diversity, or pipeline-maturity artifacts are missing. Presence of those artifacts does not establish that the pipeline beats cheap baselines or validate biology. The Phase AB ABAG- gate is available through the same surface and fails closed when claim-strength downgrade, reviewer-decision, evidence-gap, or external- handoff artifacts are missing. Presence of those artifacts does not authenticate reviewers, validate the science, establish biology, or authorize release. Domain-review outcomes can also be bound to the exact frozen pilot-evidence package JSON through pep_sha256 and the package-aware CLI path. This closes a package-revision ambiguity while preserving ID-only compatibility for legacy records; it does not authenticate reviewers or establish scientific validity.

Longer-range architecture

The mature system should become a wet-lab compression engine:

candidate generation or import
  -> dry-lab validation and ranking
  -> benchmark-challenged proxy models
  -> uncertainty-aware panel construction
  -> qualified external review
  -> structured result summaries
  -> result-input validation with retained file-level errors and identity checks
  -> calibration intake
  -> recalibration gate
  -> human decision record
  -> next-round selection

The purpose of added layers is not simulation theater.

The purpose is to improve which questions qualified humans choose to test next.

Result ingestion is fail-closed at the review boundary. Legacy directory loading remains warning-compatible for callers that only need valid records, but calibration and reporting workflows retain every schema-invalid file as structured provenance. A partial directory therefore cannot silently become a recalibration cohort; this catches input completeness problems, not bad assay science. Duplicate result IDs and duplicate panel candidate IDs also block clean intake because they make evidence identity ambiguous. Missing or non-directory result paths fail before report generation; an existing empty directory is the explicit no-results state. Control-failed observations remain in the audit report but are excluded from per-assay actual predicates, cohort metrics, and interpretable per-candidate outcome flags. Raw outcome fields and failed-result IDs remain available for audit; failed controls still block recalibration. Batch-level lab-result summaries apply the same boundary: raw qualitative counts remain available for audit, while by_usable_qualitative_result is restricted to observations whose positive and negative controls both passed. Human-readable reports label the two views separately. Lab-result reports also expose whether raw_data_sha256 is absent, partially declared, or declared for every loaded result. This is provenance visibility, not independent verification of the raw assay files and not a biological validation gate; legacy results remain accepted when the optional field is missing. An optional raw_data_file reference can be verified by supplying --raw-data-dir; the verifier hashes files independently, rejects path escape, and blocks clean calibration intake on missing or mismatched files. This binds the declaration to file bytes only and does not validate assay contents. The loader also validates assay_date as a real, canonical YYYY-MM-DD calendar date because the generic JSON Schema helper does not enforce format annotations. Invalid dates remain structured file errors and cannot enter sorted reports or calibration metrics. Intake reports also classify explicit SYNTHETIC labels by result ID. These records remain available for demonstrations and audit, but the recalibration gate fails closed when any synthetic-labeled result is present. Unclassified records are not inferred to be real wet-lab evidence. Results whose candidate IDs are absent from the submitted calibration panel are retained as orphan provenance but block clean intake because they cannot be joined to prior predictions. This prevents a broader result directory from silently inflating a panel-specific cohort. New panels may additionally carry the certificate hash already required on each lab result. When present, intake compares hashes for every tested candidate and blocks mismatches or partial opted-in coverage. Legacy panels without that optional column are reported as certificate identity not available, not silently verified. New panels may also carry a frozen panel_id in the panel and result records. Intake blocks multiple submitted panel IDs, mismatches, or partial opted-in coverage. Legacy panels without that optional field report panel identity not available.

Package map

Package Role
openamp_foundry.data Loading and normalizing candidate/reference data.
openamp_foundry.features Physicochemical feature extraction.
openamp_foundry.scoring Activity-likeness, safety-risk, novelty, feasibility, ensemble, and related scorers.
openamp_foundry.selection Ranking, diversity selection, and panel construction guards.
openamp_foundry.evidence Evidence certificate generation and validation.
openamp_foundry.reports Human-readable and machine-readable reports.
openamp_foundry.benchmark Leakage checks, baseline comparisons, ablations, and benchmark scaffolding.
openamp_foundry.generators Safe, bounded toy candidate generation.
openamp_foundry.simulation Experimental virtual-assay proxy interfaces and ablations.
openamp_foundry.calibration Structured result intake, recalibration gates, and proposal generation.
openamp_foundry.active_learning Informative next-batch selection under uncertainty.
openamp_foundry.analysis Diversity, similarity, structural-class, and audit helpers.
openamp_foundry.qc Quality-control checks for pre-synthesis, hemolysis risk, and feasibility.
openamp_foundry.utils I/O helpers, hashing, and shared utilities.

Package behavior must remain aligned with docs under docs/PROJECT_INDEX.md.

Core artifacts

Artifact Purpose Source doc
Evidence certificate Explains candidate selection or rejection. EVIDENCE_CERTIFICATE.md
Run manifest Records command, inputs, config, commit, hashes, and claim boundaries. RUN_MANIFEST_STANDARD.md
Benchmark card Documents benchmark purpose, data, baselines, and limits. BENCHMARK_GOVERNANCE.md
Dataset card Documents source, license, labels, preprocessing, bias, and release status. DATA_GOVERNANCE.md
Model or adapter card Documents model/adapter scope, benchmarks, limitations, and release status. MODEL_CARD_TEMPLATE.md
Review packet Packages artifacts for qualified external review. EXTERNAL_REVIEW_PACKET.md
Decision record Records important governance decisions. DECISION_RECORD_TEMPLATE.md

Threat model

The system is designed to reduce these failures:

Failure Mitigation
Cherry-picking candidates Predefined selection rules, manifests, and evidence certificates.
Rediscovering known references Novelty audit and nearest-neighbor reporting.
Unsafe optimization Safety policy, release policy, safety-risk penalties, human review.
Model self-confirmation Cheap-baseline comparisons and benchmark governance.
Dataset leakage Leakage checks, dataset cards, split documentation.
Overclaiming Proof ladder, claim review checklist, reviewer onboarding.
Simulation theater Virtual assay scope, simulation benchmark, ranking gates.
Unreviewed release Model release policy, release checklist, CODEOWNERS intent.
Agent drift Agent operating contract, issue labels, human-agent collaboration model.
Artifact drift Schema registry and artifact versioning policy.

Data flow

  1. Load candidate records.
  2. Normalize and validate records.
  3. Reject invalid inputs with explicit errors.
  4. Compute dry-lab features.
  5. Score independently.
  6. Apply configured ranking and safety-aware tradeoffs.
  7. Audit novelty and similarity.
  8. Select diverse candidates or panel roles.
  9. Write structured outputs.
  10. Generate evidence certificates.
  11. Generate run manifest.
  12. Generate human-readable report.
  13. Validate artifacts where schemas exist.
  14. Map claims to proof-ladder level.

Result-learning flow

When qualified result summaries exist:

  1. Ingest structured result summaries.
  2. Join results to prior predictions.
  3. Preserve controls, quality flags, and limitations at a safe abstraction level.
  4. Generate calibration intake artifact.
  5. Run recalibration gate.
  6. Record whether recalibration may be considered.
  7. Generate proposal if allowed.
  8. Require human review and decision record.
  9. Apply changes only after review.
  10. Preserve rejections as useful evidence.

Extension points

Future extension points include:

  • scorers;
  • external predictors;
  • virtual-assay proxies;
  • dataset loaders;
  • benchmark modules;
  • report generators;
  • evidence-certificate fields;
  • review packet generators;
  • calibration proposal engines.

Every extension point should answer:

  1. What does this component do?
  2. What is it not for?
  3. Does it need a card?
  4. What baseline challenges it?
  5. Can it affect ranking?
  6. What is its release status?
  7. What failure mode is expected?

Use ADAPTER_AUTHOR_GUIDE.md for external adapters.

Adapter policy

Adapters must not silently:

  • transmit sequences or sensitive metadata;
  • download model artifacts;
  • change ranking authority;
  • hide unavailable dependencies;
  • convert external scores into proof;
  • publish restricted outputs.

Default adapter mode should be off or info until gates support stronger use.

Schema and compatibility policy

Schemas and structured artifacts should follow:

Breaking changes to core artifacts require compatibility review.

Final standard

OpenAMP architecture is successful when a skeptical reviewer can inspect an artifact and understand how it was produced, what it supports, what it does not support, and what review is still required.