cbdb-inputter-agent

⚠️ Early-stage project, under active development. The core client (Milestones 1–8 — auth, mutation API, extraction/staging workflow, CLI, live-validated against a real instance, staging-batch preview) is implemented and tested, but the CLI surface, staging-file schema, and skill wiring should all still be considered subject to change. Read AGENTS.md before touching production data with this.

An authorized API client for cbdb-online-main-server (the CBDB online data-entry system, live at https://input.cbdb.fas.harvard.edu) that submits biographical records via its /api/v2/* Mutation API, in place of manually clicking through the web UI — while keeping every write attributable and auditable.

Start here:

• docs/00-target-system-brief.md — what we know about the target system's auth, API, and audit logging (includes facts confirmed live against a real instance during Milestone 7).
• docs/07-api-md-digest.md — the target system's own API.md (which is under continuing revision upstream), digested down to what actually binds this client, with a sync stamp and a re-sync procedure.
• docs/01-implementation-plan.md — this repo's architecture and milestone-by-milestone build plan.
• docs/03-extraction-review-workflow.md — how unstructured source material (e.g. a classical-Chinese biography) becomes a human-reviewable, bulk-editable staging file before anything is submitted.
• docs/04-field-whitelists.md — per-resource allowed fields and composite primary keys for all 13 supported resources.
• docs/09-code-labels-and-snapshot.md — how every code in a batch gets a human-readable name (and an address its full parent chain), using CBDB's weekly SQLite build, plus the hard line between what that snapshot may and may not be used for.
• docs/08-review-interface-design.md — the offline review page (review/batch.html) that loads a batch's review.json, lets a human resolve conflicts and edit values in bulk, and exports a decisions.json that apply-review writes back into the YAML.
• docs/06-staging-preview-design.md — the generated, read-only preview.md (with a best-effort live old-vs-new diff) that validate --staging refreshes on every run, for reviewing a batch without reading raw YAML.
• docs/10-office-aggregate-design.md and docs/11-salt-administration-design.md — the two global reference data designs: writing OFFICE_CODES through the office aggregate, and place names plus hierarchy edges through ADDR_CODES / ADDR_BELONGS_DATA. Read the second before touching either place-name table.
• cases/README.md — the index of every job this repo has been used for, what each one produced and what actually landed. The batches themselves are gitignored, so this is the only record a clone carries.
• docs/02-review-log.md — the review-agent + codex findings and fixes for every milestone, including the two real bugs Milestone 7's live validation caught that no amount of mocked testing would have found.
• AGENTS.md — hard rules for any agent (or human) working in this repo, including "Where code goes": what belongs in src/, review/, cases/ and data/, and why a directory is never named after a job.

Setup

1. cp .env.sample .env
2. Log into the target CBDB instance's web UI once, go to /profile, create a Personal Access Token, and paste it into .env as CBDB_API_TOKEN.
3. Leave CBDB_DRY_RUN=true until you've validated your input data against a local or test instance.
4. pip install -e . for the runtime package, or pip install -r requirements-dev.txt to also run the test suite (pytest).

Usage

python -m cbdb_agent validate --staging <path> | --input <path>  [--env <path>]
python -m cbdb_agent submit   --staging <path> | --input <path>  [--dry-run] [--env <path>]

See skills/cbdb-data-entry/SKILL.md for the full CLI reference (exit codes, archiving behavior, the "defer" conflict-resolution semantics) and both entry paths — already-structured JSON records, or an unstructured-source-text extraction workflow that drafts a bulk-editable YAML staging file for human review before anything is submitted.

Layout

src/cbdb_agent/      the abstract program - the only Python installed and imported
review/              the pages a human opens, plus one entry point per case
cases/<case-id>/      one job's content; most jobs have only a README
docs/                the reasoning, and docs/02-review-log.md as the append-only log
data/                generated and submitted artifacts (gitignored)

AGENTS.md's "Where code goes" is the binding version of this, with the rule that nothing under src/ or review/ may be named after a job.

Status

All 8 planned milestones are implemented and tested (758 tests, no real network calls), and — for the core write path — validated live against a real local cbdb-online-main-server instance. Work has continued past them into global reference data: the office aggregate (docs/10) and the place-name tables (docs/11). See docs/01-implementation-plan.md for the milestone list, docs/02-review-log.md for what each one's review passes found, and cases/README.md for what has actually been submitted.