Skip to content

feat(ingest): OpenAI-compatible extraction endpoints - #6

Open
lil2good wants to merge 3 commits into
mainfrom
jarvis/openai-compatible-ingest
Open

lil2good wants to merge 3 commits into
mainfrom
jarvis/openai-compatible-ingest

Conversation

@lil2good

Copy link
Copy Markdown
Contributor

Opened by Jarvis so Argus reviews Engram changes before they ship (TheDev, 10/23).

What: _call_llm can use any OpenAI-compatible /chat/completions endpoint via base_url (falls back to reasoning_content for reasoning models); Anthropic stays the default. run_ingest gains provider/base_url. Small dialectical/cli/pyproject updates.

Tested: pytest tests/: 29 passed.

Not merging until Argus and TheDev are happy.

Pending Engram work from the Hermes dev tree, now reviewed through Argus.
- _call_llm takes base_url; OpenAI-compatible /chat/completions path, reasoning_content fallback
- run_ingest gains provider/base_url
- dialectical + cli tweaks; pyproject updated
Tests: 29 passed (pytest tests/).
@nubis-agent

nubis-agent Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
{
  "schema_version": 1,
  "head_sha": "9899f57c2237eee09d93018df10bbb0b75acbd17",
  "status": "clear",
  "review_complete": true,
  "merge_confidence": 4,
  "confidence_reason": "The configured non-Anthropic endpoint is now passed through the normal CLI path without credential-pool substitution; only configuration documentation remains as optional follow-up.",
  "blocking_findings": [],
  "advisory_findings": [
    {
      "id": "NUBIS-004",
      "severity": "P3",
      "file": "config.example.json",
      "line": 6,
      "problem": "The new ingestion behavior requires `llm_base_url` (and optionally `llm_api_key_env`) for an OpenAI-compatible provider, but the checked-in config template and README still only describe the Anthropic configuration. A user copying the template has no documented way to configure the feature added by this PR.",
      "required_fix": "Optionally document the OpenAI-compatible configuration keys and an example endpoint/API-key environment variable in the config template or README.",
      "acceptance_criteria": "Documentation shows a valid non-Anthropic `llm_provider`, `llm_base_url`, model, and API-key environment-variable configuration that corresponds to the CLI ingestion path."
    }
  ],
  "blockers": [],
  "checks_run": [
    {
      "command": "argus-native-gates ci Atomlaunch/engram 9899f57c2237eee09d93018df10bbb0b75acbd17",
      "result": "Helper could not complete because it raised JSONDecodeError parsing GitHub API output; direct repository-scoped REST evidence was used instead."
    },
    {
      "command": "argus-native-gates risk --repo /home/triage/triage-agent/worktrees/engram-pr6 --base origin/main --head 9899f57c2237eee09d93018df10bbb0b75acbd17",
      "result": "mac_build_required=false; no native, bundle, sandbox, helper, or Mac-build risk paths changed."
    }
  ],
  "ci": {
    "state": "green",
    "checked_at_sha": "9899f57c2237eee09d93018df10bbb0b75acbd17",
    "checks": [
      {
        "name": "test (3.11)",
        "status": "completed",
        "conclusion": "success"
      },
      {
        "name": "test (3.12)",
        "status": "completed",
        "conclusion": "success"
      },
      {
        "name": "test (3.13)",
        "status": "completed",
        "conclusion": "success"
      }
    ]
  },
  "mac_verification": {
    "required": false,
    "reasons": [],
    "reported": false,
    "report_url": null,
    "report_sha": null
  },
  "resolved_finding_ids": [
    "NUBIS-003"
  ],
  "handoff": {
    "target": null,
    "reason": "Review is clear for the current head; NUBIS-004 is optional advice and the two prior automatic handoff rounds are already used.",
    "round": 2,
    "action": "not_needed"
  }
}

Nubis Verdict

Merge confidence: 4/5
Reason: The current head wires the configured provider and base URL through engram ingest, preserves that explicit endpoint over Hermes credential-pool fallback, and has focused coverage for both CLI and direct ingestion entry points.

Summary

  • This synchronize resolves NUBIS-003: engram/cli.py forwards llm_provider and llm_base_url, while run_ingest reads the configured non-Anthropic API-key environment variable and does not replace a configured endpoint with a credential-pool endpoint.
  • The current head remains cleanly mergeable. All three GitHub Actions test check-runs for this exact SHA completed successfully.
  • NUBIS-004 is an optional P3: document the new endpoint configuration alongside the existing Anthropic-only config template/README guidance.

Findings

  • P3 · config.example.json:6 · NUBIS-004 — Optional documentation follow-up: expose llm_base_url and the API-key environment-variable convention so users copying the checked-in template can configure the new OpenAI-compatible ingestion path.

Resolved findings

  • NUBIS-003 — The normal CLI call now passes provider and base_url at engram/cli.py:405-412. run_ingest uses the explicit configured endpoint at engram/ingest.py:365-381 and only consults the Hermes credential pool when no explicit endpoint is configured (engram/ingest.py:409-446). tests/test_ingest_endpoint.py:47-62 covers both CLI and direct calls, including an unrelated credential-pool endpoint.

Review basis

  • Inspected live PR metadata, all PR comments and prior Nubis findings, the complete effective main...9899f57c2237eee09d93018df10bbb0b75acbd17 diff, and the synchronize delta from 66506c082ea468badada55a21bad8f92ed68bd61.
  • Inspected changed endpoint/config resolution code, all run_ingest call sites, config template/README references, and focused regression tests.
  • git diff --check found no whitespace errors; no added conflict markers, debug additions, or confirmed secret-like values were found. GitHub reports the PR mergeable and clean.
  • CI gate: GitHub REST reports successful test (3.11), test (3.12), and test (3.13) check-runs for this SHA. The native-gates CI helper itself hit a JSON parser error, so its result was not used as check evidence.
  • The native/Mac gate reports no Mac-build risk; no C/C++/Objective-C or Chromium-runtime paths changed.
  • Review mode: Code-only review; no local build/test/lint/typecheck was run.

Recommendation

Code review is clear at this head. The documentation note is optional and does not block merge readiness.

Review mode: Code-only review; no local build/test/lint/typecheck was run.

Merge confidence: 4/5

@nubis-agent

nubis-agent Bot commented Oct 24, 2026

Copy link
Copy Markdown

@cursor Address the required fixes from Nubis's review of 8850577: #6 (comment). Preserve finding IDs, push safe fixes to this PR branch, and let the new commit trigger Nubis re-review. Do not merge or deploy. Builds/tests remain opt-in under the existing review policy.

Repository/PR: #6
Reviewed head: 8850577
Review: #6 (comment)

Required findings:

  • NUBIS-001 · engram/ingest.py:123 — Reasoning-only multi-line JSON is reduced to one line before json.loads, so affected extraction chunks fail. Required fix: preserve/robustly extract the complete JSON response before parsing. Acceptance: a formatted reasoning-only JSON response yields the expected entities/facts dictionary; non-JSON still fails safely.
  • NUBIS-002 · pyproject.toml:10 — The default Anthropic path and the new requests endpoint path lack declared runtime dependencies. Required fix: align declared dependencies/extras with supported provider paths. Acceptance: each documented supported install contains the selected provider client; default Anthropic ingest does not fail solely because anthropic is absent.
  • NUBIS-GATE-CI · pyproject.toml:1 — No CI checks report for this head. This is a merge gate; wait for/enable CI or obtain owner acceptance, then request a rerun. It is context only and not a coding-agent task.

- NUBIS-001: _extract_json_text keeps the complete JSON object from reasoning-only
  completions (whole text, fenced block, or balanced {...} span); non-JSON still
  returns None. Tests: tests/test_ingest_llm.py.
- NUBIS-002: anthropic and requests are core dependencies again; extras kept as aliases.
- NUBIS-GATE-CI: GitHub Actions runs pytest on 3.11-3.13 from a default install.
@nubis-agent

nubis-agent Bot commented Oct 24, 2026

Copy link
Copy Markdown

@cursor Address the required fixes from Nubis's review of 66506c0: #6 (comment). Preserve finding IDs, push safe fixes to this PR branch, and let the new commit trigger Nubis re-review. Do not merge or deploy. Builds/tests remain opt-in under the existing review policy.

Repository/PR: #6
Reviewed head: 66506c0
Review: #6 (comment)

Required finding:

  • NUBIS-003 · engram/cli.py:403 — engram ingest does not forward the configured provider/base URL to run_ingest; non-Anthropic users can fall back to Anthropic or an unrelated Hermes credential-pool endpoint. Required fix: wire the configured provider and base URL through the CLI call, or consistently load both in run_ingest, without overriding an explicit endpoint. Acceptance: a non-Anthropic llm_provider plus llm_base_url in Engram config makes engram ingest call that exact <base_url>/chat/completions endpoint without requiring/substituting a Hermes credential-pool endpoint.

CI is green for this head. The prior NUBIS-001, NUBIS-002, and CI-gate findings are resolved.

- run_ingest fills provider, llm_base_url and llm_api_key_env (default OPENAI_API_KEY) from
  the Engram config; explicit arguments still win.
- A configured/caller endpoint is never replaced by a Hermes credential-pool endpoint;
  the Anthropic env/.env key lookup only applies to the anthropic provider.
- engram ingest forwards provider and base_url from config.
- tests/test_ingest_endpoint.py: CLI ingest hits exactly <llm_base_url>/chat/completions
  even with an unrelated credential pool present.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant