diff --git a/CHANGELOG.md b/CHANGELOG.md index c170ec19..ce151ad7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,7 @@ The first release of Harness² CLI, a general-purpose version of ### Added - One `harness2` command with subcommands: `init`, `doctor`, `task`, `run`, `status`, - `stop`, `diff`, `eval`, `apply`, `judge`, `report`, `runs`, `config`, `chat`, `guide`, + `stop`, `diff`, `eval`, `apply`, `judge`, `report`, `runs`, `config`, `chat`, `guide`, `login`, `setup`, `demo`, `bridge`, `completion`. - Your own tasks: `.harness2/tasks//task.md` with optional `task.toml`, run in a clean git worktree of your repository or in a scratch directory, graded optionally by a `check` @@ -21,8 +21,9 @@ The first release of Harness² CLI, a general-purpose version of - Claude Code and command executors. A project's own harness (CLAUDE.md, `.claude/skills`, `.claude/agents`, `.claude/scripts`, hooks; or AGENTS.md, `.agents/`, `.codex/hooks.json`) is imported as the starting harness. -- Subscriptions: `claude-login/` and `chatgpt-login/` use the login of the - installed `claude` or `codex`, with the user's personal setup kept out of the runs. +- Subscriptions: `claude-login/` runs on Claude Code's own login and + `chatgpt-login/` on a Codex login kept for harness2 (`harness2 login codex`), with + personal settings, hooks, plugins and instructions kept out of the runs. - Any model provider: Anthropic, OpenAI, OpenAI- or Anthropic-compatible servers, Vertex AI and Bedrock, through provider profiles; `harness2 bridge` serves any of them to Codex. - Layered settings: defaults, `~/.config/harness2/config.toml`, `.harness2/config.toml`, diff --git a/README.md b/README.md index b33d4a77..633145bd 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,9 @@ uv tool install git+https://github.com/3ae3ae/harness2-cli # or: pipx install git+https://github.com/3ae3ae/harness2-cli ``` -By default runs use the CLI's own login, so a Claude or ChatGPT subscription is enough. You +By default runs use your subscription, so a Claude or ChatGPT plan is enough: Claude Code +runs on its own login, and Codex on a login kept for harness2 that you make once with +`harness2 login codex`. You can use an API key or another provider instead; see [Configuration](#configuration). Runs make many model calls and use money or subscription quota: the [quick start](#quick-start) shows how to see the number of calls first and how to cap them. @@ -138,6 +140,7 @@ harness2 guide --install claude | `harness2 judge` / `report` | Grade each task's adopted result, and summarise the grades | | `harness2 runs ls / show / rm` | List, inspect and delete runs | | `harness2 config show / get / set` | Read and change settings; `show --origin` says where each value comes from | +| `harness2 login` | Check the subscription logins; `login codex` signs Codex in for harness2 | | `harness2 chat` | Talk to your agent CLI about tasks, runs and results | | `harness2 guide` | The guide for AI agents that use harness2 | | `harness2 demo` | A local run with scripted editors and no model calls | @@ -161,7 +164,8 @@ channel = "claude" model = "claude-login/claude-opus-5-5" ``` -`chatgpt-login/` uses your ChatGPT subscription through Codex. API keys and other +`chatgpt-login/` uses your ChatGPT subscription through Codex (after +`harness2 login codex`). API keys and other endpoints work too: Anthropic, OpenAI, any OpenAI-compatible or Anthropic-compatible server, Vertex AI and Bedrock. To tune an agent other than Claude Code or Codex, the `command` executor runs your own command and edits the files you list. diff --git a/docs/cli.md b/docs/cli.md index 983305c3..c5f9319d 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -195,6 +195,13 @@ rubric, or to explain a run's results. |---|---| | `--with {claude,codex}` | Which CLI to start. Default: the improver's channel, if installed. | +### `login [codex]` + +Without an argument, show whether `claude-login` (Claude Code's own login) and +`chatgpt-login` (the Codex login kept for harness2) are ready. `harness2 login codex` signs +Codex in to ChatGPT for harness2, in its own home, apart from your `~/.codex`; `--device` +uses a device code on machines without a browser. + ### `guide` Print the guide for AI agents that drive harness2, followed by the current project's diff --git a/docs/configuration.md b/docs/configuration.md index f2757184..7679b621 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -52,14 +52,26 @@ all you need: | Model | Runs on | |---|---| | `claude-login/`, e.g. `claude-login/claude-sonnet-5-5` | your Claude subscription, through Claude Code's login (`claude auth login`) | -| `chatgpt-login/`, e.g. `chatgpt-login/gpt-6.1-sol` | your ChatGPT subscription, through Codex's login (`codex login`) | - -`harness2 init` picks these by default, and `harness2 doctor` checks that the CLI is logged -in. Runs still keep your personal setup out of the harness being measured: Claude Code runs -with `--setting-sources project,local --strict-mcp-config --no-session-persistence`, so your -own `~/.claude/CLAUDE.md`, skills, plugins, MCP servers and settings are not loaded and no -session history is written; Codex runs with `--ignore-user-config --ignore-rules`. Codex -still reads a global `~/.codex/AGENTS.md` if you have one, and `doctor` says so. +| `chatgpt-login/`, e.g. `chatgpt-login/gpt-6.1-sol` | your ChatGPT subscription, through a Codex login kept for harness2 (`harness2 login codex`, once) | + +`harness2 init` picks these by default, and `harness2 login` and `harness2 doctor` say +whether they are ready. Runs keep your personal setup out of the harness being measured: + +- **Claude Code** runs on its own login, with `--setting-sources project,local + --strict-mcp-config --no-session-persistence`: your `~/.claude/CLAUDE.md`, skills, plugins, + hooks, MCP servers and settings are not loaded, and no session history is written. +- **Codex** cannot switch off a personal `~/.codex/hooks.json` or `~/.codex/AGENTS.md` with + a flag, so harness2 gives it a home of its own, `~/.local/state/harness2/codex-home`. + `harness2 login codex` signs that home in to ChatGPT once. It is a separate login: your own + `~/.codex` is never read or changed. Runs also pass `--ignore-user-config --ignore-rules`. + +To run Codex on your own home anyway, with whatever hooks and instructions it holds, set +its path; `harness2 doctor` then lists the personal files every run will include: + +```toml +[providers.chatgpt-login] +home = "~/.codex" +``` Subscriptions have usage limits. When a run reaches one, it stops cleanly (exit code 4), keeps everything finished so far, and the same `harness2 run` command resumes once the diff --git a/harness2/cli/doctor.py b/harness2/cli/doctor.py index 5da655ca..cae3c435 100644 --- a/harness2/cli/doctor.py +++ b/harness2/cli/doctor.py @@ -16,9 +16,7 @@ from __future__ import annotations -import os import subprocess -from pathlib import Path from .. import providers, registry from ..core.proposer import Proposer @@ -130,16 +128,21 @@ def binary_check(binary=binary, channel=channel): "detail": "; ".join(problems) or detail, } ) - if any(config.provider(name).type == "chatgpt-login" for name in referenced): - agents = Path(os.environ.get("CODEX_HOME") or Path.home() / ".codex") / "AGENTS.md" - if agents.is_file() and (text := agents.read_text(encoding="utf-8")): - rows.append( - { - "check": "Codex global AGENTS.md", - "ok": True, - "detail": f"your global AGENTS.md ({len(text)} characters) is part of every Codex run", - } - ) + if provider.type == "chatgpt-login": + home = providers.codex_login_home(provider) + files = [ + str(path) + for name in ("AGENTS.md", "hooks.json") + if (path := home / name).is_file() and path.stat().st_size + ] + if files: + rows.append( + { + "check": "Codex home", + "ok": True, + "detail": " and ".join(files) + " are part of every Codex run", + } + ) if args.live: for (channel, model), spec in sorted(models.items()): diff --git a/harness2/cli/guide.py b/harness2/cli/guide.py index 240adff7..24b09160 100644 --- a/harness2/cli/guide.py +++ b/harness2/cli/guide.py @@ -47,8 +47,9 @@ ## Workflow 1. `harness2 init --yes [--executor claude|codex] [--auth login|api-key]` creates - .harness2/. `--auth login` uses the agent CLI's own login (a Claude or ChatGPT - subscription). Change any choice afterwards with `harness2 config set KEY VALUE`, for + .harness2/. `--auth login` uses a Claude or ChatGPT subscription: Claude Code's own login + as it is, or for Codex a separate login the person makes once with `harness2 login codex` + (it opens a browser; ask them to run it). `harness2 login --json` shows which are ready. Change any choice afterwards with `harness2 config set KEY VALUE`, for example `executor.model`. 2. Tasks. `harness2 task new [--files] [--check ""]`, then write the request in .harness2/tasks//task.md the way the person would really ask for it. Good tasks are diff --git a/harness2/cli/init.py b/harness2/cli/init.py index bb8b3f80..b39aeebc 100644 --- a/harness2/cli/init.py +++ b/harness2/cli/init.py @@ -37,7 +37,7 @@ PROVIDERS = {"claude": ("claude-login", "anthropic"), "codex": ("chatgpt-login", "openai")} AUTH = { "claude": ("your Claude subscription, through Claude Code's own login", "ANTHROPIC_API_KEY"), - "codex": ("your ChatGPT subscription, through Codex's own login", "OPENAI_API_KEY"), + "codex": ("your ChatGPT subscription, through a Codex login for harness2", "OPENAI_API_KEY"), } CONFIG = """\ @@ -264,6 +264,16 @@ def _key_summary(root: Path, models: tuple[str, ...]) -> str: loaded = S.load(None) lines = [] for name in dict.fromkeys(providers.split_model(m)[0] for m in models if "/" in m): + if name == "chatgpt-login": + home = providers.codex_login_home(loaded.provider(name)) + ok = providers.codex_logged_in(home) + lines.append( + "Codex is signed in for harness2." + if ok + else "Codex is not signed in for harness2 yet: run `harness2 login codex` once " + "(a separate login, so your own ~/.codex settings and hooks stay out of runs)." + ) + continue if name in LOGIN_CHECKS: argv, cli, fix = LOGIN_CHECKS[name] ok = shutil.which(argv[0]) and _quiet(argv) @@ -289,7 +299,6 @@ def _q(value) -> str: LOGIN_CHECKS = { "claude-login": (["claude", "auth", "status"], "Claude Code", "claude auth login"), - "chatgpt-login": (["codex", "login", "status"], "Codex", "codex login"), } diff --git a/harness2/cli/login.py b/harness2/cli/login.py new file mode 100644 index 00000000..3cc39362 --- /dev/null +++ b/harness2/cli/login.py @@ -0,0 +1,102 @@ +# Copyright 2026 The Harness2 Authors +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Show whether the subscription logins harness2 uses are ready, and set up Codex's. + +claude-login uses Claude Code's own login as it is: runs switch off your personal settings, +hooks and plugins with flags. Codex has no such switch for its hooks.json and AGENTS.md, so +chatgpt-login runs use a Codex home of their own, signed in once with `harness2 login codex`. +""" + +from __future__ import annotations + +import argparse +import os +import shutil +import subprocess +import sys + +from .. import providers + + +def add_parser(sub) -> argparse.ArgumentParser: + p = sub.add_parser( + "login", + help="check the subscription logins, or sign Codex in for harness2", + description="Without an argument, show whether claude-login and chatgpt-login are " + "ready. `harness2 login codex` signs in to ChatGPT in a Codex home kept for harness2, " + "apart from your own ~/.codex.", + ) + p.add_argument("service", nargs="?", choices=("codex",), help="sign this CLI in for harness2") + p.add_argument( + "--device", + action="store_true", + help="sign in with a device code, for machines without a browser", + ) + return p + + +def run(args, ctx) -> int: + home = providers.codex_login_home(ctx.settings.provider("chatgpt-login")) + if args.service is None: + rows = _status(home) + ctx.out(rows, lambda data: "\n".join(f"{r['provider']:<14} {r['detail']}" for r in data)) + return 0 + if shutil.which("codex") is None: + print("`codex` is not on PATH; install Codex first.", file=sys.stderr) + return 1 + if not sys.stdin.isatty(): + print("Signing in needs a terminal and a browser (or --device).", file=sys.stderr) + return 1 + home.mkdir(parents=True, exist_ok=True) + print( + f"Signing Codex in to ChatGPT for harness2. This login lives in {home}, apart from " + "your own ~/.codex, so your personal hooks and AGENTS.md stay out of harness2's runs.\n" + ) + env = {k: v for k, v in os.environ.items() if k not in ("OPENAI_API_KEY", "CODEX_API_KEY")} + subprocess.run( + ["codex", "login", *(["--device-auth"] if args.device else [])], + env={**env, "CODEX_HOME": str(home)}, + ) + if not providers.codex_logged_in(home): + print("Codex did not finish signing in.", file=sys.stderr) + return 1 + print("Signed in. Models written chatgpt-login/ now run on this login.") + return 0 + + +def _status(home) -> list[dict]: + claude = shutil.which("claude") and _ok(["claude", "auth", "status"]) + codex = providers.codex_logged_in(home) + return [ + { + "provider": "claude-login", + "ready": bool(claude), + "detail": "ready (Claude Code's own login)" + if claude + else "Claude Code is not logged in; run `claude auth login`", + }, + { + "provider": "chatgpt-login", + "ready": codex, + "detail": f"ready ({home})" if codex else "not signed in; run `harness2 login codex`", + }, + ] + + +def _ok(argv: list[str]) -> bool: + try: + return subprocess.run(argv, capture_output=True, timeout=20).returncode == 0 + except (OSError, subprocess.TimeoutExpired): + return False diff --git a/harness2/cli/main.py b/harness2/cli/main.py index aec4bc86..400c0237 100644 --- a/harness2/cli/main.py +++ b/harness2/cli/main.py @@ -39,6 +39,7 @@ "config", "chat", "guide", + "login", "setup", "demo", "bridge", diff --git a/harness2/providers.py b/harness2/providers.py index 96053697..f17b7ba3 100644 --- a/harness2/providers.py +++ b/harness2/providers.py @@ -21,9 +21,10 @@ import re import subprocess from dataclasses import dataclass +from pathlib import Path from urllib.parse import urlsplit -from .settings import ConfigError +from .settings import ConfigError, state_dir PROVIDER_TYPES = ("anthropic", "openai", "vertex", "bedrock", "claude-login", "chatgpt-login") USAGE_LIMIT = re.compile( @@ -53,6 +54,35 @@ class Provider: api_key_env: str = "" project: str = "" region: str = "" + home: str = "" + + +def codex_login_home(p: Provider) -> Path: + return Path(p.home).expanduser() if p.home else state_dir() / "codex-home" + + +def codex_logged_in(home: Path) -> bool: + try: + return ( + subprocess.run( + ["codex", "login", "status"], + env={ + **{ + k: v + for k, v in os.environ.items() + if k not in (*STRIP_ENV, "OPENAI_API_KEY", "CODEX_API_KEY") + }, + "CODEX_HOME": str(home), + }, + stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + timeout=10, + ).returncode + == 0 + ) + except Exception: + return False def split_model(ref: str) -> tuple[str, str]: @@ -182,22 +212,20 @@ def litellm_params(p: Provider, model: str) -> dict: def check(p: Provider) -> list[str]: - if p.type in ("claude-login", "chatgpt-login"): - argv = ( - ["claude", "auth", "status"] - if p.type == "claude-login" - else ["codex", "login", "status"] - ) - login = "claude auth login" if p.type == "claude-login" else "codex login" + if p.type == "chatgpt-login": + home = codex_login_home(p) + if codex_logged_in(home): + return [] + if p.home: + return [ + f"chatgpt-login: not logged in at {home}; run `codex login` with CODEX_HOME={home}" + ] + return ["chatgpt-login: not logged in for harness2; run `harness2 login codex`"] + if p.type == "claude-login": try: result = subprocess.run( - argv, - env={ - k: v - for k, v in os.environ.items() - if k not in STRIP_ENV - and (p.type != "chatgpt-login" or k not in ("OPENAI_API_KEY", "CODEX_API_KEY")) - }, + ["claude", "auth", "status"], + env={k: v for k, v in os.environ.items() if k not in STRIP_ENV}, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, @@ -207,7 +235,7 @@ def check(p: Provider) -> list[str]: return [] except (OSError, subprocess.SubprocessError): pass - return [f"{p.type}: not logged in; run `{login}`"] + return ["claude-login: not logged in; run `claude auth login`"] problems = [] if p.api_key_env and not os.environ.get(p.api_key_env): problems.append(f"{p.name}: set {p.api_key_env}") diff --git a/harness2/settings.py b/harness2/settings.py index 90f5d9a2..5aa93fe0 100644 --- a/harness2/settings.py +++ b/harness2/settings.py @@ -103,6 +103,10 @@ def cache_dir() -> Path: return Path(os.environ.get("XDG_CACHE_HOME") or Path.home() / ".cache") / "harness2" +def state_dir() -> Path: + return Path(os.environ.get("XDG_STATE_HOME") or Path.home() / ".local/state") / "harness2" + + def parse_value(raw: str) -> Any: """Parse a TOML literal, falling back to an unquoted string.""" try: @@ -241,7 +245,7 @@ def provider(self, name: str) -> providers.Provider: configured = self.section("providers") if name not in configured: raise ConfigError(f"unknown provider {name!r}; configure providers.{name}") - fields = ("type", "base_url", "api_key_env", "project", "region") + fields = ("type", "base_url", "api_key_env", "project", "region", "home") return providers.Provider( name, **{k: v for k, v in configured[name].items() if k in fields} ) diff --git a/harness2/substrate/codex.py b/harness2/substrate/codex.py index 2b8518fb..591d6254 100644 --- a/harness2/substrate/codex.py +++ b/harness2/substrate/codex.py @@ -248,6 +248,7 @@ def env( stripped = providers.STRIP_ENV if provider is not None and provider.type == "chatgpt-login": stripped += ("OPENAI_API_KEY", "CODEX_API_KEY") + home = providers.codex_login_home(provider) return { **{k: v for k, v in base_env.items() if k not in stripped}, **({"CODEX_HOME": str(home)} if home is not None else {}), diff --git a/tests/test_settings.py b/tests/test_settings.py index 25b3fbc8..978c30c0 100644 --- a/tests/test_settings.py +++ b/tests/test_settings.py @@ -65,6 +65,16 @@ def test_defaults_access_and_copy_isolation(self): self.assertEqual(resolved.get("executor.editable"), []) self.assertEqual(settings.load(None).section("executor"), settings.DEFAULTS["executor"]) + def test_state_dir_uses_xdg_or_home_without_creating_it(self): + with patch.dict(os.environ, {"HOME": str(self.root)}, clear=True): + self.assertEqual(settings.state_dir(), self.root / ".local/state/harness2") + self.assertFalse(settings.state_dir().exists()) + os.environ["XDG_STATE_HOME"] = str(self.root / "state") + self.assertEqual(settings.state_dir(), self.root / "state/harness2") + self.assertFalse(settings.state_dir().exists()) + os.environ["XDG_STATE_HOME"] = "" + self.assertEqual(settings.state_dir(), self.root / ".local/state/harness2") + def test_precedence_and_origins_across_all_six_layers(self): self.write(self.global_file, "[run]\nk = 2\nworkers = 2\n") self.write( diff --git a/tests/test_subscriptions.py b/tests/test_subscriptions.py index 4adfcc1f..976e41e1 100644 --- a/tests/test_subscriptions.py +++ b/tests/test_subscriptions.py @@ -51,6 +51,8 @@ config = home / "config.toml" record = {"kind": kind, "argv": argv, "env": dict(os.environ), "cwd": str(cwd), "stdin": "" if status else sys.stdin.read(), + "agents_exists": (home / "AGENTS.md").exists(), + "hooks_exists": (home / "hooks.json").exists(), "config_exists": config.exists(), "config": config.read_text() if config.exists() else None} with open(os.environ["FAKE_RECORD"], "a") as log: @@ -123,12 +125,15 @@ def setUp(self): path.mkdir(parents=True) (path / "config.toml").write_text("private user config") (path / "auth.json").write_text("private user auth") + (self.user_home / ".codex/AGENTS.md").write_text("Private user instructions.\n") + (self.user_home / ".codex/hooks.json").write_text('{"hooks": {}}\n') environment = patch.dict( os.environ, { "PATH": f"{binary}{os.pathsep}{os.defpath}", "HOME": str(self.user_home), "XDG_CONFIG_HOME": str(self.root / "config"), + "XDG_STATE_HOME": str(self.root / "state"), "FAKE_RECORD": str(self.record), "CLAUDE_CODE_OAUTH_TOKEN": "fixture-oauth", "ANTHROPIC_API_KEY": "ambient-anthropic", @@ -294,7 +299,12 @@ def test_chatgpt_subscription_executor_and_improver_launches(self): ], ) for record in self.records(): - self.assertNotIn("CODEX_HOME", record["env"]) + self.assertEqual( + record["env"]["CODEX_HOME"], str(self.root / "state/harness2/codex-home") + ) + self.assertFalse(record["config_exists"]) + self.assertFalse(record["agents_exists"]) + self.assertFalse(record["hooks_exists"]) self.assertTrue( all( key not in record["env"] @@ -302,8 +312,48 @@ def test_chatgpt_subscription_executor_and_improver_launches(self): ) ) self.assertEqual({p.name: p.read_bytes() for p in home.iterdir()}, before) + self.assertFalse((settings.state_dir() / "codex-home").exists()) - def test_inherited_login_homes_are_preserved(self): + def test_configured_chatgpt_home_is_used_without_writes(self): + for populated in (False, True): + with self.subTest(populated=populated): + home = self.user_home / f"custom codex-{populated}" + if populated: + home.mkdir() + for name in ("auth.json", "config.toml", "AGENTS.md", "hooks.json"): + (home / name).write_text("fixture " + name) + before = {p.name: p.read_bytes() for p in home.iterdir()} if populated else {} + adapter = self.adapter( + f"custom-{populated}", + **{ + "executor.name": "codex", + "executor.model": "chatgpt-login/model", + "improver.channel": "codex", + "improver.model": "chatgpt-login/model", + "providers.chatgpt-login.home": f"~/{home.name}", + }, + ) + with patch.dict(os.environ, {"CODEX_HOME": str(self.user_home / ".codex")}): + result = adapter.execute("one", None, "custom") + self.assertFalse((result / "codex-home").exists()) + context = self.root / f"context-{populated}" + context.mkdir() + Proposer(adapter.spec.channels["proposer"]).call("mission", context, "custom") + for record in self.records()[-2:]: + self.assertEqual(record["env"]["CODEX_HOME"], str(home)) + self.assertEqual(record["config_exists"], populated) + self.assertEqual(record["agents_exists"], populated) + self.assertEqual(record["hooks_exists"], populated) + self.assertNotIn("OPENAI_API_KEY", record["env"]) + self.assertNotIn("CODEX_API_KEY", record["env"]) + self.assertIn("--ignore-user-config", record["argv"]) + self.assertIn("--ignore-rules", record["argv"]) + self.assertIn("--ephemeral", record["argv"]) + after = {p.name: p.read_bytes() for p in home.iterdir()} if home.exists() else {} + self.assertEqual(after, before) + self.assertEqual(home.exists(), populated) + + def test_inherited_claude_home_is_preserved_and_codex_home_is_overridden(self): context = self.root / "context" context.mkdir() for kind, provider_name, variable in ( @@ -324,14 +374,19 @@ def test_inherited_login_homes_are_preserved(self): }, ) adapter.execute("one", None, "inherited") - self.assertEqual(self.records()[-1]["env"][variable], os.environ[variable]) + expected = ( + os.environ[variable] + if kind == "claude" + else str(settings.state_dir() / "codex-home") + ) + self.assertEqual(self.records()[-1]["env"][variable], expected) Proposer(adapter.spec.channels["proposer"]).call("mission", context, "inherited") - self.assertEqual(self.records()[-1]["env"][variable], os.environ[variable]) + self.assertEqual(self.records()[-1]["env"][variable], expected) def test_login_status_success_failure_missing_cli_and_timeout(self): for kind, argv, login in ( ("claude-login", ["auth", "status"], "claude auth login"), - ("chatgpt-login", ["login", "status"], "codex login"), + ("chatgpt-login", ["login", "status"], "harness2 login codex"), ): with self.subTest(provider=kind): provider = providers.Provider(kind, kind) @@ -339,9 +394,17 @@ def test_login_status_success_failure_missing_cli_and_timeout(self): record = self.records()[-1] self.assertEqual(record["argv"], argv) self.assertNotIn("CLAUDE_CONFIG_DIR", record["env"]) - self.assertNotIn("CODEX_HOME", record["env"]) + if kind == "chatgpt-login": + self.assertEqual( + record["env"]["CODEX_HOME"], str(settings.state_dir() / "codex-home") + ) + self.assertNotIn("OPENAI_API_KEY", record["env"]) + self.assertNotIn("CODEX_API_KEY", record["env"]) + else: + self.assertNotIn("CODEX_HOME", record["env"]) self.assertTrue(all(key not in record["env"] for key in providers.STRIP_ENV)) - problem = [f"{kind}: not logged in; run `{login}`"] + where = " for harness2" if kind == "chatgpt-login" else "" + problem = [f"{kind}: not logged in{where}; run `{login}`"] with patch.dict(os.environ, {"FAKE_LOGIN_RC": "1"}): self.assertEqual(providers.check(provider), problem) with patch.dict(os.environ, {"PATH": ""}): @@ -359,6 +422,33 @@ def test_login_status_success_failure_missing_cli_and_timeout(self): self.assertEqual(args[3], 'model"\nnext\x7f') self.assertEqual(tomllib.loads(args[5]), {"model_reasoning_effort": 'effort"\nnext'}) + def test_codex_logged_in_checks_only_the_requested_home(self): + home = self.root / "login home" + with patch.dict(os.environ, {"CODEX_HOME": str(self.user_home / ".codex")}): + self.assertTrue(providers.codex_logged_in(home)) + record = self.records()[-1] + self.assertEqual(record["argv"], ["login", "status"]) + self.assertEqual(record["env"]["CODEX_HOME"], str(home)) + self.assertFalse(record["config_exists"]) + self.assertFalse(record["agents_exists"]) + self.assertFalse(record["hooks_exists"]) + self.assertTrue( + all( + key not in record["env"] + for key in (*providers.STRIP_ENV, "OPENAI_API_KEY", "CODEX_API_KEY") + ) + ) + with patch.dict(os.environ, {"FAKE_LOGIN_RC": "1"}): + self.assertFalse(providers.codex_logged_in(home)) + for error in (FileNotFoundError(), subprocess.TimeoutExpired("codex", 10), ValueError()): + with ( + self.subTest(error=type(error).__name__), + patch("harness2.providers.subprocess.run", side_effect=error) as run, + ): + self.assertFalse(providers.codex_logged_in(home)) + self.assertEqual(run.call_args.kwargs["timeout"], 10) + self.assertFalse(home.exists()) + def test_api_provider_homes_and_ambient_oauth_are_unchanged(self): for kind, provider_name in (("claude", "anthropic"), ("codex", "openai")): with self.subTest(kind=kind): @@ -453,13 +543,15 @@ def doctor(self, adapter): output = io.StringIO() argv = ["-C", str(self.project), "--json"] for key, value in adapter.settings.flat().items(): - if key.startswith(("executor.", "improver.")) and not isinstance(value, list): + if key.startswith(("executor.", "improver.", "providers.")) and not isinstance( + value, list + ): argv += ["-c", f"{key}={json.dumps(value)}"] with contextlib.redirect_stdout(output): code = main([*argv, "doctor"]) return code, {row["check"]: row for row in json.loads(output.getvalue())} - def test_doctor_login_status_and_global_codex_instructions(self): + def test_doctor_login_status_uses_dedicated_codex_home(self): adapter = self.adapter( **{ "executor.name": "codex", @@ -468,24 +560,15 @@ def test_doctor_login_status_and_global_codex_instructions(self): "improver.model": "chatgpt-login/model", } ) - agents = self.user_home / ".codex/AGENTS.md" - agents.write_text("Global rules. 한글\n") code, rows = self.doctor(adapter) self.assertEqual(code, 0) - self.assertEqual( - rows["Codex global AGENTS.md"], - { - "check": "Codex global AGENTS.md", - "ok": True, - "detail": "your global AGENTS.md (17 characters) is part of every Codex run", - }, - ) + self.assertNotIn("Codex home", rows) with patch.dict(os.environ, {"FAKE_LOGIN_RC": "1"}): code, rows = self.doctor(adapter) self.assertEqual(code, 1) self.assertEqual( rows["provider chatgpt-login"]["detail"], - "chatgpt-login: not logged in; run `codex login`", + "chatgpt-login: not logged in for harness2; run `harness2 login codex`", ) code, rows = self.doctor(self.adapter()) self.assertEqual(code, 1) @@ -494,19 +577,8 @@ def test_doctor_login_status_and_global_codex_instructions(self): "claude-login: not logged in; run `claude auth login`", ) _, rows = self.doctor(self.adapter()) - self.assertNotIn("Codex global AGENTS.md", rows) - agents.write_text("") - _, rows = self.doctor(adapter) - self.assertNotIn("Codex global AGENTS.md", rows) - other = self.root / "custom-codex" - other.mkdir() - (other / "AGENTS.md").write_text("custom") - with patch.dict(os.environ, {"CODEX_HOME": str(other)}): - _, rows = self.doctor(adapter) - self.assertEqual( - rows["Codex global AGENTS.md"]["detail"], - "your global AGENTS.md (6 characters) is part of every Codex run", - ) + self.assertNotIn("Codex home", rows) + self.assertFalse((settings.state_dir() / "codex-home").exists()) self.assertTrue( all( record["argv"] in (["auth", "status"], ["login", "status"]) @@ -514,6 +586,47 @@ def test_doctor_login_status_and_global_codex_instructions(self): ) ) + def test_doctor_custom_codex_home_login_and_instruction_files(self): + home = self.user_home / "custom-codex" + home.mkdir() + adapter = self.adapter( + **{ + "executor.name": "codex", + "executor.model": "chatgpt-login/model", + "improver.channel": "codex", + "improver.model": "chatgpt-login/model", + "providers.chatgpt-login.home": "~/custom-codex", + } + ) + with patch.dict(os.environ, {"FAKE_LOGIN_RC": "1"}): + code, rows = self.doctor(adapter) + self.assertEqual(code, 1) + self.assertEqual( + rows["provider chatgpt-login"]["detail"], + f"chatgpt-login: not logged in at {home}; run `codex login` with CODEX_HOME={home}", + ) + self.assertEqual(self.records()[-1]["env"]["CODEX_HOME"], str(home)) + for files in ((), ("hooks.json",), ("AGENTS.md",), ("AGENTS.md", "hooks.json")): + with self.subTest(files=files): + for name in ("AGENTS.md", "hooks.json"): + (home / name).write_text("fixture" if name in files else "") + before = {p.name: p.read_bytes() for p in home.iterdir()} + code, rows = self.doctor(adapter) + self.assertEqual(code, 0) + if files: + self.assertEqual( + rows["Codex home"], + { + "check": "Codex home", + "ok": True, + "detail": " and ".join(str(home / name) for name in files) + + " are part of every Codex run", + }, + ) + else: + self.assertNotIn("Codex home", rows) + self.assertEqual({p.name: p.read_bytes() for p in home.iterdir()}, before) + def test_executor_usage_limits_stop_and_resume_entire_run(self): adapter = self.adapter() self.stop_and_resume(adapter, "executor") diff --git a/tests/test_ui.py b/tests/test_ui.py index 53a2549e..021f6190 100644 --- a/tests/test_ui.py +++ b/tests/test_ui.py @@ -114,6 +114,25 @@ def test_apply_json_reports_the_plan_and_refuses_to_ask(self): self.assertEqual(plan["changes"][0]["path"], "CLAUDE.md") self.assertFalse(plan["changes"][0]["applied"]) + def test_login_status_uses_the_harness2_codex_home(self): + from harness2.cli import login + + homes = [] + ctx = _Ctx(self.root) + ctx.settings = type("S", (), {"provider": lambda _self, name: None})() + with ( + patch.object(login.providers, "codex_login_home", return_value=self.root / "h"), + patch.object(login.providers, "codex_logged_in", side_effect=homes.append), + patch.object(login, "_ok", return_value=True), + patch.object(login.shutil, "which", return_value="/bin/claude"), + patch("sys.stdout", io.StringIO()) as out, + ): + args = argparse.Namespace(service=None, device=False) + self.assertEqual(login.run(args, ctx), 0) + self.assertEqual(homes, [self.root / "h"]) + self.assertIn("claude-login", out.getvalue()) + self.assertIn("harness2 login codex", out.getvalue()) + def test_guide_mentions_every_agent_contract(self): from harness2.cli import guide