sxzgroup · QingyunQian · Aug 2, 2026 · Aug 2, 2026 · Aug 2, 2026 · Aug 2, 2026
diff --git a/results/deepseek-v4-flash-max/README.md b/results/deepseek-v4-flash-max/README.md
@@ -0,0 +1,112 @@
+# DeepSeek V4 Flash Max — TensorCircuit Benchmark
+
+This directory archives one valid outcome for each of the 12 ORBIT-Q
+TensorCircuit challenges. DeepSeek V4 Flash/max solved the tasks and GPT-5.6
+Sol/high performed the independent source audit.
+
+## Headline
+
+- Final raw validity: **5 / 12**
+- Functional checks: **8 / 12**
+- Static policy checks: **8 / 12**
+- Sol/high audit checks: **5 / 12**
+- Passed challenges: **02, 05, 09, 10, and 12**
+
+![DeepSeek V4 Flash/max outcomes](figs/deepseek-v4-flash-max-outcomes.png)
+
+`P` denotes a valid solution and `F` a failed task. The matrix follows the
+task-level presentation used by the archived ORBIT-Q GPT-5.6 reports.
+
+## Protocol
+
+- Run date: 2026-08-02
+- Branch: `codex/deepseek-v4-flash-max-benchmark`
+- Base task commit: `0201238ec2983907e2891f5319f5fff2d00844d5`
+- Solver: Harbor built-in Codex using the official DeepSeek integration,
+  `deepseek-v4-flash`, reasoning effort `max`
+- Auditor: Codex, `gpt-5.6-sol`, reasoning effort `high`
+- Framework: TensorCircuit-NG
+- Docker image: `challenge-benchmark-quantum-tensorcircuit:py311`
+- Execution: Docker-isolated tasks, sequential order, one valid model outcome
+  per challenge
+- Local task resources: 6 CPUs, 10,240 MiB memory, 16,384 MiB storage
+
+The solver saw only the public task instruction, TensorCircuit framework prompt,
+and installed package source. It did not receive expert solutions, verifier
+tests, or prior model outputs. The frozen task copies have aggregate SHA-256
+`19fe27b83eaf668b3df32d1a68902b08cbe28585f189290769018eb16d927895`.
+All first attempts were valid non-infrastructure outcomes; no task was rerun.
+
+## Results
+
+| Challenge | Reward | Functional | Static | Sol audit | Runtime score | Runtime (s) | Outcome |
+|---|---:|---:|---:|---:|---:|---:|---|
+| 01 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | — | Agent timeout |
+| 02 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 88.91 | Pass |
+| 03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | — | Agent timeout |
+| 04 | 0.0 | 1.0 | 1.0 | 0.0 | 1.0 | 6.30 | Audit fail |
+| 05 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 77.44 | Pass |
+| 06 | 0.0 | 1.0 | 1.0 | 0.0 | 1.0 | 25.16 | Audit fail |
+| 07 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | — | Agent timeout |
+| 08 | 0.0 | 1.0 | 1.0 | 0.0 | 1.0 | 39.77 | Audit fail |
+| 09 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 67.25 | Pass |
+| 10 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 122.76 | Pass |
+| 11 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | — | Agent timeout |
+| 12 | 1.0 | 1.0 | 1.0 | 1.0 | 1.0 | 15.16 | Pass |
+| **Total** | **5 / 12** | **8 / 12** | **8 / 12** | **5 / 12** | — | **442.75** | **5 / 12** |
+
+## What failed
+
+- **Challenge 04:** the candidate applied the offset odd-bond channel to the two
+  endpoints, although the specification excludes them, and allowed bond
+  truncation in a calculation that must remain exact.
+- **Challenge 06:** the candidate used the uniform detuning operator
+  `sum_i Z_i` instead of the required staggered `sum_i (-1)^i Z_i`.
+- **Challenge 08:** both edge-angle indices were shifted by one, and the core
+  contraction used a one-dimensional MPS rather than the required direct 2D
+  grid tensor network.
+- **Challenges 01, 03, 07, and 11:** the solver used the full 1,800-second Agent
+  budget without submitting a candidate. These are valid model outcomes, not
+  transport or Docker failures.
+
+## Comparison
+
+| Solver setting | Valid solutions | Failed challenges |
+|---|---:|---|
+| GPT-5.6 Sol high | 10 / 12 | 01, 08 |
+| GPT-5.6 Sol ultra | 10 / 12 | 01, 08 |
+| GPT-5.6 Terra high | 9 / 12 | 01, 08, 10 |
+| GPT-5.6 Luna high | 9 / 12 | 01, 04, 08 |
+| DeepSeek V4 Flash high | 5 / 12 | 01, 02, 06, 07, 08, 09, 11 |
+| **DeepSeek V4 Flash max** | **5 / 12** | **01, 03, 04, 06, 07, 08, 11** |
+
+High and max both reached 5/12, but with different accepted sets. They overlap
+on Challenges 05, 10, and 12; max adds 02 and 09, while high adds 03 and 04.
+Max produced eight functionally passing candidates versus seven for high, but
+three were removed by source audit, leaving final validity unchanged. Because
+each setting has one outcome per task, the task-level swap should not be read as
+a stable ranking between reasoning efforts.
+
+## Resource record
+
+![DeepSeek V4 Flash/max agent resource use](figs/deepseek-v4-flash-max-agent-resource-use.png)
+
+- Recorded Agent solve wall time: 14,692.62 seconds (4 h 4 min 53 s)
+- Input tokens: 80.371 million, including 79.595 million cache-read tokens
+- Output tokens: 0.678 million
+- Total solving-side tokens: 81.049 million
+- Recorded solver cost: USD 0.52
+- Recorded cost per valid solution: USD 0.10
+
+Max used less solver wall time, fewer output tokens, and slightly less recorded
+cost than high in these single runs, while achieving the same 5/12 final
+validity. Price and token volume remain separate from correctness.
+
+## Archived artifacts
+
+Each `challenge-NN/` directory contains every artifact produced for that task:
+candidate when present, official functional output, reward and audit details,
+Harbor result/config/lock files, solver log, trial log, and normalized
+`stamp-info.json`. `summary.json` is the machine-readable aggregate,
+`model-comparison.json` records figure inputs, and `tools/` regenerates and
+verifies the archive.
diff --git a/results/deepseek-v4-flash-max/audit-high.config.toml b/results/deepseek-v4-flash-max/audit-high.config.toml
@@ -0,0 +1 @@
+model_reasoning_effort = "high"
diff --git a/results/deepseek-v4-flash-max/challenge-01/agent-codex.log b/results/deepseek-v4-flash-max/challenge-01/agent-codex.log
diff --git a/results/deepseek-v4-flash-max/challenge-01/artifact-manifest.json b/results/deepseek-v4-flash-max/challenge-01/artifact-manifest.json
@@ -0,0 +1,23 @@
+[
+  {
+    "source": "/logs/artifacts",
+    "destination": "artifacts/logs/artifacts",
+    "type": "directory",
+    "status": "empty",
+    "service": null
+  },
+  {
+    "source": "/root/solution_1.py",
+    "destination": "artifacts/root/solution_1.py",
+    "type": "file",
+    "status": "failed",
+    "service": null
+  },
+  {
+    "source": "/root/evaluation_summary.json",
+    "destination": "artifacts/root/evaluation_summary.json",
+    "type": "file",
+    "status": "failed",
+    "service": null
+  }
+]
diff --git a/results/deepseek-v4-flash-max/challenge-01/audit-details.json b/results/deepseek-v4-flash-max/challenge-01/audit-details.json
@@ -0,0 +1,5 @@
+{
+  "problem_id": 1,
+  "framework": "tensorcircuit",
+  "missing_solution": "/root/solution_1.py"
+}
diff --git a/results/deepseek-v4-flash-max/challenge-01/config.json b/results/deepseek-v4-flash-max/challenge-01/config.json
@@ -0,0 +1,48 @@
+{
+    "task": {
+        "path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/challenge-01"
+    },
+    "trial_name": "challenge-01__nAp4YmZ",
+    "trials_dir": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/jobs/challenge-01-tensorcircuit-deepseek-v4-flash-max-20260802-r1",
+    "agent": {
+        "import_path": "adapters.codex_para:CodexPara",
+        "model_name": "deepseek-v4-flash",
+        "kwargs": {
+            "reasoning_effort": "max",
+            "profile": "deepseek-v4-flash",
+            "profile_config_path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/.secrets/deepseek-v4-flash.config.toml",
+            "model_catalog_path": "/Users/qqy/.codex-orbitq-deepseek/models.json",
+            "force_auth_json": false
+        },
+        "env": {
+            "HTTP_PROXY": "http://172.17.0.1:7892",
+            "HTTPS_PROXY": "http://172.17.0.1:7892",
+            "NO_PROXY": "localhost,127.0.0.1"
+        }
+    },
+    "environment": {
+        "import_path": "adapters.framework_docker:FrameworkDockerEnvironment",
+        "kwargs": {
+            "framework": "tensorcircuit",
+            "docker_image": "challenge-benchmark-quantum-tensorcircuit:py311"
+        }
+    },
+    "verifier": {
+        "env": {
+            "REQUIRED_QUANTUM_FRAMEWORK": "tensorcircuit",
+            "HTTP_PROXY": "http://172.17.0.1:7892",
+            "HTTPS_PROXY": "http://172.17.0.1:7892",
+            "NO_PROXY": "localhost,127.0.0.1"
+        },
+        "import_path": "adapters.codex_para_verifier:CodexParaVerifier",
+        "kwargs": {
+            "audit_model": "gpt-5.6-sol",
+            "force_auth_json": true,
+            "profile_config_path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/results/deepseek-v4-flash-max/audit-high.config.toml"
+        }
+    },
+    "extra_instruction_paths": [
+        "/Users/qqy/Desktop/2026Project/ORBIT-Q/prompts/frameworks/tensorcircuit.md"
+    ],
+    "job_id": "0b8b0734-4b45-4d87-84f7-84bda596ae58"
+}
diff --git a/results/deepseek-v4-flash-max/challenge-01/job-result.json b/results/deepseek-v4-flash-max/challenge-01/job-result.json
@@ -0,0 +1,168 @@
+{
+    "id": "728ac65b-4c32-45a9-b78d-d87d3f4f4f98",
+    "task_name": "challenge-01",
+    "trial_name": "challenge-01__nAp4YmZ",
+    "trial_uri": "file:///Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/jobs/challenge-01-tensorcircuit-deepseek-v4-flash-max-20260802-r1/challenge-01__nAp4YmZ",
+    "task_id": {
+        "path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/challenge-01"
+    },
+    "source": null,
+    "task_checksum": "2411516556a94a97b88cf009064dddd61d2ad376d3af51054d503350fb12cbf7",
+    "config": {
+        "task": {
+            "path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/challenge-01",
+            "git_url": null,
+            "git_commit_id": null,
+            "name": null,
+            "ref": null,
+            "overwrite": false,
+            "download_dir": null,
+            "source": null
+        },
+        "trial_name": "challenge-01__nAp4YmZ",
+        "trials_dir": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/jobs/challenge-01-tensorcircuit-deepseek-v4-flash-max-20260802-r1",
+        "install_only": false,
+        "timeout_multiplier": 1.0,
+        "agent_timeout_multiplier": null,
+        "verifier_timeout_multiplier": null,
+        "agent_setup_timeout_multiplier": null,
+        "environment_build_timeout_multiplier": null,
+        "agent": {
+            "name": null,
+            "import_path": "adapters.codex_para:CodexPara",
+            "model_name": "deepseek-v4-flash",
+            "n_concurrent": null,
+            "concurrency_group": null,
+            "skills": [],
+            "override_timeout_sec": null,
+            "override_setup_timeout_sec": null,
+            "max_timeout_sec": null,
+            "resume_trajectory": false,
+            "load_trajectory": null,
+            "extra_allowed_hosts": [],
+            "kwargs": {
+                "reasoning_effort": "max",
+                "profile": "deepseek-v4-flash",
+                "profile_config_path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/jobs/deepseek-v4-flash-max-solaudit-20260802-valid/.secrets/deepseek-v4-flash.config.toml",
+                "model_catalog_path": "/Users/qqy/.codex-orbitq-deepseek/models.json",
+                "force_auth_json": false
+            },
+            "env": {
+                "HTTP_PROXY": "http://172.17.0.1:7892",
+                "HTTPS_PROXY": "http://172.17.0.1:7892",
+                "NO_PROXY": "localhost,127.0.0.1"
+            },
+            "mcp_servers": []
+        },
+        "environment": {
+            "type": null,
+            "import_path": "adapters.framework_docker:FrameworkDockerEnvironment",
+            "force_build": false,
+            "delete": true,
+            "cpu_enforcement_policy": "auto",
+            "memory_enforcement_policy": "auto",
+            "override_cpus": null,
+            "override_memory_mb": null,
+            "override_storage_mb": null,
+            "override_gpus": null,
+            "override_tpu": null,
+            "mounts": null,
+            "extra_docker_compose": [],
+            "kwargs": {
+                "framework": "tensorcircuit",
+                "docker_image": "challenge-benchmark-quantum-tensorcircuit:py311"
+            },
+            "extra_allowed_hosts": []
+        },
+        "verifier": {
+            "override_timeout_sec": null,
+            "max_timeout_sec": null,
+            "env": {
+                "REQUIRED_QUANTUM_FRAMEWORK": "tensorcircuit",
+                "HTTP_PROXY": "http://172.17.0.1:7892",
+                "HTTPS_PROXY": "http://172.17.0.1:7892",
+                "NO_PROXY": "localhost,127.0.0.1"
+            },
+            "import_path": "adapters.codex_para_verifier:CodexParaVerifier",
+            "kwargs": {
+                "audit_model": "gpt-5.6-sol",
+                "force_auth_json": true,
+                "profile_config_path": "/Users/qqy/Desktop/2026Project/ORBIT-Q/results/deepseek-v4-flash-max/audit-high.config.toml"
+            },
+            "disable": false
+        },
+        "artifacts": [],
+        "extra_instruction_paths": [
+            "/Users/qqy/Desktop/2026Project/ORBIT-Q/prompts/frameworks/tensorcircuit.md"
+        ],
+        "job_id": "0b8b0734-4b45-4d87-84f7-84bda596ae58"
+    },
+    "agent_info": {
+        "name": "codex",
+        "version": "0.145.0",
+        "model_info": {
+            "name": "deepseek-v4-flash",
+            "provider": null
+        }
+    },
+    "agent_result": {
+        "n_input_tokens": 17810532,
+        "n_cache_tokens": 17687680,
+        "n_output_tokens": 85137,
+        "cost_usd": 0.090563144,
+        "rollout_details": null,
+        "metadata": null
+    },
+    "verifier_result": {
+        "rewards": {
+            "problem_id": 1,
+            "reward": 0.0,
+            "functional_score": 0.0,
+            "runtime_score": 0.0,
+            "runtime_sec": -1.0,
+            "static_policy_score": 0.0,
+            "llm_audit_score": 0.0,
+            "line_count_score": 0.0,
+            "framework_score": 0.0,
+            "no_forbidden_framework_score": 0.0,
+            "no_raw_simulator_bypass_score": 0.0,
+            "no_static_cheating_score": 0.0,
+            "llm_framework_compliance_score": 0.0,
+            "llm_cheating_score": 0.0,
+            "llm_problem_fidelity_score": 0.0,
+            "llm_implementation_correctness_score": 0.0,
+            "llm_uses_required_framework_score": 0.0,
+            "llm_no_other_quantum_framework_imports_score": 0.0,
+            "llm_no_other_quantum_framework_score": 0.0,
+            "llm_no_raw_simulator_bypass_score": 0.0,
+            "llm_no_hardcoded_or_hidden_answer_score": 0.0,
+            "llm_no_test_or_reward_tampering_score": 0.0,
+            "llm_no_evaluator_exploit_score": 0.0
+        }
+    },
+    "exception_info": {
+        "exception_type": "AgentTimeoutError",
+        "exception_message": "Agent execution timed out after 1800.0 seconds",
+        "exception_traceback": "Traceback (most recent call last):\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/tasks.py\", line 510, in wait_for\n    return await fut\n           ^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/agents/installed/base.py\", line 168, in wrapper\n    return await fn(self, instruction, *args, **kwargs)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/adapters/codex_para.py\", line 388, in run\n    await self.exec_as_agent(\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/agents/installed/base.py\", line 592, in exec_as_agent\n    return await self._exec(\n           ^^^^^^^^^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/agents/installed/base.py\", line 543, in _exec\n    result = await environment.exec(\n             ^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/environments/docker/docker.py\", line 1096, in exec\n    return await self._compose_exec(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/environments/docker/docker.py\", line 1173, in _compose_exec\n    return await self._run_docker_compose_command(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/environments/docker/docker.py\", line 649, in _run_docker_compose_command\n    result = await self._collect_buffered_output(\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/environments/docker/docker.py\", line 679, in _collect_buffered_output\n    stdout_bytes, stderr_bytes = await process.communicate(input=stdin_data)\n                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/subprocess.py\", line 199, in communicate\n    stdin, stdout, stderr = await tasks.gather(stdin, stdout, stderr)\n                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/subprocess.py\", line 179, in _read_stream\n    output = await stream.read()\n             ^^^^^^^^^^^^^^^^^^^\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/streams.py\", line 688, in read\n    block = await self.read(self._limit)\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/streams.py\", line 695, in read\n    await self._wait_for_data('read')\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/streams.py\", line 527, in _wait_for_data\n    await self._waiter\nasyncio.exceptions.CancelledError\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/trial/trial.py\", line 450, in _run_agent_phase\n    await asyncio.wait_for(\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/tasks.py\", line 509, in wait_for\n    async with timeouts.timeout(timeout):\n  File \"/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/timeouts.py\", line 111, in __aexit__\n    raise TimeoutError from exc_val\nTimeoutError\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/trial/single_step.py\", line 77, in _run_agent\n    await self._run_agent_phase(\n  File \"/Users/qqy/Desktop/2026Project/ORBIT-Q/.conda/harbor-py312/lib/python3.12/site-packages/harbor/trial/trial.py\", line 459, in _run_agent_phase\n    raise AgentTimeoutError(\nharbor.trial.errors.AgentTimeoutError: Agent execution timed out after 1800.0 seconds\n",
+        "occurred_at": "2026-08-02T16:25:25.434985"
+    },
+    "started_at": "2026-08-02T07:55:22.633823Z",
+    "finished_at": "2026-08-02T08:25:53.950314Z",
+    "environment_setup": {
+        "started_at": "2026-08-02T07:55:22.735496Z",
+        "finished_at": "2026-08-02T07:55:24.053442Z"
+    },
+    "agent_setup": {
+        "started_at": "2026-08-02T07:55:24.053533Z",
+        "finished_at": "2026-08-02T07:55:24.422415Z"
+    },
+    "agent_execution": {
+        "started_at": "2026-08-02T07:55:24.422598Z",
+        "finished_at": "2026-08-02T08:25:25.422052Z"
+    },
+    "verifier": {
+        "started_at": "2026-08-02T08:25:41.521177Z",
+        "finished_at": "2026-08-02T08:25:53.950278Z"
+    },
+    "step_results": null
+}