Skip to content

Add GPT-5.6 Luna high TensorCircuit benchmark results - #21

Open
QingyunQian wants to merge 3 commits into
sxzgroup:v2from
QingyunQian:codex/gpt-5.6-luna-high-benchmark-upstream
Open

Add GPT-5.6 Luna high TensorCircuit benchmark results #21
QingyunQian wants to merge 3 commits into
sxzgroup:v2from
QingyunQian:codex/gpt-5.6-luna-high-benchmark-upstream

Conversation

@QingyunQian

@QingyunQian QingyunQian commented Aug 1, 2026

Copy link
Copy Markdown

Summary

Archive the single-trial GPT-5.6 Luna/high TensorCircuit benchmark across all 12 ORBIT-Q challenges, independently audited by GPT-5.6 Sol/high.

  • 9 / 12 final expert-adjudicated valid solutions; failures on Tasks 01, 04, and 08
  • Functional and static checks pass on all 12 tasks
  • Docker-isolated, sequential execution with one valid model outcome per task
  • Includes selected solutions, rewards, functional/static/audit outputs, logs, configs, provenance, normalized summary data, and reproducible figures
  • Preserves the raw Task 08 verifier pass while recording the final human expert adjudication as failure

Validation

  • Verified solver model gpt-5.6-luna with reasoning effort high
  • Verified audit model gpt-5.6-sol with reasoning effort high
  • Verified one non-infrastructure outcome for every task
  • Archive regeneration and consistency checks pass

See results/gpt56luna-high/README.md for the complete result table and protocol.

Result figures

Final task outcomes

GPT-5.6 Luna high final task outcomes

Agent-side resource use

GPT-5.6 Luna high agent-side resource use

@QingyunQian
QingyunQian marked this pull request as ready for review August 1, 2026 15:07
@QingyunQian QingyunQian changed the title Benchmark GPT-5.6 Luna high with Sol high audit Add GPT-5.6 Luna high TensorCircuit benchmark results Aug 1, 2026
@refraction-ray

Copy link
Copy Markdown
Member

I think problem 8 can be taken right as the LLM audit indicates

@QingyunQian
QingyunQian changed the base branch from main to v2 August 3, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants