Skip to content

Adds two direct-placement BBOPlace Frontier-CS 2.0 tasks#130

Merged
joyemang33 merged 1 commit into
mainfrom
codex/add-bboplace-direct-tasks
Jun 3, 2026
Merged

Adds two direct-placement BBOPlace Frontier-CS 2.0 tasks#130
joyemang33 merged 1 commit into
mainfrom
codex/add-bboplace-direct-tasks

Conversation

@joyemang33
Copy link
Copy Markdown
Contributor

Summary

Adds two direct-placement BBOPlace Frontier-CS 2.0 tasks:

  • bboplace_direct_ispd2005
  • bboplace_direct_iccad2015

Unlike the suite-style BBOPlace tasks, these variants ask the agent to directly produce one JSON placement for one fixed design:

  • ISPD2005 direct: adaptec1
  • ICCAD2015 direct: superblue1

Agents may write Python scripts, shell scripts, search loops, local parsers, or other helper programs during the trial, but the final graded artifact is /app/solution.json. The judge uses the same CPU-only BBOPlace MGO MP-HPWL evaluation path and relaxed MGO baselines as the suite tasks.

This also extends the Frontier-CS 2.0 adapter reference-solution handling so file submissions with non-.py paths, such as /app/solution.json, can still generate valid Harbor reference artifacts without affecting directory-style tasks like Vector DB ANN.

Please read CONTRIBUTING.md before submitting.

Type of Change

  • New research problem
  • New algorithmic problem
  • New Frontier-CS 2.0 problem
  • Bug fix
  • Documentation update
  • Other:

Testing

  • PYTHONPYCACHEPREFIX=/private/tmp/frontier-cs-pycache python3 -m py_compile 2.0/problems/bboplace_direct_ispd2005/evaluator.py 2.0/problems/bboplace_direct_iccad2015/evaluator.py adapters/frontier-cs-2.0/src/frontier_cs_2_0/adapter.py
  • Generated all four BBOPlace Harbor tasks:
    • bboplace_ispd2005
    • bboplace_iccad2015
    • bboplace_direct_ispd2005
    • bboplace_direct_iccad2015
  • python3 tools/bboplace/check_generated_tasks.py /private/tmp/frontier-cs-bboplace-generated
  • Generated vector_db_ann_relaxed as a non-BBOPlace adapter smoke test to confirm directory submissions were unaffected.
  • Ran judge-image smoke tests for both direct tasks:
    • bboplace_direct_ispd2005 on adaptec1
    • bboplace_direct_iccad2015 on superblue1
  • Ran a real Harbor Codex trial for bboplace_direct_ispd2005 with local Codex auth injection. The agent submitted multiple valid JSON artifacts and the judge returned metrics correctly; all valid placements scored 0 because their HPWL remained above the relaxed baseline.

Checklist

  • Code follows the project structure and conventions
  • Self-review completed
  • Documentation updated (if applicable)

CI Validation (for new problems)

When adding new problems, CI will automatically validate that your reference solution achieves score > 0.

  • Algorithmic problems: Include reference.cpp in your problem directory
  • Research problems: Include reference.py (or reference.cpp if language: cpp in config.yaml)
  • 2.0 problems: Include reference.py unless the problem config declares another language

These direct JSON tasks include reference.json because the configured submission path is /app/solution.json.

@joyemang33 joyemang33 force-pushed the codex/add-bboplace-direct-tasks branch from fb615a9 to 23d230f Compare June 3, 2026 01:25
@joyemang33 joyemang33 marked this pull request as ready for review June 3, 2026 01:25
@joyemang33 joyemang33 merged commit cb89d67 into main Jun 3, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant