Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,20 @@ jobs:
# what this step enforces.
- name: Pyright type check
run: uv run pyright
- name: FrontierChallenge release gates
run: |
uv run pytest -q \
-c benchmarks/frontierchallenge/pyproject.toml \
benchmarks/frontierchallenge/tests
uv run python benchmarks/frontierchallenge/scripts/check_public_leaks.py \
benchmarks/frontierchallenge --allow-empty
uv run python benchmarks/frontierchallenge/scripts/check_restricted_software.py \
benchmarks/frontierchallenge
for script in \
benchmarks/frontierchallenge/scripts/*.sh \
benchmarks/frontierchallenge/shared_images/*.sh; do
bash -n "$script"
done
- name: Unit and exec-bundle tests
env:
SANDBOX_BACKEND: bwrap
Expand Down
75 changes: 0 additions & 75 deletions benchmarks/frontierchallenge/.github/workflows/checks.yml

This file was deleted.

18 changes: 13 additions & 5 deletions benchmarks/frontierchallenge/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,26 @@
<h1 align="center">
<strong>FrontierChallenge</strong>&nbsp;&nbsp;
<img src="docs/assets/apodex-logo.png" alt="Apodex AI" width="190" align="middle">
</h1>
<div align="center">
<picture>
<img src="docs/assets/apodex-logo.png" width="30%" alt="Apodex AI">
</picture>
</div>

---

<br>

<div align="center">

**97 checkable scientific workflows for evaluating AI agents.**

[Tasks](TASKS.md) · [Quickstart](docs/quickstart.md) ·
[Hugging Face](https://huggingface.co/datasets/apodex/FrontierChallenge) ·
[Website](https://urban-chainsaw-mnormyp.pages.github.io/) · [Citation](#citation)
[Website](https://apodexai.github.io/FrontierAgent/benchmarks/FrontierChallenge/) ·
[Citation](#citation)

</div>

# FrontierChallenge

FrontierChallenge asks an agent to turn raw scientific data into specified,
machine-checkable deliverables. It covers diffraction, spectroscopy, molecular
simulation, electrochemistry, quantitative imaging, and molecular biology.
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/frontierchallenge/site/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This directory contains the dependency-free static website for the benchmark.
It lives in the public FrontierChallenge repository so website content, task
documentation, and leaderboard links can evolve together.

Published preview: <https://urban-chainsaw-mnormyp.pages.github.io/>
Published site: <https://apodexai.github.io/FrontierAgent/benchmarks/FrontierChallenge/>

## Preview locally

Expand Down
Loading