configs/ladder-3b-kaggle.yaml trains on solutions_py_decontaminated, so the
model only ever sees Python. Most competitive programmers write C++, and the
tighter time limits on hard problems often make Python infeasible regardless of
whether the algorithm is right.
The dataset has a solutions config that is not Python-restricted, so the data
side is mostly a config change. The judge is where the work is:
eval/sandbox.py hardcodes sys.executable — it needs a compile step and a
language-dispatched run command- Compilation failure is a distinct verdict from runtime error, and conflating
them would hide a whole class of model failure
- Compile time should not count against the run timeout
data/formatting.py asks for "a single Python code block"; the prompt and
extract_code's fence matching both need to follow the target language
Worth landing the judge side first and confirming it against the dataset's own
C++ reference solutions, the same way the Python judge was validated.
configs/ladder-3b-kaggle.yamltrains onsolutions_py_decontaminated, so themodel only ever sees Python. Most competitive programmers write C++, and the
tighter time limits on hard problems often make Python infeasible regardless of
whether the algorithm is right.
The dataset has a
solutionsconfig that is not Python-restricted, so the dataside is mostly a config change. The judge is where the work is:
eval/sandbox.pyhardcodessys.executable— it needs a compile step and alanguage-dispatched run command
them would hide a whole class of model failure
data/formatting.pyasks for "a single Python code block"; the prompt andextract_code's fence matching both need to follow the target languageWorth landing the judge side first and confirming it against the dataset's own
C++ reference solutions, the same way the Python judge was validated.