Question about MCU evaluation hyperparameters for reproducing JARVIS-VLA results

[wantbook-book]

Hi, thank you for releasing the code and the JARVIS-VLA model.

I am trying to reproduce the MCU evaluation results reported in the paper. Following the MCU task setting, I added the following tasks to the JARVIS-VLA codebase and evaluated CraftJarvis/JarvisVLA-Qwen2-VL-7B with the parameters below.

Evaluation settings

Task Type	History Num	Max Frames	Action Chunk Len	Temperature
Mine	2	500	1	0.6
Craft, Smelt	4	400	1	0.9

Evaluation results

Model	Mine: Iron Ore	Mine: Obsidian	Mine: Stone	Craft: Ladder	Craft: Crafting Table from Planks	Craft: Crafting Table from Logs	Smelt: Beef	Smelt: Iron Ingot
JARVIS-VLA-Qwen2	15/30	15/30	2/30	9/30	8/30	5/30	24/30	18/30

However, my reproduced results are still lower than the results reported in the paper.

Could you please clarify the exact evaluation hyperparameters used for the MCU benchmark in the paper?

Thank you very much for your help!