My Commend:
accelerate launch --num_processes=8 --main_process_port=12345 -m lmms_eval
--model qwen2_5_vl
--model_args=pretrained=Model/Qwen2.5-VL-3B-Instruct
--tasks mme_cot_reason
--batch_size 1 --log_samples --log_samples_suffix output_cot --output_path ./logs/
My error:
Traceback (most recent call last):
File "/anaconda/envs/vlm-r1/lib/python3.10/site-packages/lmms_eval/main.py", line 329, in cli_evaluate
results, samples = cli_evaluate_single(args)
File "anaconda/envs/vlm-r1/lib/python3.10/site-packages/lmms_eval/main.py", line 462, in cli_evaluate_single
raise ValueError(
ValueError: Tasks not found: mme_cot_reason. Try lmms-eval --tasks {list_groups,list_subtasks,list_tags,list} to list out all available names for task groupings; only (sub)tasks; tags; or all of the above, or pass '--verbosity DEBUG' to troubleshoot task registration issues.
2025-04-20 05:59:51.860 | ERROR | main:cli_evaluate:348 - Error during evaluation: Tasks not found: mme_cot_reason. Try lmms-eval --tasks {list_groups,list_subtasks,list_tags,list} to list out all available names for task groupings; only (sub)tasks; tags; or all of the above, or pass '--verbosity DEBUG' to troubleshoot task registration issues.. Please set --verbosity=DEBUG to get more information.
My Commend:
accelerate launch --num_processes=8 --main_process_port=12345 -m lmms_eval
--model qwen2_5_vl
--model_args=pretrained=Model/Qwen2.5-VL-3B-Instruct
--tasks mme_cot_reason
--batch_size 1 --log_samples --log_samples_suffix output_cot --output_path ./logs/
My error:
Traceback (most recent call last):
File "/anaconda/envs/vlm-r1/lib/python3.10/site-packages/lmms_eval/main.py", line 329, in cli_evaluate
results, samples = cli_evaluate_single(args)
File "anaconda/envs/vlm-r1/lib/python3.10/site-packages/lmms_eval/main.py", line 462, in cli_evaluate_single
raise ValueError(
ValueError: Tasks not found: mme_cot_reason. Try
lmms-eval --tasks {list_groups,list_subtasks,list_tags,list}to list out all available names for task groupings; only (sub)tasks; tags; or all of the above, or pass '--verbosity DEBUG' to troubleshoot task registration issues.2025-04-20 05:59:51.860 | ERROR | main:cli_evaluate:348 - Error during evaluation: Tasks not found: mme_cot_reason. Try
lmms-eval --tasks {list_groups,list_subtasks,list_tags,list}to list out all available names for task groupings; only (sub)tasks; tags; or all of the above, or pass '--verbosity DEBUG' to troubleshoot task registration issues.. Please set--verbosity=DEBUGto get more information.