vllm issue: image_grid_thw[image_index][0], list index out of range

[HeinZingerRyo]

Hi authors, thanks for your great work. I want to reproduce the web UI demo part, but I'm encountering an index error. The version of vllm is 0.7.3

The full info is as follows:

> CUDA_VISIBLE_DEVICES=1  vllm serve ./qwen2.5-vl-7b-instruct  --served-model-name qwen2.5-vl-7b-instruct --host 127.0.0.1  --port 8000  --tensor-parallel-size 1  --trust-remote-code
INFO 09-28 08:16:03 __init__.py:207] Automatically detected platform cuda.
INFO 09-28 08:16:03 api_server.py:912] vLLM API server version 0.7.3
INFO 09-28 08:16:03 api_server.py:913] args: Namespace(subparser='serve', model_tag='./qwen2.5-vl-7b-instruct', config='', host='127.0.0.1', port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, chat_template_content_format='auto', response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_request_id_headers=False, enable_auto_tool_choice=False, enable_reasoning=False, reasoning_parser=None, tool_call_parser=None, tool_parser_plugin='', model='./qwen2.5-vl-7b-instruct', task='auto', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=True, allowed_local_media_path=None, download_dir=None, load_format='auto', config_format=<ConfigFormat.AUTO: 'auto'>, dtype='auto', kv_cache_dtype='auto', max_model_len=None, guided_decoding_backend='xgrammar', logits_processor_pattern=None, model_impl='auto', distributed_executor_backend=None, pipeline_parallel_size=1, tensor_parallel_size=1, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=None, enable_prefix_caching=None, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_partial_prefills=1, max_long_partial_prefills=1, long_prefill_token_threshold=0, max_num_seqs=None, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, hf_overrides=None, enforce_eager=False, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, disable_mm_preprocessor_cache=False, enable_lora=False, enable_lora_bias=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['qwen2.5-vl-7b-instruct'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, scheduling_policy='fcfs', scheduler_cls='vllm.core.scheduler.Scheduler', override_neuron_config=None, override_pooler_config=None, compilation_config=None, kv_transfer_config=None, worker_cls='auto', generation_config=None, override_generation_config=None, enable_sleep_mode=False, calculate_kv_scales=False, additional_config=None, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, enable_prompt_tokens_details=False, dispatch_function=<function ServeSubcommand.cmd at 0x7fce98f61c60>)
INFO 09-28 08:16:03 api_server.py:209] Started engine process with PID 3667826
`torch_dtype` is deprecated! Use `dtype` instead!
INFO 09-28 08:16:07 __init__.py:207] Automatically detected platform cuda.
`torch_dtype` is deprecated! Use `dtype` instead!
INFO 09-28 08:16:08 config.py:549] This model supports multiple tasks: {'embed', 'generate', 'score', 'reward', 'classify'}. Defaulting to 'generate'.
WARNING 09-28 08:16:08 arg_utils.py:1197] The model has a long context length (128000). This may cause OOM errors during the initial memory profiling phase, or result in low performance due to small KV cache space. Consider setting --max-model-len to a smaller value.
INFO 09-28 08:16:12 config.py:549] This model supports multiple tasks: {'generate', 'reward', 'classify', 'score', 'embed'}. Defaulting to 'generate'.
WARNING 09-28 08:16:12 arg_utils.py:1197] The model has a long context length (128000). This may cause OOM errors during the initial memory profiling phase, or result in low performance due to small KV cache space. Consider setting --max-model-len to a smaller value.
INFO 09-28 08:16:12 llm_engine.py:234] Initializing a V0 LLM engine (v0.7.3) with config: model='./qwen2.5-vl-7b-instruct', speculative_config=None, tokenizer='./qwen2.5-vl-7b-instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=128000, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=qwen2.5-vl-7b-instruct, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=False, chunked_prefill_enabled=False, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":256}, use_cached_outputs=True, 
INFO 09-28 08:16:14 cuda.py:229] Using Flash Attention backend.
INFO 09-28 08:16:14 model_runner.py:1110] Starting to load model ./qwen2.5-vl-7b-instruct...
WARNING 09-28 08:16:14 vision.py:94] Current `vllm-flash-attn` has a bug inside vision module, so we use xformers backend instead. You can run `pip install flash-attn` to use flash-attention backend.
INFO 09-28 08:16:14 config.py:3054] cudagraph sizes specified by model runner [1, 2, 4, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160, 168, 176, 184, 192, 200, 208, 216, 224, 232, 240, 248, 256] is overridden by config [256, 128, 2, 1, 4, 136, 8, 144, 16, 152, 24, 160, 32, 168, 40, 176, 48, 184, 56, 192, 64, 200, 72, 208, 80, 216, 88, 120, 224, 96, 232, 104, 240, 112, 248]
Loading safetensors checkpoint shards:   0% Completed | 0/5 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:  20% Completed | 1/5 [00:00<00:02,  1.70it/s]
Loading safetensors checkpoint shards:  40% Completed | 2/5 [00:01<00:01,  1.54it/s]
Loading safetensors checkpoint shards:  60% Completed | 3/5 [00:01<00:01,  1.55it/s]
Loading safetensors checkpoint shards:  80% Completed | 4/5 [00:02<00:00,  1.58it/s]
Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:02<00:00,  2.07it/s]
Loading safetensors checkpoint shards: 100% Completed | 5/5 [00:02<00:00,  1.82it/s]

INFO 09-28 08:16:17 model_runner.py:1115] Loading model weights took 15.6270 GB
The image processor of type `Qwen2VLImageProcessor` is now loaded as a fast processor by default, even if the model checkpoint was saved with a slow processor. This is a breaking change and may produce slightly different outputs. To continue using the slow processor, instantiate this class with `use_fast=False`. Note that this behavior will be extended to all models in a future release.
INFO 09-28 08:17:05 worker.py:267] Memory profiling takes 47.90 seconds
INFO 09-28 08:17:05 worker.py:267] the current vLLM instance can use total_gpu_memory (79.15GiB) x gpu_memory_utilization (0.90) = 71.24GiB
INFO 09-28 08:17:05 worker.py:267] model weights take 15.63GiB; non_torch_memory takes 0.09GiB; PyTorch activation peak memory takes 21.27GiB; the rest of the memory reserved for KV Cache is 34.25GiB.
INFO 09-28 08:17:06 executor_base.py:111] # cuda blocks: 40077, # CPU blocks: 4681
INFO 09-28 08:17:06 executor_base.py:116] Maximum concurrency for 128000 tokens per request: 5.01x
INFO 09-28 08:17:09 model_runner.py:1434] Capturing cudagraphs for decoding. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI. If out-of-memory error occurs during cudagraph capture, consider decreasing `gpu_memory_utilization` or switching to eager mode. You can also reduce the `max_num_seqs` as needed to decrease memory usage.
Capturing CUDA graph shapes: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:12<00:00,  2.77it/s]
INFO 09-28 08:17:22 model_runner.py:1562] Graph capturing finished in 13 secs, took 1.91 GiB
INFO 09-28 08:17:22 llm_engine.py:436] init engine (profile, create kv cache, warmup model) took 64.59 seconds
INFO 09-28 08:17:22 api_server.py:958] Starting vLLM API server on http://127.0.0.1:8000
INFO 09-28 08:17:22 launcher.py:23] Available routes are:
INFO 09-28 08:17:22 launcher.py:31] Route: /openapi.json, Methods: HEAD, GET
INFO 09-28 08:17:22 launcher.py:31] Route: /docs, Methods: HEAD, GET
INFO 09-28 08:17:22 launcher.py:31] Route: /docs/oauth2-redirect, Methods: HEAD, GET
INFO 09-28 08:17:22 launcher.py:31] Route: /redoc, Methods: HEAD, GET
INFO 09-28 08:17:22 launcher.py:31] Route: /health, Methods: GET
INFO 09-28 08:17:22 launcher.py:31] Route: /ping, Methods: POST, GET
INFO 09-28 08:17:22 launcher.py:31] Route: /tokenize, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /detokenize, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/models, Methods: GET
INFO 09-28 08:17:22 launcher.py:31] Route: /version, Methods: GET
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/chat/completions, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/completions, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/embeddings, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /pooling, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /score, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/score, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/audio/transcriptions, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /rerank, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v1/rerank, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /v2/rerank, Methods: POST
INFO 09-28 08:17:22 launcher.py:31] Route: /invocations, Methods: POST
INFO:     Started server process [3667636]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     127.0.0.1:50638 - "GET /v1/models HTTP/1.1" 200 OK
INFO 09-28 08:20:26 chat_utils.py:332] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.
INFO 09-28 08:20:26 logger.py:39] Received request chatcmpl-62731522ef1d486484bf3a902fce2039: prompt: '<|im_start|>system\nYou are using a Mobile device. You are able to use a Action Space Operator to interact with the mobile based on the given task and screenshot.\n\n## Action Space\nYour available "Next Action" only include:\n- click(point=[x,y]): Click on the coordinate point specified on the screen (x,y).\n- long_press(point=[x,y]): Long press the screen to specify coordinates (x,y).\n- type(text=\'hello world\'): Types a string of text.\n- scroll(start_point=[x1,y1], end_point=[x2,y2]): Scroll the screen, (x1,y1) is the starting coordinate position, (x2,y2) is the end coordinate position. In particular, when y1=y2, you can swipe left and right on the desktop to switch pages, which is very helpful for finding a specific application.\n- press_home(): Back to Home page.\n- press_back(): Back to previous page.\n- finished(answer=\'\'): Submit the task regardless of whether it succeeds or fails. The answer parameter is to summarize the content of the reply to the user.\n- call_user(question=\'\'): Submit the task and call the user when the task is unsolvable, or when you need the user\'s help.\n- wait(): Wait for loading to complete.\n\n## Note\n- Action click, long_press and scroll must contain coordinates within.\n- You may be given some history plan and actions, this is the response from the previous loop.\n- You should carefully consider your plan base on the task, screenshot, and history actions.\n- Write a small plan and finally summarize your next action (with its target element) in one sentence in `Thought` part.\n\n## Suggestions\n- If you need to open an APP, when the home page is not available, you can scroll down to the search page to find the corresponding APP.\n- When the screen of the previous operation is not responsive, you need to avoid performing the same action in the next step.\n- Shopping or life services apps, you should make use of the in-app search function as much as possible to find quickly.\n- Reduce the execution steps as much as possible, and find the optimal execution path to achieve the task goal.\n\n## Format\nTask: The task description.\nObservation: The mobile screenshot or user response.\nThought: The process of thinking.\nAction: The next action. Must be one of the Action Space.\n\n**Be aware that Observation, Thought, and Action will be repeated.**\n\nNow, let\'s begin!<|im_end|>\n<|im_start|>user\nTask: open wechatObservation: <|vision_start|><|image_pad|><|vision_end|><|vision_start|><|image_pad|><|vision_end|><|im_end|>\n<|im_start|>assistant\n', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=0.3, top_p=1.0, top_k=-1, min_p=0.0, seed=None, stop=['Observation'], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=1024, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None), prompt_token_ids: None, lora_request: None, prompt_adapter_request: None.
INFO 09-28 08:20:28 engine.py:280] Added request chatcmpl-62731522ef1d486484bf3a902fce2039.
CRITICAL 09-28 08:20:28 launcher.py:104] MQLLMEngine is already dead, terminating server process
INFO:     127.0.0.1:57966 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR 09-28 08:20:28 engine.py:140] IndexError('list index out of range')
ERROR 09-28 08:20:28 engine.py:140] Traceback (most recent call last):
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py", line 138, in start
ERROR 09-28 08:20:28 engine.py:140]     self.run_engine_loop()
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py", line 201, in run_engine_loop
ERROR 09-28 08:20:28 engine.py:140]     request_outputs = self.engine_step()
ERROR 09-28 08:20:28 engine.py:140]                       ^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py", line 219, in engine_step
ERROR 09-28 08:20:28 engine.py:140]     raise e
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/engine/multiprocessing/engine.py", line 210, in engine_step
ERROR 09-28 08:20:28 engine.py:140]     return self.engine.step()
ERROR 09-28 08:20:28 engine.py:140]            ^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/engine/llm_engine.py", line 1391, in step
ERROR 09-28 08:20:28 engine.py:140]     outputs = self.model_executor.execute_model(
ERROR 09-28 08:20:28 engine.py:140]               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/executor/executor_base.py", line 139, in execute_model
ERROR 09-28 08:20:28 engine.py:140]     output = self.collective_rpc("execute_model",
ERROR 09-28 08:20:28 engine.py:140]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
ERROR 09-28 08:20:28 engine.py:140]     answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 09-28 08:20:28 engine.py:140]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/utils.py", line 2196, in run_method
ERROR 09-28 08:20:28 engine.py:140]     return func(*args, **kwargs)
ERROR 09-28 08:20:28 engine.py:140]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/worker_base.py", line 394, in execute_model
ERROR 09-28 08:20:28 engine.py:140]     inputs = self.prepare_input(execute_model_req)
ERROR 09-28 08:20:28 engine.py:140]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/worker_base.py", line 379, in prepare_input
ERROR 09-28 08:20:28 engine.py:140]     return self._get_driver_input_and_broadcast(execute_model_req)
ERROR 09-28 08:20:28 engine.py:140]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/worker_base.py", line 341, in _get_driver_input_and_broadcast
ERROR 09-28 08:20:28 engine.py:140]     self.model_runner.prepare_model_input(
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/model_runner.py", line 1628, in prepare_model_input
ERROR 09-28 08:20:28 engine.py:140]     model_input = self._prepare_model_input_tensors(
ERROR 09-28 08:20:28 engine.py:140]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/model_runner.py", line 1216, in _prepare_model_input_tensors
ERROR 09-28 08:20:28 engine.py:140]     self.builder.add_seq_group(seq_group_metadata)
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/model_runner.py", line 760, in add_seq_group
ERROR 09-28 08:20:28 engine.py:140]     per_seq_group_fn(inter_data, seq_group_metadata)
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/worker/model_runner.py", line 715, in _compute_multi_modal_input
ERROR 09-28 08:20:28 engine.py:140]     MRotaryEmbedding.get_input_positions(
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 854, in get_input_positions
ERROR 09-28 08:20:28 engine.py:140]     MRotaryEmbedding.get_input_positions_tensor(
ERROR 09-28 08:20:28 engine.py:140]   File "/home/xxx/micromamba/envs/vllm/lib/python3.12/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 914, in get_input_positions_tensor
ERROR 09-28 08:20:28 engine.py:140]     image_grid_thw[image_index][0],
ERROR 09-28 08:20:28 engine.py:140]     ~~~~~~~~~~~~~~^^^^^^^^^^^^^
ERROR 09-28 08:20:28 engine.py:140] IndexError: list index out of range
INFO:     Shutting down
INFO:     Waiting for application shutdown.
INFO:     Application shutdown complete.
INFO:     Finished server process [3667636]

I have tested that qwen is successfully deployed via vllm:

curl http://127.0.0.1:8000/v1/models
{"object":"list","data":[{"id":"qwen2.5-vl-7b-instruct","object":"model","created":1759047522,"owned_by":"vllm","root":"./qwen2.5-vl-7b-instruct","parent":null,"max_model_len":128000,"permission":[{"id":"modelperm-f6b8e5164b254a14932e215cfa579f66","object":"model_permission","created":1759047522,"allow_create_engine":false,"allow_sampling":true,"allow_logprobs":true,"allow_search_indices":false,"allow_view":true,"allow_fine_tuning":false,"organization":"*","group":null,"is_blocking":false}]}]}%        

I have searched for other possibly related issues but I cannot find a proper solution. I wonder if this is some prompt issue with vllm or qwen? Could you please offer some help? Thanks for your reply in advance.

[lokinko]

I suspect this might be due to an outdated version of vllm. There are some differences in the way images are inserted between vllm version 0.7 and later versions. We recommend upgrading vllm to version 0.8 or higher. If you still encounter this issue, please provide the portion of your code where you handle the response.

[HeinZingerRyo]

Hi @lokinko,

Thanks for your reply!

I didn’t make any modifications to the code. I simply ran the web UI demo exactly as described in your README, using the original codebase.

I also tried switching the vllm version to 0.10.2, but still encountered the same error:
image_grid_thw[image_index][0] list index out of range.

At the moment, I’m using the Qwen APIs instead of vllm to deploy MLLMs, and using APIs just simply works, so this issue is not urgent for me :)

[NingLi670]

Thank you for your patience. We have updated our code to support VLLM versions 0.8 and above. If you still encounter other problems during use, please feel free to ask.