Skip to content

position_embeddings need check valid #165

Description

@lix19937

Describe the bug

https://github.com/NVIDIA/TensorRT-Edge-LLM/blob/release/0.9.1/tensorrt_edgellm/models/qwen3_vl/modeling_qwen3_vl_visual.py#L223

if position_embeddings is None or [], it will dump.

    def forward(
        self,
        hidden_states: torch.Tensor,
        cu_seqlens: torch.Tensor,
        max_seqlen_carrier: Optional[torch.Tensor],
        position_embeddings: Tuple[torch.Tensor, torch.Tensor], # << ---------
        kv_lengths: Optional[torch.Tensor] = None,
    ) -> torch.Tensor:
        seq_length = hidden_states.shape[0]
        q, k, v = self.qkv(hidden_states).reshape(
            seq_length, 3 * self.num_heads,
            self.head_dim).split(self.num_heads, dim=1)
        cos, sin = position_embeddings   
        q, k = apply_rotary_pos_emb_vision(q, k, cos, sin)     # << ---------

need add a judge

if position_embeddings:   # << ---------
        cos, sin = position_embeddings   
        q, k = apply_rotary_pos_emb_vision(q, k, cos, sin)     

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions