repositioning fixes, tensor shape mismatch

[nataxcan]

Accuracy numbers were wrong on our evals, now they're good again.

The issue was in _perform_repositioning in gpu_model_runner.py in the v1 API.

Basically if there's not too many repositioning requests, the vectors from different layers of the model are concatenated along the batch dimension before RoPE is applied twice to them.

Problem was: I forgot that the "positions" tensor telling the RoPE function what positional encodings to apply to the vectors needs to be repeated as its batch size does not match the big tensor's. Normally PyTorch would broadcast that tensor, but the RoPE function calls the .view function on the key vectors such that this is a wrong assumption.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[tdoublep]

LGTM