How to Finetune StreamingChat with SVBench Dataset

[dog14230pp]

Hi authors, thank you for releasing such an excellent work!!

I'm currently trying to finetune the StreamingChat models using your SVBench dataset. However, as far as I can tell, there's limited documentation on finetuning StreamingChat (I appreciate that you kindly provided training scripts).

Could you kindly provide some guidance on how to use the SVBench dataset for model training?

Here are a few issues I'm encountering currently:

1. How can I format the dataset to follow the structure described in the InternVL chat data format guide?
   Also, which JSON files should I use (e.g., in Streaming, Dialogue, etc.)?

2. The dataset includes many QA pairs along with timestamps (e.g., qac_timestamps_start in Dialogue, and timestamps in Streaming), but I'm not sure how to align or combine them properly for training.
   For example: how many frames should be sampled per QA pair?

3. Should we use Multi-Image Data or Video Data format as described in InternVL documentation?

Thanks so much for your help, and I look forward to your insights 🙏
Hope we can have a productive discussion! 💯