Skip to content

mtp guided expert weight prefetch #18

Description

@ycui7

it would be nice to support expert weight prefetch when SSD expert weight streaming is enabled. The tokens predicted by the MTP head could be used to deduce what expert weights are needed in next round, and this information could be used to prefetch weight on SSD before they are needed by the GPU, this in theory should improve throughput.

MTP might predict the wrong token, but it not necessarily means the wrong token will lead the wrong expert weight. The next expert weight prediction might be more forgiving than next token prediction.

there are maybe one or two people that have been working on this feature on llama.cpp, but so far, no working demo has been shown publicly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions