Skip to content

Toppings (Delta + LoRA support) #6

@xzyaoi

Description

@xzyaoi

Stage 1 (in PR #10):

  • System Controller for Topping Registration
  • Toppings Manager (Detects, Loads, Swap toppings)
  • Kernels (Delta + LoRA kernel @enothum)
  • Statistics Logger

Stage 2:

  • FP16 full swap support
  • Correctness verification
  • Improved stats logger

Step 3:

  • Tensor Parallelism support

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions