Skip to content

Implement APIs for Packing Sequences Within a Single Batch #19

@mrapplexz

Description

@mrapplexz
  • Add cu_seqlens to SDPA and GDN layers
  • Implement some user API that controls packing dataset items into a single sequence
  • Make current models support specifying seqlens for packed sequences

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions