Skip to content

【P0】验证本地LLM与Embedding模型加载推理 #21

Description

@Karovia

验证ONNX Runtime Mobile集成,支持量化的Embedding模型(bge-small-zh-v1.5)和LLM模型(Qwen2-0.5B)加载、推理,封装chat()/embedding()接口与云端服务商规范对齐

验收标准

  1. ONNX Runtime Mobile可正常加载量化模型,无崩溃或异常
  2. bge-small-zh-v1.5 Embedding生成正常,输出维度正确(512维)
  3. Qwen2-0.5B LLM推理正常,支持流式输出,响应速度满足移动端要求(首字<300ms)
  4. 接口规范与原云端服务商完全对齐,前端/业务层无感知
  5. 支持模型推理参数配置(温度、最大生成长度等)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions