Skip to content

Client-side ONNX inference (zero backend) #7

Description

@ONDAAA

Run embedding model directly in the browser via ONNX Runtime Web / WebAssembly.

Benefits:

  • Zero server costs for inference
  • Zero latency (no network roundtrip)
  • Works offline
  • No rate limiting concerns

Approach:

  • Export MiniLM to ONNX format (~30MB)
  • Bundle with extension or lazy-download on first use
  • Use onnxruntime-web for inference
  • Keep backend only for auth/billing

Trade-off: larger extension size (~30MB vs ~50KB)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions