feat: Expose Pochi LLM as local OpenAI-compatible API for terminal scripts

[wsxiaoys]

Summary

Enable user scripts running in the integrated terminal to access Pochi's LLM capabilities via a standard API (OpenAI compatible) without requiring separate API keys.

Proposed Solution

1. Local LLM Server:

   • Implement a lightweight HTTP server within the VSCode extension (e.g., using hono).
   • This server should listen on a local port and implement the OpenAI Chat Completion API (/v1/chat/completions).
   • It should proxy requests to the internal Pochi model infrastructure (using createModel and streamText from ai SDK).

2. Environment Variables:

   • Inject environment variables into the VSCode terminal session using vscode.EnvironmentVariableCollection.
   • Variables to set:
     • OPENAI_BASE_URL: http://127.0.0.1:<port>/v1
     • OPENAI_API_KEY: A randomly generated key for the session.
     • POCHI_OPENAI_API_BASE / POCHI_OPENAI_API_KEY: Pochi specific aliases.

3. Authentication & Security:

   • The local server should validate the Authorization header against the generated API key to ensure only authorized scripts (from the user's terminal) can access it.

4. Configuration:

   • Add a new configuration option in pochi.advanced to enable/disable this feature (e.g., pochi.advanced.llmServer.enabled).
   • The server should only start if this option is enabled.

5. Benefits:

   • Allows users to run scripts (e.g., Python scripts using openai library, or other tools) that leverage Pochi's LLM access.
   • No need for users to configure external API keys for these scripts.
   • Usage counts towards Pochi's quota.

Implementation Details

• Dependencies: Add hono, @hono/node-server to packages/vscode.
• Configuration: Update package.json to include the new setting.
• Server Implementation: Create a LLMServer class that starts the server on activation (if enabled) and manages the port/lifecycle.
• Integration: Register the server in extension.ts, listen for configuration changes, and update environment variables.

Example Usage

User runs a python script in the terminal:

from openai import OpenAI
client = OpenAI() # Automatically picks up OPENAI_BASE_URL and OPENAI_API_KEY injected by Pochi
response = client.chat.completions.create(
    model="pochi/swift",
    messages=[{"role": "user", "content": "Hello"}]
)
print(response.choices[0].message.content)

🤖 Generated with Pochi