fix: require model discovery before AI SDK usage#17
Open
tonychang04 wants to merge 1 commit intomainfrom
Open
Conversation
… defaults AI skills were defaulting to hardcoded model IDs (e.g., claude-3.5-haiku) instead of querying which models are actually configured. Now skills instruct agents to run `npx @insforge/cli metadata --json` or query `ai.configs` table first. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WalkthroughThe pull request updates documentation for AI model discovery, shifting from HTTP endpoint-based configuration to CLI-driven metadata queries and direct database table access. It removes hardcoded model IDs from SDK examples, replaces them with variables, and introduces explicit guidance on the Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
skills/insforge/ai/sdk-integration.md (1)
32-131:⚠️ Potential issue | 🟠 MajorIntroduce an explicit
Usage Examplessection header.Examples are present, but the required
Usage Examplessection heading is missing. Please wrap sections like “Chat Completions”, “Embeddings”, and “Image Generation” under## Usage Examplesto satisfy the documentation format contract.As per coding guidelines: "Documentation files must maintain consistent format: Setup, Usage Examples, Best Practices, Common Mistakes sections".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@skills/insforge/ai/sdk-integration.md` around lines 32 - 131, Add a top-level "## Usage Examples" header and move the existing "Chat Completions", "Embeddings", and "Image Generation" sections so they are subsections under that header (e.g., "## Usage Examples" followed by "### Chat Completions", "### Embeddings", "### Image Generation"); ensure the document follows the required section order (Setup, Usage Examples, Best Practices, Common Mistakes) and that the headings in the diff (Chat Completions, Embeddings, Image Generation) are converted to subordinate headings under the new Usage Examples section.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@skills/insforge/ai/backend-configuration.md`:
- Around line 1-62: The AI Backend Configuration document is missing the
required headings; add concise "Setup", "Usage Examples", and "Common Mistakes"
sections to the existing content so it follows the repo's enforced documentation
structure: create a Setup section explaining prerequisites and how to run the
example commands (e.g., npx `@insforge/cli` metadata --json and the SQL/HTTP
options), a Usage Examples section showing minimal examples of using a returned
model_id in SDK/CLI/HTTP calls, and a Common Mistakes section listing typical
pitfalls (e.g., using shortened model_id, attempting AI calls when no models are
active). Ensure the new headings appear alongside the existing "Discovering
Available Models", "Best Practices", and "When No Models Are Configured"
sections.
---
Outside diff comments:
In `@skills/insforge/ai/sdk-integration.md`:
- Around line 32-131: Add a top-level "## Usage Examples" header and move the
existing "Chat Completions", "Embeddings", and "Image Generation" sections so
they are subsections under that header (e.g., "## Usage Examples" followed by
"### Chat Completions", "### Embeddings", "### Image Generation"); ensure the
document follows the required section order (Setup, Usage Examples, Best
Practices, Common Mistakes) and that the headings in the diff (Chat Completions,
Embeddings, Image Generation) are converted to subordinate headings under the
new Usage Examples section.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7978184f-303b-41e8-bc3e-670337b681e4
📒 Files selected for processing (2)
skills/insforge/ai/backend-configuration.mdskills/insforge/ai/sdk-integration.md
Comment on lines
1
to
62
| # AI Backend Configuration | ||
|
|
||
| HTTP endpoints to check AI configurations. Requires admin authentication. | ||
| Check which AI models are configured for a project. | ||
|
|
||
| ## Authentication | ||
| ## Discovering Available Models | ||
|
|
||
| All admin endpoints require: | ||
| ``` | ||
| Authorization: Bearer {admin-token-or-api-key} | ||
| ### Option 1 — CLI (recommended) | ||
|
|
||
| ```bash | ||
| npx @insforge/cli metadata --json | ||
| ``` | ||
|
|
||
| ## List AI Configurations | ||
| The `ai.configurations` section lists all models with `modelId` and `enabled` status. | ||
|
|
||
| Check which AI models are available for this project. | ||
| ### Option 2 — Raw SQL | ||
|
|
||
| ``` | ||
| GET /api/ai/configurations | ||
| Authorization: Bearer {admin-token} | ||
| ``` | ||
| Query the `ai.configs` table directly: | ||
|
|
||
| Response example: | ||
| ```json | ||
| { | ||
| "configurations": [ | ||
| { | ||
| "id": "config-uuid", | ||
| "name": "Claude Haiku", | ||
| "modelId": "anthropic/claude-3.5-haiku", | ||
| "enabled": true | ||
| }, | ||
| { | ||
| "id": "config-uuid-2", | ||
| "name": "GPT-4", | ||
| "modelId": "openai/gpt-4", | ||
| "enabled": true | ||
| } | ||
| ] | ||
| } | ||
| ```bash | ||
| npx @insforge/cli db query "SELECT model_id, provider, is_active, input_modality, output_modality FROM ai.configs WHERE is_active = true" | ||
| ``` | ||
|
|
||
| ## Quick Reference | ||
|
|
||
| | Task | Endpoint | | ||
| |------|----------| | ||
| | List AI configs | `GET /api/ai/configurations` | | ||
| **Table: `ai.configs`** | ||
|
|
||
| --- | ||
| | Column | Type | Description | | ||
| |--------|------|-------------| | ||
| | `model_id` | VARCHAR(255) | Unique model identifier (use this in SDK calls) | | ||
| | `provider` | VARCHAR(255) | AI provider (e.g., `openai`, `anthropic`, `google`) | | ||
| | `is_active` | BOOLEAN | Whether the model is enabled | | ||
| | `input_modality` | TEXT[] | Supported input types: `text`, `image`, `audio`, `video`, `file` | | ||
| | `output_modality` | TEXT[] | Supported output types: `text`, `image`, `audio`, `video`, `file` | | ||
| | `system_prompt` | TEXT | Optional default system prompt | | ||
|
|
||
| ## Best Practices | ||
|
|
||
| 1. **Always check available configurations first** before implementing AI features | ||
| - Call `GET /api/ai/configurations` to see which models are enabled | ||
| - Only use model IDs that appear in the configurations list | ||
| ### Option 3 — HTTP endpoint (requires admin auth) | ||
|
|
||
| 2. **Verify the exact model ID** from the configurations response | ||
| - Model IDs must match exactly (e.g., `anthropic/claude-3.5-haiku`, not `claude-haiku`) | ||
| ``` | ||
| GET /api/ai/configurations | ||
| Authorization: Bearer {admin-token} | ||
| ``` | ||
|
|
||
| ## Common Mistakes | ||
| ## Best Practices | ||
|
|
||
| | Mistake | Solution | | ||
| |---------|----------| | ||
| | Using a model ID that isn't configured | Check configurations first, use only enabled models | | ||
| | Guessing model IDs | Always verify against `GET /api/ai/configurations` | | ||
| | Assuming all models are available | Each project has its own configured models | | ||
| 1. **Always check available models first** before implementing AI features | ||
| 2. **Use exact `model_id`** from the query response — do not shorten or guess | ||
| 3. Each project has its own configured models — do not assume availability | ||
|
|
||
| ## When No Configurations Exist | ||
| ## When No Models Are Configured | ||
|
|
||
| If `GET /api/ai/configurations` returns an empty list or the requested model is not configured: | ||
| If the query returns no results: | ||
|
|
||
| 1. **Do not attempt to use AI features** - they will fail | ||
| 2. **Instruct the user** to configure AI models on the InsForge Dashboard: | ||
| - Go to the InsForge Dashboard → AI Settings | ||
| - Add and enable the desired AI model configurations | ||
| - Save changes | ||
| 3. **After configuration**, verify by calling `GET /api/ai/configurations` again | ||
| 1. **Do not attempt to use AI features** — they will fail | ||
| 2. **Instruct the user** to configure AI models on the InsForge Dashboard → AI Settings | ||
| 3. **After configuration**, verify by querying again | ||
|
|
||
| ## Recommended Workflow | ||
|
|
||
| ``` | ||
| 1. Check configurations → GET /api/ai/configurations | ||
| 1. Check available models → npx @insforge/cli metadata --json | ||
| OR query ai.configs table | ||
| 2. If empty or missing model → Instruct user to configure on Dashboard | ||
| 3. If model exists → Proceed with SDK integration | ||
| 3. If model exists → Use exact model_id in SDK calls | ||
| ``` |
There was a problem hiding this comment.
Add required documentation sections to match repo format.
This file currently omits the required Setup, Usage Examples, and Common Mistakes sections. Please add those headings (even if brief) so the module follows the enforced documentation structure.
As per coding guidelines: "Documentation files must maintain consistent format: Setup, Usage Examples, Best Practices, Common Mistakes sections".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@skills/insforge/ai/backend-configuration.md` around lines 1 - 62, The AI
Backend Configuration document is missing the required headings; add concise
"Setup", "Usage Examples", and "Common Mistakes" sections to the existing
content so it follows the repo's enforced documentation structure: create a
Setup section explaining prerequisites and how to run the example commands
(e.g., npx `@insforge/cli` metadata --json and the SQL/HTTP options), a Usage
Examples section showing minimal examples of using a returned model_id in
SDK/CLI/HTTP calls, and a Common Mistakes section listing typical pitfalls
(e.g., using shortened model_id, attempting AI calls when no models are active).
Ensure the new headings appear alongside the existing "Discovering Available
Models", "Best Practices", and "When No Models Are Configured" sections.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
claude-3.5-haiku) instead of querying which models are actually configured for the projectsdk-integration.mdto replace all hardcoded model IDs withMODEL_IDplaceholders and add a "Discover Available Models First" section with CLI (npx @insforge/cli metadata --json) and raw SQL (ai.configstable) approachesbackend-configuration.mdto document theai.configstable schema and prioritize CLI/SQL discovery over the HTTP endpointTest plan
npx @insforge/cli metadata --jsonreturns AI configurationsnpx @insforge/cli db query "SELECT model_id FROM ai.configs WHERE is_active = true"works🤖 Generated with Claude Code
Note
Require model discovery before AI SDK usage in documentation
npx @insforge/cli metadata --json) or direct SQL againstai.configs, replacing the previous focus on admin HTTP endpoints.MODEL_IDvalues sourced from metadata orai.configsqueries.is_active,input_modality, andoutput_modalityfields fromai.configsbefore making SDK calls.Macroscope summarized 4cbd844.
Summary by CodeRabbit