Skip to content

feat: Add OpenAI-compatible API support to AI Writer operation #250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
aurkinet wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: Add OpenAI-compatible API support to AI Writer operation #250

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2737c0b
feat: Add OpenAI-compatible API support to AI Writer operation
aurkinet Sep 22, 2025
597b6fd
Merge branch 'main' into feature/openai-compatible-support
bryantgillespie Mar 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions packages/ai-writer-operation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AI Writer Operation

Generate text based on a written prompt within Directus Flows with this custom operation, powered by [OpenAI's Text Generation API]([https://.com](https://openai.com/product)), [Anthropic](https://www.anthropic.com/) [MistralAi (via Replicate)](https://replicate.com/mistralai/mistral-7b-v0.1) and [Meta's LLama (via Replicate)](https://replicate.com/meta/meta-llama-3.1-405b-instruct).
Generate text based on a written prompt within Directus Flows with this custom operation, powered by [OpenAI's Text Generation API](https://openai.com/product), [Anthropic](https://www.anthropic.com/), [MistralAi (via Replicate)](https://replicate.com/mistralai/mistral-7b-v0.1), [Meta's LLama (via Replicate)](https://replicate.com/meta/meta-llama-3.1-405b-instruct), and any OpenAI-compatible API.

![The AI Writer operation, showing a masked OpenAI API Key field, model and prompt selection fields, and a multiline text input.](https://raw.githubusercontent.com/directus-labs/extensions/main/packages/ai-writer-operation/docs/options.png)

Expand All @@ -10,11 +10,23 @@ This operation contains some configuration options - an Api-Key, a selection of
![The output showing a string that has been grammatically fixed.](https://raw.githubusercontent.com/directus-labs/extensions/main/packages/ai-writer-operation/docs/output.png)

### API-Keys
You can generate your API-Keys on the follosing sites:
You can generate your API-Keys on the following sites:
- [OpenAI](https://platform.openai.com/api-keys)
- [Anthropic](https://console.anthropic.com/settings/workspaces/default/keys)
- [Replicate](https://replicate.com/account/api-tokens)

### OpenAI-Compatible APIs
The extension now supports any OpenAI-compatible API by selecting "OpenAI Compatible (Custom)" as the AI Provider. This allows you to use services like:
- Local models (Ollama, LM Studio, etc.)
- Cloud providers (Azure OpenAI, Google Vertex AI, etc.)
- Other OpenAI-compatible services (Groq, Together AI, etc.)

When using a custom provider, you'll need to:
1. Select "OpenAI Compatible (Custom)" as the AI Provider
2. Enter your custom API endpoint (e.g., `https://api.example.com/v1`)
3. Provide your API key
4. Choose a model from the dropdown or select "Custom Model" to enter a specific model name

Comment on lines +19 to +29

Copilot AI Mar 10, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README implies Azure OpenAI / Vertex AI work directly as “OpenAI-compatible APIs”, but Azure’s endpoints/auth (api-key header + api-version + deployments path) and Vertex AI are not Chat Completions-compatible by default. Please clarify that this works with services that expose the OpenAI Chat Completions-compatible endpoint + Bearer auth (or mention that Azure/Vertex require a compatibility/proxy layer).

Suggested change
The extension now supports any OpenAI-compatible API by selecting "OpenAI Compatible (Custom)" as the AI Provider. This allows you to use services like:
- Local models (Ollama, LM Studio, etc.)
- Cloud providers (Azure OpenAI, Google Vertex AI, etc.)
- Other OpenAI-compatible services (Groq, Together AI, etc.)
When using a custom provider, you'll need to:
1. Select "OpenAI Compatible (Custom)" as the AI Provider
2. Enter your custom API endpoint (e.g., `https://api.example.com/v1`)
3. Provide your API key
4. Choose a model from the dropdown or select "Custom Model" to enter a specific model name
The extension supports providers that expose an **OpenAI Chat Completions-compatible HTTP API** (OpenAI-style paths and request/response format, using `Authorization: Bearer <api-key>`), by selecting "OpenAI Compatible (Custom)" as the AI Provider. This allows you to use services like:
- Local models (Ollama, LM Studio, etc.) when configured to expose an OpenAI-compatible endpoint
- Cloud providers (Azure OpenAI, Google Vertex AI, etc.) **via an OpenAI-compatible/translation layer or proxy** that normalizes their endpoints and auth to the OpenAI Chat Completions format
- Other OpenAI-compatible services (Groq, Together AI, etc.) that implement the Chat Completions API
When using a custom provider, you'll need to:
1. Select "OpenAI Compatible (Custom)" as the AI Provider
2. Enter your custom API endpoint (e.g., `https://api.example.com/v1`) that implements the OpenAI Chat Completions-compatible interface
3. Provide your API key (sent as a Bearer token)
4. Choose a model from the dropdown or select "Custom Model" to enter a specific model name
> Note: Azure OpenAI and Google Vertex AI do **not** use the OpenAI Chat Completions URL structure or authentication scheme by default. To use them here, you must either:
> - Use a compatibility/proxy layer that exposes an OpenAI Chat Completions-compatible endpoint, or
> - Use a provider-managed endpoint explicitly documented as OpenAI Chat Completions-compatible.

Copilot uses AI. Check for mistakes.
## Custom Prompts

For a completely custom prompt using the "Create custom prompt" type, you will need to create a **system** message at the start of the message thread so that the Text Generation API knows how it should respond. Examples of initial system prompts can be found in the config objects of each built-in prompt in the [source code of this extension](https://github.com/directus-labs/extension-ai-writer-operation/tree/production/src/prompts). OpenAI also provides a solid overview of [how to write good prompts](https://platform.openai.com/docs/guides/prompt-engineering).
Expand Down
4 changes: 3 additions & 1 deletion packages/ai-writer-operation/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,9 @@
"urls": [
"https://api.openai.com/v1/**",
"https://api.anthropic.com/v1/**",
"https://api.replicate.com/v1/**"
"https://api.replicate.com/v1/**",
"https://**/v1/**",
"http://**/v1/**"
]
},
"sleep": {}
Expand Down
35 changes: 30 additions & 5 deletions packages/ai-writer-operation/src/Provider/OpenAi.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,43 @@ import { Provider } from './Provider';

export class OpenAi extends Provider {
constructor(options: AiWriterOperationOptions) {
if (!options.apiKeyOpenAi) {
throw new InvalidPayloadError({ reason: 'OpenAI API Key is missing' });
}
// Determine if this is a custom OpenAI-compatible provider or standard OpenAI
const isCustomProvider = options.aiProvider === 'openai-compatible';

Copilot AI Mar 10, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isCustomProvider is checked with a case-sensitive equality (options.aiProvider === 'openai-compatible'), but getProvider() routes here using a case-insensitive comparison. If aiProvider ever comes through with different casing (e.g., via API/DB), this will incorrectly treat it as standard OpenAI and require apiKeyOpenAi / use the OpenAI endpoint. Normalize casing here (e.g., compare options.aiProvider?.toLowerCase()), or pass a canonical provider value into the provider classes.

Suggested change
const isCustomProvider = options.aiProvider === 'openai-compatible';
const isCustomProvider = options.aiProvider?.toLowerCase() === 'openai-compatible';

Copilot uses AI. Check for mistakes.

if (isCustomProvider) {
if (!options.apiKeyCustom) {
throw new InvalidPayloadError({ reason: 'Custom API Key is missing' });
}

if (!options.customEndpoint) {
throw new InvalidPayloadError({ reason: 'Custom Endpoint is missing' });
}

// Ensure the endpoint ends with /chat/completions for OpenAI-compatible APIs
const endpoint = options.customEndpoint.endsWith('/chat/completions')
? options.customEndpoint
: `${options.customEndpoint.replace(/\/$/, '')}/chat/completions`;
Comment on lines +22 to +24

Copilot AI Mar 10, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The endpoint normalization can produce a duplicated path when the user provides a full /chat/completions/ URL with a trailing slash (e.g., .../chat/completions/ becomes .../chat/completions/chat/completions). Consider trimming trailing slashes before the endsWith('/chat/completions') check (or using new URL() path joining) to make this robust.

Suggested change
const endpoint = options.customEndpoint.endsWith('/chat/completions')
? options.customEndpoint
: `${options.customEndpoint.replace(/\/$/, '')}/chat/completions`;
const baseEndpoint = options.customEndpoint.replace(/\/+$/, '');
const endpoint = baseEndpoint.endsWith('/chat/completions')
? baseEndpoint
: `${baseEndpoint}/chat/completions`;

Copilot uses AI. Check for mistakes.

super(options, 'https://api.openai.com/v1/chat/completions', options.apiKeyOpenAi);
super(options, endpoint, options.apiKeyCustom);
} else {
if (!options.apiKeyOpenAi) {
throw new InvalidPayloadError({ reason: 'OpenAI API Key is missing' });
}

super(options, 'https://api.openai.com/v1/chat/completions', options.apiKeyOpenAi);
}
}

public async messageRequest(): Promise<string> {
const messages = this.getMessages();

// Use custom model name if provided, otherwise use the selected model
const modelName = this.options.model === 'custom' && this.options.customModelName
? this.options.customModelName
Comment on lines +39 to +41

Copilot AI Mar 10, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When model is set to 'custom' but customModelName is missing/empty, the request will be sent with model: 'custom'. Add server-side validation to require a non-empty customModelName when model === 'custom' (don’t rely on the app UI’s conditional required field).

Suggested change
// Use custom model name if provided, otherwise use the selected model
const modelName = this.options.model === 'custom' && this.options.customModelName
? this.options.customModelName
// Validate that a custom model name is provided when using a custom model
if (this.options.model === 'custom' && !this.options.customModelName) {
throw new InvalidPayloadError({
reason: 'Custom Model Name is missing',
});
}
// Use custom model name if provided, otherwise use the selected model
const modelName = this.options.model === 'custom' && this.options.customModelName
? this.options.customModelName

Copilot uses AI. Check for mistakes.
: this.options.model!;

const requestBody: RequestBody = {
model: this.options.model!,
model: modelName,
messages,
max_completion_tokens: this.options.maxToken || 0,
};
Expand Down
4 changes: 2 additions & 2 deletions packages/ai-writer-operation/src/Provider/ProviderFactory.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ export function getProvider(options: AiWriterOperationOptions) {
return new Anthropic(options);
}

if (options.aiProvider.toLowerCase() === 'openai') {
if (options.aiProvider.toLowerCase() === 'openai' || options.aiProvider.toLowerCase() === 'openai-compatible') {
return new OpenAi(options);
}

if (options.aiProvider.toLowerCase() === 'replicate') {
return new Replicate(options);
}

throw new Error(`Unsoported AI Provider ${options.aiProvider}`);
throw new Error(`Unsupported AI Provider ${options.aiProvider}`);

Copilot AI Mar 10, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For unsupported providers, this throws a generic Error while other validation errors use InvalidPayloadError. Consider throwing InvalidPayloadError here as well (with a clear reason) so API consumers get consistent error types/status codes.

Suggested change
throw new Error(`Unsupported AI Provider ${options.aiProvider}`);
throw new InvalidPayloadError({ reason: `Unsupported AI Provider: ${options.aiProvider}` });

Copilot uses AI. Check for mistakes.
}
3 changes: 3 additions & 0 deletions packages/ai-writer-operation/src/api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ export interface AiWriterOperationOptions {
apiKeyAnthropic?: string | null;
apiKeyOpenAi?: string | null;
apiKeyReplicate?: string | null;
apiKeyCustom?: string | null;
customEndpoint?: string | null;
model?: string | null;
customModelName?: string | null;
promptKey?: string | null;
system?: string | null;
json_mode?: boolean;
Expand Down
50 changes: 50 additions & 0 deletions packages/ai-writer-operation/src/app.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,14 @@ export default defineOperationApp({
return replicateModels;
}

if (provider === 'openai-compatible') {
// For custom providers, allow any model name to be entered
return [
{ text: 'Custom Model (enter model name below)', value: 'custom' },
...openAiModels, // Include OpenAI models as examples
];
}

return [];
};

Expand All @@ -103,6 +111,10 @@ export default defineOperationApp({
text: 'Open AI',
value: 'openai',
},
{
text: 'OpenAI Compatible (Custom)',
value: 'openai-compatible',
},
{
text: 'Replicate (Meta & Mistral)',
value: 'replicate',
Expand Down Expand Up @@ -153,6 +165,32 @@ export default defineOperationApp({
hidden: context.aiProvider !== 'replicate',
},
},
{
field: 'apiKeyCustom',
name: 'Custom API Key',
type: 'string',
meta: {
required: context.aiProvider === 'openai-compatible',
options: {
masked: true,
},
width: 'full',
interface: 'input',
hidden: context.aiProvider !== 'openai-compatible',
},
},
{
field: 'customEndpoint',
name: 'Custom Endpoint',
type: 'string',
meta: {
required: context.aiProvider === 'openai-compatible',
width: 'full',
interface: 'input',
hidden: context.aiProvider !== 'openai-compatible',
note: 'Enter the base URL of your OpenAI-compatible API (e.g., https://api.example.com/v1). The /chat/completions endpoint will be automatically appended.',
},
},
{
field: 'model',
name: 'AI Model',
Expand All @@ -166,6 +204,18 @@ export default defineOperationApp({
width: 'half',
},
},
{
field: 'customModelName',
name: 'Custom Model Name',
type: 'string',
meta: {
required: context.aiProvider === 'openai-compatible' && context.model === 'custom',
width: 'half',
interface: 'input',
hidden: !(context.aiProvider === 'openai-compatible' && context.model === 'custom'),
note: 'Enter the exact model name as expected by your API provider.',
},
},
{
field: 'maxToken',
name: 'Max Token',
Expand Down