Skip to content

feat: add OrcaRouter third-party format #1316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sailt-k wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

feat: add OrcaRouter third-party format #1316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions common/adapters.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ export const THIRDPARTY_HANDLERS: { [svc in ThirdPartyFormat]: AIAdapter } = {
gemini: 'kobold',
arli: 'kobold',
'lm-studio': 'openai',
orcarouter: 'openai',
}

export const BASIC_PROMPT_ONLY: { [svc in ThirdPartyFormat]?: boolean } = {
Expand All @@ -176,6 +177,7 @@ export const THIRDPARTY_FORMATS = [
'arli',
'gemini',
'lm-studio',
'orcarouter',
] as const

export const AI_ADAPTERS = [
Expand Down Expand Up @@ -382,6 +384,7 @@ export const FORMAT_LABEL: { [key in ThirdPartyFormat]: string } = {
tabby: 'TabbyAPI',
vllm: 'vLLM',
'lm-studio': 'LM Studio',
orcarouter: 'OrcaRouter',
}

export const INSTRUCT_SERVICES: { [key in AIAdapter]?: boolean } = {
Expand Down
7 changes: 7 additions & 0 deletions common/requests/util.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,13 @@ export function getOaiCompatibleUrl(
preset: Partial<AppSchema.GenSettings>,
isThirdParty?: boolean
) {
// OrcaRouter is an OpenAI-compatible gateway with a fixed endpoint. When its
// format is selected without a custom URL, default to the OrcaRouter API so
// users only need to provide their API key.
if (isThirdParty && preset.thirdPartyFormat === 'orcarouter' && !preset.thirdPartyUrl) {
return { url: 'https://api.orcarouter.ai/v1', changed: true }
}

if (isThirdParty && preset.thirdPartyUrl) {
if (!preset.providerId && preset.thirdPartyUrlNoSuffix) {
return { url: preset.thirdPartyUrl, changed: true }
Expand Down
3 changes: 2 additions & 1 deletion srv/adapter/openai.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@ export const handleOAI: ModelAdapter = async function* (opts) {
const isChatFormat =
gen.thirdPartyFormat === 'openai-chat' ||
gen.thirdPartyFormat == 'openai-chatv2' ||
gen.thirdPartyFormat === 'lm-studio'
gen.thirdPartyFormat === 'lm-studio' ||
gen.thirdPartyFormat === 'orcarouter'
const useChat = (isThirdParty && isChatFormat) || gen.service === 'openai'

if (useChat) {
Expand Down
1 change: 1 addition & 0 deletions web/pages/Settings/Provider/index.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,7 @@ const EditConnectionDetails: Field<{ show: boolean; close: () => void }> = (prop
'featherless',
'arli',
'gemini',
'orcarouter',
]

const options = formats.map((id) => ({ label: FORMAT_LABEL[id], value: id as string }))
Expand Down