RosettaAI SDK

RosettaAI is a unified TypeScript SDK designed for seamless interaction with multiple Large Language Model (LLM) providers like OpenAI (including Azure), Anthropic, Google Generative AI, and Groq, as well as custom OpenAI-compatible providers. It provides a consistent interface (generate, stream, embed, generateSpeech, transcribe, listModels, etc.) that abstracts away the provider-specific implementations, allowing you to switch between backends with minimal code changes.

Built with Node.js v20+ and TypeScript v5.5+ in mind, it emphasizes type safety, robustness, and adherence to modern backend development best practices.

Key Features

Unified API: Interact with different LLMs using a single, consistent interface for common tasks.
Provider Support: Seamlessly switch between major AI providers:
- OpenAI (Standard & Azure)
- Anthropic
- Google Generative AI
- Groq
- ElevenLabs (TTS & STT)
- Custom OpenAI-Compatible Providers (e.g., LM Studio, Novita, GPUStack)
Core Functionality:
- Chat Completions (Streaming & Non-Streaming)
- Text Embeddings
- Tool Use / Function Calling
- Multimodal Input (Images)
- Text-to-Speech (TTS) via OpenAI/Azure, Groq, and ElevenLabs
- Speech-to-Text (STT) via OpenAI/Azure, Groq, and ElevenLabs
- Audio Translation via OpenAI/Azure & Groq
- Voice Listing (ElevenLabs)
- Model Listing (Built-in & Custom Providers)
Advanced Features (Provider-dependent):
- JSON Mode / Structured Output (json_object and json_schema modes; OpenAI/Azure/Google/Anthropic)
- Grounding / Citations (Google)
- Thinking Steps (Anthropic)
- Programmatic Tool Calling with container reuse and code-execution stream events (Anthropic)
Provider Passthrough: Forward provider-specific parameters (e.g., presence_penalty, top_k, seed) directly to the underlying API via extraParams, without losing unmapped fields.
Type Safe: Leverages TypeScript's strong typing for improved developer experience, autocompletion, and compile-time error checking.
Robust Error Handling: Provides classified errors (ConfigurationError, ProviderAPIError, UnsupportedFeatureError, MappingError) for easier debugging and programmatic handling.
Flexible Configuration: Easily configure API keys and defaults via .env files or direct constructor arguments.

Supported Features Matrix

This matrix provides a general guide to feature support across providers. Provider capabilities and model support change frequently, so always consult the official provider documentation for the latest details. Custom provider support depends on their specific API implementation.

Feature	OpenAI (Azure)	Anthropic	Google	Groq	ElevenLabs	Custom (OpenAI-Compat)	Notes
Chat (Generate)	✅	✅	✅	✅	❌	✅
Chat (Stream)	✅	✅	✅	✅	❌	✅
Image Input	✅	✅	✅	⚠️	❌	⚠️	Groq/Custom support varies by model
Tool Use	✅	✅	✅	✅	❌	✅	Implementation details differ slightly
Programmatic Tool Calling	❌	✅	❌	❌	❌	❌	Anthropic only; requires provider/model support
Embeddings	✅	❌	✅	✅	❌	⚠️	Custom support varies
JSON Mode	✅	⚠️	⚠️	⚠️	❌	✅	Anthropic: `json_schema` only; see Structured Output
Grounding/Citations	❌	❌	✅	❌	❌	❌	Via Google Search tool integration
Thinking Steps	❌	✅	❌	❌	❌	❌	Anthropic specific feature
TTS	✅	❌	❌	✅	✅	❌	High-quality TTS with 70+ languages
TTS (Streaming)	✅	❌	❌	❌	✅	❌	Real-time audio streaming
STT	✅	❌	⚠️	✅	✅	❌	99 languages, diarization, timestamps
STT (Translate)	✅	❌	❌	✅	❌	❌	To English
Voice Listing	❌	❌	❌	❌	✅	❌	List available TTS voices
Model Listing	✅	✅	✅	✅	❌	✅	Custom via OpenAI-compat `/models`

✅ = Supported | ⚠️ = Partial/Limited/Via Prompting/Varies | ❌ = Not Supported

Note: This SDK aims to provide a common interface but cannot enable features unsupported by the underlying provider API.

Installation

npm install @missionsquad/rosetta-ai
# or
yarn add @missionsquad/rosetta-ai
# or
pnpm add @missionsquad/rosetta-ai

Ensure you have Node.js v20 or later installed.

Configuration

RosettaAI can be configured using environment variables (loaded via dotenv or set directly) or by passing a configuration object to the constructor.

1. Environment Variables (.env file):

Create a .env file in your project root. See .env.example for all available options.

API Keys (Required for desired providers):

# .env
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=AIza...
GROQ_API_KEY=gsk_...
OPENAI_API_KEY=sk-... # For standard OpenAI OR Azure if AZURE_OPENAI_API_KEY is not set

# Custom Provider Example Keys
NOVITA_API_KEY=your_novita_key
GPUSTACK_API_KEY=your_gpustack_key
# LMSTUDIO_API_KEY= # Often not needed for local

Azure OpenAI (Alternative to Standard OpenAI): If using Azure, provide these instead of/in addition to OPENAI_API_KEY. RosettaAI will prioritize Azure if its key and endpoint are set.

AZURE_OPENAI_API_KEY=your_azure_openai_api_key
AZURE_OPENAI_ENDPOINT=https://your-resource-name.openai.azure.com/
AZURE_OPENAI_API_VERSION=2024-05-01-preview # Check Azure docs for appropriate version
AZURE_OPENAI_DEPLOYMENT_NAME=your-gpt-deployment-name # Default CHAT deployment ID
ROSETTA_AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=your-embedding-deployment-name # Default EMBEDDING deployment ID

Custom Provider Base URLs (Optional): Set base URLs if they differ from defaults or if not set in constructor config.

# NOVITA_BASE_URL=https://api.novita.ai/v3/openai
# GPUSTACK_BASE_URL=https://gpu.crypto-tech.cloud/v1-openai
# LMSTUDIO_BASE_URL=http://localhost:1234/v1

Optional Defaults: Set default models for each provider to avoid specifying them in every call.

# Chat Models
ROSETTA_DEFAULT_ANTHROPIC_MODEL=claude-3-haiku-20240307
ROSETTA_DEFAULT_GOOGLE_MODEL=gemini-1.5-flash-latest
ROSETTA_DEFAULT_GROQ_MODEL=llama3-8b-8192
ROSETTA_DEFAULT_OPENAI_MODEL=gpt-4o-mini
ROSETTA_DEFAULT_NOVITA_MODEL=meta-llama/llama-3.1-8b-instruct # Example custom default
ROSETTA_DEFAULT_GPUSTACK_MODEL=ibm-granite-3.2-8b-instruct # Example custom default
ROSETTA_DEFAULT_LMSTUDIO_MODEL=lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF # Example custom default

# Embedding Models
ROSETTA_DEFAULT_EMBEDDING_GOOGLE_MODEL=text-embedding-004
ROSETTA_DEFAULT_EMBEDDING_OPENAI_MODEL=text-embedding-3-small
ROSETTA_DEFAULT_EMBEDDING_GROQ_MODEL=nomic-embed-text-v1.5
ROSETTA_DEFAULT_EMBEDDING_GPUSTACK_MODEL=nomic-embed-text-v1.5 # Example custom default

# Audio Models
ROSETTA_DEFAULT_TTS_OPENAI_MODEL=tts-1
ROSETTA_DEFAULT_STT_OPENAI_MODEL=whisper-1
ROSETTA_DEFAULT_STT_GROQ_MODEL=whisper-large-v3

2. Constructor Configuration:

Pass configuration options directly when creating the RosettaAI instance. Constructor arguments override environment variables.

import { RosettaAI, Provider, RosettaAIConfig, CustomProviderConfig } from 'rosetta-ai-sdk'
import { OpenAICompatibleMapper } from 'rosetta-ai-sdk/dist/core/mapping/openai-compatible.mapper' // Adjust path if needed

// Example Custom Provider Config
const lmstudioConfig: CustomProviderConfig = {
  providerKey: 'lmstudio',
  mapper: OpenAICompatibleMapper,
  supportedFeatures: ['generate', 'stream', 'tool_use', 'list_models'], // Supports listing
  baseURL: process.env.LMSTUDIO_BASE_URL || 'http://localhost:1234/v1',
  apiKey: process.env.LMSTUDIO_API_KEY || undefined,
  defaultModel: process.env.ROSETTA_DEFAULT_LMSTUDIO_MODEL || 'local-model'
  // modelListPath: '/models' // Default, can omit
}

const customWithPathConfig: CustomProviderConfig = {
  providerKey: 'custom-path',
  mapper: OpenAICompatibleMapper,
  supportedFeatures: ['generate', 'list_models'],
  baseURL: 'https://api.example.com/v2',
  apiKey: 'key-123',
  defaultModel: 'model-a',
  modelListPath: '/openai-compat/models' // Custom path for model listing
}

const customWithUrlConfig: CustomProviderConfig = {
  providerKey: 'custom-url',
  mapper: OpenAICompatibleMapper,
  supportedFeatures: ['generate', 'list_models'],
  baseURL: 'https://api.example.com/v2', // Base for generate
  apiKey: 'key-456',
  defaultModel: 'model-b',
  modelListUrl: 'https://models.example.org/all' // Absolute URL for listing
}

// Main Config
const config: RosettaAIConfig = {
  openaiApiKey: 'sk-...',
  googleApiKey: 'AIza...',
  // Anthropic and Groq will be loaded from environment if keys exist there

  defaultModels: {
    [Provider.OpenAI]: 'gpt-4o-mini',
    [Provider.Google]: 'gemini-1.5-flash-latest',
    lmstudio: 'local-model-override' // Override default for lmstudio
  },

  // Example: Explicit Azure configuration
  azureOpenAIApiKey: 'azure-key',
  azureOpenAIEndpoint: 'https://your-azure.openai.azure.com/',
  azureOpenAIApiVersion: '2024-05-01-preview',
  azureOpenAIDefaultChatDeploymentName: 'my-gpt4-deployment',
  azureOpenAIDefaultEmbeddingDeploymentName: 'my-embedding-deployment',

  // Register Custom Providers
  customProviders: [lmstudioConfig, customWithPathConfig, customWithUrlConfig],

  // Optional: Override default retries/timeout
  defaultMaxRetries: 3,
  defaultTimeoutMs: 90000 // 90 seconds
}

const rosetta = new RosettaAI(config)

Quick Start

import { RosettaAI, Provider, RosettaMessage } from 'rosetta-ai-sdk'
import dotenv from 'dotenv'

// Load .env file (if using)
dotenv.config()

async function main() {
  // Initialize (reads config from constructor or process.env)
  const rosetta = new RosettaAI()

  // --- Basic Chat Completion (Non-Streaming) ---
  console.log('\n--- Basic Generation ---')
  try {
    const messages: RosettaMessage[] = [
      { role: 'system', content: 'You are a helpful assistant.' },
      { role: 'user', content: 'What is the capital of France?' }
    ]

    const result = await rosetta.generate({
      provider: Provider.OpenAI, // Or Provider.Anthropic, Provider.Google, Provider.Groq, 'lmstudio'
      // model: 'gpt-4o-mini', // Optional: uses configured default if not set
      messages: messages,
      maxTokens: 50
    })

    console.log(`[${result.model}] Response:`)
    console.log(result.content)
    console.log('Usage:', result.usage ? JSON.stringify(result.usage) : 'N/A')
  } catch (error) {
    console.error(`Generation failed: ${error.name} - ${error.message}`)
  }

  // --- Streaming Chat Completion ---
  console.log('\n--- Streaming Generation ---')
  try {
    const stream = rosetta.stream({
      provider: Provider.Groq, // Or any other configured provider
      // model: 'llama3-8b-8192', // Optional
      messages: [{ role: 'user', content: 'Write a short haiku about TypeScript.' }],
      maxTokens: 60
    })

    console.log(`[Streaming Response]`)
    let fullContent = ''
    for await (const chunk of stream) {
      switch (chunk.type) {
        case 'content_delta':
          process.stdout.write(chunk.data.delta)
          fullContent += chunk.data.delta
          break
        case 'message_stop':
          console.log(`\n--- Stream Stopped (Reason: ${chunk.data.finishReason}) ---`)
          break
        case 'final_usage':
          console.log('\nFinal Usage:', chunk.data.usage ? JSON.stringify(chunk.data.usage) : 'N/A')
          break
        case 'error':
          console.error('\nStream Error:', chunk.data.error.message)
          return // Stop processing on error
        // Add cases for other chunk types (tool_call_*, json_*, thinking_*, etc.) if needed
        default:
          // console.log(`Chunk: ${chunk.type}`); // Log other chunk types if interested
          break
      }
    }
    console.log('\n--- Stream Complete ---')
    // console.log("Final Accumulated Content:", fullContent);
  } catch (error) {
    // Errors during stream *setup* (e.g., invalid config) are caught here
    console.error(`Streaming setup failed: ${error.name} - ${error.message}`)
  }

  // --- List Models ---
  console.log('\n--- Model Listing ---')
  try {
    const providerToList = Provider.OpenAI // Or 'lmstudio', Provider.Groq, etc.
    if (rosetta.getConfiguredProviders().includes(providerToList)) {
      const models = await rosetta.listModels(providerToList)
      console.log(`Models for ${providerToList}:`)
      models.data.slice(0, 5).forEach(m => console.log(` - ${m.id} (Owned by: ${m.owned_by})`))
      if (models.data.length > 5) console.log(' ... and more')
    } else {
      console.log(`Provider ${providerToList} not configured, skipping model list.`)
    }
  } catch (error) {
    console.error(`Model listing failed: ${error.name} - ${error.message}`)
  }
}

main().catch(console.error)

Core Concepts & Usage Examples

Initialization

import { RosettaAI, Provider, RosettaAIConfig, CustomProviderConfig } from 'rosetta-ai-sdk'
import { OpenAICompatibleMapper } from 'rosetta-ai-sdk/dist/core/mapping/openai-compatible.mapper'
import dotenv from 'dotenv'

dotenv.config() // Load .env

// Option 1: Automatic configuration from environment variables
const rosettaAuto = new RosettaAI()

// Option 2: Explicit configuration via constructor
const customConfig: CustomProviderConfig = {
  providerKey: 'my-custom',
  mapper: OpenAICompatibleMapper,
  supportedFeatures: ['generate', 'list_models'],
  baseURL: 'http://localhost:11434/v1', // Example: Ollama endpoint
  apiKey: 'ollama', // Example API key if needed
  modelListPath: '/api/tags' // Example: Ollama model listing path (might need custom mapper logic)
}
const config: RosettaAIConfig = {
  openaiApiKey: 'sk-...',
  anthropicApiKey: 'sk-ant-...',
  defaultModels: {
    [Provider.OpenAI]: 'gpt-4o-mini',
    [Provider.Anthropic]: 'claude-3-haiku-20240307',
    'my-custom': 'llama3'
  },
  customProviders: [customConfig]
}
const rosettaManual = new RosettaAI(config)

// Check configured providers
console.log('Available Providers:', rosettaManual.getConfiguredProviders())

Chat Completions

`generate()` (Non-Streaming)

Use generate for simple request-response interactions.

import { RosettaAI, Provider, RosettaMessage, ProviderKey } from 'rosetta-ai-sdk'
// ... initialization ...

const messages: RosettaMessage[] = [
  { role: 'system', content: 'You are a concise poet.' },
  { role: 'user', content: 'Write a 2-line poem about the moon.' }
]

try {
  const providerKey: ProviderKey = Provider.Anthropic // Or 'my-custom', Provider.OpenAI, etc.
  const result = await rosetta.generate({
    provider: providerKey,
    model: 'claude-3-haiku-20240307', // Specify model or use default
    messages: messages,
    maxTokens: 30,
    temperature: 0.8
  })

  console.log(result.content)
  console.log('Finish Reason:', result.finishReason)
  console.log('Usage:', result.usage)
} catch (error) {
  // Handle errors (see Error Handling section)
  console.error(error)
}

Cancelling streaming requests (all providers)

Both stream() (text/chat) and streamSpeech() (audio) return a cancellable async iterable. Call stream.cancel(reason?) to stop iteration and abort the upstream provider request. You can also access the underlying AbortSignal via stream.signal if you need to attach your own handlers.

const stream = rosetta.stream({ provider: Provider.OpenAI, model: 'gpt-4o-mini', messages })

setTimeout(() => stream.cancel('no longer needed'), 2_000)

for await (const chunk of stream) {
  // handle chunk
}

Provider wiring:

OpenAI / Azure OpenAI / Groq / Anthropic / OpenAI-compatible custom providers: Streaming chat requests are issued with { signal: stream.signal }, so canceling aborts the HTTP request immediately.
Google Generative AI: The async generator from generateContentStream is closed when stream.cancel() is called.
OpenAI streaming text-to-speech: The abort signal is passed to audio.speech.create, and the response body iterator closes when canceled.
ElevenLabs streaming TTS (custom provider): Streams check the shared abort signal and invoke the SDK's abort() helper when available before yielding an audio_stop event.

The same cancel behavior applies to any custom mapper you register; the SDK forwards the abort signal and a cleanup hook so mapper implementations can release their streaming resources.

`stream()` (Streaming)

Use stream to process responses chunk-by-chunk, ideal for real-time applications.

import { RosettaAI, Provider, RosettaMessage, StreamChunk, ProviderKey } from 'rosetta-ai-sdk'
// ... initialization ...

const messages: RosettaMessage[] = [{ role: 'user', content: 'Explain the concept of "event loop" in Node.js simply.' }]

try {
  const providerKey: ProviderKey = Provider.OpenAI // Or 'my-custom', Provider.Groq, etc.
  const stream = rosetta.stream({
    provider: providerKey,
    model: 'gpt-4o-mini',
    messages: messages,
    maxTokens: 150
  })

  let fullResponse = ''
  let toolArgs = '' // Example accumulator for tool arguments

  for await (const chunk of stream) {
    switch (chunk.type) {
      case 'message_start':
        console.log(`Stream started (Model: ${chunk.data.model})`)
        break
      case 'content_delta':
        process.stdout.write(chunk.data.delta)
        fullResponse += chunk.data.delta
        break
      case 'tool_call_start':
        console.log(`\nTool Call Start: ${chunk.data.toolCall.function.name} (ID: ${chunk.data.toolCall.id})`)
        toolArgs = '' // Reset for new tool call
        break
      case 'tool_call_delta':
        process.stdout.write(chunk.data.functionArgumentChunk)
        toolArgs += chunk.data.functionArgumentChunk
        break
      case 'tool_call_done':
        console.log(`\nTool Call Done (ID: ${chunk.data.id}). Args: ${toolArgs}`)
        break
      case 'message_stop':
        console.log(`\nStream Stop Reason: ${chunk.data.finishReason}`)
        break
      case 'final_usage':
        console.log(`\nFinal Usage: ${JSON.stringify(chunk.data.usage)}`)
        break
      case 'final_result':
        console.log('\nFinal Aggregated Result Received.')
        // console.log(chunk.data.result); // Access the complete GenerateResult object
        break
      case 'error':
        console.error(`\nStream Error: ${chunk.data.error.message}`)
        break
      // Add cases for json_*, thinking_*, citation_* as needed
      default:
        const _: never = chunk // Exhaustiveness check
        console.log(`\nUnknown chunk type: ${(_ as any).type}`)
    }
  }
  console.log('\n--- Stream Complete ---')
} catch (error) {
  // Handle stream setup errors
  console.error(error)
}

Embeddings

Generate vector representations of text using embed. Supported by OpenAI/Azure, Google, Groq, and potentially custom providers.

import { RosettaAI, Provider, EmbedParams, ProviderKey } from 'rosetta-ai-sdk'
// ... initialization ...

const textsToEmbed = ['RosettaAI simplifies LLM interactions.', 'TypeScript adds static typing to JavaScript.']

try {
  const providerKey: ProviderKey = Provider.OpenAI // Or Provider.Google, Provider.Groq, 'my-custom-embed-provider'
  const params: EmbedParams = {
    provider: providerKey,
    model: 'text-embedding-3-small', // Use appropriate model
    input: textsToEmbed
    // dimensions: 256 // Optional: For OpenAI models supporting reduced dimensions
  }

  const result = await rosetta.embed(params)

  console.log(`Generated ${result.embeddings.length} embeddings.`)
  result.embeddings.forEach((vec, i) => {
    console.log(`Embedding ${i + 1} (Dim: ${vec.length}): [${vec.slice(0, 3).join(', ')}...]`)
  })
  console.log('Usage:', result.usage)
  console.log('Model Used:', result.model)
} catch (error) {
  console.error(error)
}

Tool Use / Function Calling

Instruct models to use predefined tools (functions) to interact with external systems or data.

import { z } from 'zod'
import { RosettaAI, Provider, RosettaTool, RosettaMessage, ProviderKey } from 'rosetta-ai-sdk'
// ... initialization ...

// 1. Define your tool
const getWeatherTool: RosettaTool = {
  type: 'function',
  function: {
    name: 'getCurrentWeather',
    description: 'Get the current weather for a location.',
    parameters: {
      type: 'object',
      properties: { location: { type: 'string', description: 'City and state/country' } },
      required: ['location']
    },
    zodSchema: z.object({
      location: z.string()
    })
  }
}

// 2. Implement the tool function
async function getCurrentWeather(location: string): Promise<string> {
  console.log(`[TOOL] Getting weather for ${location}...`)
  // Simulate API call
  await new Promise(r => setTimeout(r, 100))
  return JSON.stringify({ temperature: 72, condition: 'Sunny' })
}

// 3. Conversation Loop
async function runToolConversation() {
  const messages: RosettaMessage[] = [{ role: 'user', content: "What's the weather in San Francisco?" }]
  const maxTurns = 5 // Prevent infinite loops
  const providerKey: ProviderKey = Provider.OpenAI // Or 'my-custom', Provider.Anthropic, etc.

  for (let i = 0; i < maxTurns; i++) {
    console.log(`\nTurn ${i + 1}: Sending request...`)
    try {
      const response = await rosetta.generate({
        provider: providerKey,
        messages: messages,
        tools: [getWeatherTool],
        toolChoice: 'auto' // Let the model decide
      })

      console.log('Assistant:', response.content ?? '[No text content]')
      messages.push({ role: 'assistant', content: response.content, toolCalls: response.toolCalls })

      if (response.toolCalls && response.toolCalls.length > 0) {
        console.log('Tool Calls Requested:', response.toolCalls.length)
        const toolResults: RosettaMessage[] = []
        for (const call of response.toolCalls) {
          if (call.function.name === 'getCurrentWeather') {
            try {
              const args = JSON.parse(call.function.arguments)
              const result = await getCurrentWeather(args.location)
              toolResults.push({ role: 'tool', toolCallId: call.id, content: result })
            } catch (e) {
              console.error(`Tool execution error: ${e.message}`)
              toolResults.push({ role: 'tool', toolCallId: call.id, content: JSON.stringify({ error: e.message }) })
            }
          }
        }
        messages.push(...toolResults) // Add results for the next turn
      } else {
        console.log('\n--- Conversation End ---')
        break // Exit loop if no tool calls
      }
    } catch (error) {
      console.error(error)
      break
    }
  }
}

runToolConversation()

Anthropic Programmatic Tool Calling

RosettaAI also supports Anthropic's programmatic tool calling flow through the same generate(...) and stream(...) APIs. Enable it with programmaticToolCalling: true on a supported Anthropic model.

const result = await rosetta.generate({
  provider: Provider.Anthropic,
  model: 'claude-sonnet-4-6',
  messages: [{ role: 'user', content: 'Use tools to look up the weather in Paris.' }],
  tools: [getWeatherTool],
  programmaticToolCalling: true
})

if (result.toolCalls?.length) {
  console.log(result.toolCalls[0].caller) // { type: 'code_execution_20260120', toolId: '...' } when provider-supplied
}

console.log(result.container) // { id, expiresAt? } when Anthropic returns a reusable container

When programmaticToolCalling is enabled for Anthropic:

Rosetta injects Anthropic code execution automatically and maps RosettaTool.allowedCallers onto Anthropic's allowed_callers
Returned tool calls may include caller metadata describing whether the invocation was direct or originated from code execution
Final results may include container metadata that must be reused on follow-up tool-result turns
Streaming may emit container_info, code_execution_start, code_execution_delta, and code_execution_result chunks in addition to the normal text and tool-call events

Important constraints:

This is Anthropic-only and depends on the target model supporting programmatic tool calling
Do not combine it with a forced specific toolChoice; the Anthropic mapper rejects that combination
Do not pass extraParams.disable_parallel_tool_use = true; the Anthropic mapper rejects that combination
If you build your own multi-turn tool loop on top of RosettaAI, preserve Anthropic assistant rawResponse.content blocks by passing them back as RosettaMessage.rawContentBlocks, and reuse the returned container on the next request

Multimodal (Image Input)

Send images along with text prompts to multimodal models (OpenAI, Anthropic, Google).

import { RosettaAI, Provider, RosettaMessage, RosettaImageData, ImageMimeType, ProviderKey } from 'rosetta-ai-sdk'
import fs from 'fs/promises'
import path from 'path'
// ... initialization ...

async function describeImage(imagePath: string) {
  try {
    // 1. Load and encode the image
    const buffer = await fs.readFile(imagePath)
    const base64Data = buffer.toString('base64')
    const ext = path.extname(imagePath).toLowerCase()
    const mimeType: ImageMimeType = ext === '.png' ? 'image/png' : 'image/jpeg' // Add more types as needed
    const imageData: RosettaImageData = { mimeType, base64Data }

    // 2. Construct the message
    const messages: RosettaMessage[] = [
      {
        role: 'user',
        content: [
          // Content is an array for multimodal
          { type: 'text', text: 'Describe this image in detail.' },
          { type: 'image', image: imageData }
        ]
      }
    ]

    // 3. Generate response
    const providerKey: ProviderKey = Provider.OpenAI // Or Provider.Anthropic, Provider.Google
    const result = await rosetta.generate({
      provider: providerKey,
      model: 'gpt-4o-mini', // Use a vision-capable model
      messages: messages,
      maxTokens: 150
    })

    console.log('Image Description:', result.content)
  } catch (error) {
    console.error(error)
  }
}

// Ensure you have an image file (e.g., logo.png) in the same directory or provide the correct path
describeImage(path.join(__dirname, 'logo.png'))

Structured Output (JSON Mode & JSON Schema)

Request the model to respond in structured JSON format using the responseFormat parameter. The SDK supports two structured output modes and behavior varies by provider.

JSON Mode (`json_object`)

Forces the model to produce valid JSON. Describe the desired shape in the prompt; the model is not constrained to a specific schema.

import { RosettaAI, Provider, ProviderKey } from 'rosetta-ai-sdk'
// ... initialization ...

const result = await rosetta.generate({
  provider: Provider.OpenAI,
  model: 'gpt-4o-mini',
  messages: [
    { role: 'system', content: 'Extract the user\'s name and age from the text. Respond in JSON with keys "name" and "age".' },
    { role: 'user', content: 'My name is Alice and I am 30 years old.' }
  ],
  responseFormat: { type: 'json_object' }
})

const parsed = JSON.parse(result.content!)
console.log(parsed) // { name: 'Alice', age: 30 }

For Google, you can also provide an optional JSON schema to guide the json_object response:

const result = await rosetta.generate({
  provider: Provider.Google,
  model: 'gemini-1.5-flash-latest',
  messages: [
    { role: 'user', content: 'Extract the name and age from: "Bob is 25 years old."' }
  ],
  responseFormat: {
    type: 'json_object',
    schema: {
      type: 'object',
      properties: {
        name: { type: 'string' },
        age: { type: 'number' }
      },
      required: ['name', 'age']
    }
  }
})

JSON Schema Mode (`json_schema`)

Constrains the model output to conform to a specific JSON Schema. The model is guaranteed to produce output that validates against the provided schema.

// OpenAI strict structured output
const result = await rosetta.generate({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'Extract the name and age from: "Bob is 25 years old."' }
  ],
  responseFormat: {
    type: 'json_schema',
    json_schema: {
      name: 'person_extraction',     // Required for OpenAI (SDK defaults to 'response')
      strict: true,                   // Enable strict mode (SDK defaults to true)
      schema: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          age: { type: 'number' }
        },
        required: ['name', 'age'],
        additionalProperties: false
      }
    }
  }
})

// Anthropic structured output via json_schema
const result = await rosetta.generate({
  provider: Provider.Anthropic,
  model: 'claude-sonnet-4-20250514',
  messages: [
    { role: 'user', content: 'Extract the name and age from: "Bob is 25 years old."' }
  ],
  responseFormat: {
    type: 'json_schema',
    json_schema: {
      schema: {
        type: 'object',
        properties: {
          name: { type: 'string' },
          age: { type: 'number' }
        },
        required: ['name', 'age']
      }
    }
  }
})

Streaming Structured Output

Structured output is also available during streaming via json_delta and json_done chunk types:

const stream = rosetta.stream({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'List 3 colors as JSON array.' }],
  responseFormat: { type: 'json_object' }
})

for await (const chunk of stream) {
  switch (chunk.type) {
    case 'json_delta':
      // Partial JSON as it streams in
      console.log('Snapshot:', chunk.data.snapshot)
      break
    case 'json_done':
      // Final parsed result
      console.log('Parsed:', chunk.data.parsed)
      break
  }
}

Provider Support

Mode	OpenAI (Azure)	Anthropic	Google	Groq	Custom (OpenAI-Compat)
`json_object`	✅	❌	✅	⚠️	✅
`json_schema`	✅	✅	✅	❌	✅
Stream JSON (`json_delta`/`json_done`)	✅	❌	✅	❌	✅

Provider Notes:

OpenAI/Azure: Both json_object and json_schema are natively supported. For json_object, the schema field is informational only — describe the desired shape in the prompt. For json_schema, the SDK defaults name to 'response' and strict to true if omitted.
Anthropic: Supports json_schema only (via the output_config API parameter). Throws UnsupportedFeatureError for json_object — use json_schema instead. The name and strict fields are ignored; only schema is sent.
Google: Both json_object and json_schema map to application/json MIME type. Schema is natively used to constrain output. The SDK normalizes JSON Schema for Google compatibility (removes additionalProperties, $schema, converts anyOf/oneOf to enum where possible).
Groq: json_object support is unconfirmed; a warning is logged. json_schema is not supported.
Custom (OpenAI-Compatible): Passes responseFormat through via mapToOpenAIResponseFormat(). Support depends on the underlying provider.

Extra Parameters (Provider Passthrough)

The extraParams field allows you to pass provider-specific parameters that are not part of the unified RosettaAI interface directly through to the underlying provider API. This is useful for parameters like presence_penalty, frequency_penalty, repetition_penalty, top_k, seed, logprobs, and any other provider-specific options.

// Example: Passing OpenAI-specific parameters
const result = await rosetta.generate({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Tell me a story.' }],
  temperature: 0.7,
  extraParams: {
    presence_penalty: 0.6,
    frequency_penalty: 0.3,
    seed: 42,
    logprobs: true,
    top_logprobs: 3
  }
})

// Example: Passing Anthropic-specific parameters
const result2 = await rosetta.generate({
  provider: Provider.Anthropic,
  model: 'claude-3-haiku-20240307',
  messages: [{ role: 'user', content: 'Hello' }],
  extraParams: {
    top_k: 40,
    metadata: { user_id: 'user-123' }
  }
})

// Example: Passing Google-specific parameters
const result3 = await rosetta.generate({
  provider: Provider.Google,
  model: 'gemini-1.5-flash-latest',
  messages: [{ role: 'user', content: 'Hello' }],
  extraParams: {
    candidateCount: 1,
    topK: 40
  }
})

Behavior:

Extra parameters are spread into the final provider API payload.
Explicitly mapped fields (temperature, topP, maxTokens, etc.) always take precedence over colliding keys in extraParams.
For Google, extraParams are spread into the config object (where generation parameters reside), not the top-level request.
extraParams is also available on EmbedParams and audio params (TranscribeParams, TranslateParams).
The SDK does not validate extraParams against the provider API — you are responsible for ensuring the parameters are valid for the target provider.

Audio (TTS, STT, Translation)

Text-to-Speech (TTS)

Generate speech from text using generateSpeech (non-streaming) or streamSpeech. Supported by OpenAI/Azure, Groq, and ElevenLabs (see ElevenLabs Audio Provider below).

import { RosettaAI, Provider, SpeechParams } from 'rosetta-ai-sdk'
import fs from 'fs/promises'
import path from 'path'
// ... initialization ...

async function generateAudio() {
  const outputDir = path.join(__dirname, 'audio_output')
  await fs.mkdir(outputDir, { recursive: true })
  const filePath = path.join(outputDir, 'hello.mp3')

  try {
    // OpenAI TTS
    const params: SpeechParams = {
      provider: Provider.OpenAI,
      input: 'Hello from RosettaAI!',
      voice: 'alloy', // Choose a voice
      // model: 'tts-1-hd', // Optional model override
      responseFormat: 'mp3'
    }

    const audioBuffer = await rosetta.generateSpeech(params)
    await fs.writeFile(filePath, audioBuffer)
    console.log(`Audio saved to ${filePath}`)

    // Groq TTS (wav output only)
    const groqParams: SpeechParams = {
      provider: Provider.Groq,
      model: 'playai-tts',
      input: 'Hello from Groq TTS!',
      voice: 'Fritz-PlayAI',        // See supported voices below
      responseFormat: 'wav'          // Groq TTS only supports wav
    }

    const groqAudio = await rosetta.generateSpeech(groqParams)
    await fs.writeFile(path.join(outputDir, 'hello_groq.wav'), groqAudio)

    // Example: Streaming TTS (OpenAI/ElevenLabs)
    // const stream = rosetta.streamSpeech(params);
    // const writeStream = (await fs.open(path.join(outputDir, 'hello_stream.mp3'), 'w')).createWriteStream();
    // for await (const chunk of stream) {
    //   if (chunk.type === 'audio_chunk') writeStream.write(chunk.data);
    // }
    // writeStream.end();
    // console.log('Streamed audio saved.');
  } catch (error) {
    console.error(error)
  }
}

// Groq supported TTS voices:
// Arista-PlayAI, Atlas-PlayAI, Basil-PlayAI, Briggs-PlayAI, Calum-PlayAI,
// Celeste-PlayAI, Cheyenne-PlayAI, Chip-PlayAI, Cillian-PlayAI, Deedee-PlayAI,
// Fritz-PlayAI, Gail-PlayAI, Indigo-PlayAI, Mamaw-PlayAI, Mason-PlayAI,
// Mikail-PlayAI, Mitch-PlayAI, Quinn-PlayAI, Thunder-PlayAI

generateAudio()

Speech-to-Text (STT) & Translation

Transcribe audio to text using transcribe or translate audio to English using translate. Supported by OpenAI/Azure, Groq, and ElevenLabs (STT only, see ElevenLabs Audio Provider below for advanced features like diarization and audio event tagging).

import { RosettaAI, Provider, TranscribeParams, TranslateParams, RosettaAudioData, ProviderKey } from 'rosetta-ai-sdk'
import fs from 'fs/promises'
import path from 'path'
// ... initialization ...

async function processAudio(audioPath: string) {
  try {
    // 1. Prepare audio data
    const buffer = await fs.readFile(audioPath)
    const audioData: RosettaAudioData = {
      data: buffer,
      filename: path.basename(audioPath),
      mimeType: 'audio/mpeg' // Adjust based on your file type (mp3, wav, etc.)
    }

    // 2. Transcribe
    console.log('\n--- Transcription ---')
    const transcribeProvider: ProviderKey = Provider.Groq // Or Provider.OpenAI
    const transcribeParams: TranscribeParams = {
      provider: transcribeProvider,
      audio: audioData
      // model: 'whisper-large-v3', // Optional
      // language: 'en', // Optional language hint
    }
    const transcription = await rosetta.transcribe(transcribeParams)
    console.log(`[${transcribeParams.provider}] Transcription: ${transcription.text}`)

    // 3. Translate (to English)
    console.log('\n--- Translation ---')
    const translateProvider: ProviderKey = Provider.Groq // Or Provider.OpenAI
    const translateParams: TranslateParams = {
      provider: translateProvider,
      audio: audioData
      // model: 'whisper-large-v3', // Optional
    }
    const translation = await rosetta.translate(translateParams)
    console.log(`[${translateParams.provider}] Translation: ${translation.text}`)
  } catch (error) {
    console.error(error)
  }
}

// Ensure you have an audio file (e.g., sample_audio.mp3) or use the TTS output
processAudio(path.join(__dirname, 'sample_audio.mp3')) // Provide path to your audio file

ElevenLabs Audio Provider

ElevenLabs provides state-of-the-art Text-to-Speech (TTS) and Speech-to-Text (STT) capabilities with advanced features. To use ElevenLabs, configure it as a custom provider.

Setup:

Install the ElevenLabs SDK:

npm install @elevenlabs/elevenlabs-js

Set your API key in .env:

ELEVENLABS_API_KEY=your_elevenlabs_api_key
ROSETTA_DEFAULT_TTS_ELEVENLABS_MODEL=eleven_flash_v2_5  # Optional
ROSETTA_DEFAULT_STT_ELEVENLABS_MODEL=scribe_v1         # Optional

Configure the ElevenLabs provider:

import { RosettaAI, RosettaAIConfig, CustomProviderConfig } from 'rosetta-ai-sdk'
import { ElevenLabsMapper } from 'rosetta-ai-sdk' // Exported from main package

const elevenlabsConfig: CustomProviderConfig = {
  providerKey: 'elevenlabs',
  mapper: ElevenLabsMapper,
  supportedFeatures: ['tts', 'stt', 'list_voices'],
  apiKey: process.env.ELEVENLABS_API_KEY,
  defaultTtsModel: 'eleven_flash_v2_5',  // or 'eleven_multilingual_v2'
  defaultSttModel: 'scribe_v1'
}

const rosetta = new RosettaAI({
  customProviders: [elevenlabsConfig]
})

Text-to-Speech with ElevenLabs:

import { SpeechParams } from 'rosetta-ai-sdk'
import fs from 'fs/promises'

async function elevenlabsTTS() {
  const params: SpeechParams = {
    provider: 'elevenlabs',
    model: 'eleven_flash_v2_5', // Low latency model
    input: 'Hello from ElevenLabs! This is high-quality, natural-sounding speech.',
    voice: '21m00Tcm4TlvDq8ikWAM', // Rachel voice (use listVoices to find voice IDs)
    responseFormat: 'mp3',
    ttsOptions: {
      stability: 0.5,          // 0-1: Lower = more variable/expressive
      similarityBoost: 0.75,   // 0-1: Higher = closer to original voice
      style: 0.3,              // 0-1: Exaggeration of style (v3 models)
      useSpeakerBoost: true    // Boost clarity for low-quality audio
    }
  }

  // Non-streaming
  const audioBuffer = await rosetta.generateSpeech(params)
  await fs.writeFile('elevenlabs_output.mp3', audioBuffer)

  // Streaming (for lower latency)
  const stream = rosetta.streamSpeech(params)
  const chunks: Buffer[] = []
  for await (const chunk of stream) {
    if (chunk.type === 'audio_chunk') chunks.push(chunk.data)
  }
  await fs.writeFile('elevenlabs_streamed.mp3', Buffer.concat(chunks))
}

Text Normalization:

ElevenLabs TTS includes automatic text normalization that improves pronunciation of numbers, currencies, phone numbers, and URLs. This is enabled by default and can be disabled per-request:

// Normalization is ON by default:
// "$42.50" -> "42.50 dollars"
// "555-555-5555" -> "5 5 5, 5 5 5, 5 5 5 5"
// "example.com/path" -> "example dot com slash path"

// Disable normalization for a specific request:
const params: SpeechParams = {
  provider: 'elevenlabs',
  model: 'eleven_flash_v2_5',
  input: 'The price is $42.50',
  voice: '21m00Tcm4TlvDq8ikWAM',
  ttsNormalize: false  // Send raw text to ElevenLabs
}

Output Format Mapping:

Rosetta Format	ElevenLabs Format	Description
`'mp3'`	`mp3_44100_128`	Standard MP3 at 44.1kHz/128kbps
`'opus'`	`opus_48000_128`	Opus at 48kHz/128kbps (web streaming)
`'wav'`	`pcm_44100`	Uncompressed PCM at 44.1kHz
`'pcm'`	`pcm_16000`	PCM at 16kHz (telephony)
`'aac'`, `'flac'`	SDK default	Not directly supported; falls back

Speech-to-Text with ElevenLabs:

ElevenLabs STT supports 99 languages with advanced features:

Speaker Diarization: Identify up to 32 different speakers
Audio Event Tagging: Detect laughter, applause, and other sounds
Word-level Timestamps: Precise timing for each word
High Accuracy: State-of-the-art Scribe v1 model

import { TranscribeParams, RosettaAudioData } from 'rosetta-ai-sdk'
import fs from 'fs/promises'

async function elevenlabsSTT() {
  const audioBuffer = await fs.readFile('conversation.mp3')
  const audioData: RosettaAudioData = {
    data: audioBuffer,
    filename: 'conversation.mp3',
    mimeType: 'audio/mpeg'
  }

  const params: TranscribeParams = {
    provider: 'elevenlabs',
    model: 'scribe_v1',
    audio: audioData,
    language: 'en',              // Optional: ISO-639-1 language code
    diarize: true,               // Enable speaker identification
    tagAudioEvents: true         // Tag non-speech sounds like [laughter]
  }

  const result = await rosetta.transcribe(params)

  console.log('Transcription:', result.text)
  console.log('Language:', result.language)

  // Access word-level details with speaker IDs and timestamps
  if (result.words) {
    result.words.forEach((word: any) => {
      console.log(`[${word.speaker_id}] ${word.text} (${word.start}s - ${word.end}s)`)
    })
  }
}

List Available Voices:

async function listElevenLabsVoices() {
  const voices = await rosetta.listVoices('elevenlabs')

  console.log(`Found ${voices.data.length} voices:`)
  voices.data.forEach(voice => {
    console.log(`- ${voice.name} (${voice.id})`)
    console.log(`  Description: ${voice.description}`)
    console.log(`  Category: ${voice.category}`)
    console.log(`  Preview: ${voice.previewUrl}`)
  })
}

Supported ElevenLabs Models:

TTS Models:
- eleven_flash_v2_5 - Fastest, lowest latency (~75ms), recommended for real-time
- eleven_turbo_v2_5 - Balanced quality and speed (~250ms)
- eleven_multilingual_v2 - High quality, 70+ languages
- eleven_v3 - Most expressive and emotionally rich
STT Model:
- scribe_v1 - State-of-the-art transcription with 99 language support

For more details, see the ElevenLabs example.

Model Listing

List available models for configured providers (built-in and custom).

import { RosettaAI, Provider, ProviderKey, RosettaModelList } from 'rosetta-ai-sdk'
// ... initialization ...

async function listProviderModels(providerKey: ProviderKey) {
  console.log(`\n--- Listing Models for: ${providerKey} ---`)
  try {
    const models: RosettaModelList = await rosetta.listModels(providerKey)
    console.log(`Found ${models.data.length} models:`)
    models.data.slice(0, 5).forEach(m => console.log(`  - ${m.id} (Owned by: ${m.owned_by})`))
    if (models.data.length > 5) console.log('  ...')
  } catch (error) {
    console.error(`Error listing models for ${providerKey}: ${error.message}`)
  }
}

async function listAllProviderModels() {
  console.log('\n--- Listing Models for ALL Configured Providers ---')
  try {
    const allResults = await rosetta.listAllModels()
    for (const providerKey in allResults) {
      console.log(`\nProvider: ${providerKey}`)
      const result = allResults[providerKey]
      if (result instanceof Error) {
        // Check if it's an error object
        console.error(`  Error: ${result.message}`)
      } else if (result) {
        console.log(`  Found ${result.data.length} models:`)
        result.data.slice(0, 5).forEach(m => console.log(`    - ${m.id}`))
        if (result.data.length > 5) console.log('    ...')
      }
    }
  } catch (error) {
    console.error(`Unexpected error during listAllModels: ${error.message}`)
  }
}

// Example calls
// listProviderModels(Provider.OpenAI);
// listProviderModels('lmstudio'); // Example custom provider key
// listAllProviderModels();

Error Handling

RosettaAI throws specific error types to help you handle issues gracefully.

import {
  RosettaAI,
  Provider,
  RosettaAIError,
  ConfigurationError,
  ProviderAPIError,
  UnsupportedFeatureError,
  MappingError
} from 'rosetta-ai-sdk'
// ... initialization ...

async function safeGenerate() {
  try {
    const result = await rosetta.generate({
      provider: Provider.OpenAI,
      model: 'invalid-model-id', // Intentionally invalid
      messages: [{ role: 'user', content: 'Test' }]
    })
    console.log(result.content)
  } catch (error) {
    if (error instanceof ConfigurationError) {
      console.error(`Configuration Error: ${error.message}`)
      // e.g., Missing API key, invalid deployment ID
    } else if (error instanceof ProviderAPIError) {
      console.error(
        `Provider API Error (${error.provider || error.customProvider}): Status ${error.statusCode ??
          'N/A'}, Code: ${error.errorCode ?? 'N/A'} - ${error.message}`
      )
      // e.g., Rate limit, authentication error, invalid request to provider
      // console.error("Underlying error:", error.underlyingError); // Log original error if needed
    } else if (error instanceof UnsupportedFeatureError) {
      console.error(
        `Unsupported Feature Error: ${error.provider || error.customProvider} does not support ${error.feature}.`
      )
      // e.g., Trying TTS with Groq, Embeddings with Anthropic
    } else if (error instanceof MappingError) {
      console.error(`Internal SDK Mapping Error: ${error.message}`)
      // e.g., Failed to convert data between RosettaAI and provider format
    } else if (error instanceof RosettaAIError) {
      // Catch any other base SDK errors
      console.error(`RosettaAI Error: ${error.name} - ${error.message}`)
    } else {
      // Catch unexpected errors
      console.error('Unexpected Error:', error)
    }
  }
}

safeGenerate()

OpenAI Responses API (Stateful Interface)

RosettaAI provides access to OpenAI's Responses API as a separate, OpenAI-specific interface. This is a stateful, agent-ready API designed for modern AI interactions.

Key Differences from Chat Completions:

Stateful conversations: Use previous_response_id to chain turns without replaying message history
Separated concerns: instructions (developer/system intent) + input (user content)
Built-in tools: web_search, file_search, image_generation, code_interpreter
Semantic streaming: Rich event types beyond simple content deltas
OpenAI-only: Not available for other providers (by design)

Basic Usage

import { RosettaAI, Provider } from 'rosetta-ai-sdk'
// ... initialization ...

// Simple response
const result = await rosetta.createResponse({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  instructions: 'You are a helpful coding assistant',
  input: 'Explain async/await in JavaScript',
  max_tokens: 200
})

console.log(result.output_text)
console.log(`Response ID: ${result.id}`) // Use for stateful conversations

Stateful Conversations

// First turn
const turn1 = await rosetta.createResponse({
  provider: Provider.OpenAI,
  instructions: 'You are a math tutor',
  input: 'What is 15 + 27?'
})

console.log(turn1.output_text) // "15 + 27 = 42"

// Second turn - reference previous response (no history replay!)
const turn2 = await rosetta.createResponse({
  provider: Provider.OpenAI,
  instructions: 'You are a math tutor',
  input: 'Now multiply that by 3',
  previous_response_id: turn1.id // Stateful!
})

console.log(turn2.output_text) // "42 × 3 = 126"

Built-in Tools

// Web search
const result = await rosetta.createResponse({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  input: 'What are the latest TypeScript features?',
  tools: [{ type: 'web_search' }],
  tool_choice: 'auto'
})

// Image generation
const imageResult = await rosetta.createResponse({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  input: 'Generate a logo with a mountain and sun',
  tools: [
    {
      type: 'image_generation',
      options: { size: '1024x1024', quality: 'hd' }
    }
  ],
  tool_choice: { type: 'image_generation' }
})

// Code interpreter
const codeResult = await rosetta.createResponse({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  input: 'Analyze this CSV data: ...',
  tools: [{ type: 'code_interpreter' }]
})

Custom Function Tools

import { ResponsesTool } from 'rosetta-ai-sdk'
import { z } from 'zod'

const getWeatherTool: ResponsesTool = {
  type: 'function',
  name: 'getCurrentWeather',
  description: 'Get current weather for a location',
  parameters: {
    type: 'object',
    properties: {
      location: { type: 'string' },
      unit: { type: 'string', enum: ['celsius', 'fahrenheit'] }
    },
    required: ['location']
  },
  zodSchema: z.object({
    location: z.string(),
    unit: z.enum(['celsius', 'fahrenheit']).optional()
  })
}

const result = await rosetta.createResponse({
  provider: Provider.OpenAI,
  input: "What's the weather in San Francisco?",
  tools: [getWeatherTool],
  tool_choice: 'auto'
})

if (result.tool_calls) {
  for (const call of result.tool_calls) {
    const args = JSON.parse(call.function.arguments)
    // Execute your function...
    const weather = await getCurrentWeather(args)

    // Continue conversation with result
    const followUp = await rosetta.createResponse({
      provider: Provider.OpenAI,
      input: JSON.stringify(weather),
      previous_response_id: result.id
    })
  }
}

Structured Output

const result = await rosetta.createResponse({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  instructions: 'Extract package information',
  input: 'Install typescript version 5.5.4',
  response_format: {
    type: 'json_schema',
    json_schema: {
      name: 'PackageInfo',
      strict: true,
      schema: {
        type: 'object',
        properties: {
          package_name: { type: 'string' },
          version: { type: 'string' }
        },
        required: ['package_name', 'version'],
        additionalProperties: false
      }
    }
  }
})

const parsed = JSON.parse(result.output_text)
console.log(parsed) // { package_name: 'typescript', version: '5.5.4' }

Streaming Responses

const stream = rosetta.streamResponse({
  provider: Provider.OpenAI,
  model: 'gpt-4o',
  instructions: 'You are a creative writer',
  input: 'Write a haiku about TypeScript',
  stream: true
})

for await (const chunk of stream) {
  switch (chunk.type) {
    case 'response.created':
      console.log(`Stream started: ${chunk.data.id}`)
      break

    case 'response.output_text.delta':
      process.stdout.write(chunk.data.delta)
      break

    case 'response.output_text.done':
      console.log('\n[Text complete]')
      break

    case 'response.tool_call.start':
      console.log(`\nTool call: ${chunk.data.name}`)
      break

    case 'response.tool_call.delta':
      process.stdout.write(chunk.data.delta)
      break

    case 'response.tool_call.done':
      console.log(`\nTool call complete: ${chunk.data.arguments}`)
      break

    case 'response.completed':
      console.log(`\nUsage: ${JSON.stringify(chunk.data.usage)}`)
      break

    case 'response.failed':
      console.error(`Failed: ${chunk.data.error.message}`)
      break

    case 'error':
      console.error(`Error: ${chunk.data.error.message}`)
      break
  }
}

When to use Responses API vs Chat Completions:

Use Responses API for: Agent applications, stateful conversations, built-in tools, complex workflows
Use Chat Completions for: Provider-agnostic code, simple Q&A, maximum portability

See examples/responses-api.ts for a complete demonstration.

For Anthropic chat streaming via rosetta.stream(...), StreamChunk also includes programmatic tool calling events:

container_info
code_execution_start
code_execution_delta
code_execution_result

API Reference

Detailed documentation for all exported classes, methods, types, and interfaces is available via JSDoc comments within the source code. Use your IDE's IntelliSense or generate HTML documentation using TypeDoc.

Key exports include:

Client: RosettaAI
Enums: Provider
Configuration: RosettaAIConfig, ProviderOptions, CustomProviderConfig, ModelListingSourceConfig
Core Parameters: GenerateParams, EmbedParams, SpeechParams, TranscribeParams, TranslateParams
Core Results: GenerateResult, EmbedResult, TranscriptionResult, RosettaModel, RosettaModelList
Streaming: StreamChunk, AudioStreamChunk
Common Types: ProviderKey, RosettaMessage, RosettaContentPart, RosettaImageData, RosettaAudioData, RosettaTool, RosettaToolCallRequest, TokenUsage, Citation
Responses API (OpenAI): CreateResponseParams, ResponseResult, ResponsesStreamChunk, ResponsesTool, ResponsesInputItem, ResponsesToolChoice, ResponsesFormat
Errors: RosettaAIError, ConfigurationError, ProviderAPIError, UnsupportedFeatureError, MappingError, InvalidToolDefinitionError, ToolArgumentValidationError

Examples

Runnable examples demonstrating various features can be found in the /examples directory:

basic-chat.ts: Simple non-streaming generation.
streaming-chat.ts: Handling streaming responses and various chunk types.
tool-use.ts: Function calling/tool execution loop.
image-input.ts: Sending images to multimodal models.
embeddings.ts: Generating text embeddings.
audio.ts: Text-to-Speech and Speech-to-Text/Translation.
structured-output.ts: Requesting and validating JSON output.
list-models.ts: Listing available models for configured providers.
responses-api.ts: OpenAI Responses API with stateful conversations, built-in tools, and semantic streaming.
custom-provider-*.ts: Examples for integrating custom OpenAI-compatible providers (Novita, GPUStack, LM Studio).

To run an example:

Ensure you have configured your API keys in a .env file (see Configuration).
Make sure any required sample files (e.g., logo.png, sample_audio.mp3) exist in the examples directory if needed by the specific example.
Run the build command: npm run build
Execute the example using: npm run example:<name> (e.g., npm run example:basic, npm run example:stream, npm run example:responses, npm run example:listmodels).

Development

Clone: git clone https://github.com/MissionSquad/rosetta-ai-sdk.git
Install: cd rosetta-ai-sdk && npm install
Build: npm run build (Compiles TypeScript to JavaScript in /dist)
Test: npm test (Runs Jest tests)
Lint: npm run lint (Checks code style and potential type issues)
Format: npm run format (Formats code using Prettier)

Contributing

Contributions are welcome! Please follow these steps:

Fork the repository.
Create a new branch (git checkout -b feature/your-feature-name).
Make your changes.
Ensure tests pass (npm test) and linting is clean (npm run lint).
Add tests for new features or bug fixes.
Commit your changes (git commit -m 'Add some feature').
Push to the branch (git push origin feature/your-feature-name).
Open a Pull Request.

Please adhere to the established code style and architectural patterns.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Name		Name	Last commit message	Last commit date
Latest commit History 176 Commits
.github/workflows		.github/workflows
examples		examples
src		src
tests		tests
.env.example		.env.example
.eslintignore		.eslintignore
.eslintrc		.eslintrc
.gitignore		.gitignore
.prettierrc		.prettierrc
LICENSE		LICENSE
README.md		README.md
babel.config.json		babel.config.json
claude.md		claude.md
integration_guide.md		integration_guide.md
jest.config.json		jest.config.json
package-lock.json		package-lock.json
package.json		package.json
tsconfig.json		tsconfig.json
types-file.sh		types-file.sh
yarn.lock		yarn.lock

Folders and files

Latest commit

History

Repository files navigation

RosettaAI SDK

Key Features

Supported Features Matrix

Installation

Configuration

Quick Start

Core Concepts & Usage Examples

Initialization

Chat Completions

generate() (Non-Streaming)

Cancelling streaming requests (all providers)

stream() (Streaming)

Embeddings

Tool Use / Function Calling

Anthropic Programmatic Tool Calling

Multimodal (Image Input)

Structured Output (JSON Mode & JSON Schema)

JSON Mode (json_object)

JSON Schema Mode (json_schema)

Streaming Structured Output

Provider Support

Extra Parameters (Provider Passthrough)

Audio (TTS, STT, Translation)

Text-to-Speech (TTS)

Speech-to-Text (STT) & Translation

ElevenLabs Audio Provider

Model Listing

Error Handling

OpenAI Responses API (Stateful Interface)

Basic Usage

Stateful Conversations

Built-in Tools

Custom Function Tools

Structured Output

Streaming Responses

API Reference

Examples

Development

Contributing

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

`generate()` (Non-Streaming)

`stream()` (Streaming)

JSON Mode (`json_object`)

JSON Schema Mode (`json_schema`)

Packages