Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.llm
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ You can add messages using either `{ user: "..." }` or `{ role: "user", content:
### Predicted Outputs
Pass `prediction` to support [Predicted Outputs](https://platform.openai.com/docs/guides/latency-optimization#use-predicted-outputs):
```ruby
ai.chat_completion(openai: "gpt-4o", params: { prediction: "..." })
ai.chat_completion(openai: "gpt-4o", prediction: "..." )
```

### Prompt Caching
When using Anthropic models, you can specify `cache_at`. Messages above that size get sent as ephemeral multipart segments.
```ruby
ai.chat_completion(params: { cache_at: 1000 })
ai.chat_completion(cache_at: 1000)
```

## Function Dispatch
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ chat_completion(openai: "gpt-4.1-nano", messages: [{ user: "What is the meaning
Raix supports [Predicted Outputs](https://platform.openai.com/docs/guides/latency-optimization#use-predicted-outputs) with the `prediction` parameter for OpenAI.

```ruby
>> ai.chat_completion(openai: "gpt-4o", params: { prediction: })
>> ai.chat_completion(openai: "gpt-4o", prediction:)
```

### Prompt Caching
Expand All @@ -68,7 +68,7 @@ Raix supports [Anthropic-style prompt caching](https://openrouter.ai/docs/prompt
Note that there is a limit of four breakpoints, and the cache will expire within five minutes. Therefore, it is recommended to reserve the cache breakpoints for large bodies of text, such as character cards, CSV data, RAG data, book chapters, etc. Raix does not enforce a limit on the number of breakpoints, which means that you might get an error if you try to cache too many messages.

```ruby
>> my_class.chat_completion(params: { cache_at: 1000 })
>> my_class.chat_completion(cache_at: 1000)
=> {
"messages": [
{
Expand Down Expand Up @@ -672,7 +672,7 @@ class StructuredResponse
})

transcript << { user: "Analyze the person named #{name}" }
chat_completion(params: { response_format: format })
chat_completion(response_format: format)
end
end

Expand Down
23 changes: 15 additions & 8 deletions lib/raix/chat_completion.rb
Original file line number Diff line number Diff line change
Expand Up @@ -40,15 +40,22 @@ module ChatCompletion

# This method performs chat completion based on the provided transcript and parameters.
#
# @param params [Hash] The parameters for chat completion.
# @option loop [Boolean] :loop (false) Whether to loop the chat completion after function calls.
# @option params [Boolean] :json (false) Whether to return the parse the response as a JSON object. Will search for <json> tags in the response first, then fall back to the default JSON parsing of the entire response.
# @option params [Boolean] :openai (false) Whether to use OpenAI's API instead of OpenRouter's.
# @option params [Boolean] :raw (false) Whether to return the raw response or dig the text content.
# @option params [Array] :messages (nil) An array of messages to use instead of the transcript.
# @option tools [Array|false] :available_tools (nil) Tools to pass to the LLM. Ignored if nil (default). If false, no tools are passed. If an array, only declared tools in the array are passed.
# @param loop [Boolean] Whether to loop the chat completion after function calls.
# @param json [Boolean] Whether to return the response as a JSON object.
# Will search for <json> tags in the response first, then fall back to the
# default JSON parsing of the entire response.
# @param openai [Boolean|String] Whether to use OpenAI's API instead of
# OpenRouter's. Pass a String to specify the model name.
# @param raw [Boolean] Whether to return the raw response or dig the text
# content.
# @param messages [Array] An array of messages to use instead of the
# transcript.
# @param available_tools [Array|false] Tools to pass to the LLM. Ignored if
# nil (default). If false, no tools are passed. If an array, only declared
# tools in the array are passed.
# @param params [Hash] Additional API parameters.
# @return [String|Hash] The completed chat response.
def chat_completion(params: {}, loop: false, json: false, raw: false, openai: false, save_response: true, messages: nil, available_tools: nil)
def chat_completion(loop: false, json: false, raw: false, openai: false, save_response: true, messages: nil, available_tools: nil, **params)
# set params to default values if not provided
params[:cache_at] ||= cache_at.presence
params[:frequency_penalty] ||= frequency_penalty.presence
Expand Down
12 changes: 7 additions & 5 deletions lib/raix/prompt_declarations.rb
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,13 @@ def prompts
# @raise [RuntimeError] If no prompts are defined.
#
# @param prompt [String] The prompt to use for the chat completion.
# @param params [Hash] Parameters for the chat completion.
# @param raw [Boolean] Whether to return the raw response.
# @param openai [Boolean|String] Whether to use OpenAI directly.
# @param params [Hash] Additional API parameters to pass through to
# {ChatCompletion#chat_completion}.
#
# TODO: SHOULD NOT HAVE A DIFFERENT INTERFACE THAN PARENT
def chat_completion(prompt = nil, params: {}, raw: false, openai: false)
def chat_completion(prompt = nil, raw: false, openai: false, **params)
raise "No prompts defined" unless self.class.prompts.present?

loop_count = 0
Expand Down Expand Up @@ -107,7 +109,7 @@ def chat_completion(prompt = nil, params: {}, raw: false, openai: false)
# set the stream if necessary
self.stream = instance_exec(&current_prompt.stream) if current_prompt.stream.present?

execute_ai_request(params:, raw:, openai:, transcript:, loop_count:)
execute_ai_request(raw:, openai:, transcript:, loop_count:, **params)
end

next unless current_prompt.until.present? && !instance_exec(&current_prompt.until)
Expand All @@ -133,8 +135,8 @@ def chat_completion(prompt = nil, params: {}, raw: false, openai: false)
last_response
end

def execute_ai_request(params:, raw:, openai:, transcript:, loop_count:)
chat_completion_from_superclass(params:, raw:, openai:).then do |response|
def execute_ai_request(raw:, openai:, transcript:, loop_count:, **params)
chat_completion_from_superclass(raw:, openai:, **params).then do |response|
transcript << { assistant: response }
@last_response = send(current_prompt.name, response)
self.stream = nil # clear it again so it's not used for the next prompt
Expand Down
2 changes: 1 addition & 1 deletion spec/raix/chat_completion_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ def initialize
end

context "with predicted outputs" do
let(:completion) { subject.chat_completion(openai: "gpt-4o", params: { prediction: }) }
let(:completion) { subject.chat_completion(openai: "gpt-4o", prediction:) }
let(:prediction) do
"THE MEANING OF LIFE CAN VARY GREATLY FROM PERSON TO PERSON, OFTEN INVOLVING THE PURSUIT OF HAPPINESS, CARE OF OTHERS, AND PERSONAL GROWTH!."
end
Expand Down
2 changes: 1 addition & 1 deletion spec/raix/function_dispatch_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@ def complete(...)
# With OpenAI:
expect { WhatIsTheWeather.new.chat_completion(openai: "gpt-4o", loop: true) }.to raise_error(/unauthorized function call/i)
# With OpenRouter:
expect { WhatIsTheWeather.new.chat_completion(openai: false, params: { model: "gpt-4o" }, loop: true) }.to raise_error(/unauthorized function call/i)
expect { WhatIsTheWeather.new.chat_completion(openai: false, model: "gpt-4o", loop: true) }.to raise_error(/unauthorized function call/i)
ensure
Raix.configuration.openai_client = previous_clients[:openai]
Raix.configuration.openrouter_client = previous_clients[:openrouter]
Expand Down
Loading