Skip to content

Migrate from chat.completions to Responses API#65

Open
filip-komarzyniec wants to merge 10 commits into
mainfrom
RHOAIENG-60206-AutoRAG-migrate-to-OGX-Responses-API
Open

Migrate from chat.completions to Responses API#65
filip-komarzyniec wants to merge 10 commits into
mainfrom
RHOAIENG-60206-AutoRAG-migrate-to-OGX-Responses-API

Conversation

@filip-komarzyniec
Copy link
Copy Markdown
Collaborator

@filip-komarzyniec filip-komarzyniec commented May 19, 2026 •
edited
Loading

Description

Motivation

  • Use the built-in agent loop working under the hood of Responses API,
  • stay on top of the industry standards

Changes

  • API calls change
  • small refactoring of various names so that they better represent current behaviour

Testing

Commit with updated tests will be added once manual tests are completed.

Checklist

  • Tests added/updated
  • Documentation updated
  • Code follows style guide
  • All checks passing

@filip-komarzyniec filip-komarzyniec requested review from LukaszCmielowski and jakub-walaszczyk and removed request for jakub-walaszczyk May 19, 2026 14:17
@jakub-walaszczyk
Copy link
Copy Markdown
Collaborator

jakub-walaszczyk commented May 19, 2026

@filip-komarzyniec please restore changes that introduce formatting for 80 characters per line. 120 has been fine here, and these changes make noise in the PR

@filip-komarzyniec
source code transitioned from chat/completions API to Responses API
f46209b 
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>
@filip-komarzyniec
changed returned pattern content; formatter and linter related changes
f8dc86d 
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>
@filip-komarzyniec
updated tests to match recent changes
9932186 
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>
@filip-komarzyniec
fixup! source code transitioned from chat/completions API to Response…
59d7320 
…s API

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>
@filip-komarzyniec filip-komarzyniec force-pushed the RHOAIENG-60206-AutoRAG-migrate-to-OGX-Responses-API branch from 3a91573 to 59d7320 Compare May 19, 2026 14:52
@filip-komarzyniec
formatter and linter enforced changes
8317802 
Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@filip-komarzyniec filip-komarzyniec force-pushed the RHOAIENG-60206-AutoRAG-migrate-to-OGX-Responses-API branch from adbece5 to 8317802 Compare May 19, 2026 14:55
@filip-komarzyniec
fixup! updated tests to match recent changes
4d362cd 
Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
jakub-walaszczyk
jakub-walaszczyk requested changes May 20, 2026
View reviewed changes
Copy link
Copy Markdown
Collaborator

@jakub-walaszczyk jakub-walaszczyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the requested changes and consider handling chroma within the ai4rag context.

Comment thread ai4rag/rag/foundation_models/ogx.py
Comment thread ai4rag/core/experiment/experiment.py Outdated
Comment thread ai4rag/core/experiment/experiment.py Outdated
Comment thread ai4rag/core/experiment/experiment.py Outdated
Comment thread ai4rag/core/experiment/experiment.py Outdated
Comment thread ai4rag/rag/foundation_models/base_model.py
Comment thread ai4rag/rag/foundation_models/base_model.py
Comment thread ai4rag/rag/foundation_models/ogx.py
Comment thread ai4rag/rag/vector_store/ogx.py Outdated
Comment thread dev_utils/mocks.py
filip-komarzyniec
filip-komarzyniec commented May 20, 2026
View reviewed changes
Comment thread ai4rag/rag/retrieval/retriever.py Outdated
@filip-komarzyniec
changes mentioned in PR review #1
8e74789 
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>
@filip-komarzyniec
updated test files to reflect newest changes
0cde922 
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>
@filip-komarzyniec filip-komarzyniec force-pushed the RHOAIENG-60206-AutoRAG-migrate-to-OGX-Responses-API branch from 38210dc to 0cde922 Compare May 21, 2026 13:18
@filip-komarzyniec
unwanted formatting changes reverted
71460bd 
Signed-off-by: Filip Komarzyniec <fkomarzy@redhat.com>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@filip-komarzyniec filip-komarzyniec force-pushed the RHOAIENG-60206-AutoRAG-migrate-to-OGX-Responses-API branch from 52fead2 to 71460bd Compare May 21, 2026 14:00
@filip-komarzyniec
fixed invalid parameter in resposnes API call; related tests updated
07f3715 
rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
jakub-walaszczyk
jakub-walaszczyk reviewed May 22, 2026
View reviewed changes
Comment thread ai4rag/utils/constants.py
"""Constants used for setting the generation (inference) parameters for chat models only."""

MAX_COMPLETION_TOKENS = 2048
MAX_TOKENS = 2048
Copy link
Copy Markdown
Collaborator

@jakub-walaszczyk jakub-walaszczyk May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the parameter name is now max_output_tokens why do we use max_tokens? Wouldn't it be better to reflect the name of the parameter?

Copy link
Copy Markdown
Collaborator Author

@filip-komarzyniec filip-komarzyniec May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would, if we dropped the chat.completions support. The problem is that in the older API (completions) this param is called max_completion_tokens. In newer API (responses) it's max_output_tokens.

Now that we have some abstract constants class, I've decided to call it just max_tokensthere because it's consumed by both API interfaces.
What's more, the max_tokens is a deprecated name once used by chat.completions API so it's not totally made up by me.

The whole problem is caused by the fact that we decided earlier to leave the chat method support. No one uses it now so maybe it's better to just replace it with responses-aligned methods.

Copy link
Copy Markdown
Collaborator

@jakub-walaszczyk jakub-walaszczyk May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's replace it and keep proper parameters names, Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@LukaszCmielowski LukaszCmielowski Awaiting requested review from LukaszCmielowski

@jakub-walaszczyk jakub-walaszczyk Awaiting requested review from jakub-walaszczyk

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@filip-komarzyniec @jakub-walaszczyk