OpenAI Agent Support

[Z3r0Sum]

Discussion: Adding OpenAI model support

Summary

I was looking into adding first-class support for an OpenAI-based agent, and can follow most of the patterns outlined in the current code for existing agents. However, there are some fundamental differences worth discussing since OpenAI's closed-source models (GPT-5.2, GPT-5.1, etc.) cannot flow through Vertex AI.

Existing Agent Patterns Easily Incorporated

• An OpenAI executor
• The Tool abstraction for provider/callbacks
• Metaagent routing
• Basic integration tests via judge

Key Architectural Considerations

This is the big one. Claude and Gemini both authenticate through Vertex AI using GCP workload identity — no API keys, no secrets to manage. A Cloud Run service just works via ambient credentials.

We can do a few things here (updated with recent learnings):

1. Vertex AI Model Garden MaaS — Use gpt-oss models via the OpenAI-compatible endpoint on Vertex AI. Same GCP workload identity auth, no API keys. The trade-off is these are open-weight models — much cheaper ($0.07–0.09/M input tokens) but "less capable" than OpenAI's closed-source offerings. This is also tightly coupled to GCP (which a lot of the other parts of the project are, so may not be a big deal). It appears after further exploration that there's only a few OSS models that support function calling: https://docs.cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling and none of the OpenAI OSS models fall under this category. I initially had this working through the basic integration tests within judge; however, that doesn't actually do any tool calling and when testing e2e, I discovered the limitation.
2. Direct OpenAI API — Use closed-source models (GPT-4o, o3, o4) via OpenAI's public API with an OPENAI_API_KEY. For production on Cloud Run, the recommendation is to store the key in GCP Secret Manager and mount it via the container config's secret_key_ref. This avoids hardcoding keys and enables rotation without redeployment, but it's still an additional secret to manage that doesn't exist for the Vertex-backed providers. This also opens go-driftlessaf up to use cases outside the GCP ecosystem — for example, users who want to drive reconcilers via Argo Events, Kafka, or other event systems could interface directly with OpenAI without requiring Vertex AI / GCP.
3. Both — Support both paths, with the model prefix determining the provider (openai/gpt-oss-* → Vertex AI MaaS, gpt-*/o1-*/o3-*/o4-* → direct OpenAI API).

Thoughts on this approach?

Closing

Let me know what you think — I have a working implementation that supports the closed model implementation, where I use OpenAI directly to generate a simple gRPC service based on a new repo's issue. I wrote a quick local Go script to bypass some of the other pieces for quicker testing/iteration that leverages a form of the github-issue-materializer. I find it quite awesome, so kudos for laying a solid foundation!

Edit: I'm having a few issues with the open source gpt models in regards to the tool callback. I'll need to look into it a bit more so as of now only the OpenAI closed source models are working properly when opening a PR based on issue reconciliation (explained above - the OpenAI OSS models do not support function callback 😢 ).

[mosabua]

@Z3r0Sum I am not certain about the technical details, but from a higher level perspective we definitely want DriftlessAF to support more models and more deployment methods.

This could mean supporting OpenAI, but could also mean any other open source or closed source models. I could even mean running a model embedded as part of the deployment stack rather than calling out to an external model. And that deployment stack could run on any cloud provider ideally or even on prem or locally.

That said we have no current plans for implementing this.

PS: Sorry for the late reply.