Post on the new voting pattern and dynamic chat model selection

[mariofusco]

No description provided.

[mariofusco]

/cc @geoand @cescoffier

[github-actions]

🙈 The PR is closed and the preview is expired.

[geoand]

Interesting!

Two totally different things come to mind:

• Blog posts are great, but we really need to keep the actual documentation up to date with these advancements, as we can't expect users to jump between documentation and blog posts.
• On the technical side, it's not clear to me (and unless I missed it, it's not explained) what happens in a voting situation when one or more agents fail to respond or respond too late

[mariofusco]

* Blog posts are great, but we really need to keep the actual documentation up to date with these advancements, as we can't expect users to jump between documentation and blog posts.

Both the voting pattern and the dynamic chat model selection are documented in langchain4j.

I'm bumping the quarkus extension to langchain4j 1.15 right now and I will update the documentation there with the dynamic chat model selection also there. I would leave out from the extension documentation these additional agentic pattern, because there isn't any difference with what provided out-of-the-box by langchain4j.

Note that the in my opinion the actual new interesting feature is the dynamic chat model selection, while this new voting pattern is mostly to showcase one of the possible application with a real-world example. This is a quite common required that I heard a few time while attending conferences: can we switch the model used by an agent based on the state of the system? And to me the most natural implementation of this was to allow to select the model as a function of the AgenticScope.

* On the technical side, it's not clear to me (and unless I missed it, it's not explained) what happens in a voting situation when one or more agents fail to respond or respond too late

At the moment you will wait for the slowest voters, while if there is an error the outcome of the voter will be simply discarded by the aggregation function. Regarding these concerns I believe that they could be covered with quarkus built-in fault tolerance that at the moment is not fully working with agents as reported here quarkiverse/quarkus-langchain4j#2457 and I need to further investigate this.

[geoand]

LGTM, but I would like @cescoffier to also have a look

[cescoffier]

I would wait until we have a proper integration in Quarkus.
At the moment, while this is published on the Quarkus blog, it never mentions Quarkus beyond the link in line 11. There's no CDI, no @RegisterAiService, no Quarkus config, no mention of the quarkus-langchain4j extension. At least something like: In a Quarkus application, these agents are CDI beans managed by the quarkus-langchain4j extension, which handles model configuration via application.properties.

[mariofusco]

I would wait until we have a proper integration in Quarkus. At the moment, while this is published on the Quarkus blog, it never mentions Quarkus beyond the link in line 11. There's no CDI, no @RegisterAiService, no Quarkus config, no mention of the quarkus-langchain4j extension. At least something like: In a Quarkus application, these agents are CDI beans managed by the quarkus-langchain4j extension, which handles model configuration via application.properties.

I just bumped the quarkus extension to langchain4j 1.15.0 that includes these new features. The post is in the same style of this other one https://quarkus.io/blog/agentic-ai-patterns/ but yes I could rewrite it using Quarkus annotation if required.

[cescoffier]

It would be great to show how it works in Quarkus, in a Quarkus way, and what about the "enterprise" parts (metrics,s tracing..)

[geoand]

Definitely!

[mariofusco]

Sure, I will update the code snippet to use the quarkus extension. Regarding metrics I already added a snapshot of the tracing of the execution.

[mariofusco]

@geoand @cescoffier I updated the article as requested, please give it a second look.

[cescoffier]

I'm not a big fan of this. @geoand any idea how we could improve?

[mariofusco]

Me neither, but unfortunately I didn't find a better way, it is not possible to directly inject the chat models directly into the agent.

[kdubois]

+1 on this ... It would be great if we can find a more elegant way to supply the model from the application properties, ideally with the @ModelName annotation... FWIW this is the temporary "solution"/hack I came up with in my project 😃

    @ChatModelSupplier
    static ChatModel chatModel() {
        return RemediationModel.model;
    }

    @ApplicationScoped
    class RemediationModel {
        private static ChatModel model;

        @Inject
        void init(@ModelName("remediation") ChatModel remediationModel) {
            model = remediationModel;
        }
    }

[geoand]

@mariofusco do you have the code anywhere so I can play with it?

[mariofusco]

@kdubois For fixed chat models you can already annotate the agent with @ModelName and it works, I implemented it with this PR quarkiverse/quarkus-langchain4j#2412

@geoand I don't have this application using the Quarkus extension anywhere, it's only in a test using vanilla langchain4j with the declarative API. I will put this on project in my personal github, so I could also link it in the article.

[geoand]

Okay, thanks. I'll use that to come up with something that can improve the situation here.

But as Clement said in another comment, this blog post should reference a Quarkus LangChain4j application, not vanilla LangChain4j

[mariofusco]

Agreed, I'm putting it together right now, I'll keep you posted.

[geoand]

🙏🏽

[mariofusco]

I put the complete quarkus project here and updated the post to link it.

[geoand]

@mariofusco see quarkiverse/quarkus-langchain4j#2486

[cescoffier]

Before merging, update the publication date.

[geoand]

We probably want to either wait for my PR to be available, or at least update the blog post when it is

[mariofusco]

I updated the publication date to tomorrow. It's up to you to decide if you want to publish it as it and then fix it when @geoand 's improvement will be available or simply wait for it and publish the definitive version.

[geoand]

Sounds good

[cescoffier]

let's merge it like this!