Bug: The program experiences lag at different points every time it runs.

[RAO-sy]

Running the example “Asynchronous Execution” very slow and lags in various places, even though we use the different LLM api. May I ask what this is all about?

[dhh1995]

Thanks for the feedback!
Did you mean this example?

It is an example of using the StringFuture operations to parallelize the LLM calls.
The expected behavior is that all LLM calls (x * y = ?) should be run in parallel, and the response will be concatenated.
However, since there is no format control for that response, the output may become messy for different LLMs.

May I know the API you are using and the version of APPL? Have you run other examples before that?