Repository navigation
Models and calls: reasoning that stays valid, the settings' lead in every chat, backups for calls that hang - #21
Merged
Merged
Conversation
…nged A provider binds a reasoning's signature to everything before it: the instructions, the tools offered and every earlier message, as they were. Each change iq made to one of those got the next call refused, or recovered only after a refused call and an alarming notice. The reasoning is now taken off before the call wherever iq knows the prefix changed: the answers after the first result set apart, all of them when the tools offered differ from the last call's, and all of them when a chat comes back under instructions written otherwise (the person's rules or a skill edited, another place). The fallback after a refusal stays for what iq cannot foresee, in two stages: without the reasoning of the earlier turns, then without any; its notice says it calmly.
A chat came back on the settings' lead only when its model was never chosen, so that one started with `/new <model>` kept an old lead forever; and a chat kept the effort it was saved with whatever the settings said since. Every chat the person leads now comes back on the settings' lead and on their effort, each change said as iq starts and the move kept with the chat's choices: a model or an effort chosen for it lasts until iq starts again. The agents a lead started keep what it chose for them.
A cheap lead often read the wrong skill, or none, and did the work its own way. Giving every skill whole would cost a lot of context and blur them together. `lead_skills = [...]` names a few short skills given whole in the instructions of the chats the person starts, after the list every agent is given; the others, and every skill for the agents a chat starts, are read on demand as before. One that cannot be given is said so in its place.
The namer could keep a name made of the request's first words, a verb
of asking or a politeness, and that name then showed where the person
writes and reads costs.
The namer is told never to use a verb of asking nor a politeness, in
any language, with a pull request named by its number as an example.
Where the person writes, their own chat is "the chat" ("Write to the
chat…"), an agent it started goes by its name; the turn's costs call
the chat's own spending "chat". The status line, naming the chat
written to among several, keeps its name.
…ackup of the person's A call got no answer for ten minutes, and iq stopped the turn, as it never sends a call again by itself. The provider's own fallback policies need an organisation's admin, and how long they wait before falling back is not said. In the settings, `[backups]` gives a model the models a call goes to in turn, best the same model in another region so that the cache and the reasoning are lost once at most, and `backup_after` (180 seconds unless set) how long a call to a model with a backup left waits. Past it, or at once when the provider is out of capacity instead of waiting through its tries, the call goes to the next backup the person allows, with a notice and without the reasoning; the backup answers for the rest of the turn and is billed as the model that answered. A refusal goes to no backup, and the last backup is waited for as before. Backups are checked as iq starts with the other models it uses. A call given up on may still be billed by the provider, and a strong model at high effort on a large context can be slow without hanging: `backup_after` is the same for every model for now.
The provider's dashboard listed iq's calls as made by no user agent: its HTTP client sent none. The client now says `User-Agent: iq/<version>`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Models and calls: reasoning that stays valid, the settings' lead in every chat, a few skills given whole, backups for calls that hang.
What
lead_skills = [...]: a few short skills given whole in the instructions of the person's chats; the others, and all of them for the agents a chat starts, read on demand.[backups]andbackup_after(180 s): a call that gets no answer in time, or whose provider is out of capacity (at once, without the tries), goes to the model's next backup the person allows, with a notice and without the reasoning, for the rest of the turn, billed as the model that answered. A refusal goes to none; the last backup is waited for as before. A call given up on may still be billed, andbackup_afteris one for every model for now.User-Agent: iq/<version>.Checked
Every commit passes
scripts/check.sh, each point with tests of its own against scripted models (a provider out of capacity or slow for one model, a refused signature, results set apart, tools added, a chat resumed elsewhere) and a local server for the user agent. Not checked against a real provider: the backups, and Anthropic's own refusals.First of three stacked pull requests.