Feedback wanted: ReFrame default model selection

[CTOUT]

What changed

After some experimentation with model identifiers, the ReFrame agent model array has gone through a few iterations. Here is where we landed:

model: [GPT-5.4 mini (copilot), Claude Haiku 4.5 (copilot), Auto (copilot)]

Why these models

• GPT-5.4 mini - 400K context, 0.33x pricing, no deprecation flag. Good at structured config/JSON parsing and PowerShell generation.
• Claude Haiku 4.5 - 200K context, 0.33x pricing. Anthropic models tend to follow multi-step instructions reliably, which suits ReFrame's confirm-before-apply workflow.
• Auto - final fallback; lets Copilot choose if the above are unavailable.

GPT-4.1 was considered (0x cost) but has a deprecation warning and expires shortly, so it was excluded.

The tradeoff

This setting locks the model for all users. You cannot override it at prompt time. If you want a different model (e.g. Claude Sonnet, GPT-5.5), you would need to fork the agent file.

We still want your feedback

• Do GPT-5.4 mini or Claude Haiku 4.5 give you good results for game config analysis?
• Have you hit cases where a more capable model was needed?
• Would you prefer we remove the lock entirely and let your Copilot picker drive the model?
• Any other models you think would be a better default?

Drop your thoughts below!