Skip to content

[codex] Reapply active LLM profile after save#961

Open
neubig wants to merge 5 commits into
mainfrom
codex/reapply-active-llm-profile
Open

[codex] Reapply active LLM profile after save#961
neubig wants to merge 5 commits into
mainfrom
codex/reapply-active-llm-profile

Conversation

@neubig
Copy link
Copy Markdown
Member

@neubig neubig commented May 30, 2026

Summary

  • Reapply the active LLM profile after saving it so updated model/API key settings are copied back into agent_settings.llm.
  • Route this through the existing activation mutation so React Query and SettingsService cache invalidation stay consistent.
  • Add tests for the active-profile reapply decision and update the rename test to assert the mutation path.

Root Cause

Conversation start reads agent_settings.llm, not the profile row directly. Saving an active profile updated the profile record, but did not re-activate it unless it was renamed, so new conversations could still use stale encrypted LLM settings.

Validation

  • npm test -- __tests__/components/settings/llm-profiles/llm-settings-local-view.test.tsx
  • npx vitest run __tests__/components/settings/llm-profiles/llm-settings-local-view.test.tsx
  • npm run typecheck
  • npm run build
  • git diff --check

Visual Evidence

Before/after active LLM profile save behavior

The recording compares main against this PR using a controlled local agent-server API: before the fix, saving the active profile updates the profile row while agent_settings.llm stays stale; after the fix, the active profile is reapplied and the settings used for conversation start match the saved profile.

AI disclosure: this visual-evidence update was added by an AI agent (OpenHands) on behalf of the user.


🐳 Docker images for this PR

GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas

Component Value
Image ghcr.io/openhands/agent-canvas
Architectures amd64, arm64
Agent Server ghcr.io/openhands/agent-server:1.24.0-python
Automation openhands-automation==1.0.0a5
Commit 61e92591d7d306a58b85aedf9af5b11df30ad03e

Pull (multi-arch manifest)

# Multi-arch manifest — Docker automatically pulls the correct architecture
docker pull ghcr.io/openhands/agent-canvas:sha-61e9259

Run

docker run -it --rm \
  -p 8000:8000 \
  ghcr.io/openhands/agent-canvas:sha-61e9259

All tags pushed for this build

ghcr.io/openhands/agent-canvas:sha-61e9259-amd64
ghcr.io/openhands/agent-canvas:codex-reapply-active-llm-profile-amd64
ghcr.io/openhands/agent-canvas:pr-961-amd64
ghcr.io/openhands/agent-canvas:sha-61e9259-arm64
ghcr.io/openhands/agent-canvas:codex-reapply-active-llm-profile-arm64
ghcr.io/openhands/agent-canvas:pr-961-arm64
ghcr.io/openhands/agent-canvas:sha-61e9259
ghcr.io/openhands/agent-canvas:codex-reapply-active-llm-profile
ghcr.io/openhands/agent-canvas:pr-961

About Multi-Architecture Support

  • Each tag (e.g., sha-61e9259) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., sha-61e9259-amd64) are also available if needed

@vercel
Copy link
Copy Markdown

vercel Bot commented May 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agent-canvas Ready Ready Preview, Comment Jun 1, 2026 2:09pm

Request Review

@neubig neubig marked this pull request as ready for review May 30, 2026 16:39
Copy link
Copy Markdown
Contributor

all-hands-bot commented May 31, 2026

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

Copy link
Copy Markdown
Contributor

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: [codex] Reapply active LLM profile after save

🟡 Acceptable — The fix is correct and well-motivated. A few readability and test-coverage points worth addressing.


What the PR does

The root cause is clearly stated: conversation start reads , not the profile row directly. Saving an active profile was updating the profile store but not syncing those values back into . The fix: after saving any active profile, re-run the activation mutation (which copies LLM config into ) and route it through so React Query cache invalidation stays consistent.

This is correct and the scope is appropriately minimal.


Issues

[IMPROVEMENT OPPORTUNITIES]

** condition is harder to read than it needs to be**

The expression:

The clause is effectively unreachable in edit mode (because renaming to an existing profile name, including the active profile's name, is blocked by the guard). It is only meaningful in create mode when is /, where already produces — making the second clause a redundant repeat of the first.

This simpler form communicates the intent without the puzzle:

[TESTING GAPS]

Missing test: create-mode, is null/undefined, matches the active profile

The new test suite covers three cases (save active without rename, rename active profile, inactive profile), but not the fourth reachable path: creating a new profile (e.g. after deleting the active one) where is and equals the stale . The function's behaviour for this case is currently untested.

No runtime evidence in PR description

For a behavioural bug fix that changes when is synced (affecting every conversation started after saving), the description should include an section with a short before/after demonstration — e.g. steps to reproduce the original bug and a screen recording or console log showing the corrected sync. Tests prove the decision logic; they do not prove the end-to-end conversation-start behaviour.


What's good

  • Extracting as a pure, exported function is the right call — it's directly testable and the three covered cases are meaningful.
  • Switching from to is the correct approach. Going through the mutation hook keeps React Query's cache consistent rather than bypassing it with a raw service call, and this is exactly what the existing rename path should have been doing.
  • The rename-test update (mock instead of ) more accurately reflects the real code path.
  • added to deps is consistent with how is already treated, so no new instability is introduced.

[RISK ASSESSMENT]

  • [Overall PR] ⚠️ Risk Assessment: 🟢 LOW — The change adds one extra mutation call on the happy path when saving an active profile. Worst case is a spurious extra activation request; no data loss or breaking API changes. Error handling is already covered by the surrounding try/catch.

VERDICT:
Worth merging with the test gap and evidence section addressed. The core logic is sound.

KEY INSIGHT:
The broadened condition (save active profile → always re-apply, not just on rename) is the right fix, but extract the redundant branch so the function's semantics are self-evident.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation


Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

  1. Add a file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing. See the customization docs for the required frontmatter format.
  2. Re-request a review — the reviewer reads guidelines from the PR branch, so your changes take effect immediately.

Was this review helpful? React with 👍 or 👎 to give feedback.

Comment thread src/components/features/settings/llm-profiles/llm-settings-local-view.tsx Outdated
Simplify the active-profile reapply predicate and cover the recreate-active-profile path from review feedback.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 31, 2026

PR Artifacts Cleaned Up

The .pr/ directory has been removed after approval.

Copy link
Copy Markdown
Member Author

neubig commented May 31, 2026

CI is green on aa519d52, all review threads are resolved, and the before/after GIF has been added to the PR description. all-hands-bot, please re-review and approve #961 if all comments have been addressed.\n\n_This comment was generated by an AI agent (OpenHands) on behalf of the user._

@neubig neubig requested a review from all-hands-bot May 31, 2026 19:05
Copy link
Copy Markdown
Contributor

all-hands-bot commented May 31, 2026

Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

Copy link
Copy Markdown
Contributor

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: [codex] Reapply active LLM profile after save

Looks good to merge — the root cause is correctly diagnosed, the fix is minimal and correctly scoped, and the follow-up commit (45f66a0) addressed both concerns raised in the previous review round.


Current state

shouldReapplyProfileAfterSave is now expressed as three clean guard clauses:

if (!activeProfileName) return false;
if (originalName) return activeProfileName === originalName;   // edit mode
return activeProfileName === savedName;                         // create mode

All four reachable branches are covered by unit tests: save-without-rename, rename, create-with-active-name, and inactive profile. The function is exported and directly testable, which is the right design.

Mutation routing through activateProfile.mutateAsync instead of a raw ProfilesService.activateProfile call is correct — it keeps React Query cache invalidation consistent with every other activation in the component. The renamed-profile test now correctly mocks mockActivateMutateAsync rather than the service, accurately reflecting the real code path.


Minor observations (non-blocking)

  • Untested guard: The if (!activeProfileName) return false branch is not directly tested. A single case — { activeProfileName: null, originalName: 'x', savedName: 'x' } → false — would complete branch coverage. Low priority given the simplicity of the guard.

  • Error UX after partial failure: If activateProfile.mutateAsync throws after the profile save already succeeded, the surrounding catch will surface a generic save-error toast even though the profile data was persisted. This is a pre-existing pattern (out of scope here) but worth noting for a future polish pass.


Summary

The fix broadens the re-activation condition from "rename of active profile" to "any save of the active profile", routes it through the mutation hook for cache consistency, extracts the predicate as a pure exported function with good test coverage, and includes visual evidence. Ready to merge.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

@neubig neubig enabled auto-merge (squash) May 31, 2026 19:30
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

📸 Snapshot Test Report

✅ All snapshots match the main branch baselines.

Category Count
🔴 Changed 0
🆕 New 0
✅ Unchanged 73
Total 73
✅ Unchanged snapshots (73)

archived-conversation

  • conversation-panel-with-archived-badges
  • conversation-view-archived
  • conversation-view-sandbox-error

automations

  • automations-delete-modal
  • automations-list-active-inactive
  • automations-no-automations
  • automations-search-no-results

backends-extended

  • backend-add-blank-disabled
  • backend-add-cloud-advanced-open
  • backend-add-cloud-no-key-disabled
  • backend-add-cloud-with-key-enabled
  • backend-add-form-partially-filled
  • backend-add-invalid-url-disabled
  • backend-add-local-ready
  • backend-add-name-only-disabled
  • backend-add-two-column-layout
  • backend-add-whitespace-host-disabled
  • backend-after-switch
  • backend-cancel-nothing-saved
  • backend-dropdown-two-backends
  • backend-edit-prefilled
  • backend-manage-after-removal
  • backend-manage-two-listed
  • backend-remove-cancelled
  • backend-remove-confirmation
  • backend-switch-overlay

backends

  • backend-add-modal
  • backend-manage-modal
  • backend-selector-open

changes-tab

  • changes-deleted-file
  • changes-diff-viewer
  • changes-empty

collapsible-thinking

  • reasoning-content-collapsed
  • reasoning-content-expanded
  • think-action-collapsed
  • think-action-expanded

mcp-page

  • mcp-custom-server-1-editor-open
  • mcp-custom-server-2-url-filled
  • mcp-custom-server-3-all-filled
  • mcp-custom-server-4-installed
  • mcp-custom-server-editor
  • mcp-empty-installed
  • mcp-search-filtered
  • mcp-slack-install-1-marketplace
  • mcp-slack-install-2-modal
  • mcp-slack-install-3-filled
  • mcp-slack-install-4-installed

onboarding

  • onboarding-step-0-choose-agent
  • onboarding-step-1-check-backend
  • onboarding-step-2-setup-llm
  • onboarding-step-3-say-hello

projects-workspace-browser

  • projects-workspace-browser

settings-page

  • add-backend-modal
  • analytics-consent-modal
  • home-screen
  • settings-app-page
  • settings-page

settings-secrets

  • secrets-add-form-filled
  • secrets-add-form
  • secrets-after-save
  • secrets-delete-confirm
  • secrets-list

settings-verification

  • condenser-settings
  • verification-settings-off
  • verification-settings-on

sidebar

  • sidebar-collapsed
  • sidebar-conversation-panel
  • sidebar-filter-menu

skills-page

  • skills-empty
  • skills-loaded
  • skills-no-match
  • skills-search-filtered
  • skills-type-filter

Generated by the Snapshot Tests workflow. This comment was created by an AI agent (OpenHands) on behalf of the repo maintainers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants