Skip to content

Add resource QA CLI - #529

Open
ChinmayK0607 wants to merge 3 commits into
mainfrom
chinmay/resource-qa-cli-v1
Open

Add resource QA CLI#529
ChinmayK0607 wants to merge 3 commits into
mainfrom
chinmay/resource-qa-cli-v1

Conversation

@ChinmayK0607

@ChinmayK0607 ChinmayK0607 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Adds hud qa agents for Environment and Taskset agent discovery.
  • Adds hud qa run with exact result selection, evidence reuse, waiting, overwrite control, JSON output, and distinct quality/request/execution exit codes.
  • Adds hud qa results for resource-scoped result inspection.
  • Encodes list-valued query parameters as repeated FastAPI query keys and documents the command surface.

Why

Resource QA workflows currently require direct API calls. The CLI should expose the existing Platform QA execution path without introducing another launcher, worker, result contract, or lifecycle.

Follow-up to HUD-2355.

Impact

Users can discover, run, wait for, and inspect first-class Environment and Taskset QA agents from the SDK. The CLI fails closed on incomplete Platform responses, pins exact reused attempts, and reports transport failures separately from quality failures.

Requires hud-monorepo#953 to deploy first. There are no SDK protocol or manifest compatibility changes.

Validation

Fail-to-pass coverage:

  • The hud qa command did not exist on the base branch.
  • Repeated subject IDs were encoded as a Python list instead of repeated query keys.
  • Incomplete reused-result responses could score the wrong historical attempt.
  • Network and timeout failures exited as quality failures.

Pass-to-pass coverage:

  • 895 passed in the full SDK test suite, with 6 pre-existing warnings.
  • 21 passed in the focused QA CLI and Platform transport tests.
  • Ruff format and lint passed for all changed files.
  • Pyright passed for the changed production modules.
  • hud qa --help and hud qa run --help smoke tests passed.

Made with Cursor

ChinmayK0607 and others added 2 commits August 3, 2026 11:44
Expose existing Platform QA workflows for environment and taskset resources with strict result selection, waiting, and machine-readable output.

Co-authored-by: Cursor <cursoragent@cursor.com>
Use the result payload returned by resource selection so no-wait output does not hide a stored quality verdict behind the terminal execution status.

Co-authored-by: Cursor <cursoragent@cursor.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 81c7394. Configure here.

Comment thread hud/cli/qa.py
Classify valid non-resource agents as unsupported caller input instead of a platform execution failure.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant