fix(uipath-functions): teach required pyproject authors + add Critical Rules - #2405
fix(uipath-functions): teach required pyproject authors + add Critical Rules#2405AlexBizon wants to merge 2 commits into
Conversation
…l Rules Three of twelve uipath-functions tasks failed the 2026-07-31 nightly on gpt-5.6-terra (simple-echo 0.52, e2e-lifecycle 0.64, file-attachment-input 0.41). All three failed on one assertion: FAIL: pyproject.toml has no `authors` entry Five check scripts under tests/tasks/uipath-functions/ require `authors`, but SKILL.md never mentioned it — the scaffold omits the field and nothing told the agent to add it. Sonnet infers it; gpt-5.6-terra does not. Also adds the Critical Rules section the skill was missing entirely (mandated by .claude/rules/skill-structure.md, and its absence is a red flag in .claude/rules/skill-review.md). Phrasing note: `authors` is tied to scaffolding, not to packing. The three failing tasks never pack, so guidance worded as "set it before packing" lets an agent correctly skip it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @AlexBizon's task in 2m 10s —— View job SummaryAdds a Change-by-Change Review1.
|
| Area | Status | Notes |
|---|---|---|
| Frontmatter | OK | name matches folder, description at 389 chars (under 1024 cap) |
| E2E Tests | OK | Not applicable — no new CLI workflows; existing tests in tests/tasks/uipath-functions/ cover the behavior. The authors assertion in those tests is what motivated this fix |
| Skill Body | OK | Critical Rules added in correct position, content is accurate and prescriptive |
| References & Assets | OK | No references or assets changed |
| Repo Hygiene | OK | CODEOWNERS entry exists, no secrets, change scoped to the modified skill |
Issues for Manual Review
- Coder-eval confirmation pending. The PR description notes that a
gpt-5.6-terrarerun is needed to confirm CR2 resolves the three failing tasks. The hypothesis is sound (the rule makesauthorsimpossible to miss), but the proof run hasn't landed yet.
Conclusion
Clean, well-scoped fix. The root-cause analysis is thorough, the Critical Rules section is well-written, and the authors template addition directly addresses the test failures. The only actionable follow-up is deduplicating the pre-existing Important Notes section against the new Critical Rules, which is out of scope for this targeted fix.
Approve — no blocking issues.
|
…hijack `uip function new -l py` passes through to `uipath new`. When uipath-langchain is present, its langgraph_new_middleware intercepts the command and scaffolds a LangGraph agent instead of a Function (UiPath/uipath-python#1543, open since 2026-04-02). tests/docker/Dockerfile:118 installs uipath-langchain globally to speed up coded-agent fixtures, so this fires in every uipath-functions eval task. The agent is handed langgraph.json + an LLM main.py with no `functions` map, no `class Input`, and no `traced` import. Sometimes it notices and rewrites; sometimes it burns the whole turn budget. Observed on run 30615869105: skill-functions-simple-echo MAX_TURNS_EXHAUSTED (40) score 0.143 FAIL: main.py is missing `class Input` skill-functions-tracing-redaction MAX_TURNS_EXHAUSTED (40) score 0.000 FAIL: main.py must import `traced` via `from uipath.tracing import traced` Adds a one-line `ls langgraph.json` check after scaffold plus the recovery steps, and folds the warning into Critical Rule 1. Turns a 40-turn flail into a two-edit correction. This mitigates the symptom in the skill. The durable fixes are #1543 upstream, and installing uipath-langchain into an isolated venv in the eval image rather than globally. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second finding: the eval image guarantees the LangGraph scaffold hijackCoder-eval run 30615869105 on this branch came back 10/12. The two Root cause
# Pre-install Python packages needed by coded-agent sandbox fixtures so agents
# don't trigger a live uv-sync download ...
RUN pip install --no-cache-dir "uipath-langchain>=0.9.26"Installed for coded-agent fixture speed, but it poisons every functions task in the same image. Reproduced locally on uip 1.200.0-dev.8046 — So the agent is handed a scaffold with no What this PR now does about itAdds a one-line detection check after scaffold plus explicit recovery steps, and folds the warning into Critical Rule 1. A two-edit correction instead of a 40-turn flail. This is symptom mitigation. The durable fixes are elsewhere:
Re-run dispatched: https://github.com/UiPath/skills/actions/runs/30617651902 Caveat on verificationThese runs use |
Why
3/12
uipath-functionstasks failed the 2026-07-31_04-38-51 nightly ongpt-5.6-terra:skill-functions-file-attachment-inputskill-functions-simple-echoskill-functions-python-e2e-lifecycleAll three failed on the same single assertion:
Every
command_executedandskill_triggeredcriterion passed. Turn counts were well under budget (10/35, 13/25, 13/31) — the agent finished cleanly and lost only therun_commandcriterion.Root cause: five check scripts under
tests/tasks/uipath-functions/requireauthors, butSKILL.mdnever mentioned it.uip function newomits the field. Sonnet infers it;gpt-5.6-terradoes not.What changed
authorsin the Step 5pyproject.tomltemplate, plus an explicit note that the scaffold omits it.Critical Rulessection. The skill had none — required by.claude/rules/skill-structure.md(§ SKILL.md Body Structure) and flagged as a red flag by.claude/rules/skill-review.md.Deliberate wording choice:
authorsis tied to scaffolding, not packing. The three failing tasks never pack, so guidance phrased as "set it before packing" lets an agent correctly conclude it is unnecessary. Rule 2 says to add it as part of scaffolding, "including ones you only run locally."Relationship to #1979
#1979 contains the same one-line
authorstemplate addition. It branched from main ~382 commits back, so it predates main'suip functions->uip functiondoc rename — but it never edited those verb lines, and on uip 1.200.0-dev.8046 the CLI reports usage asuip function|functions, i.e. both forms remain live aliases. So there is no functional regression either way; it is a docs-convention difference only. #1979's other two fixes (deploy_tenant turn budget, in_flow_register entrypoint resolution) remain valuable and are not covered here. Recommend rebasing #1979 and dropping only its duplicatedauthorshunk.Note for whoever reviews #1979: the comment claiming "Ran 5 times all suite and all tests passed" holds for 4 of the 5 linked runs. Run 29086622276 concluded failure at 11/12 —
smoke-trigger/business-daysscored 0.000 on activation recall while the suite itself reportedpass_rate=80% (4/5) gate=PASS. That red is the verdict-step bug in item 1 below, not a defect in #1979's fixes.Verification
Validators pass locally:
hooks/validate-skill-descriptions.sh— 389 charsscripts/check-skill-status.py— OK, 24 skillsscripts/check-cli-verbs.py— 0 High, 0 Medium (catalog uip 1.200.0-dev.8042)Coder-eval on the
uipath-functionstree pending; the hypothesis is that Critical Rule 2 makes the field impossible to miss, but that needs a run ongpt-5.6-terrato confirm.Not addressed here
Two separate findings from the same investigation, worth their own issues:
run-coder-eval.ymlverdict ignores suite gates. The step exits non-zero unless everytask.jsonisSUCCESS, so a rate-gated recall suite can never be green if one row misses — even when the suite reportsgate=PASS. Hit this onsmoke-trigger/business-days(suitepass_rate=80% (4/5) gate=PASS, run still red).uipath-functions. The skill showsinvoke_activitywithobject_pathpre-filled and no route from an activity name to a path;bindings.jsonis mentioned only as a generated filename. The content exists underuipath-platformanduipath-agents, but the self-contained-skill rule forbids linking to it.🤖 Generated with Claude Code