Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@

`Key` identifies the backing app. `resource list` does not return `systemName`, `deployVersion`, or the action schema — fetch those in Step 3 with `uip solution resources get`.

**Solution-internal apps require the UNFILTERED list.** `--kind`, `--source`, and `--search` filters currently apply only to the remote catalog — a `Source: "Local"` app is dropped by every filtered query and appears only in a plain `uip solution resources list --output json`. Run the unfiltered list (from the solution directory) before concluding an app is absent, and always pass the row's `Key` — not the name — to `resources get` (name lookups fail for solution-internal apps).

**App not in the list — STOP.** If the named app has no row in `uip solution resources list` (or Step 3's `resources get` exits non-zero / "was not found"), the app cannot back an escalation. Report `<APP_NAME> was not found in the solution or the resource catalog — the escalation cannot be wired` and stop. Do NOT retry with alternate search terms, other endpoints, raw API calls, or re-authentication, and do NOT write the escalation (or an escalate guardrail action) referencing the missing app — a single failed lookup is terminal for this run.

### Step 3 — Fetch the app's action schema

Pass the `Key` from Step 2 to `uip solution resources get` — one CLI-native call returns the app spec, including the action schema. Works for both external (`Source: "Remote"`) and solution-internal (`Source: "Local"`) apps, even before the app is imported into the solution. Run from the solution directory (or pass `--solution-folder <path>`).
Expand Down Expand Up @@ -170,7 +174,7 @@
"properties": {
"resourceKey": "<appId-guid>", // the `Key` from `uip solution resources list` (== the key passed to `resources get`)
"appName": "<appName>", // `Name` from `uip solution resources list` (== `Spec.Name` from `resources get`)
"folderName": "Shared/Approvals", // literal Folder from `uip solution resources list`. External: Orchestrator folder (e.g., "Shared/Approvals"). Solution-internal: "solution_folder". uip agent refresh translates this to folderPath in the App binding inside bindings_v2.json.

Check warning on line 177 in skills/uipath-agents/references/lowcode/capabilities/escalation/escalation.md

View workflow job for this annotation

GitHub Actions / uipath-agents

Possibly stale `uip agent refresh translates this to folderPath in the App binding inside` (valid prefix: `agent refresh`)
"appVersion": 1, // integer — the `version` field of the parsed `Spec.ActionSchema` (NOT `Spec.Version`, the package semver)
"isActionableMessageEnabled": false,
"actionableMessageMetaData": null
Expand Down Expand Up @@ -229,7 +233,7 @@

### Step 6b — Inline agents only: wire the escalation flow node

**Skip if the agent is standalone.** If the escalation is on an **inline** agent (embedded in a flow), the `resource.json` alone is never reached at runtime — you MUST also add an escalation flow node connected to the autonomous node's `escalation` handle. The registry exposes the escalation node as concrete variants (e.g. `uipath.agent.resource.escalation.coded-action-app` for an Action Center / Workflow Action app) — there is no bare `uipath.agent.resource.escalation` node. Discover the available variant with `uip maestro flow registry search "escalation" --output json` (pick the one with `AvailableOnTenant: true`), fetch its manifest with `uip maestro flow registry get <NodeType> --output json`, then hand the node + edge authoring to the `uipath-maestro-flow` skill (Critical Rule 16 — this skill does not author `.flow` graphs directly). Run Step 7's refresh/validate with `--inline-in-flow` plus `--bindings-target <FlowProjectDir>/bindings_v2.json`. See [../inline-in-flow/inline-in-flow.md](../inline-in-flow/inline-in-flow.md).

Check warning on line 236 in skills/uipath-agents/references/lowcode/capabilities/escalation/escalation.md

View workflow job for this annotation

GitHub Actions / uipath-agents

Possibly stale `uip maestro flow registry search escalation` (valid prefix: `maestro flow registry search`)

### Step 7 — Refresh, validate, and refresh solution resources

Expand All @@ -256,7 +260,7 @@
### Step 8 — Bundle and upload

```bash
uip solution bundle . -d ./dist --output json

Check warning on line 263 in skills/uipath-agents/references/lowcode/capabilities/escalation/escalation.md

View workflow job for this annotation

GitHub Actions / uipath-agents

Possibly stale `uip solution bundle` (valid prefix: `solution`)
uip solution upload ./dist/<SOLUTION_NAME>.uis --output json
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,14 @@ cases:
Name the catalog-recommended action for that scope in the description. Severity `judgment` — a guardrail that breaks
the agent or leaves a security gap can be Critical; a milder ineffectiveness is Warning/Info.

**The catalog verdict IS the evidence — do not verify SDK behavior empirically.** Source read + catalog comparison
fully decides this check. Do NOT import or execute the project's dependencies, inspect `uipath_langchain` /
`langchain` package sources, fetch SDK documentation sites, or run the agent's factory code to "prove" an action is
ineffective — that forensic detour adds no admissible evidence to the finding and routinely burns the entire review
budget. Once the configured action diverges from the catalog's recommendation for that scope, write the finding and
move on to the report (the report file is the deliverable; an unwritten report scores zero regardless of analysis
depth).

### Relevance + Wiring Check → `CODED_GUARDRAIL_MISAPPLIED`

Two ways a format-valid guardrail can be misapplied:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"SolutionId": "e5ca1a7e-0000-4000-8000-000000000000", "Projects": []}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"DocVersion": "1.0.0",
"StudioMinVersion": "2025.04.0",
"SolutionId": "e5ca1a7e-0000-4000-8000-000000000000",
"Projects": [],
"AutomationHubIdeaUrl": null
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
{
"docVersion": "1.0.0",
"resource": {
"name": "EscalationWorksApp",
"kind": "app",
"type": "workflow Action",
"apiVersion": "apps.uipath.com/v1",
"projectKey": "e5ca1a7e-0005-4005-8005-000000000005",
"isOverridable": true,
"dependencies": [
{
"name": "EscalationWorksApp",
"kind": "appVersion"
},
{
"name": "EscalationWorksApp",
"kind": "process"
}
],
"runtimeDependencies": [],
"files": [],
"folders": [
{
"fullyQualifiedName": "solution_folder"
}
],
"spec": {
"name": "EscalationWorksApp",
"description": "Escalation works queue app (generic single input/output form).",
"type": "workflow Action",
"version": null,
"appSystemName": null,
"appVersionRef": {
"key": "e5ca1a7e-0002-4002-8002-000000000002"
},
"codeBehindProcess": {
"key": "e5ca1a7e-0003-4003-8003-000000000003"
},
"actionSchema": "{\"version\": 1, \"inputs\": [{\"name\": \"input\", \"type\": \"System.String\", \"description\": \"Generic input value\"}], \"inOuts\": [], \"outputs\": [{\"name\": \"output\", \"type\": \"System.String\", \"description\": \"Generic output value\"}], \"outcomes\": [{\"name\": \"Submit\"}, {\"name\": \"Reject\"}]}",
"manualActivationActions": null
},
"locks": [],
"key": "e5ca1a7e-0001-4001-8001-000000000001"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"docVersion": "1.0.0",
"resource": {
"name": "EscalationWorksApp",
"kind": "appVersion",
"apiVersion": "apps.uipath.com/v2",
"projectKey": "e5ca1a7e-0005-4005-8005-000000000005",
"isOverridable": true,
"dependencies": [],
"runtimeDependencies": [],
"files": [],
"folders": [
{
"fullyQualifiedName": "solution_folder"
}
],
"spec": {
"name": "EscalationWorksApp",
"description": null,
"publishNote": null,
"version": null,
"projectId": null,
"isAppPublic": false,
"packageFile": null,
"packageKey": null
},
"locks": [],
"key": "e5ca1a7e-0002-4002-8002-000000000002"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
{
"docVersion": "1.0.0",
"resource": {
"name": "EscalationWorksApp",
"kind": "package",
"apiVersion": "orchestrator.uipath.com/v1",
"projectKey": "e5ca1a7e-0005-4005-8005-000000000005",
"isOverridable": true,
"dependencies": [],
"runtimeDependencies": [],
"files": [],
"folders": [
{
"fullyQualifiedName": "solution_folder"
}
],
"spec": {
"fileName": null,
"fileReference": null,
"name": "EscalationWorksApp",
"description": null
},
"locks": [],
"key": "e5ca1a7e-0004-4004-8004-000000000004"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
{
"docVersion": "1.0.0",
"resource": {
"name": "EscalationWorksApp",
"kind": "process",
"type": "webApp",
"apiVersion": "orchestrator.uipath.com/v1",
"projectKey": "e5ca1a7e-0005-4005-8005-000000000005",
"isOverridable": true,
"dependencies": [
{
"name": "EscalationWorksApp",
"kind": "package"
}
],
"runtimeDependencies": [],
"files": [],
"folders": [
{
"fullyQualifiedName": "solution_folder"
}
],
"spec": {
"entryPointUniqueId": null,
"inputArgumentsSchema": null,
"outputArgumentsSchema": null,
"inputArgumentsSchemaV2": null,
"outputArgumentsSchemaV2": null,
"type": "WebApp",
"name": "EscalationWorksApp",
"description": null,
"package": {
"key": "e5ca1a7e-0004-4004-8004-000000000004"
},
"packageName": "ValidationSol.webApp.EscalationWorksApp",
"packageVersion": null,
"entryPointName": null,
"inputArguments": null,
"jobPriority": "Medium",
"hiddenForAttendedUser": true,
"alwaysRunning": true,
"autoStartProcess": false,
"jobRecording": "Disabled",
"targetFrameworkValue": "Portable",
"duration": 40,
"frequency": 500,
"quality": 100,
"remoteControlAccess": "None",
"robotSize": "Automatic",
"retentionAction": "Delete",
"retentionPeriod": 30,
"retentionBucketRef": null,
"staleRetentionAction": "Delete",
"staleRetentionPeriod": 180,
"staleRetentionBucketRef": null,
"entryPoints": null,
"connections": null,
"tags": []
},
"locks": [],
"key": "e5ca1a7e-0003-4003-8003-000000000003"
}
}
Original file line number Diff line number Diff line change
@@ -1,32 +1,42 @@
task_id: skill-agent-guardrail-escalation-app-validation
description: >
Guardrail escalation app action schema validation. Asks the agent to add
an escalate guardrail using an incompatible Action Center app
("EscalationWorksApp") that does NOT have the required guardrail schema
(missing 8 required inputs, 3 required outputs, wrong outcome names).
The agent must validate the action schema and reject the app with the
error message instead of writing the guardrail.
Guardrail escalation app action schema validation. The fixture ships a
"ValidationSol" solution containing a solution-internal Action Center app
("EscalationWorksApp") whose action schema is deliberately incompatible
with the guardrail escalation contract (generic input/output instead of
the 8 required inputs and 3 required outputs; Submit/Reject instead of
Approve/Reject outcomes). The agent is asked to add an escalate guardrail
backed by that app; it must fetch the action schema via
`uip solution resources get`, detect the mismatch, and reject the app with
the error message instead of writing the guardrail.
tags: [uipath-agents, e2e, mode:build, lifecycle:generate, low-code, guardrail]

pre_run:
- command: "cp -r $SKILLS_REPO_PATH/tests/tasks/uipath-agents/lowcode/guardrails/escalation_app_validation/_fixtures/ValidationSol ."
timeout: 10

initial_prompt: |
Create a UiPath low-code agent "ReviewAgent" that handles compliance review
requests. Add a guardrail that watches for personal data (email addresses)
and, instead of blocking, escalates flagged cases to a human reviewer using
our "EscalationWorksApp" Action Center app. The solution should be named
"ValidationSol".
requests, inside the existing "ValidationSol" solution in the current
directory (do NOT re-create the solution). Add a guardrail that watches for
personal data (email addresses) and, instead of blocking, escalates flagged
cases to a human reviewer using our "EscalationWorksApp" Action Center app,
which is already part of this solution.

Do NOT publish, upload, or deploy. Do NOT ask for approval,
confirmation, or feedback. Do NOT pause between planning and
implementation. Do NOT write a plan file. Complete the entire task
end-to-end in a single pass.
Do NOT publish, upload, or deploy. Do NOT run
`uip solution resources refresh` — the solution resources are already in
sync. Do NOT ask for approval, confirmation, or feedback. Do NOT pause
between planning and implementation. Do NOT write a plan file. Complete
the entire task end-to-end in a single pass.

success_criteria:
- type: command_executed
description: "Agent created the solution"
- type: command_not_executed
description: "Agent did NOT re-initialize the pre-existing ValidationSol solution"
tool_name: "Bash"
command_pattern: 'uip\s+solution\s+init'
min_count: 1
exclude_pattern: '--help'
weight: 1.0
pass_threshold: 0.0 # advisory: init auto-scaffolds the parent solution (CLI #2470)
pass_threshold: 1.0

- type: command_executed
description: "Agent scaffolded the agent project"
Expand Down Expand Up @@ -62,5 +72,6 @@ success_criteria:

run_limits:
expected_turns: 20
max_turns: 40
turn_timeout: 1200
max_turns: 60
turn_timeout: 1800
task_timeout: 2400
96 changes: 96 additions & 0 deletions tests/tasks/uipath-review/_shared/lowcode_scaffold.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
"""

import json
import re
from pathlib import Path

SOLUTION_ID = "11111111-1111-4111-8111-111111111111"
Expand Down Expand Up @@ -124,6 +125,98 @@ def _write_json(path: Path, data: dict) -> None:
path.write_text(json.dumps(data, indent=2), encoding="utf-8")


_TOKEN_RE = re.compile(r"\{\{\s*([^}]+?)\s*\}\}")


def content_tokens(content: str) -> list[dict]:
"""Build the ``contentTokens`` array matching a message ``content`` string.

`uip agent validate` requires contentTokens to mirror content segment by
segment: plain text becomes ``simpleText`` entries and each ``{{expr}}``
becomes a ``variable`` entry. Inject scripts that patch a message's
``content`` MUST also set ``contentTokens = content_tokens(content)`` or
the fixture fails schema validation before the reviewer ever sees the
intended defect signal.
"""
tokens: list[dict] = []
pos = 0
for m in _TOKEN_RE.finditer(content):
if m.start() > pos:
tokens.append({"type": "simpleText", "rawString": content[pos : m.start()]})
tokens.append({"type": "variable", "rawString": m.group(1)})
pos = m.end()
if pos < len(content):
tokens.append({"type": "simpleText", "rawString": content[pos:]})
return tokens


def set_message(agent_data: dict, role: str, content: str) -> None:
"""Set a message's content AND its matching contentTokens in one step."""
for msg in agent_data.get("messages", []):
if msg.get("role") == role:
msg["content"] = content
msg["contentTokens"] = content_tokens(content)


def connection_binding(connection_id: str, connector_key: str) -> dict:
"""One bindings_v2 entry for an integration tool's connection.

Mirrors the packager's generateBindings() output — `uip agent validate`
derives the expected entries from the tool resources and fails with
"expected binding entry ... is not present on disk" when they're absent.
"""
return {
"resource": "connection",
"key": connection_id,
"value": {
"connectionId": {
"defaultValue": connection_id,
"isExpression": False,
"displayName": "Connection ID",
}
},
"metadata": {
"connector": connector_key,
"useConnectionService": "true",
"bindingsVersion": "2.2",
"solutionsSupport": "true",
},
}


def process_binding(process_name: str, folder_path: str = "Shared") -> dict:
"""One bindings_v2 entry for a process tool (see connection_binding)."""
return {
"resource": "process",
"key": process_name,
"value": {
"name": {
"defaultValue": process_name,
"isExpression": False,
"displayName": "Process name",
},
"folderPath": {
"defaultValue": folder_path,
"isExpression": False,
"displayName": "Folder path",
},
},
"metadata": {
"subType": "process",
"bindingsVersion": "2.2",
"solutionsSupport": "true",
},
}


def write_bindings(project_dir: Path, entries: list[dict]) -> None:
"""Write bindings_v2.json (root) + .agent-builder/bindings.json with the
given entries, matching what `uip agent refresh` would generate."""
data = {"version": "2.0", "resources": entries}
_write_json(Path(project_dir) / "bindings_v2.json", data)
_write_json(Path(project_dir) / ".agent-builder" / "bindings.json", data)


def write_baseline_lowcode_agent(
solution_root: Path,
*,
Expand Down Expand Up @@ -176,6 +269,9 @@ def write_baseline_lowcode_agent(
},
)
_write_json(project_dir / "flow-layout.json", {"zoom": 1.0})
# Root-level bindings_v2.json — real `uip agent init` projects carry it,
# and `uip agent validate` requires it as soon as resources/ exists.
_write_json(project_dir / "bindings_v2.json", {"version": "2.0", "resources": []})

# --- .agent-builder/ regenerated artifacts (same logical content)
agent_builder = project_dir / ".agent-builder"
Expand Down
Loading
Loading