Adversarial red-team arena for UiPath agents. Go safe or go home.
Submission for UiPath AgentHack 2026, Track 3 (UiPath Test Cloud).
The complete solution is built and deployed in the UiPath Labs (staging) org provisioned for this submission:
Tenant DefaultTenant. What is live there:
- Solution (Maestro Case
FightArena+ Maestro FlowRoundOrchestrator+ Agent Builder agentsMetroBankCSRandRefereeAgent) is deployed and activated in the Orchestrator folderShared/Gauntlet. - Coded App
gauntletappis deployed and live at https://hackathon26_038.staging.uipath.host/gauntletapp/. Open it from the Apps menu in the portal.
UiPath Agent Evaluations tell you whether your agent passes the tests you wrote. They don't tell you what happens when a real attacker shows up with a prompt you never imagined.
Gauntlet closes that gap. A Red Coach agent (powered by a frontier LLM) plays an adversary against any UiPath Agent, Maestro Flow, or external target. When the existing attack corpus stops scoring hits, the Coach invents new attack personas mid-fight. Every winning attack is automatically persisted to UiPath Test Manager as a permanent regression test. Every failing fight opens an Action Center task with a concrete fix recommendation written by a second LLM-driven agent.
The result is an agentic test factory that grows the regression suite by itself, with every fight double-tagged against OWASP LLM Top-10 and MITRE ATLAS so the coverage matrix is something a compliance officer can actually read.
Agent safety today is mostly vibes-based. Teams ship an agent, write a few eval rubrics, and hope nothing breaks. Gauntlet replaces "hope" with a continuous adversarial loop:
| Today | With Gauntlet |
|---|---|
| Static eval rubrics that humans wrote | Self-play that invents new evals |
| "Did we test prompt injection?" → maybe? | OWASP / MITRE coverage matrix on every release |
| Bug found in prod → manual repro → manual test case | Auto-populated Test Manager regression entry |
| Failure → developer manually patches prompt | Fix Recommender drafts the patch, opens Action Center task |
Tenant status at a glance. Robustness score, top critical findings, recent fights, quick-action tiles, mini coverage heatmap.
The Coach picks the persona × posture combination with the highest expected reward (Thompson sampling), or asks the LLM to invent a new one. The risk-weighted coverage table shows exactly where Blue is weakest.
Single fight or batch of up to 10. Pick the Red persona, the scenario, and the Blue policy posture (Standard / Lenient / Naive / External LangGraph target).
Open any losing fight, read the Fix Recommender's patch proposal (root cause, suggested system-prompt patch, regression test), file it to UiPath Action Center for human approval.
Persona × posture heatmap, severity trend, fix-recommender pipeline status, per-day adversarial call volume.
Every fight is double-tagged. The Audit view answers "what kinds of attacks have we tested for, and how did Blue do?" in a form a compliance officer can read.
Full transcripts, referee verdicts, fix-proposal links. Click any row to drill into the full conversation.
| Component | Role | Where it lives |
|---|---|---|
| UiPath Automation Cloud | Execution and orchestration plane | UiPath Labs tenant: staging.uipath.com/hackathon26_038/DefaultTenant |
Maestro Case (FightArena) |
Long-running fight orchestration; rounds-as-tasks | uipath/gauntlet/FightArena/ |
Maestro Flow (RoundOrchestrator) |
Single round end-to-end: Red attack → Blue response → judge → score | uipath/gauntlet/RoundOrchestrator/ |
Agent Builder (MetroBankCSR) |
Blue target. Reference customer-service agent (system under test) | uipath/gauntlet/MetroBankCSR/ |
Agent Builder (RefereeAgent) |
Judge. Scores each round against the rubric | uipath/gauntlet/RefereeAgent/ |
| Coded Agents (Python, LangGraph) | Red Coach (gauntlet coach), Fix Recommender (gauntlet fix) |
src/gauntlet/ |
Coded App (gauntletapp) |
Operator surface: Threat Dashboard, Coach Lab, Fix Lab, Analytics, Audit, Logs | uipath/gauntlet-console/ |
| Test Manager | Persistent regression set, auto-populated by Coach on winning attacks | Imported via scripts/import_runs_to_test_manager.py |
| Action Center | Human-in-the-loop fix approval; tasks opened from Fix Lab | Live API call from browser via @uipath/uipath-typescript |
UiPath TypeScript SDK (@uipath/uipath-typescript) |
Browser-side calls from Coded App to Maestro instances, Test Manager, Action Center | uipath/gauntlet-console/src/lib/uipath.ts |
uip CLI (latest) |
Login, pack, publish, deploy across all artifacts | Used in setup steps below |
A combination. Gauntlet uses both coded agents and low-code agents from Agent Builder. It is deliberately polyglot to prove the arena is framework-neutral.
-
Coded Agents (Python, LangGraph + a frontier LLM):
gauntlet coach. Adversarial Red Coach with risk-weighted persona selection (src/gauntlet/coach.py)gauntlet fix. Fix Recommender (src/gauntlet/fix.py)gauntlet referee(local mirror). Judge logic (src/gauntlet/referee.py)- External Blue target reference implementation in LangGraph (src/gauntlet/blue_team_external.py)
-
Low-code Agents (UiPath Agent Builder):
MetroBankCSR. The Blue target (system under test)RefereeAgent. The in-tenant judge
-
Low-code Orchestration (UiPath Maestro):
FightArena. Maestro Case for long-running fight lifecycleRoundOrchestrator. Maestro Flow for per-round execution
-
Coded App (TypeScript + React):
gauntletapp. The operator surface that ties them all together
- UiPath Automation Cloud tenant with Test Cloud, Maestro, Agent Builder, Action Center, Coded Apps, and Test Manager enabled
- Node.js 20+
- Python 3.11+
uipCLI (latest; install withnpm i -g @uipath/cli). The solutionpack/deploy runcommands below need a recent build.- LLM API key (frontier-model access)
git clone https://github.com/tdries/uipath-hackathon-gauntlet.git
cd uipath-hackathon-gauntlet
cp .env.example .env
# Fill in ANTHROPIC_API_KEY (required)Only ANTHROPIC_API_KEY is required to run Gauntlet. The other keys in .env.example are optional: OPENAI_API_KEY powers the optional multi-model bake-off, and HEYGEN_API_KEY / ELEVENLABS_API_KEY were used only for demo-video production, so leave them blank. The UIPATH_* fields are filled automatically by uip login in the next step.
uip login
# Follow the browser flow. The CLI writes credentials to ~/.uipath/credentials.python -m venv .venv
source .venv/bin/activate
pip install -e .This exposes the gauntlet CLI. The fight / coach / fix commands make live LLM calls, so they need ANTHROPIC_API_KEY set in .env:
gauntlet --help # list every command
gauntlet personas # list the 8 red-team personas
gauntlet scenarios # list the 15 fight scenarios
# Run one live fight: gauntlet fight <persona> --scenario <scenario>
# --blue-mode is one of: standard | lenient | naive | external
gauntlet fight fake-ceo --scenario acme-roofing-bec --blue-mode naive
# Let the Coach read the corpus and invent a brand-new attack persona, then fight it:
gauntlet coach --auto-fight
# Draft a fix proposal for a losing fight (pass any run file from runs/):
gauntlet fix runs/20260516-163715-fight-d6c8ae26-fake-ceo-lenient.jsonNo API key handy? The repo already ships 75 recorded fights (9 with fix proposals) under runs/. Run
gauntlet leaderboardto summarize them, or open the Coded App offline (see Local development) to browse every screen without spending a token.
The full UiPath solution (Maestro Case + Flow + both Agent Builder agents) lives in uipath/gauntlet/. That folder holds the solution manifest (gauntlet.uipx) and each project. You pack the folder into a deployable .zip, publish the .zip, then deploy it:
cd uipath/gauntlet
# 1. Pack the solution folder (it contains gauntlet.uipx) into a .zip under ./dist
uip solution pack . ./dist -n gauntlet -v 1.0.0
# pack prints the exact .zip path it wrote; use that path in the next step
# 2. Publish the packed .zip to the UiPath solution feed
uip solution publish ./dist/gauntlet_1.0.0.zip
# 3. Deploy it. Creates an Orchestrator folder, provisions resources, and activates
uip solution deploy run --name gauntlet \
--package-name gauntlet --package-version 1.0.0 \
--folder-name Gauntlet --parent-folder-path Shared
# (find published package names/versions with: uip solution packages list)This deploys:
FightArena(Maestro Case)RoundOrchestrator(Maestro Flow)MetroBankCSR(Agent Builder agent, the Blue target)RefereeAgent(Agent Builder agent, the judge)
cd uipath/gauntlet-console
npm install
npm run build # produces dist/
uip codedapp pack dist -n gauntletapp -v 1.0.0 # pack dist/ into a .nupkg
uip codedapp publish --name gauntletapp # register the package in your tenant
uip codedapp deploy --name gauntletapp # deploy / upgrade the appThis publishes and deploys gauntletapp to your tenant. Open it from the Apps tab in UiPath Automation Cloud.
Tenant note: the app's
uipath.jsonis configured for the UiPath Labs orghackathon26_038/ tenantDefaultTenant(base URLhttps://staging.api.uipath.com). Update uipath/gauntlet-console/uipath.json (orgName,tenantName,baseUrl, andclientIdfor your own External Application) before publishing to your own tenant.
Once everything is deployed:
- Open
gauntletappin your UiPath tenant. - On the Dashboard, click Run a fight. Pick a Red persona (e.g.
aggressive-lawyer), a scenario, and the Blue posture. Click Replay this fight. - Open Coach Lab from the sidebar. Paste your LLM API key into the modal (stays in
sessionStorage, never sent to Gauntlet servers). Click Run live (add key). The Coach invents a new attack persona in front of you. - On the Defend tab, open any losing fight. The Fix Recommender has already drafted a patch. Click File to Action Center to create a real Action Center task in your tenant.
- Browse the Audit tab for OWASP LLM Top-10 / MITRE ATLAS coverage, and Logs for the full corpus of 75 recorded fights.
To run the Coded App locally against your tenant (without redeploying):
cd uipath/gauntlet-console
npm run dev
# → http://localhost:5173The app falls back to the bundled offline corpus when not authenticated, so judges can also inspect every screen without needing a tenant.
Gauntlet is the adversarial complement to UiPath Agent Evaluations. Where Agent Evaluations score your agent against a static rubric, Gauntlet scores it against an opponent that's actively trying to break it, and turns every successful attack into a permanent test case.
Built end-to-end with an agentic coding tool. Thematically right: an agent building a tool to keep agents honest.
.
├── src/gauntlet/ Python package (Coach, Fix, Referee, Runner, CLI)
├── uipath/
│ ├── gauntlet/ UiPath solution
│ │ ├── FightArena/ Maestro Case
│ │ ├── RoundOrchestrator/ Maestro Flow
│ │ ├── MetroBankCSR/ Agent Builder agent (Blue target)
│ │ └── RefereeAgent/ Agent Builder agent (judge)
│ └── gauntlet-console/ React Coded App (`gauntletapp`)
├── personas/ Red attack persona library (8 YAML personas)
├── scenarios/ Fight scenarios (15 YAML scenarios)
├── scripts/ Test Manager import helper
└── docs/
├── ARCHITECTURE.md Detailed system architecture
├── PERSONAS.md Red-team persona reference
└── screenshots/ In-app screenshots referenced above
Licensed under the MIT License, one of the two licenses (MIT or Apache 2.0) permitted by the UiPath AgentHack 2026 rules. See LICENSE.
Where each required element lives, so judges can verify at a glance:
| Requirement | Status | Section |
|---|---|---|
| Public GitHub repo with all project files | ✅ | github.com/tdries/uipath-hackathon-gauntlet |
| What the project does | ✅ | Project description |
| Which UiPath components it uses | ✅ | UiPath components used |
| Coded agents, low-code Agent Builder, or a combination | ✅ A combination of both | Agent type |
| Prerequisites | ✅ | Prerequisites |
| Setup instructions | ✅ | Setup instructions |
| MIT or Apache 2.0 license | ✅ MIT | License |
| UiPath Labs environment URL | ✅ | staging.uipath.com/hackathon26_038 |