Skip to content

feat: add disambiguation check skill - #157

Open
Karima (KarimaKT) wants to merge 1 commit into
microsoft:mainfrom
KarimaKT:feature/disambiguation-skill
Open

feat: add disambiguation check skill#157
Karima (KarimaKT) wants to merge 1 commit into
microsoft:mainfrom
KarimaKT:feature/disambiguation-skill

Conversation

@KarimaKT

Copy link
Copy Markdown

What

Adds a standalone disambiguation check skill (\skills/disambiguation/SKILL.md) that scans all routable surfaces in a Copilot Studio agent for routing collisions before pushing.

Why

When agents grow in topics, tools, and child agents, overlapping names, descriptions, and output schemas cause the orchestrator to deliberate longer (adding latency) or pick the wrong surface. These collisions are invisible to YAML validation — the files are structurally valid, the routing is just ambiguous.

What it checks

Check Severity Example
Combined definition collisions HIGH Two topics both output \�mployeeId\ — user asks 'What's the employee ID?' and both match
Instruction interference HIGH Instructions describe full ticket-creation behavior AND a CreateTicket tool exists — orchestrator short-circuits routing
Cross-scope collisions HIGH Parent has a BenefitsFAQ topic AND a BenefitsExpert child agent
Output schema collisions MEDIUM Same output field name in multiple surfaces
Tool ↔ Topic collisions MEDIUM Topic says 'Creates IT tickets' and a CreateTicket tool also creates tickets

How it works

  • Runs per scope — parent agent and each child agent checked independently
  • Only checks active surfaces (those with \modelDescription\ or \�eginDialog.description)
  • Reports findings with sample user queries that trigger each collision and suggested fixes
  • Includes a Quick Scan section for lightweight inline checks from other skills

Integration

  • Added a pre-push disambiguation suggestion to \manage-agent\ skill — when routing-relevant changes are detected, the user is prompted (not blocked) to run the check before pushing.

Files changed

  • \skills/disambiguation/SKILL.md\ — new standalone skill
  • \skills/manage-agent/SKILL.md\ — added disambiguation suggestion before push

Adds a standalone skill that scans all routable surfaces in a Copilot Studio
agent for routing collisions by evaluating the full AI definition (name,
description, inputs/outputs) across topics, tools, and child agents.

Checks for:
- Combined definition collisions (shared outputs, overlapping descriptions)
- Instruction interference (instructions short-circuiting tool routing)
- Cross-scope collisions (parent topics vs child agent descriptions)
- Output schema collisions
- Tool vs topic collisions

Runs per-scope (parent + each child agent independently) and only checks
active surfaces (those with modelDescription).

Also adds a pre-push disambiguation suggestion to manage-agent skill.
@KarimaKT
Karima (KarimaKT) force-pushed the feature/disambiguation-skill branch from f19a388 to b6d5bb7 Compare May 1, 2026 04:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant