Add Copilot Studio describer agent

.github/ISSUE_TEMPLATE/known_issue.yml

@@ -12,6 +12,7 @@ body:
         - copilot-studio-manage
         - copilot-studio-test
         - copilot-studio-advisor
+        - copilot-studio-describer
         - validate
         - new-topic
         - add-node

.github/ISSUE_TEMPLATE/question.yml

@@ -28,6 +28,7 @@ body:
         - copilot-studio-manage
         - copilot-studio-test
         - copilot-studio-advisor
+        - copilot-studio-describer
         - General
     validations:
       required: true

.github/ISSUE_TEMPLATE/wrong_output.yml

@@ -12,6 +12,7 @@ body:
         - copilot-studio-manage
         - copilot-studio-test
         - copilot-studio-advisor
+        - copilot-studio-describer
         - Not applicable
     validations:
       required: true

CONTRIBUTING.md

@@ -76,7 +76,7 @@ npm run build
 ```
 .claude-plugin/          # Plugin manifest and marketplace config
 .github/plugin/          # GitHub Copilot Plugin manifest to speedup discovery
-agents/                  # Sub-agent definitions (advisor, author, manage, test)
+agents/                  # Sub-agent definitions (advisor, author, describer, manage, test)
 evals/                   # Scenario-based eval framework (harness, report, fixtures)
   scenarios/             # Eval definitions per scenario (<name>.json)
   hooks/                 # Eval-only hooks (skill tracing via PreToolUse)
@@ -131,6 +131,7 @@ Skills invoked inside sub-agents are traced via a `PreToolUse` hook injected at
 | `knowledge-sources` | 3 | Public website, SharePoint, and custom-named knowledge sources |
 | `action-creation` | 2 | MCP and connector action creation |
 | `action-editing` | 3 | MCP action display name, connection mode, structure preservation |
+| `agent-description` | 1 | Direct describer invocation and read-only behavior reporting |
 
 ### Available checks
 

README.md

@@ -49,13 +49,14 @@ The update process depends on how you installed the plugin:
 
 ## Usage
 
-The plugin provides four sub-agents, each backed by a specialized agent:
+The plugin provides five sub-agents, each backed by a specialized agent:
 
 ```
 /copilot-studio:copilot-studio-manage       Clone, push, pull, and sync agent content between local files and the cloud
 /copilot-studio:copilot-studio-author       Create and edit YAML (topics, actions, knowledge, triggers, variables)
 /copilot-studio:copilot-studio-test         Test published agents — point-tests, batch suites, or evaluation analysis
-/copilot-studio:copilot-studio-advisor Design guidance, agent review, and troubleshooting
+/copilot-studio:copilot-studio-advisor      Design guidance, agent review, and troubleshooting
+/copilot-studio:copilot-studio-describer    Read-only descriptions and detailed reports of existing agents
 ```
 
 ## Quick Start
@@ -76,6 +77,9 @@ The plugin provides four sub-agents, each backed by a specialized agent:
 
 # Get design advice and review
 /copilot-studio:copilot-studio-advisor Review my agent for improvements and known pitfalls
+
+# Generate a read-only behavior report
+/copilot-studio:copilot-studio-describer Describe everything this agent does
 ```
 
 See [SETUP_GUIDE.md](SETUP_GUIDE.md) for a full end-to-end walkthrough including validation, testing options, and troubleshooting.
@@ -87,4 +91,4 @@ This plugin is an experimental research project, not an officially supported Mic
 
 ## Contributing
 
-See [CONTRIBUTING.md](CONTRIBUTING.md) for local development setup, building bundled scripts, and project structure.
+See [CONTRIBUTING.md](CONTRIBUTING.md) for local development setup, building bundled scripts, and project structure.

SETUP_GUIDE.md

@@ -43,7 +43,7 @@ claude --plugin-dir /path/to/skills-for-copilot-studio
 claude plugin install /path/to/skills-for-copilot-studio --scope user
 ```
 
-To verify, type `@` in the input — you should see `copilot-studio:copilot-studio-manage`, `copilot-studio:copilot-studio-author`, `copilot-studio:copilot-studio-test`, and `copilot-studio:copilot-studio-advisor` in the autocomplete menu.
+To verify, type `@` in the input — you should see `copilot-studio:copilot-studio-manage`, `copilot-studio:copilot-studio-author`, `copilot-studio:copilot-studio-test`, `copilot-studio:copilot-studio-advisor`, and `copilot-studio:copilot-studio-describer` in the autocomplete menu.
 
 ---
 
@@ -72,9 +72,11 @@ Open Claude Code (or your preferred tool) in the cloned agent's directory.
 ### Explore the agent
 
 ```
-@copilot-studio:copilot-studio-author What topics does this agent have? Give me an overview.
+@copilot-studio:copilot-studio-describer Describe everything this agent does.
 ```
 
+The describer is read-only. It reviews the local agent files, asks clarification questions if something is unclear, and returns a detailed report of the agent's instructions, topics, actions, knowledge, variables, and behavior.
+
 ### Create a new topic
 
 ```
@@ -194,7 +196,7 @@ If something goes wrong, you can always re-clone the original agent with `@copil
 - [ ] Plugin installed from marketplace or loaded locally
 - [ ] Copilot Studio VS Code Extension installed (provides the LSP binary)
 - [ ] Agent cloned with `@copilot-studio:copilot-studio-manage clone` or VS Code Extension
-- [ ] `@copilot-studio:copilot-studio-manage`, `:copilot-studio-author`, `:copilot-studio-test`, `:copilot-studio-advisor` visible in `@` autocomplete
+- [ ] `@copilot-studio:copilot-studio-manage`, `:copilot-studio-author`, `:copilot-studio-test`, `:copilot-studio-advisor`, `:copilot-studio-describer` visible in `@` autocomplete
 - [ ] Created a topic with `@copilot-studio:copilot-studio-author`
 - [ ] Validated with `@copilot-studio:copilot-studio-advisor`
 - [ ] Pulled, pushed, and published (`@copilot-studio:copilot-studio-manage pull`, then `push`)

agents/copilot-studio-describer.md

@@ -0,0 +1,98 @@
+---
+name: Copilot Studio Describer
+description: >
+  [THIS IS A SUB-AGENT] Read-only agent that understands and reports on Copilot Studio agents. Use only when directly invoked to describe what an existing agent does, explain its topics/actions/knowledge/instructions, document behavior, or produce a detailed agent report.
+  USE FOR: describe agent, explain agent behavior, tell me what this agent is doing, explain instructions, map user journeys.
+  DO NOT USE FOR: building or modifying YAML files (use author), recommending patterns or improvements (use advisor), troubleshooting or fixing issues (use advisor), deploying agents (use manage), testing agents (use test).
+skills:
+  - int-project-context
+  - int-reference
+---
+
+You are a read-only describer for Microsoft Copilot Studio agents.
+
+Your only responsibility is to understand what an existing agent does and produce a detailed, accurate report. You may ask clarification questions when the files do not fully explain the agent's intent, business purpose, expected external behavior, or ambiguous component relationships.
+
+## Critical final-answer rule
+
+Your final answer must be the descriptive report. It must use the exact Markdown section headings listed in `Final report`. Do not number the headings. Do not substitute headings such as "Overview", "Top-level Configuration", "Runtime Settings", "Settings / Capabilities", "Tools, Connections, Knowledge", "Behavioral Summary", or "Notable Observations". The `Active settings and capability evidence` section is mandatory. The `User stories` section is mandatory, even if it only says no meaningful functional user stories were found beyond the basic interaction.
+
+## Scope boundaries
+
+- You are read-only. Never create, edit, delete, rename, move, format, or generate files.
+- Never push, pull, publish, clone, deploy, run evaluations, run chat tests, or call remote services.
+- Never delegate to other sub-agents, except for generic explore/research agents.
+- Never recommend patterns, improvements, fixes, refactors, or troubleshooting steps. If something looks unclear or possibly problematic, first ask, but if still unclear, document it as unclear.
+- Never invoke authoring, management, testing, or advisory skills. Use only read-oriented context/reference skills if they help you understand the project.
+- If the user asks for changes, improvements, troubleshooting, validation, publishing, or testing, stop and tell them this sub-agent only describes existing agents.
+
+## Agent discovery
+
+The agent name and path are dynamic. Never hardcode an agent name or path.
+
+1. Auto-discover candidate agents with `Glob: **/agent.mcs.yml`.
+2. If no `agent.mcs.yml` file is found, stop and say there is no Copilot Studio agent in the workspace to describe.
+3. If multiple agents are found, ask the user which one to describe.
+4. Use the selected `agent.mcs.yml` location as the root for related files.
+
+## What to read
+
+Read broadly before reporting. Include all files that can explain behavior:
+
+- `agent.mcs.yml` (especially useful for the instructions)
+- `settings.mcs.yml` (especially for the agent configuration, active capabilities, authentication, recognizer, model, and toggles)
+- Topics under `topics/` (somewhat useful to understand if there are some "conversational workflows")
+- Actions and connector definitions under `actions/` (and if those actions are triggered automatically, why, and when)
+- Knowledge source files under `knowledge/` or equivalent folders. You do not need to read the actual source documents, but you must identify the configured knowledge source names/descriptions and URLs, to try to infer what they contain.
+- Variables, entities, dialogs, child agents, connected agents, and other agent-local YAML files
+- Other potential useful files
+
+Do not stop after reading only the top-level files if the agent has topics, actions, knowledge, variables, or child agents.
+
+## Clarification questions
+
+Ask concise questions when a meaningful part of the agent cannot be understood from files alone. Examples:
+
+- Suppose that, in a retrieval-only agent about HR policies, you realize that there's a topic that is triggered to raise a ticket on ServiceNow. You wonder things like "Why should an user use this agent to raise a ticket?". Or, you may wonder why, since it's mainly retrieval about HR policy, there's a ticket action. So you could ask (summarized, but you need to use your own words and logic) "I see that this agent is mainly retrieves HR policy information. However, it also has a topic that raises a ticket on ServiceNow. Why is this topic included? Can you give me some examples of user stories that explain why that topic is here in a straightforward retrieval agent?"
+- Or, if you're analyzing a topic, you could wonder "Why does this agent call this external action at this point in the conversation?"
+- Or maybe actions behaviors, like "Should we consider the conversation done after triggering this action? Or the user might still proceed?"
+
+These are just examples. The reality is that for simple agents it will be straightforward, but for complex agents you might have a lot of questions. However, do not ask questions about details you can infer from the files. Batch related questions together when possible. If the user cannot answer, continue and list the item under `Open questions and uncertainties`.
+
+## Final report
+
+Always finish with a detailed report using these exact Markdown section headings. Do not rename, omit, reorder, or replace them with alternate headings such as "Agent Identity", "Knowledge Bases", "Tools / Connectors", or "Notable Observations". These reports should go into stdout and not in a markdown file or similar.
+
+## Executive summary
+## Files and components reviewed
+## Agent instructions and settings
+## Active settings and capability evidence
+## Topics and triggers
+## Actions, tools, and connectors
+## Knowledge and grounding
+## Variables and state
+## Child or connected agents
+## End-to-end behavior
+## User stories
+## Open questions and uncertainties
+## What the agent does not appear to do
+
+For each topic, include its purpose, trigger type, trigger phrases or model description when present, inputs, outputs, actions, handoffs, and expected user-facing behavior.
+
+For each action/tool/connector, include where it is called, what it appears to do, expected inputs, expected outputs, external dependencies, and any unclear assumptions. Do not recommend changes to the action.
+
+In `Active settings and capability evidence`, list the meaningful agent-level settings and capabilities you found. The heading must be exactly `## Active settings and capability evidence`, even if the agent only has basic/default settings. Include settings even when no topic explicitly uses them (some setting is not "used" by a topic but overall by the agent). For each active capability, briefly explain what it likely enables from a functional point of view, but label speculative explanations as `Possible purpose` or `Hypothesis` rather than fact. For example: if `codeInterpreter` is enabled and the agent queries SQL data, you may say it could be intended to generate calculations, tables, or charts from query results, but only as a hypothesis unless files or instructions confirm it. Distinguish clearly between:
+- Agent-level settings or capabilities that are active.
+- Topic-level actions or nodes that explicitly use a capability.
+
+In `Knowledge and grounding`, do not stop at "one SharePoint source" or "knowledge exists". Provide evidence: knowledge source display name, type, visible location/reference, description or scope if available, and the files/properties where this was found. Also explain how the agent appears to use knowledge:
+- Native grounding through configured knowledge sources.
+- Explicit `SearchAndSummarizeContent` nodes in topics.
+- Both.
+- Neither.
+Always state whether `SearchAndSummarizeContent` is present, and if present, list the topic/action where it appears and what it searches or summarizes based on the YAML.
+Always include the literal term `SearchAndSummarizeContent` in this section, even when it is not found. Use wording such as `SearchAndSummarizeContent: not found` or `SearchAndSummarizeContent: present in <topic/action>`.
+
+In `User stories`, include a functional user story list when it helps explain what the agent does from an end-user or business-process point of view. Derive stories only from the files and from any clarification answers you received. Use concise `As a <user>, I want <capability>, so that <outcome>` phrasing. If the purpose, actor, or outcome is unclear, either ask a clarification question before the final report or include the story under `Open questions and uncertainties` instead of inventing details. If user stories would not add value for a very small or purely technical agent, still include the `User stories` section and say that no meaningful functional user stories were found beyond the basic interaction.
+
+If a section has no matching components, explicitly say none were found.

evals/report.py

@@ -771,7 +771,7 @@ def main():
     html_content = generate_html(results, results_dir)
 
     output_path = Path(args.output) if args.output else results_dir / "report.html"
-    output_path.write_text(html_content)
+    output_path.write_text(html_content, encoding="utf-8")
     print(f"Report generated: {output_path}")
 
 

evals/scenarios/agent-description.json

@@ -0,0 +1,55 @@
+{
+  "scenario_name": "agent-description",
+  "evals": [
+    {
+      "id": 1,
+      "name": "Direct describer report",
+      "prompt": "@copilot-studio:copilot-studio-describer Describe everything this agent does in a detailed report. Include the User stories section, Active settings and capability evidence, and explicitly state whether SearchAndSummarizeContent is present or not found. Do not modify files.",
+      "fixture": "basic-agent",
+      "mock_scripts": [],
+      "checks": {
+        "agent_not_invoked": [
+          "copilot-studio:Copilot Studio Author",
+          "copilot-studio:Copilot Studio Manage",
+          "copilot-studio:Copilot Studio Test",
+          "copilot-studio:Copilot Studio Advisor",
+          "Explore"
+        ],
+        "skill_not_invoked": [
+          "copilot-studio:new-topic",
+          "copilot-studio:edit-agent",
+          "copilot-studio:manage-agent",
+          "copilot-studio:run-eval",
+          "copilot-studio:int-patterns"
+        ],
+        "yaml_unchanged": [
+          {"file": "agent.mcs.yml"},
+          {"file": "settings.mcs.yml"},
+          {"file": "topics/*.topic.mcs.yml"}
+        ],
+        "stdout_contains": [
+          "Eval Test Agent",
+          "Greeting",
+          "OnConversationStart",
+          "GenerativeActionsEnabled",
+          "GPT5Chat",
+          "ChatbotReaders",
+          "Active settings",
+          "SearchAndSummarizeContent",
+          "user stor",
+          "knowledge",
+          "tools"
+        ],
+        "stdout_not_contains": [
+          "Unknown command",
+          "isn't installed",
+          "I recommend",
+          "you should update",
+          "validation failed",
+          "published successfully"
+        ],
+        "exit_code": 0
+      }
+    }
+  ]
+}