Add agent architecture advisor - #283
Conversation
Initial addition of the Agent Architecture Advisor skill for Copilot Studio and Azure AI Foundry. Includes agent reasoning instructions (SKILL.md), human-facing overview (README.md), metadata, Python analysis scripts (ingest_agent.py, analyze_topics.py, cost_model.py), requirements.txt, and comprehensive reference documentation. Enables agent architecture review, design, migration, and cost modeling with evidence-based recommendations.
…and detail on functionality
There was a problem hiding this comment.
Pull request overview
Adds a new submission under submissions/agent-architecture-advisor/ providing an “Agent Architecture Advisor” skill: a structured, rules-driven architecture review/design workflow (with references) plus deterministic Python helpers for ingestion, topic analysis, and cost/break-even modeling.
Changes:
- Introduces the
agent-architecture-advisorskill (SKILL.md) with multi-mode routing (Design / Review / Modernize) and an opinionated report contract. - Adds deterministic analysis scripts (
ingest_agent.py,analyze_topics.py,cost_model.py) plus an optionalrequirements.txt. - Adds a substantial set of reference materials (
references/*) and a human-facingREADME.md, plusmetadata.jsonfor gallery listing.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| submissions/agent-architecture-advisor/SKILL.md | Skill instructions: mode routing + shared reasoning core + report/output contract. |
| submissions/agent-architecture-advisor/scripts/requirements.txt | Optional dependency list (PyYAML) for higher-fidelity YAML parsing. |
| submissions/agent-architecture-advisor/scripts/ingest_agent.py | Ingests Copilot Studio/Foundry exports into a normalized JSON model for downstream analysis. |
| submissions/agent-architecture-advisor/scripts/analyze_topics.py | Deterministic topic-level measurements (collisions, orchestration graph, config checks, complexity, variables, grounding). |
| submissions/agent-architecture-advisor/scripts/cost_model.py | Deterministic cost model + break-even calculation with rate-card overrides. |
| submissions/agent-architecture-advisor/references/token-optimization.md | Advisory guidance for token/weight cost levers and how to present them. |
| submissions/agent-architecture-advisor/references/requirement-model.md | Requirement schema + elicitation/inference guidance shared across modes. |
| submissions/agent-architecture-advisor/references/reference-links.json | Allowlisted “source of truth” external links for guidance/pricing fetch protocol. |
| submissions/agent-architecture-advisor/references/rate-card.md | Human-readable explanation of the cost model’s scaffold rates/assumptions. |
| submissions/agent-architecture-advisor/references/rate-card.json | Machine-readable fallback rates/assumptions consumed by cost_model.py. |
| submissions/agent-architecture-advisor/references/platform-capabilities.md | Platform capability envelope reference used for ceiling classification. |
| submissions/agent-architecture-advisor/references/output-templates.md | Canonical report template and mode-specific adjustments. |
| submissions/agent-architecture-advisor/references/migration-classic-to-agentic.md | Reference for classic→agentic modernization (Mode C) mapping and outputs. |
| submissions/agent-architecture-advisor/references/hybrid-patterns.md | EXTEND/hybrid reference architectures and boundary contract requirements. |
| submissions/agent-architecture-advisor/references/harness-rules.md | Skill scope harness + agentic harness capability model + operating loop rules. |
| submissions/agent-architecture-advisor/references/foundry-design-templates.md | Foundry target-state spec templates for EXTEND/MIGRATE outputs. |
| submissions/agent-architecture-advisor/references/ceiling-rules.md | Ruleset for classifying findings (GOVERNANCE/CEILING/COST/DESIGN/CONFIG) and deriving verdicts. |
| submissions/agent-architecture-advisor/references/agent-quality-dimensions.md | Cross-cutting operational dimensions and their mapping to rules. |
| submissions/agent-architecture-advisor/README.md | Human-facing overview, usage positioning, prompts, and installation notes. |
| submissions/agent-architecture-advisor/metadata.json | Gallery metadata (name/description/platforms/tags/authors/version/dates). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
@microsoft-github-policy-service agree |
Explicit addition of foundry exported agent file Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated no new comments.
Suppressed comments (3)
submissions/agent-architecture-advisor/SKILL.md:83
- This section says “five batched rounds” but then lists 6 rounds (1–6). The mismatch can cause the skill to skip the operational-readiness elicitation step.
Elicit the requirement model through five batched rounds. Ask 3–5 questions per round
submissions/agent-architecture-advisor/README.md:11
- The README claims the skill can be used in “GitHub Copilot”, but this repo’s submission metadata platform list appears limited to values like “Copilot Studio”, “Cowork”, and “Scout” (and the PR metadata lists only those). This creates a support/expectations mismatch for gallery consumers.
This skill can be used across Copilot Studio, Copilot Cowork, Scout, and GitHub
Copilot. The same architecture-review reasoning applies regardless of which
host invokes it.
submissions/agent-architecture-advisor/scripts/ingest_agent.py:656
- In the .yaml/.yml ingestion path, the file is re-read and
ingest_foundry(...)is called unconditionally. This overwrites the earlier Copilot Studio topic parse (format/platform) and can also add spurious parse warnings/unreadable entries when PyYAML is absent.
text = path.read_text(encoding="utf-8", errors="replace")
ingest_foundry(text, model, path.name)
|
Thanks for putting this together, Anamika Kumari (@Annamika) ! There's real depth here and a thoughtful mapping of the process from ideation to production-ready agent building, and the questions one should ask themselves in between. My main thought here is on scope. The way we tend to frame it: a skill is a reusable procedure that earns its place on one specific job. Reading this end to end, one trigger opens onto a whole practice, review, design, diagnosis, platform and design choice, harness upgrade considerations for existing agents (in the case of MCS), and cost at scale, each with its own machinery. That is closer to an army of skills around one subject than a single procedure, as you can see with some of those specific jobs living as a standalone skill elsewhere in the gallery. I believe holding that much in one place is also what makes the parts start to disagree: harness-rules.md talk about extended agentic loop and orchestration capabilities in Copilot Studio's GHCP harness, while hybrid-patterns.md still treats some of these as a ceiling for Copilot Studio and reframes its scope differently. As much as decision trees are very helpful for users, from an architectural advisory standpoint we try to be careful with scenario>design strict patterns. Similar concern applies to the harness migration/upgrade: a fixed 1:1 mapping in a space that is moving fast, and tools like https://github.com/[microsoft/copilot-studio-plugin](https://github.com/microsoft/copilot-studio-plugin) are built to carry exactly that mapping forward (even for terminology matters, as we have moved from "migrating" from "classic" to "upgrading" from "standard harness". So rather than a list of edits, my suggestion is to let each of these procedures be the size it actually is, a focused review, a modernizer that defers to the plugin, a cost model that stands alone. Scoped that way the contradictions mostly dissolve on their own, and the strong ideas at the center get enough room. Happy to talk it through. Thank you! |
|
Hi Adriana Trujillo (@adrianatruji) Thanks for the review the scope point is fair, and it holds up against the gallery. Most of what I bundled already exists as focused skills (AI Platform Advisor, EADA, Harness Picker, Topic Blueprint, the test planners, Knowledge Readiness). One trigger over all of that is the problem you flagged. So rather than patch, I'd extract one focused skill: Copilot Studio Agent Linter - reads an exported agent and finds structural defects: trigger-phrase collisions, unreachable topics, missing fallback/escalation, variable-lifecycle issues. Deterministic, one job. I'll push the update to the same PR for review. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
submissions/agent-architecture-advisor/scripts/cost_model.py:204
foundry_cost()currently adds the “supporting uplift” ontofixed_monthly(fixed + supporting) even thoughsupportingis computed from(variable + fixed). That makes the reported fixed cost volume-dependent and undercounts Foundry per-conversation cost (sincecost_per_conversationexcludes the variable portion of the uplift). This distorts the break-even calculation and the cost comparison logic.
fixed = SEARCH_FIXED_MONTHLY + HOSTING_FIXED_MONTHLY
supporting = (variable + fixed) * SUPPORTING_UPLIFT
total = variable + fixed + supporting
return {
| if fmt == "solution_zip": | ||
| ingest_solution_zip(path, model, verbose) | ||
| elif fmt == "agent_yaml": | ||
| text = path.read_text(encoding="utf-8", errors="replace") | ||
|
|
||
| # Heuristic: Foundry agent definitions often have top-level 'model'/'model_deployment' and 'tools'. | ||
| # Prefer failing with an explicit warning over mis-parsing as a Copilot Studio topic. | ||
| looks_like_foundry = ( | ||
| re.search(r"^\s*(model|model_deployment)\s*:", text, re.M) | ||
| and re.search(r"^\s*tools\s*:", text, re.M) | ||
| ) | ||
|
|
||
| if looks_like_foundry: | ||
| ingest_foundry(text, model, path.name) | ||
| else: | ||
| model["source"]["format"] = "copilot_studio_agent_yaml" | ||
| model["agent"]["platform"] = "copilot_studio" | ||
| model["source"]["parsed_files"].append(path.name) | ||
| parse_topic_yaml(text, path.name, model, verbose) | ||
| text = path.read_text(encoding="utf-8", errors="replace") | ||
| ingest_foundry(text, model, path.name) | ||
| elif fmt == "customizations_xml": | ||
| text = path.read_text(encoding="utf-8", errors="replace") |
Adds the Agent Architecture Advisor skill for reviewing, designing, and modernizing Copilot Studio and Azure AI Foundry agents.
The skill: