Add a0p-troubleshooter skill for RCA and fix planning

[Copilot]

Adds a first-class troubleshooting/RCA skill to the a0p skills framework, covering CI failures, runtime crashes, DNS/Pages issues, Fleet agent anomalies, PCNA/EDCM pipeline errors, and "it stopped working" regressions.

New files

• .agents/skills/a0p-troubleshooter/SKILL.md — Skill body with:

  • 11 failure classification classes (ci_build, agent_run, pcna_edcm, dns_https, etc.)
  • Structured JSON output schema (summary → hypotheses → evidence → root cause + confidence → fix plan)
  • Memory rules: what to persist in semi-permanent instances vs. what to never store (secrets, keys, credential blobs)
  • Multi-model hub split: T1 Hypothesizer → T2 Skeptic → T2 Fix Author → T2 Synthesizer
  • a0p-specific failure pattern playbook (console-tab guard, four-place route registration, frozen Fleet runs, PCNA coherence lock, custom domain CNAME overwrite)

• evals/evals.json — 5 full eval cases for the skill-creator harness (CI regression guard, SQLAlchemy pool exhaustion, GitHub Pages 404, Fleet frozen run, missing console tab)

• evals/eval_set.json — 12-entry trigger eval set for run_eval.py (8 should-trigger, 4 should-not-trigger)

Housekeeping

• Added __pycache__/ and *.pyc to .gitignore (were missing project-wide)

Discovery / wiring

No registry changes needed — local a0p-* skills are discovered purely by filesystem presence under .agents/skills/. The skills-lock.json is only for externally-sourced skills.

Running the eval

# Trigger eval (requires claude CLI)
PYTHONPATH=.agents/skills/skill-creator \
  python .agents/skills/skill-creator/scripts/run_eval.py \
  --eval-set .agents/skills/a0p-troubleshooter/evals/eval_set.json \
  --skill-path .agents/skills/a0p-troubleshooter/ \
  --runs-per-query 3 --verbose

# Full skill-creator eval harness
# Point run_eval.py at evals/evals.json for end-to-end graded runs

Original prompt

Create and wire in a new troubleshooting skill into the existing skills framework in repo The-Interdependency/a0.

Goals:

1. Add a new skill directory under .agents/skills/ named something like a0p-troubleshooter (or consistent with existing naming conventions).
2. Add a SKILL.md that defines a troubleshooting/RCA capability suitable for the platform (multi-model hub + semi-permanent instances). The SKILL.md must:
   • follow the repository’s existing skill conventions (match structure/patterns used by existing skills like .agents/skills/a0p-model-selector/SKILL.md)
   • define trigger conditions, input expectations, operating rules, output schema (JSON), memory rules (don’t store secrets), and optional multi-model role splitting.
3. Wire the skill into whatever registry/loader/config the project uses so it is discoverable and usable by the agent platform (find the existing skill discovery mechanism and update it accordingly).
4. Add a small evaluation set (if the repo has an established pattern for skill eval) so it can be tested with the existing skill-creator scripts. Use representative troubleshooting prompts (CI failure, stack trace, DNS/Pages issue). Ensure the eval set is placed in the correct location and referenced appropriately.
5. Ensure lint/tests (if any for skills) pass.

Context:

• The repo already contains a skills framework in .agents/skills/ and skill tooling in .agents/skills/skill-creator/scripts/ (e.g. run_eval.py).
• The platform is a multi-model, multimodal hub with persistent ZFAE agent definition and tools.

Deliverables:

• New skill directory with SKILL.md.
• Any necessary registry/config updates.
• Eval set additions.
• Brief documentation in the PR description about how to run the eval for the new skill.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: what happened to my custom domain?

Assistant: Most commonly, nothing “mysterious” happened to the domain itself—your GitHub Pages custom domain usually disappears/gets unset (or stops working) because Pages was unpublished/disabled or the DNS check started failing, so GitHub can’t keep serving your site at that domain.

Here are the main causes to check (in order), based on how GitHub Pages custom domains work:

1) Your Pages site got unpublished/disabled

If a Pages site is disabled but your DNS still points at GitHub, the domain can look “gone” from the site and GitHub warns about takeover risk. GitHub lists a couple common reasons a Pages site may be automatically disabled (especially around private repos + plan changes or transfers).
See: About custom domains and GitHub Pages

What to do

• Go to repo → Settings → Pages
• Confirm Pages is enabled and the Source/Build is set correctly for your site.

2) Your DNS records no longer point to GitHub Pages correctly

When you set/change a custom domain, GitHub runs an automatic DNS check; if DNS isn’t correct, GitHub may not be able to provision/keep HTTPS and the domain setup will be “not verified” / not working.
See: Securing your GitHub Pages site with HTTPS

What to do

• If you use an apex domain (example.com): ensure your A/AAAA (or ALIAS/ANAME) records match GitHub’s documented values.
• If you use a subdomain (www.example.com): ensure there’s a CNAME to wayseer00.github.io.

3) You’re expecting a custom domain to be set on the “wrong” repo

GitHub distinguishes user/organization sites vs project sites:

• User site must be in a repo named <owner>.github.io (for you: wayseer00.github.io).
  See: What is GitHub Pages?

If your repo is named github.wayseer.io, it’s not the standard user-site repo name—so depending on how you configured Pages, you might actually be dealing with a project site configuration and/or a mismatch with where the domain is set.

Also: if you set a custom domain on your user site, it can become the default for your other Pages sites unless overridden.
See: About custom domains and GitHub Pages

---

Fast clarifying questions (so I can tell you the exact fix)

1. What is the custom domain you were using (e.g., github.wayseer.io or www.wayseer.io)?
2. In Settings → Pages, does it say Pages is publishing from a branch/folder, or does it show an error / “Pages is disabled”?
3. Is this intended to be your user site (wayseer00.github.io) or a project site?

If you paste what you see in the Pages settings panel (the “Custom...

This pull request was created from Copilot chat.

[Copilot]

Pull request overview

Adds a new “a0p-troubleshooter” skill intended to guide RCA/troubleshooting and fix planning for common a0p/platform failure modes, plus supporting evaluation inputs and minor repo hygiene.

Changes:

• Added new skill definition at .agents/skills/a0p-troubleshooter/SKILL.md with trigger guidance, operating rules, a JSON output schema, memory rules, and a0p-specific failure playbooks.
• Added evaluation inputs for the skill (eval_set.json trigger set, plus an additional evals.json case file).
• Updated .gitignore to ignore Python bytecode artifacts (__pycache__/, *.pyc).

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 3 comments.

File	Description
.gitignore	Ignores Python bytecode/cache outputs repo-wide.
.agents/skills/a0p-troubleshooter/SKILL.md	Introduces the troubleshooting/RCA skill content and required JSON output contract.
.agents/skills/a0p-troubleshooter/evals/evals.json	Adds richer eval case definitions (currently not clearly wired to existing scripts).
.agents/skills/a0p-troubleshooter/evals/eval_set.json	Adds trigger-eval query set consumable by run_eval.py.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The skill text instructs using tools get_job_logs, get_file_contents, and search_code, but these tool names don’t appear to exist in this repo’s tool registry. This will make the skill’s evidence-collection guidance unusable in practice. Please update this rule to reference the actual tools/APIs used here (e.g., github_api for Actions/log retrieval, bash_run for repo grep/file reads, and web_search for DNS/provider lookups), or phrase it generically without naming non-existent tools.

Suggested change

	`get_job_logs`, config files via `get_file_contents`, code via
	`search_code`, DNS records via web lookup. Cite your sources.
	`github_api`, config files and repository code via `bash_run`,
	DNS records and provider/web checks via `web_search`. Cite your sources.

[Copilot]

Section reference looks incorrect: this points readers to “§6 Quick Reference”, but the Quick reference section is §8. Please update the cross-reference so it doesn’t send readers to the wrong place.

Suggested change

	8. **Cite a0p-specific file paths** where relevant (see §6 Quick Reference).
	8. **Cite a0p-specific file paths** where relevant (see §8 Quick Reference).

[Copilot]

This evals.json format (with expected_output/expectations) doesn’t appear to be consumed by the current skill-creator scripts in this repo (I can’t find any references to these keys, and run_eval.py/run_loop.py only read the eval_set.json query list). If this file is meant to drive an end-to-end graded harness, it likely needs corresponding wiring/docs in the existing scripts; otherwise consider removing or renaming it to avoid suggesting it’s active test coverage when it isn’t.

Suggested change

	"prompt": "My GitHub Actions deploy workflow is failing. Here's the error from the console-tab regression guard step:\n\n```\nERROR: Tab 'diagnostics' declared in UI_META has no sections and no custom renderer.\nDeploy blocked.\n```\n\nWhat went wrong and how do I fix it?",
	"expected_output": "Identifies the CI failure class, explains the four-place registration requirement and console-tab guard rule, provides a concrete fix (add sections to UI_META or add a CUSTOM_TAB_RENDERERS entry), and gives verification steps.",
	"expectations": [
	"The output classifies the failure as ci_build or similar",
	"The output mentions check-console-tabs.mjs or the console-tab regression guard",
	"The output mentions CUSTOM_TAB_RENDERERS in client/src/pages/console.tsx",
	"The output mentions adding sections to UI_META as a fix option",
	"The output includes a verification step",
	"The output follows the JSON schema with summary, classification, hypotheses, fix_plan"
	]
	},
	{
	"id": 2,
	"prompt": "I'm seeing this traceback in the FastAPI logs:\n\n```\nTraceback (most recent call last):\n File \"python/routes/my_module.py\", line 47, in get_data\n result = await db.execute(select(MyModel).where(MyModel.user_id == user_id))\n File \"python/database.py\", line 23, in execute\n async with self._session() as session:\nsqlalchemy.exc.OperationalError: (asyncpg.exceptions.TooManyConnectionsError) sorry, too many clients already\n```\n\nThis started about 30 minutes ago after we deployed a new route module.",
	"expected_output": "Classifies as runtime_crash or db_migration, identifies connection pool exhaustion, links to the recent deploy as the likely trigger, proposes checking for missing session cleanup in the new route, and suggests a mitigation (restart + connection pool config).",
	"expectations": [
	"The output classifies the failure as runtime_crash or db_migration",
	"The output identifies connection pool exhaustion as the root cause or primary hypothesis",
	"The output links the new route module deploy to the timing of the failure",
	"The output suggests checking for unclosed database sessions or missing async context managers",
	"The output proposes a mitigation (restart, pool size config, or session cleanup)",
	"The output includes verification steps",
	"The output follows the JSON schema"
	]
	},
	{
	"id": 3,
	"prompt": "My custom domain interdependentway.org stopped working. GitHub Pages is showing a 404. I deployed to the gh-pages branch yesterday.",
	"expected_output": "Classifies as github_pages or dns_https, explains the CNAME file overwrite pattern (most common cause after a deploy), gives exact steps to verify and restore the CNAME file, and checks DNS record configuration.",
	"expectations": [
	"The output classifies the failure as github_pages or dns_https",
	"The output identifies CNAME file overwrite during deploy as the primary hypothesis",
	"The output explains that the CNAME file must contain only the bare domain name at the publish source root",
	"The output mentions checking Settings → Pages to confirm Pages is still enabled",
	"The output includes DNS verification steps",
	"The output follows the JSON schema with at least two hypotheses"
	]
	},
	{
	"id": 4,
	"prompt": "An agent run in the Fleet is stuck in 'running' status even though I can see it hasn't emitted a log event in 15 minutes. The agent process appears to have crashed. How do I diagnose this?",
	"expected_output": "Classifies as agent_run, explains the frozen 'running' pattern (finally block raised before status update), references run_logger and run_context, gives steps to fix the status in the DB and prevent recurrence.",
	"expectations": [
	"The output classifies the failure as agent_run",
	"The output explains that the finally block likely raised before the status update",
	"The output mentions wrapping the status UPDATE in its own try block as the fix",
	"The output references python/services/run_logger.py or python/services/run_context.py",
	"The output suggests a mitigation to manually update the run status",
	"The output follows the JSON schema"
	]
	"prompt": "My GitHub Actions deploy workflow is failing. Here's the error from the console-tab regression guard step:\n\n```\nERROR: Tab 'diagnostics' declared in UI_META has no sections and no custom renderer.\nDeploy blocked.\n```\n\nWhat went wrong and how do I fix it?"
	},
	{
	"id": 2,
	"prompt": "I'm seeing this traceback in the FastAPI logs:\n\n```\nTraceback (most recent call last):\n File \"python/routes/my_module.py\", line 47, in get_data\n result = await db.execute(select(MyModel).where(MyModel.user_id == user_id))\n File \"python/database.py\", line 23, in execute\n async with self._session() as session:\nsqlalchemy.exc.OperationalError: (asyncpg.exceptions.TooManyConnectionsError) sorry, too many clients already\n```\n\nThis started about 30 minutes ago after we deployed a new route module."
	},
	{
	"id": 3,
	"prompt": "My custom domain interdependentway.org stopped working. GitHub Pages is showing a 404. I deployed to the gh-pages branch yesterday."
	},
	{
	"id": 4,
	"prompt": "An agent run in the Fleet is stuck in 'running' status even though I can see it hasn't emitted a log event in 15 minutes. The agent process appears to have crashed. How do I diagnose this?"