Skip to content

feat(mcp): serve the reference tables from gw1://meta - #39

Merged
Graphmaxer merged 1 commit into
mainfrom
fix/audit-2026-08-11-chatgpt
Aug 11, 2026
Merged

feat(mcp): serve the reference tables from gw1://meta#39
Graphmaxer merged 1 commit into
mainfrom
fix/audit-2026-08-11-chatgpt

Conversation

@Graphmaxer

Copy link
Copy Markdown
Owner

Triage of a fourth external audit (ChatGPT, 2026-08-11 — snapshot without .git, no npm access, so it never ran the suite). It rated the repo 7.4/10 and raised three "P1"s; one was real, and its sharpest form was not the one reported.

The real finding

gw1://meta served _meta.json alone. The README promised four reference tables, but the load-bearing version is that UNKNOWN_ATTRIBUTE tells the caller "Title tracks and each profession's attribute lines are listed in the gw1://meta resource" — a dead pointer inside an LLM-facing error, which defeats the one-round-trip self-correction the project designs for. So: enrich, don't trim the prose.

  • Payload composed at module scope (a resource read must not rebuild it — same rule as the tool schema literals) from the gw-data tables: provenance + professions + attributes (title tracks 102-109 included) + campaigns + skill types, plus a one-line note on the id ranges.
  • Deliberately not written into _meta.json: one key per generator, so composing tables in would be an unmanaged copy with no generator and no refresh path.
  • The resource's own title/description were updated too — they ship in resources/list, and enriching the payload alone would just move the bug one layer over.
  • Resources carry no outputSchema and are not in tools/list, so this costs zero fixed context.

Two prose fixes

  • /privacy claimed "no personal data is collected" absolutely while recording a caller-controlled client name. The blob stays (it is what showed 63% of traffic was one uptime monitor); the absolute claim goes, replaced by naming who supplies the value, its bounds (charset-restricted, 64 chars, no version), and that Analytics Engine has no per-row deletion.
  • README: /exportaccount copies your character name — said in the paragraph that tells you to paste the JSON into a conversation.

Rejected, with reasons recorded in CLAUDE.md

  • "pnpm --filter @gw1-mcp/gw-data update is broken, no such script"false: update is a pnpm builtin (verified on 11.11.0). The command is correct.
  • Drop the client-name dimension (their recommended fix) — deletes the capability that answers whether any directory listing sends real clients.
  • Auto-merge of data imports — already gated by upstream JSON Schemas, the name gate over all five constant tables, assertCoherentSnapshot, the growth gate (no longer fail-open) and the golden fixtures; the guessed-threshold successor was measured and rejected.
  • Licensing — the counsel-gated register entry; the audit adds nothing to it.

Verification

pnpm -r typecheck, 376 tests (107 / 79 / 118 / 72), pnpm lint, pnpm fmt:check — all green. New assertions: gw1://meta must carry each of the four tables, and /privacy must not regrow the absolute claim.

Method note worth keeping: this pass never ran the suite, so its strongest claims were about prose consistency — and that is exactly where its one real finding was.

🤖 Generated with Claude Code

The resource served _meta.json alone, while the README promised professions,
attribute lines, campaigns and skill types — and, worse, UNKNOWN_ATTRIBUTE told
the caller to enumerate title tracks there. That pointer was a dead end in an
LLM-facing error, which is the reason to enrich rather than trim the prose: a
model that typo'd an attribute name gets suggestions, but nothing to enumerate
from.

The payload is composed at module scope (a resource read must not rebuild it,
same rule as the tool schema literals) from the gw-data tables. Deliberately NOT
written into _meta.json: that file is one key per generator, and composing tables
into it would create an unmanaged copy with no generator.

Two prose fixes from the same audit pass:

- /privacy claimed "no personal data is collected" absolutely while recording a
  CALLER-CONTROLLED client name. The blob stays — it is what showed 63% of
  traffic was one uptime monitor — but the absolute claim goes, replaced by
  naming who supplies the value, its bounds, and the fact that Analytics Engine
  has no per-row deletion.
- README now says /exportaccount copies your character name, since the same
  paragraph tells you to paste the JSON into a conversation.
@codecov

codecov Bot commented Aug 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.48%. Comparing base (ee745c6) to head (77b98bb).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #39      +/-   ##
==========================================
+ Coverage   87.46%   87.48%   +0.01%     
==========================================
  Files          16       16              
  Lines         766      767       +1     
  Branches      227      227              
==========================================
+ Hits          670      671       +1     
  Misses          8        8              
  Partials       88       88              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@codspeed-hq

codspeed-hq Bot commented Aug 11, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 22 untouched benchmarks


Comparing fix/audit-2026-08-11-chatgpt (77b98bb) with main (ee745c6)

Open in CodSpeed

@Graphmaxer
Graphmaxer merged commit 449b511 into main Aug 11, 2026
16 checks passed
@Graphmaxer
Graphmaxer deleted the fix/audit-2026-08-11-chatgpt branch August 11, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant