QEP-5: Code of AI Use

[mmcky]

Adds QEP-5: Code of AI Use as a Draft. Discussion: #12.

AI-assisted contributions are welcome at QuantEcon — our maintainers use these tools daily and in the open. This QEP asks every contributor to accept six norms in exchange: a human is in the loop (chooses the task, reads the result, submits it, answers review), meaningful AI involvement is disclosed with a hand-added Assisted-by: trailer, the contributor owns and can explain the work, claims about tests and benchmarks are verified before they are made, nobody crawls the tracker, duplicates, or farms pull requests, and a lapse is put right by amending the pull request.

It is policy, not infrastructure. The norms bind from acceptance, like the Code of Conduct, without anyone signing anything and without anything being built. The problem it answers is cost, not safety: an unreviewed submission is an extractive contribution — it costs more to review than it returns.

Scope

The discussion issue originally proposed a contributor registry and a pull-request gate alongside the policy. Both are deliberately out of scope, for two reasons set out in What this is not and Alternatives considered: the norms bind whether or not anyone attests to them, so a registry would notify rather than bind; and GitHub's native volume controls (org-level pull-request limits in particular) cover the single-account case for free. The full registry design is retained in QuantEcon/infrastructure#5; two constraints from it are carried into the QEP so they survive. At acceptance, #12 closes with a pointer there and the revival trigger — the native controls proving insufficient — rather than a new discussion issue.

Revised 2026-09-10

Rebased onto main (the hand-added README index row is gone; the index is generated post-merge). The text was revised against the two comments below and @oyamad's suggestions from the 8 Sep review round; the comment of 2026-09-10 on this PR lists what changed. The revision was drafted with AI assistance and the document now carries its own Disclosure section.

Revised 2026-09-11

Revised against @jstac's comment of 10 Sep; the comment of 2026-09-11 on this PR lists what changed. In short: a marker-fenced Code in brief above the full norms, three visibility channels in Adoption (a single Assisted-by: <tool> (<model>) or Assisted-by: none line in the template, the AGENTS.md block, and a comment-only ask on external PRs that fires only when the line is missing), norm 2 letting the tool add the trailer, the compliance study in Alternatives, and an Amendments to QEP-2 section for the good first issue reservation. Rollout is tracked on #33, with QuantEcon/cli#50 and QuantEcon/meta#385 for the two mechanisms.

Points worth arguing about

Norm 1 binds ownership, not attendance. LLVM's test: a human in the loop, not a human at the keyboard. Consequence: a cron-scheduled pipeline whose operator discloses, owns, and answers review is compliant; one that files and vanishes is not. The draft as first written would have made QuantEcon.py#923 a violation of a change we merged.

Norm 3's test carve-out is now bounded. Tests stand in for line-by-line reading only where a test would fail if the change were wrong. Deletions, behaviour-preserving refactors, and changes the issue asked to be judged site by site must be read — #923 is the counter-example, where dead-code removal satisfied the original carve-out with no reading at all.

Norm 5 adopts LLVM's good first issue prohibition, with the label as the fence. It is deliberately not widened to bug or audit issues: a maintainer who wants any issue reserved for hands-on learning labels it, and Adoption makes that deliberate. The alternative — a fuzzy "tractable enumerated issue" category — has no boundary a contributor can see.

Scope now covers issues, review comments, and proposals, in one sentence after the norms. LLVM covers these and the cost argument is identical. This is the one change with no prior discussion on the thread.

Organisation-operated automation is carved out, applying the accountability rule rather than waiving it: a named maintainer owns the output, the automation is declared in a public register, and it carries QEP-2's automated label. That register is what QEP-4 (#8) would supply; the two drafts should agree before either is accepted.

Field test

Per the practice proposed in #14, the draft's disclosure ask was tried informally on one pull request in one repository (QuantEcon.py#923, merged 9 Sep). That exercised norms 1, 2, and 3 and the enforcement path. Norms 4, 5 (the good first issue rule), and 6, the organisation-automation carve-out, and the scope beyond pull requests are untested.

Process

Draft. #23 (QEP-1 v3) removes the decision deadline and requires acceptance by a Core Maintainer other than the author, with no objection outstanding; this PR follows that rule whether or not #23 has merged first. Before a decision, the draft goes to the whole team, and possibly the community, as @oyamad suggested.

qep-checks parity: new QEP, unversioned (implicitly v0); no README row.

🤖 Generated with Claude Code

[mmcky]

Field note: first disclosure ask under this draft — QuantEcon/QuantEcon.py#923, merged 9 Sep.

While this QEP is in Draft we tried its disclosure ask informally on a live PR, which is the field test qeps#14 proposes. The PR was a small cleanup removing eight flake8 F841 unused-local findings from four numerical modules. It carried no disclosure and had agent fingerprints in its description, so we asked a neutral question on the thread — was AI involved, and which model or tool — and got a complete answer within the hour, offered in good faith and with an explicit undertaking to follow whatever format we settle on.

What was disclosed: the change was produced by a third-party agent runtime operating as an autonomous "OSS sprint assistant" on a cron schedule, with GPT-5.5 as the active model. The agent read the issue and PR context, wrote the patch, ran the local validation, and drafted the PR text.

Four things this tells us about the draft as written.

1. Norm 1 is the operative norm here, not norm 2 — and the motivation's central claim is now confirmed rather than inferred. The audit inferred "issue-crawling automation rather than individual experimentation" from fingerprints; this is that pattern, described by the contributor in their own words. Under norm 1 as drafted — "Unattended agents and automated pipelines must not open pull requests or crawl our issue tracker for work" — the contribution is a violation. It is also correct, minimal, in scope, and we merged it. The draft has no way to say that. The enforcement ladder starts at a rectification comment and is built around getting a PR amended; nothing covers "the work is good, the process isn't", which is the case we hit on the very first try.

2. The trailer assumption has a hole. Norm 2 points at Co-Authored-By: <tool>, "which Claude Code and Copilot already emit automatically". This pipeline emitted Signed-off-by: and no AI trailer at all. For third-party or in-house runtimes the automatic path does not exist, so the marker appears only if the operator adds it by hand. A greppable-marker audit will therefore systematically under-count exactly the unattended pipelines we most want visible, because the well-behaved Claude and Copilot cases are the ones that self-report. That makes the pull-request-template checkbox in Adoption carry more weight than the trailer for this class, and it argues for wording norm 2 as a trailer naming the tool, added by hand where the tool does not emit one.

3. Detection by fingerprint is weak; asking works. What flagged this PR was the prose, not the code — a TDD "RED/GREEN" template applied to a lint cleanup where no test was ever red. The diff carried no signal whatsoever: the issue enumerated all eight sites, so a human working from that list produces the same patch. That asymmetry will widen as agents get better at writing descriptions. Against it, one neutral question produced full voluntary disclosure in under an hour. This is direct support for the QEP's central bet — norms plus a visible disclosure path, applied by maintainers reading pull requests — and against any mechanism that depends on identifying agents.

4. A cheap counter-example to norm 3's test carve-out. The issue behind this PR asked for a per-site human decision in terms: in numerical code an assigned-and-discarded matrix is sometimes the residue of an incomplete edit, so each site needs examining rather than blanket deletion, and any site revealing a genuine logic gap gets its own issue. The agent deleted all eight without examining any. The answer happened to be right — a maintainer checked each site and all eight are genuinely dead — but the judgement the issue asked for was supplied after the fact by a reviewer, not by the contributor. Note what the tests contributed: nothing. Removing a dead binding cannot fail a test, so norm 3's "tests can stand in for line-by-line reading" would have been fully satisfied by a change in which no reading occurred. This is a smaller and clearer illustration than the #949 case and belongs in the norm 3 discussion.

What to do with this class of case. Three options; I think the middle one is right.

Enforce norm 1 literally — close such pull requests on process even when the change is good. Consistent, but it costs us correct work and the goodwill of a contributor who answered openly the moment we asked.

Move norm 1 from scheduling to accountability — what matters is not whether a human was present at submission but whether a human owns the change and answers review on it. Keep "don't crawl the tracker for work" as the volume rule, and make the binding test ownership and responsiveness, which is already norm 3's territory. On that reading this pull request is compliant once disclosed, while a pipeline that files and vanishes is not.

Keep norm 1 strict and add an explicit accept-once path — merge on the merits, state plainly that future submissions need a human in the loop. This is effectively what we did, and if we want it available it should be written down rather than improvised case by case.

One boundary question for norm 5. The suggestion to take the good first issue rule from LLVM's AI Tool Use Policy is a good one, but note it would not have covered this pull request: the issue was labelled bug, filed from the July tech-debt audit. Enumerated audit issues are simultaneously the most tractable targets for an agent and some of the better teaching material we have. If the reservation is meant to protect learning opportunities, good first issue alone is too narrow a fence.

One further data point. The same pipeline has a second pull request open on QuantEcon.py, and it shows the opposite failure mode — scope beyond what the issue asked for, which a reviewer now has to trim back. Under-supplied judgement in one case, over-supplied scope in the other; both land on maintainer attention, which is the cost this QEP is about.

[mmcky]

@oyamad suggestions on this QEP, from the 8 Sep review round. No strong opinion on the details, but he likes LLVM's AI Tool Use Policy (https://llvm.org/docs/AIToolPolicy.html, linked from #12), the good first issue rule in particular. On process: invite opinions from the whole team, and possibly the community, before the Core Maintainers decide.

Four things in that policy bear on the draft.

Its good first issue rule is a prohibition; ours is not. LLVM: "using AI tools to fix issues labelled as 'good first issues' is forbidden", reserving them for newcomers. Norm 5 ends with "The idea behind issues labelled as good first issue is they are good candidates for humans to learn the codebase" — an explanation, not an obligation. The fence is narrow either way: the issue behind QuantEcon/QuantEcon.py#923 was labelled bug.

Its binding test is a human in the loop, not human attendance — review the output before submitting, and be able to answer questions during review. That is the second of the three options above. It also matches norm 3: passing maintainer feedback to an LLM to generate replies is explicitly discouraged.

It recommends an Assisted-by: trailer. Two advantages over Co-Authored-By: — it is added by hand, so it does not depend on the tool emitting anything, and it makes no authorship claim in the contributor graph. This touches ground the Alternatives considered section covers, though a trailer is still a marker rather than a narrative.

"Extractive contributions" — submissions whose review burden exceeds their value to the project — is a sharper name for the cost argument in our Summary.

On process: the 28 Aug deadline lapsed while this PR was a Draft, and there are no comments on it or on #12 from anyone else. Acting on the suggestion means rebasing past QEP-3, marking this ready, putting it to the team, deciding whether to open it to the community, and setting a fresh window for lazy consensus.

[mmcky]

Revised 2026-09-10 — the feedback above is incorporated. Rebased onto main (README row dropped; the index is generated now). What changed, against the two comments above and @oyamad's suggestions:

• Norm 1 now binds a human in the loop — chooses, reads before requesting review, submits, answers review — rather than human attendance (the middle option above; also LLVM's test). "Don't crawl the tracker" moved to norm 5 as the volume rule.
• Norm 2 names a hand-added Assisted-by: <tool> trailer. A tool-emitted Co-Authored-By: still counts, but the text says outright that third-party and in-house runtimes emit nothing, so the operator adds it.
• Norm 3's test carve-out is bounded to changes a test could fail. Deletions, behaviour-preserving refactors, and site-by-site judgements must be read.
• Norm 5 adopts LLVM's good first issue prohibition. The label is the fence; it is not widened to bug or audit issues — a maintainer who wants an issue reserved labels it, and Adoption makes that deliberate.
• Enforcement gains a merge-on-merits path: correct work is merged and the lapse recorded in a comment that counts toward repeated. Norm 6 now covers an already-merged PR.
• Motivation records the QuantEcon.py#923 confirmation. Summary uses extractive contribution. Alternatives adds "adopt LLVM's policy wholesale", QEMU's provenance ban, and "detect agents rather than ask".
• New: the norms cover issues, review comments, and proposals (one sentence); a Disclosure section closes the document.

Kept out of the QEP and put in the PR body instead: the evidence-sample statement from #14 (one PR, one repo; norms 4–6 and the carve-out untested) and the team-wide invitation before a decision. At acceptance, #12 closes with a pointer to the retained registry design in QuantEcon/infrastructure#5.

[Copilot]

🔵 Needs a closer look

It adopts a binding, org-wide governance policy whose normative substance is a human-judgment decision requiring Core Maintainer review under QEP-1, even though no objective defects were found.

Pull request overview

This PR adds a new Draft proposal, QEP-5: Code of AI Use, as a single Markdown file under qeps/. It establishes an org-wide policy that welcomes AI-assisted contributions while binding every contributor to six norms (human-in-the-loop ownership, disclosure via an Assisted-by: trailer, verified claims, no crawling/duplicating/farming, and rectification of lapses). It frames the problem as maintainer review cost rather than safety, and deliberately scopes out a contributor registry and PR gate in favour of GitHub's native volume controls. The document follows the repo conventions defined in QEP-1 and enforced by qep-checks (unversioned new QEP, no self-added README row, ascending ordered-list markers).

Changes:

• Introduces the six-norm Code of AI Use, an Enforcement ladder, a "What this is not" scoping section, Alternatives considered, and an Adoption section.
• Carves out organisation-operated automation via QEP-2's automated label and a named-maintainer accountability rule, and reserves good first issue (QEP-2) for hands-on learning.
• Adds a self-referential Disclosure section noting the draft itself was AI-assisted.

File summaries

File	Description
qeps/qep-0005-code-of-ai-use.md	New Draft QEP defining the Code of AI Use policy, its enforcement, scope, alternatives, and adoption obligations; frontmatter, header table, links, and ordered lists all conform to repo/CI conventions.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jstac]

Hi guys, the following is written by Claude under my direction. My main concern was that this might end up being ignored, so how can we get it out in front of contributors and their AIs.

Feel free to ignore, these are pure suggestions.

Claude's recommendations:

• A short quotable form of the norms. Six one-sentence imperatives (~150 words) above the current text, so we can copy them word-for-word wherever they appear.

• Put the disclosure checkbox in QuantEcon/.github. A public org-level .github repo supplies a default CONTRIBUTING.md and PULL_REQUEST_TEMPLATE.md to every repo that lacks its own — one PR, whole org.

• AGENTS.md too. Those defaults show on the GitHub website but aren't in the repository, so they don't come down with a git clone — an AI tool working from a checkout sees neither. Measured: agents opened a policy file they weren't handed automatically in 3.5% of runs, but disclosure rises to 77–97% once the rule is in front of them (arXiv:2607.26819). AGENTS.md is what these tools read automatically, so putting the requirement there gets Assisted-by: added by the tool rather than remembered by the contributor. Needs to go in each repo — there's no org-level version.

• A QEP-2 amendment for good first issue. Norm 5 gives the label a reservation meaning that QEP-2's table doesn't carry. A one-line bump in this branch keeps it visible to anyone reading the label standard.

Something like MicroPython's, where you pick one and the second commits you to owning the code, not just disclosing:

• I did not use generative AI tools when creating this PR.
• I used generative AI tools when creating this PR, but a human has checked the code and is responsible for the code and the description above.

[mmcky]

thanks @jstac these are good comments. I'll work through them tomorrow and update.

[mmcky]

Revised 2026-09-11 against @jstac's comment above. All four suggestions taken, one of them in a different place than proposed. Per item:

• Short quotable form. Added as The Code in brief: six one-sentence imperatives in a marker-fenced > [!IMPORTANT] block at the top of the Proposal, above the full norms, with an absolute link back to the QEP so it survives being copied. It is normative and the full norms govern where they differ. The markers (<!-- qep-appendix: code-in-brief --> … <!-- /qep-appendix -->) let a tool extract it as a substring; QEP-2's yml was the first appendix and this is the second, which was the agreed trigger for making the pattern a QEP-1 standard. That amendment is tracked on QEP-5 adoption: execution checklist #33 and lands after QEP-1 v3: stamp from v0; drop the decision deadline; generate the index #23.
• Org-level .github. Right mechanism, kept out of the QEP text: the QEP states the obligation and the rollout lives on the adoption checklist QEP-5 adoption: execution checklist #33. Checked: none of QuantEcon.py or the three lecture repos has a pull-request template, so the org default reaches all four; QuantEcon.py has its own CONTRIBUTING.md, so that one is edited directly.
• AGENTS.md. The strongest of the four. Adoption now requires the Code in brief in the file agents read from a checkout, and norm 2 lets the tool add the trailer with the responsibility staying human. Delivery is QuantEcon/cli#50: a qe gh agents sync beside qe gh labels sync that stamps the block with the QEP version in its marker and reports drift. On the paper: it is real (Yang, He, and Zhou, arXiv:2607.26819) and the abstract confirms the direction, so the QEP cites the finding rather than the percentages, which I could not verify from the abstract. It also reports that agents never refused to contribute under an AI ban, which now sits in Alternatives against the Gentoo/QEMU option.
• QEP-2 amendment. Confirmed absent from QEP-2. Added as an Amendments to QEP-2 section on this branch, applied as a file edit at landing (QEP-2's first substantive amendment, so v1), per the convention that a QEP changing another carries the edit itself rather than a forward reference from an accepted QEP to a draft.
• MicroPython's checkbox pair. Adopted as the recommended template wording in Adoption; the second box is an ownership attestation, which is norm 3, not only a disclosure.

One thing the discussion added beyond the four. The template reaches only the web UI, and agents open pull requests through gh or the API with their own body, as the QuantEcon.py#923 pipeline did. So Adoption now names three channels: the template for people in the web UI, the AGENTS.md block for agents at the moment they act, and a comment-only neutral ask on pull requests from outside the organisation for everything else. That last one automates the question that produced full disclosure within the hour in the field test. It holds nothing, decides nothing, and never checks out the pull request's code, so the rejection of a gate stands; the no-automation claim narrows to "nothing that gates or decides". Design and research are on QuantEcon/meta#385.

Commit e1d04b3. This round was drafted with AI assistance, as the Disclosure section says.

[mmcky]

Follow-up, 2026-09-11 (commit 8824139, then a merge of main for QEP-1 v3). After discussion, the checkbox pair proposed above is replaced by a single line. Norm 2 now asks every pull request for one machine-readable line in the description or a commit message: Assisted-by: <tool> (<model>) where AI tools meaningfully contributed, harness first and model in parentheses, or Assisted-by: none where they did not. The explicit negative is there because silence is ambiguous. The template carries the empty line with both forms and a fixed attestation sentence beside it; a box to tick is not needed, since norm 3 binds either way and the template's job is to make sure the author has seen it. The neutral ask on external pull requests (QuantEcon/meta#385) fires only where the description has no such line, and asks for the line plus a reply confirming a human has read the pull request. So where the template does its job, the bot never speaks. The three-channel structure is unchanged.

[mmcky]

thanks @jstac I think we are iterating towards a good end point.

The motivation section is still quite large -- do you think we should reduce words?

[jstac]

I think this looks good @mmcky ! Nice work.

[mmcky]

thanks @jstac - I will leave this open until Monday, 14th September and then I will merge.