docs: Update the AI code guidelines - #3249
Conversation
PR Summary by QodoEstablish comprehensive AI contribution policy
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Code Review by Qodo
1. Description ban contradicts itself
|
|
|
||
| The following are strictly prohibited and will result in immediate closure of a pull request or Issue and potentially a block from the organisation: | ||
|
|
||
| - **Automated Pull Request Descriptions:** Using AI to write pull requests descriptions that are vague, overly flowery, or fail to accurately describe the technical changes. We want to hear from *you* - the developer - why this change matters (see points 1 & 2). |
There was a problem hiding this comment.
1. Description ban contradicts itself 🐞 Bug ≡ Correctness
Section 2 says AI must not be used to create a pull request description except for translation, while the strict prohibition only rejects AI-written descriptions that are vague, flowery, or inaccurate. Consequently, an accurate AI-generated description is simultaneously disallowed and outside the stated prohibited-use criteria, making enforcement ambiguous.
Agent Prompt
## Issue description
The policy gives conflicting rules for AI-generated pull request descriptions: one section rejects any AI use except translation, while the prohibited-uses section rejects only vague, flowery, or inaccurate AI descriptions. State one consistent rule, including whether partial AI assistance is allowed and retaining the translation exception.
## Issue Context
This is a central rule of the new AI policy and directly affects whether maintainers should close a pull request.
## Fix Focus Areas
- AI_POLICY.md[24-24]
- AI_POLICY.md[40-42]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
| Filename | Overview |
|---|---|
| AI_POLICY.md | Adds the detailed AI contribution policy, but the new file lacks the required GPL license header. |
| CONTRIBUTING.md | Replaces duplicated AI guidance with a concise link to the dedicated policy. |
Prompt To Fix All With AI
### Issue 1
AI_POLICY.md:1
**Missing GPL license header**
The newly added `AI_POLICY.md` starts directly with its document heading and omits the repository-required GPL license header, leaving the policy document without its required licensing notice.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Reviews (1): Last reviewed commit: "docs: Update the AI code guidelines" | Re-trigger Greptile
| @@ -0,0 +1,55 @@ | |||
| # AI Contribution Policy | |||
|
|
|||
| Today, AI-assisted tools are being deployed in many coding environments. Some of the "lower-end" features like keyword-driven code completion can be useful aids; other tools make it very easy to generate contributions without the submitter fully understanding how the code works, or the consequences of specific implementation choices. These latter forms of AI assistance cause major overhead for project maintainers. | |||
There was a problem hiding this comment.
A test for each clause: does it still say something with 'AI' removed? Most of these do. Worth asking whether most of this belongs in CONTRIBUTING.md, with a short AI file left for provenance.
There was a problem hiding this comment.
That is true, but this document is here to solve a specific problem and it's good to name the cause of the problem directly, we don't see low-effort human slop really so there is no need to impose more guidelines on "regular" contributions
| - You are responsible for verifying that the code is human readable, maintainable and logically correct. | ||
| - All code needs to be tested and verified. | ||
|
|
||
| "The AI generated it and it works for me" is never an acceptable answer to a reviewer's question. Copy-and-pasting to and from an AI chatbot during the process of code review is not acceptable (unless this is only for translation to and from English). |
There was a problem hiding this comment.
The copy-paste ban could be rewritten. Linguistic support, surface edits, meaning-preserving edits and rendering are all valid reasons to copy-paste, the translation exception is already reaching for some of these.
Maybe something like: "If you cannot verify an answer well enough to defend it under follow-up questioning, say so rather than presenting it as your own"
There was a problem hiding this comment.
That's fair, could use some nuance
| @@ -0,0 +1,55 @@ | |||
| # AI Contribution Policy | |||
There was a problem hiding this comment.
Should we maybe call it LLM instead of AI? Because it is not AI. It is just the marketing term fort it, but not technically correct. At least in the technical description. The header title can probably stay "AI" for a broader understanding of the topic matter.
There was a problem hiding this comment.
Technically as well LLM's are an AI system, it's an umbrella term.
Artificial Intelligence (AI) → Machine Learning (ML) → Deep Learning → Generative AI → Large Language Models (LLMs)
There was a problem hiding this comment.
It is not Artificial Intelligence. It does not think. It is a complex predictor.
There was a problem hiding this comment.
It is not Artificial Intelligence. It does not think. It is a complex predictor.
not to argue with that concept, but i belive it do think or emulate the thinking process of the human, also i agree that it is just a very complex predictor, with massive amounts of data.
There was a problem hiding this comment.
So this is becoming philosophical. LLM’s are the tool of today but in a few years (or sooner) it may be something else. ML is bit less common of a term I think. We can do LLM for now, that’s fine.
|
|
||
| The following are strictly prohibited and will result in immediate closure of a pull request or Issue and potentially a block from the organisation: | ||
|
|
||
| - **Automated Pull Request Descriptions:** Using AI to write pull requests descriptions that are vague, overly flowery, or fail to accurately describe the technical changes. We want to hear from *you* - the developer - why this change matters (see points 1 & 2). |
There was a problem hiding this comment.
It is not clear what "overly flowery" means.
This change expands upon the existing AI code generation guidelines and turns it into an AI policy.
Notably it disallows pull requests that have descriptions that are entirely AI generated (unless translated), as well as disallowing copy-pasting answers from an AI chatbot during the review process.
It also describes a way of how this policy would be enforced.
This new policy incorporates the existing TSH guidelines on AI code generation and is based on the Mastodon AI policy and the CloudNativePG AI Policy.