Reword the moderation rules as statements, not decisions - #1508
Merged
Merged
Conversation
Each rule now says what is true of a submission ("this submission is an
advertisement") instead of what to do about it ("this submission should be
rejected because..."). Approve and reject are this app's concepts, not the
model's. Within each rule, all the true cases come before all the false
cases rather than alternating.
Lowers MIN_REJECT_PROBABILITY from 0.375 to 0.3 to match the new scores.
On a balanced 300-story sample this auto-publishes 127 of 150
human-approved stories, up from 112, while letting through the same number
of stories a human rejected. The admin flag badge threshold is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
jboolean
commented
Sep 22, 2026
|
|
||
| Both backends use the official [`@openrouter/sdk`](https://www.npmjs.com/package/@openrouter/sdk) package, not raw HTTP calls, so requests and responses are typed and validated. | ||
|
|
||
| ## How to word a rule |
Owner
Author
There was a problem hiding this comment.
This is obvious. Just describe the tooling, remove this useless section.
jboolean
commented
Sep 22, 2026
|
|
||
| Within a rule, put every case that makes it true first, then every case that makes it false. Do not alternate. | ||
|
|
||
| Both conventions were measured against 300 real stories. Together they took the auto-approve rate from 112 of 150 to 127 of 150 at the same rate of letting bad stories through. |
Owner
Author
There was a problem hiding this comment.
Please STOP PUTTING INTERMEDIATE STATE AND HISTORIC NONSENSE IN PERMANENT CODE ARTIFACTS.
jboolean
commented
Sep 22, 2026
| criteria: { true: string; false: string }; | ||
| } | ||
|
|
||
| /** |
Owner
Author
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Each moderation rule now says what is true of a submission ("this submission is an advertisement") instead of what to do about it ("this submission should be rejected because it is an advertisement"). Approve and reject are this app's concepts, not the model's. Within each rule, all the true cases come before all the false cases instead of alternating.
Lowers
MIN_REJECT_PROBABILITYfrom 0.375 to 0.3 to match the new scores. On a balanced 300-story sample this auto-publishes 127 of 150 human-approved stories, up from 112, while letting through the same number of stories a human rejected. Scores are flat from 0.26 to 0.42, so the exact threshold is not load-bearing.The admin flag badge threshold is unchanged at 0.5. It flags 147 of 300 stories at 88% precision, against 144 at 91% before.
Experiment tool
moderation-experiment/rules.tsgets the same rule text, since it is a hand-synced copy of the production file. Its README gains a section on how to word a rule.🤖 Generated with Claude Code