Our pre-claim live smoke test of v1.2.0 FAILED: 266 of 1,093 images on assumed-real pages were flagged AI (24.3 percent), concentrated on professional photography served through image CDNs (Unsplash worst, Wikipedia completely clean at 0 of 264). Full findings, isolation experiments, and the fix plan: docs/live-smoke-2026-08-16.md.
Three causes identified so far:
- DINO probe saturation on current-era commercial photography (trained hard negatives were circa-2020 stock)
- CommunityForensics spiking on CDN-processed photos with large low-texture fields (real drone shot of water at CF 0.84)
- Part of the flagged set may be genuinely AI: Unsplash hosts labeled AI content in its 2026 feed, so the 24.3 percent needs ground-truthing before it is treated as the true false-positive rate
Work in progress: ground-truthed live corpus via Unsplash's public labels, probe retrain with current-distribution hard negatives, a permanent live-stress guard in the eval gate, and a full re-smoke. No bounty claim until the re-smoke passes.
How to help:
- Reproduce any number with the Node harness (npm ci, npm run fetch-model, npm run eval -- ); it computes byte-identical model inputs to the extension
- Post live counterexamples here: pages where real photos flag or AI images pass, ideally with exact image URLs
- Recipes for labeled, redistributable live-CDN real-photo corpora directly improve the guard sets
Our pre-claim live smoke test of v1.2.0 FAILED: 266 of 1,093 images on assumed-real pages were flagged AI (24.3 percent), concentrated on professional photography served through image CDNs (Unsplash worst, Wikipedia completely clean at 0 of 264). Full findings, isolation experiments, and the fix plan: docs/live-smoke-2026-08-16.md.
Three causes identified so far:
Work in progress: ground-truthed live corpus via Unsplash's public labels, probe retrain with current-distribution hard negatives, a permanent live-stress guard in the eval gate, and a full re-smoke. No bounty claim until the re-smoke passes.
How to help: