Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Win POIDH Arbitrum bounty 323 by shipping a privacy-first local MV3 Chrome exten
- **Manifest V3.** `onnxruntime-web` in an offscreen document. WebGPU with WASM fallback (probe the adapter; do not latch a WebGPU error).
- **Auto-scan ordinary webpages.** Confidence on every badge: AI / OK / uncertain.
- **Hybrid is allowed:** neural + C2PA + EXIF/XMP + PNG/JPEG comments + weak URL hints. A URL hint alone must not cross 0.65.
- **Current fusion: CF-primary, three tiers plus view agreement.** CommunityForensics TTA takes the maximum raw sigmoid from inspected views, with one rule: a lone view in [0.65, 0.85) does not carry an AI verdict and falls back to the runner-up (live CDN-processed real photos spike single crops); views >= 0.85 keep single-view authority and early-exit. The adaptive extras band is [0.15, 0.85). CF is authoritative at >= 0.65 after agreement. Sub-floor tier: below CF 0.02, rescue only when CF >= 0.0005 AND DINO >= 0.995. Strong tier: CF in [0.02, 0.20) needs DINO >= 0.96. Normal tier: CF in [0.20, 0.65) needs DINO >= 0.70. The flat-graphic gate blocks every rescue tier. Bands derived under four simultaneous guards: public bench, 240-image stress set, a held-out live-CDN guard (camera-EXIF-verified Unsplash variants), and a held-out product-CDN guard (IKEA/Amazon imagery). **Do not restore plain max(CF, DINO), remove the graphic gate or the agreement rule, or loosen these bands without rerunning the bench, the stress set, the live guard, AND the product guard.**
- **Current fusion: CF-primary, three tiers plus view agreement.** CommunityForensics TTA takes the maximum raw sigmoid from inspected views, with one rule: a lone view in [0.65, 0.95) does not carry an AI verdict and falls back to the runner-up (live CDN-processed real photos spike single crops); views >= 0.95 keep single-view authority and early-exit. After a fallback, DINO must be >= 0.96 to lift again (a 0.76 probe on a 0.40 runner-up is the Unsplash editorial pattern). The adaptive extras band is [0.15, 0.95). The page queue never sheds TTA to center-only: adaptive already skips extras on confident reals, and shedding disabled agreement on busy Unsplash walls. CF is authoritative at >= 0.65 after agreement. Sub-floor tier: below CF 0.02, rescue only when CF >= 0.0005 AND DINO >= 0.995. Strong tier: CF in [0.02, 0.20) needs DINO >= 0.96. Normal tier: CF in [0.20, 0.65) needs DINO >= 0.70 unless agreement just fell back. The flat-graphic gate blocks every rescue tier. Bands derived under four simultaneous guards: public bench, 240-image stress set, a held-out live-CDN guard (camera-EXIF-verified Unsplash variants), and a held-out product-CDN guard (IKEA/Amazon imagery). Those four fixtures are not in git; `eval/fetch-live-guard.mjs` builds a smaller runnable stand-in. **Do not restore plain max(CF, DINO), remove the graphic gate or the agreement rule, restore center-only load-shed, or loosen these bands without rerunning the bench, the stress set, the live guard, AND the product guard.**
- **Overlay:** badge store must be an iterable `Map`, not a `WeakMap`. Reposition on scroll / resize / `visualViewport` / mutations. Never wrap images.
- **Load-unpacked users do not auto-update.** GitHub Releases zip + popup banner is the update path. Do not assume CWS.
- **Cross-device:** macOS (owner), Windows, Linux. WASM must work when WebGPU has no adapter.
Expand Down Expand Up @@ -76,7 +76,7 @@ Win POIDH Arbitrum bounty 323 by shipping a privacy-first local MV3 Chrome exten
| `src/c2pa-reader.js` | C2PA reader |

- Issue 23 (fixed in PR 24 / v1.0.7): overlay WeakMap drift + DINO max false positives.
- The public 893-image fixture with the live-guarded policy: 87.9% BA, 76.0% TPR, 99.8% TNR (harness-verified, adaptive mode, implemented view-agreement path). Stress set: 3 FPs in 240 images. Live-CDN guard (132 held-out camera-EXIF Unsplash variants, AVIF plus JPEG): 4 FPs, 3.0%, vs 9.2% for v1.2.0 on identical bytes. Product-CDN guard (100 held-out IKEA/Amazon images): 3 FPs, 3.0%, two of three CF-driven. Probe v5 trained on 11,721 rows including ~1.9k hard-negative reals, 118 verified-real live CDN images, and 198 product-CDN negatives. Smoke history: v1.2.0 24.3% flags on assumed-real pages (FAIL), v1.3.0 9.0% confirmed-real FPs (FAIL, product imagery), v1.3.1 3.21% confirmed-real FPs (PASS on accuracy) with a WebGPU worker wedge under GPU contention; v1.3.2 (current) ships the wedge watchdog and one-way WASM fallback, non-scoring, benchmark numbers re-affirmed. See docs/live-smoke-2026-08-16.md for the full isolation work.
- The public 893-image fixture with the live-guarded policy: 87.9% BA, 76.0% TPR, 99.8% TNR (last full harness run, v1.3.2 adaptive path, early-exit 0.85). That fixture is not in this checkout and was not re-scored for v1.3.3. Stress set: 3 FPs in 240 images (same caveat). Live-CDN guard (132 held-out camera-EXIF Unsplash variants, AVIF plus JPEG): 4 FPs, 3.0%, vs 9.2% for v1.2.0 on identical bytes; that 3.0% was measured with full adaptive TTA, not the center-only load-shed the live Unsplash masonry actually used. Product-CDN guard (100 held-out IKEA/Amazon images): 3 FPs, 3.0%, two of three CF-driven. Runnable stand-in scored 2026-08-17 (39 held-out Unsplash CDN JPEGs + 16 DALL-E 3): v1.3.2 3/39 live FPs (7.7%) and 43.8% TPR; production 1/39 (2.6%) and the same 43.8% TPR. Remaining FP is a strong-tier DINO rescue (CF 0.020, DINO 0.994). Probe v5 trained on 11,721 rows including ~1.9k hard-negative reals, 118 verified-real live CDN images, and 198 product-CDN negatives. Smoke history: v1.2.0 24.3% flags on assumed-real pages (FAIL), v1.3.0 9.0% confirmed-real FPs (FAIL, product imagery), v1.3.1 3.21% confirmed-real FPs (PASS on accuracy) with a WebGPU worker wedge under GPU contention; v1.3.2 shipped the wedge watchdog. A v1.3.2 live Unsplash featured-feed pass still failed (15 AI of 26 badges, 81-94%). v1.3.3 (current) never sheds TTA, raises early-exit to 0.95, and blocks middling DINO lifts after agreement fallback. See docs/live-smoke-2026-08-16.md for the isolation work.

---

Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ Two independent neural heads cover complementary failure modes, plus determinist

1. **CommunityForensics head:** ViT-Small official FP32 ONNX (CLIP 384). `p(AI) = sigmoid(logit)`. Near-zero false positives on real photos, but under-scores several modern generators (Flux, GPT-4o-image, photoreal DALL-E 3).
2. **DINOv2 probe head:** frozen DINOv2-small backbone (224 center view) with a transparent logistic head over CLS+mean-pooled features (`models/probe/dino-probe.json`: plain standardize/weights/bias, no lookup tables). Trained on ~11.7k images from public datasets across Flux, SD3.5, SDXL-era, Midjourney, DALL-E 3, GPT-4o-image and diverse real photos, including ~1.9k hard-negative reals (stock photography, product catalogs, interiors, high-saturation nature) plus 118 verified-real live-CDN and 198 product-CDN images that teach the head not to fire on professional real photos, with web-realistic JPEG/resize augmentation. Features are extracted through the same Pillow-exact resize the extension ships, so training matches serving exactly. This head carries the modern generators.
3. **Neural fusion:** CF-primary with three rescue tiers. When CommunityForensics is confident AI (`>= 0.65` after view agreement), its score wins. Between `0.02` and `0.20`, DINO can only rescue if it is highly confident (`p(AI) >= 0.96`); between `0.20` and `0.65`, DINO can lift at `p(AI) >= 0.70`. Below the `0.02` floor a rescue additionally requires CF to be at least faintly awake (`>= 0.0005`) and DINO to be saturated (`>= 0.995`): CF emits hard zeros on real photos it is certain about, while AI images in its blind spots still elicit a faint response, so a flatlined CF is itself evidence of a real photo and is never overridden. On flat graphics and catalog art (low palette / high flat-run pixels), a graphic gate suppresses every DINO rescue tier when CF stays below `0.65`. Displayed confidence is this raw fused probability; the AI verdict stays at raw `>= 0.65` with no remapping and no logit bias. The rescue bands were re-derived under four guards at once: the public bench, a 240-image full-resolution stock and catalog stress set, a held-out live-CDN guard of camera-EXIF-verified editorial photos, and a held-out product-CDN guard of IKEA and Amazon imagery.
4. **Adaptive TTA with view agreement:** the DINO pass and the official 440 center crop always run. Extra CommunityForensics views (440 corners + 512 center) run when a head is at least mildly suspicious (CF center in `[0.15, 0.85)` or DINO `>= 0.15`), so confident reals cost two passes total. Aggregation is the max of sigmoids with one honesty rule: a lone view in `[0.65, 0.85)` does not carry an AI verdict by itself and falls back to the runner-up view, because live CDN-processed real photos can spike a single crop. Any view at `>= 0.85` keeps single-view authority and early-exits. Under heavy queue load (more than 12 pending) CF drops to center-only; the DINO pass still runs.
3. **Neural fusion:** CF-primary with three rescue tiers. When CommunityForensics is confident AI (`>= 0.65` after view agreement), its score wins. Between `0.02` and `0.20`, DINO can only rescue if it is highly confident (`p(AI) >= 0.96`); between `0.20` and `0.65`, DINO can lift at `p(AI) >= 0.70`, except when that CF score is a view-agreement fallback, in which case DINO must still be `>= 0.96` (a middling probe must not re-flag a disagreed CDN spike). Below the `0.02` floor a rescue additionally requires CF to be at least faintly awake (`>= 0.0005`) and DINO to be saturated (`>= 0.995`): CF emits hard zeros on real photos it is certain about, while AI images in its blind spots still elicit a faint response, so a flatlined CF is itself evidence of a real photo and is never overridden. On flat graphics and catalog art (low palette / high flat-run pixels), a graphic gate suppresses every DINO rescue tier when CF stays below `0.65`. Displayed confidence is this raw fused probability; the AI verdict stays at raw `>= 0.65` with no remapping and no logit bias. The rescue bands were re-derived under four guards at once: the public bench, a 240-image full-resolution stock and catalog stress set, a held-out live-CDN guard of camera-EXIF-verified editorial photos, and a held-out product-CDN guard of IKEA and Amazon imagery. Those four fixtures are not in git.
4. **Adaptive TTA with view agreement:** the DINO pass and the official 440 center crop always run. Extra CommunityForensics views (440 corners + 512 center) run when a head is at least mildly suspicious (CF center in `[0.15, 0.95)` or DINO `>= 0.15`), so confident reals cost two passes total. Aggregation is the max of sigmoids with one honesty rule: a lone view in `[0.65, 0.95)` does not carry an AI verdict by itself and falls back to the runner-up view, because live CDN-processed real photos can spike a single crop into the 0.81-0.94 band. Any view at `>= 0.95` keeps single-view authority and early-exits. The page queue never sheds TTA to center-only; adaptive already skips extras on confident reals, and shedding was the live Unsplash masonry failure.
5. **Metadata:** C2PA, EXIF/XMP/IPTC, generator text in PNG/JPEG, weak URL hints. Strong metadata forces 0.95-0.99; a URL hint alone cannot cross 65%.

## Tests
Expand All @@ -85,7 +85,7 @@ Eval harness: `npm run eval -- ./path/to/labeled-folder` after `npm run fetch-mo

The legacy raw max result is included to make the tradeoff visible, not as a product claim. It caused unacceptable false positives on live stock and catalog images, so production keeps the CF guard. The historical 85.0% row was measured through a Node resize the extension never ran; later rows are computed by the same Pillow-exact resize the extension ships, byte for byte. On a 240-image full-resolution stock, catalog, and product photo stress set the production policy shows 3 false positives, zero attributable to a DINO rescue. On a held-out live-CDN guard of 132 camera-EXIF-verified Unsplash editorial variants it shows 4 false positives (3.0 percent), where the v1.2.0 configuration measured 9.2 percent on identical bytes. On a held-out product-CDN guard of 100 IKEA and Amazon images it shows 3 false positives (3.0 percent), two of them CF-driven. Public fixtures are directional only and are not a claim about Kenny's private held-out set.

**Live-web evaluation (published openly):** our 2026-08-16 live-site smoke test of v1.2.0 FAILED on CDN-processed professional photography, and the full findings, isolation experiments, and fixes are in [docs/live-smoke-2026-08-16.md](docs/live-smoke-2026-08-16.md). Three smoke rounds measured 24.3 percent flags on assumed-real pages (v1.2.0), 9.0 percent confirmed-real false positives (v1.3.0, concentrated in product imagery), and 3.21 percent (v1.3.1, clean-profile rerun). The fixes behind that arc: the view-agreement rule, live-CDN and product-CDN hard negatives in the probe, and two new permanent guard sets. v1.3.2 adds a WebGPU watchdog with automatic WASM fallback after the final rerun exposed a worker stall under GPU contention; scoring is unchanged. Live counterexamples are welcome as issues.
**Live-web evaluation (published openly):** our 2026-08-16 live-site smoke test of v1.2.0 FAILED on CDN-processed professional photography, and the full findings, isolation experiments, and fixes are in [docs/live-smoke-2026-08-16.md](docs/live-smoke-2026-08-16.md). Three smoke rounds measured 24.3 percent flags on assumed-real pages (v1.2.0), 9.0 percent confirmed-real false positives (v1.3.0, concentrated in product imagery), and 3.21 percent (v1.3.1, clean-profile rerun). A later live pass of the public v1.3.2 zip on the Unsplash featured feed still failed (15 AI of 26 badges, ordinary editorial photos at 81-94 percent). The 3.0 percent held-out live-CDN guard was scored with full adaptive TTA; the live page queued more than 12 images and shed to center-only, which disabled agreement. v1.3.3 never sheds TTA, raises the single-view early-exit from 0.85 to 0.95, and blocks middling DINO lifts after an agreement fallback. On a runnable stand-in built in this checkout (39 held-out Unsplash CDN JPEGs + 16 DALL-E 3, `eval/fetch-live-guard.mjs`): v1.3.2 fused 3/39 live FPs (7.7 percent) at 43.8 percent TPR; production fused 1/39 (2.6 percent) at the same 43.8 percent TPR. The remaining FP is an existing strong-tier DINO rescue (CF 0.020, DINO 0.994). The 893-image bench numbers above are the last full v1.3.2 adaptive-path run; that fixture is not in git and was not re-scored here. A full live Unsplash re-smoke of v1.3.3 has not been published. Live counterexamples are welcome as issues.

## Limitations

Expand Down
11 changes: 7 additions & 4 deletions STORE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

**Author:** Luis Felipe Abarca
**Project license:** MIT (bundled assets have their own notices)
**Version:** 1.3.2
**Version:** 1.3.3

Unpacked install from GitHub (`npm run fetch-model && npm run build`, then Load unpacked from `dist/`) remains the POIDH proof path. This file is listing copy for a future Chrome Web Store upload. The extension is **not** claimed to be on the Chrome Web Store.

Expand All @@ -26,8 +26,11 @@ How it works
65%; between 2% and 20% DINO may only lift if it is near-saturated (>= 96%);
between 20% and 65% DINO may lift at >= 70%. A flat-graphic guard prevents
DINO from overriding CF on catalog art and UI-like images. CommunityForensics
TTA uses the maximum raw sigmoid from inspected crops. The decision threshold
is raw 65% p(AI), with no score remapping.
TTA uses the maximum raw sigmoid from inspected crops, with view
agreement: a lone crop in [65%, 95%) falls back to the runner-up, and
a middling DINO score cannot lift that fallback. The page queue never
sheds TTA to center-only. The decision threshold is raw 65% p(AI),
with no score remapping.
- Hybrid metadata: C2PA digitalSourceType, EXIF/XMP/IPTC, generator text in PNG/JPEG, and weak URL hints. A URL hint alone cannot push a score over 65%.
- Overlay badges on large page images. Popup drop zone for local files. Works fully offline after install; the zip includes all model weights.

Expand Down Expand Up @@ -103,7 +106,7 @@ Do not screenshot private or copyrighted photos you do not have rights to. Use t

1. `npm ci && npm run package`
2. Open the Chrome Web Store Developer Dashboard (one-time developer registration is a Google process; this repo does not submit for you).
3. New item, upload `release/clueside-1.3.2.zip`
3. New item, upload `release/clueside-1.3.3.zip`
4. Paste the short and detailed descriptions above
5. Set privacy policy URL to the hosted `docs/privacy.html`
6. Attach screenshots and the 128px icon
Expand Down
2 changes: 1 addition & 1 deletion docs/POIDH-323.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,4 +182,4 @@ Known competing claims and reject patterns. Do not copy their failure modes.

Other competitors named in maintainer threads: anudit, RealGuard, Rajesh, PixelWitness. Several use score remapping or Platt-style calibration that paints low raw scores as 65%.

This repo's current policy (v1.3.2) uses max-of-inspected-view CF TTA with a view-agreement rule (a lone view in [0.65, 0.85) falls back to the runner-up; views >= 0.85 keep single-view authority), a CF-primary three-tier rescue (0.02 rescue floor; strong tier [0.02, 0.20) with DINO >= 0.96; normal tier [0.20, 0.65) with DINO >= 0.70; below the floor, rescue only when CF >= 0.0005 and DINO >= 0.995), a flat-graphic guard, and `DEFAULT_THRESHOLD` 0.65 on raw fused p(AI). Live validation ran as a three-round smoke series on 2026-08-16: v1.2.0 flagged 24.3 percent of assumed-real pages (FAIL), v1.3.0 measured 9.0 percent confirmed-real FPs (FAIL, product imagery), v1.3.1 measured 3.21 percent (PASS on accuracy, with 64.29 percent detection on Lummi's AI feed and offline zero-network PASS) while exposing a WebGPU worker wedge under GPU contention, fixed in v1.3.2 with a watchdog and automatic WASM fallback. Four permanent guards (public bench, stress set, live-CDN, product-CDN) now gate every scoring change. See docs/live-smoke-2026-08-16.md for the full history.
This repo's current policy (v1.3.3) uses max-of-inspected-view CF TTA with a view-agreement rule (a lone view in [0.65, 0.95) falls back to the runner-up; views >= 0.95 keep single-view authority; after a fallback DINO must be >= 0.96 to lift), no center-only load-shed on busy pages, a CF-primary three-tier rescue (0.02 rescue floor; strong tier [0.02, 0.20) with DINO >= 0.96; normal tier [0.20, 0.65) with DINO >= 0.70 unless agreement just fell back; below the floor, rescue only when CF >= 0.0005 and DINO >= 0.995), a flat-graphic guard, and `DEFAULT_THRESHOLD` 0.65 on raw fused p(AI). Live validation ran as a three-round smoke series on 2026-08-16: v1.2.0 flagged 24.3 percent of assumed-real pages (FAIL), v1.3.0 measured 9.0 percent confirmed-real FPs (FAIL, product imagery), v1.3.1 measured 3.21 percent (PASS on accuracy, with 64.29 percent detection on Lummi's AI feed and offline zero-network PASS) while exposing a WebGPU worker wedge under GPU contention, fixed in v1.3.2 with a watchdog and automatic WASM fallback. A later live pass of the public v1.3.2 zip on the Unsplash featured feed still failed (15 AI of 26 badges). The 3.0 percent held-out live-CDN guard did not match that feed because the live page shed TTA to center-only. Four permanent guards (public bench, stress set, live-CDN, product-CDN) still gate scoring changes; those fixtures are not in git. See docs/live-smoke-2026-08-16.md for the full history.
Loading