From 92326d8f2461b39393f9d7059ac3fce5e1b5db20 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 13:38:37 +0200 Subject: [PATCH 01/24] docs(specs): design for MegaDetector as a model choice --- ...-05-16-megadetector-model-choice-design.md | 256 ++++++++++++++++++ 1 file changed, 256 insertions(+) create mode 100644 docs/specs/2026-05-16-megadetector-model-choice-design.md diff --git a/docs/specs/2026-05-16-megadetector-model-choice-design.md b/docs/specs/2026-05-16-megadetector-model-choice-design.md new file mode 100644 index 00000000..ffcb5d69 --- /dev/null +++ b/docs/specs/2026-05-16-megadetector-model-choice-design.md @@ -0,0 +1,256 @@ +# MegaDetector as a Biowatch model choice — design + +## Background + +Biowatch currently offers three model choices in the AI Models tab: SpeciesNet (worldwide), DeepFaune (Europe), and Manas (Himalayas). All three combine an object detector with a species classifier. + +[MegaDetector](https://github.com/agentmorris/MegaDetector) is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, with **no species classification**. Inside Biowatch it would fill a different role: a fast blank-filter that lets a user separate frames with subjects from empty frames, then manually annotate species on the non-blank frames using Biowatch's existing annotation UI. + +MegaDetector v6 (`MDV6-yolov10x.pt`) is already bundled inside the DeepFaune and Manas tarballs and loaded via `ultralytics.YOLO` in `run_deepfaune_server.py`. The Python loading code is therefore already proven inside our `common` conda environment. + +## Goals + +1. Ship MegaDetector v6 (YOLOv10x) as a standalone model choice in the AI Models tab. +2. Match the established model-integration pattern (one Python server script, one `switch` case in `server.ts`, one HF tarball, one `mlmodels.js` entry). +3. Map MD's detector labels (`animal`/`person`/`vehicle` + implicit `blank`) into Biowatch's species-centric data model with **zero changes** to the observation/annotation/export pipelines — by treating the three labels as pseudo-species. +4. Suppress UI affordances that only make sense for true species classifiers (the "View species" panel) via an opt-out flag, not a special-case in every consumer. + +## Non-goals + +- No conda env bump. MD reuses `common/0.1.4` — `ultralytics` and `torch` are already pinned for DeepFaune/Manas. +- No detector-weight sharing across models. MegaDetector ships its own tarball; the copy bundled inside DeepFaune/Manas is untouched. The marginal ~110 MB is acceptable in exchange for the model-isolation property the codebase keeps today. +- No "MD-then-classifier" workflow primitive. MD is a standalone model; chaining is out of scope. +- No MDv5a packaging. Only MDv6 (YOLOv10x) ships. If a user needs v5 for reproducibility, that becomes a separate model entry later. +- No automated reclassification of existing observations. + +## Source-of-truth artifact + +Upstream weights: `https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10x.pt`. + +We rehost this single file on `huggingface.co/earthtoolsmaker/megadetector` (repo already created by the maintainer), as a tarball `6.0.tar.gz` containing a top-level `6.0/` directory with `MDV6-yolov10x.pt` inside it. This matches the layout convention of the other three model tarballs and gives us a stable, offline-deterministic download URL. + +Re-hosting is permitted: MegaDetector is MIT-licensed. + +## Tarball layout + +``` +6.0/ + MDV6-yolov10x.pt +``` + +Top-level directory name (`6.0/`) matches the model version. No `README.md` shipped (consistent with how `build-speciesnet-tarball.py` strips Kaggle's README). + +## Output schema + +The MD server emits one prediction object per image, matching the shape `inference-consumer.js` and `prediction.js` already consume: + +```json +{ + "filepath": "/path/to/image.jpg", + "detections": [ + { "label": "animal", "conf": 0.94, "xywhn": [0.5, 0.6, 0.3, 0.4], "xyxy": [...] } + ], + "classifications": {}, + "prediction": "animal", + "prediction_score": 0.94, + "model_version": "6.0" +} +``` + +- `prediction` is the label of the highest-confidence detection that passes `detectionConfidenceThreshold` (0.2 — MD's official default for v6). When no detection passes the threshold, `prediction` is `"blank"` and `prediction_score` is `null`. This mirrors how `run_deepfaune_server.py` emits `"blank"` when there are no detections. +- `classifications` is always an empty object — there is no classifier. Existing consumers already tolerate empty classifications (DeepFaune emits `{}` for blank frames). +- `detections[].xywhn` is the same center-format normalized bbox as DeepFaune/Manas, so the existing `xywhn` branch in `src/main/utils/bbox.js` handles it unchanged. + +## Pseudo-species mapping + +The three MD labels are stored as if they were species: + +| `prediction` field | Treated as scientific name | Common name | +|---|---|---| +| `animal` | `animal` | `Animal` | +| `person` | `person` | `Person` | +| `vehicle` | `vehicle` | `Vehicle` | +| `blank` | `blank` | (existing blank handling) | + +A new common-names source file resolves these: + +`src/shared/commonNames/sources/megadetector.json`: + +```json +{ + "modelId": "megadetector", + "modelVersion": "6.0", + "source": "MegaDetector v6 categories (animal / person / vehicle)", + "entries": [ + { "scientific": "animal", "common": "Animal" }, + { "scientific": "person", "common": "Person" }, + { "scientific": "vehicle", "common": "Vehicle" } + ] +} +``` + +User-facing consequence: in the Media tab the user filters `species = animal`, finds the non-blank frames, opens each one, and assigns the real species via the existing annotation UI. The MD prediction stays as the model's raw output in `model_outputs`; the user's annotation lives on the observation, same as any model. + +## Code changes + +### `python-environments/common/run_megadetector_server.py` (NEW) + +Structurally a stripped DeepFaune runner. CLI flags: `--filepath-detector-weights` and `--detection-confidence-threshold` (default 0.2). Reuses `VideoCapableLitAPI`, `safe_imread`, `to_detection_record`, and `propagate_extra_fields` from `utils.py` / `detection_utils.py`. **Does not** reuse `select_best_animal_detection`: that helper filters specifically for the `animal` class, but MD needs the best detection across all three classes (`animal`, `person`, `vehicle`). The server selects the top-confidence detection inline. No classifier import, no classifier load, no classifier inference. + +The `predict()` path: + +1. Run YOLO detector on the image. +2. Read class labels from the YOLO model's `names` attribute (same pattern as `run_deepfaune_server.py` line 440 — `class_names = detections.names`). For MDv6 these are `{0: "animal", 1: "person", 2: "vehicle"}`, but reading from the model rather than hard-coding the mapping insulates us if upstream ever renumbers. +3. Build `detection_records` from the YOLO output. +4. Pick the highest-confidence detection whose `conf >= --detection-confidence-threshold` — `prediction` becomes that detection's `label`, `prediction_score` becomes its `conf`. Default to `prediction: "blank"`, `prediction_score: null` when no detection passes. +5. Yield the dict shape shown in "Output schema" above. + +**Why the threshold appears in two places.** `mlmodels.js` carries `detectionConfidenceThreshold: 0.2` as it does for every other model — `inference-consumer.js` reads it from there (`this.model.detectionConfidenceThreshold` at server.ts/inference-consumer.js:154) to filter which detections become observations. The Python server *also* needs the threshold, because for MD the `prediction` field is itself derived from "is there a detection above threshold?" — there's no classifier to fall back on as a separate signal of "blank vs. not". The two values must agree; both are 0.2 for MD. `server.ts` passes the JS-side value through to the Python script via the CLI flag, so they stay in sync. + +### `src/main/services/ml/server.ts` + +Add `startMegaDetectorHTTPServer()` modeled on `startDeepFauneHTTPServer()`: + +```ts +interface MegaDetectorServerOptions { + port: number + detectorWeightsFilepath: string + detectionConfidenceThreshold: number + timeout: number + pythonEnvironment: { reference: { id: string; version: string } } +} + +export async function startMegaDetectorHTTPServer({ + port, + detectorWeightsFilepath, + detectionConfidenceThreshold, + timeout, + pythonEnvironment +}: MegaDetectorServerOptions): Promise<{ process: ChildProcess; shutdownApiKey: string }> { + // mirrors startDeepFauneHTTPServer; script path → run_megadetector_server.py +} +``` + +Add a new `case 'megadetector'` in the `switch` inside `startMLModelHTTPServer()`: + +```ts +case 'megadetector': { + const port = await resolveServerPort(is.dev ? 8003 : null) + const localInstallPath = getMLModelLocalInstallPath({ ...modelReference }) + const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10x.pt') + const model = findModel({ ...modelReference }) + const { process: pythonProcess, shutdownApiKey } = await startMegaDetectorHTTPServer({ + port, + detectorWeightsFilepath, + detectionConfidenceThreshold: model.detectionConfidenceThreshold, + timeout: 30, + pythonEnvironment + }) + registerActiveServer({ pid: pythonProcess.pid, port, shutdownApiKey, modelId: modelReference.id }) + return { port, process: pythonProcess, shutdownApiKey } +} +``` + +Dev port `8003` extends the existing fixed-port sequence (`8000` SpeciesNet, `8001` DeepFaune, `8002` Manas). + +### `src/shared/mlmodels.js` + +Add a fourth entry to `modelZoo`: + +```js +{ + reference: { id: 'megadetector', version: '6.0' }, + pythonEnvironment: { id: 'common', version: '0.1.4' }, + name: 'MegaDetector', + size_in_MB: , + files: 1, + downloadURL: + 'https://huggingface.co/earthtoolsmaker/megadetector/resolve/main/6.0.tar.gz?download=true', + description: + "MegaDetector is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, without identifying species. Useful as a fast blank-filter before manual species annotation.", + website: 'https://github.com/agentmorris/MegaDetector', + logo: 'megadetector', + detectionConfidenceThreshold: 0.2, + region: 'worldwide', + species_count: 3, + detectionOnly: true +} +``` + +The new `detectionOnly: true` flag is the *only* place the rest of the app needs to special-case MD. No `species_data` key — `detectionOnly` short-circuits the species panel before it would try to load one. + +### `src/main/utils/bbox.js` + +`megadetector` joins `manas` and `deepfaune` in the existing `xywhn` case — three labels falling through to the same block: + +```js +case 'manas': +case 'megadetector': +case 'deepfaune': { + // existing xywhn → top-left conversion +} +``` + +`detectModelType()` already has a generic xywhn fallback, but for clarity we add an explicit branch: `if (version === '6.0' && prediction.detections?.[0]?.xywhn) return 'megadetector'`. + +### `src/renderer/src/models/ModelCard.jsx` + +One conditional on `model.detectionOnly`: + +- Replace the `v{version} · {size} · {species_count} species` line with `v{version} · {size} · Detection only · 3 categories`. +- Skip the `▸ View {species_count} species` toggle and the `` mount. + +### `src/renderer/src/models/ModelSelect.jsx` + +Same conditional in the rich-card dropdown rows — show "Detection only" instead of the species count. + +### `src/renderer/src/models/SpeciesPanel.jsx` + +No changes. `detectionOnly: true` means the panel is never mounted for MD. + +### Logo asset + +Add `src/renderer/src/assets/logos/megadetector.png` (maintainer provides). The existing logo resolution code is data-driven on the `logo` field — no code change needed beyond placing the file. + +## Build script + +`scripts/build-megadetector-tarball.py` — same shape as `build-speciesnet-tarball.py`, using only the Python standard library: + +1. Download `MDV6-yolov10x.pt` from `https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10x.pt`. +2. Verify SHA256 against the hash of the copy already shipped in the DeepFaune tarball (locked at build time after first run — encoded as a constant in the script the same way `MEGADETECTOR_SHA256` is in `build-speciesnet-tarball.py`). +3. Place the weights at `6.0/MDV6-yolov10x.pt`. +4. `tar -czf dist/6.0.tar.gz 6.0/`. +5. Print final size + SHA256 + file list for the `mlmodels.js` update. + +The upload step (`huggingface-cli upload …` to `earthtoolsmaker/megadetector`) stays manual — credentials are user-scoped. Maintainer (Arthur) uploads. + +## Verification + +1. **Build script self-check**: tarball contains exactly one file at `6.0/MDV6-yolov10x.pt`; SHA256 of inner file matches the hash also shipped in the DeepFaune tarball. +2. **`npm test`**: covers `mlmodels.test.js` (new entry valid), `bbox.test.js` (new megadetector case routes to xywhn), `downloadState.test.js`, common-names resolver tests (megadetector source loads). +3. **`cd python-environments/common && make lint && make format && make test`**: includes a new `tests/test_megadetector_server.py`. Test cases: + - Start the server with the bundled weights. + - POST a known-animal image → `prediction == "animal"`, `prediction_score > 0.2`, at least one detection with `label == "animal"`. + - POST a known-blank image → `prediction == "blank"`, `prediction_score is None`, `detections == []` (or all below threshold). +4. **Manual smoke test**: fresh install → download MegaDetector (model card shows "Detection only · 3 categories", no "View species" link) → import ~20 mixed images (animals + people + a vehicle + blanks) → Sources tab shows correct counts per pseudo-species → open one `animal` frame → re-annotate it with a real species via the existing annotation UI → confirm the user's annotation persists and the original MD prediction is preserved in `model_outputs`. +5. **Multi-model regression check**: with MegaDetector installed alongside SpeciesNet, kick off an import using SpeciesNet, then a second import using MegaDetector. Confirm both servers can coexist (different ports) and predictions land under the correct topic in the queue. + +## Release notes + +User-facing changelog should call out: + +1. **New model: MegaDetector v6.** Worldwide animal/person/vehicle detector. Useful as a fast blank-filter before manual species annotation. +2. MegaDetector does not identify species — it only detects whether a frame contains an animal, a person, or a vehicle. Use it to triage large folders, then annotate species manually inside Biowatch. +3. ~110 MB download. Reuses the existing `common` Python environment, so no extra environment install if you already have SpeciesNet/DeepFaune/Manas. + +## Documentation updates + +- `docs/http-servers.md` — add MegaDetector to the "Supported Models" table (Focus: "Blank filter / worldwide detection"; Species Coverage: "3 categories (animal, person, vehicle)"). Add `run_megadetector_server.py` to the project-structure listing. +- `docs/specs/2026-05-16-megadetector-model-choice-design.md` — this document (checked in). + +## Open dependencies on the user + +1. Run `python scripts/build-megadetector-tarball.py` once it lands. +2. Upload the resulting `6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector/main`. +3. Provide the final tarball size in MB for `size_in_MB` in `mlmodels.js`. +4. Provide a `megadetector.png` logo for `src/renderer/src/assets/logos/`. From cf37a5f904232011ffa0522e0dd28a8c5106920e Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 13:46:13 +0200 Subject: [PATCH 02/24] docs(plans): implementation plan for MegaDetector model choice --- ...26-05-16-megadetector-model-choice-plan.md | 1421 +++++++++++++++++ 1 file changed, 1421 insertions(+) create mode 100644 docs/plans/2026-05-16-megadetector-model-choice-plan.md diff --git a/docs/plans/2026-05-16-megadetector-model-choice-plan.md b/docs/plans/2026-05-16-megadetector-model-choice-plan.md new file mode 100644 index 00000000..e9ca577b --- /dev/null +++ b/docs/plans/2026-05-16-megadetector-model-choice-plan.md @@ -0,0 +1,1421 @@ +# MegaDetector Model Choice Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Ship MegaDetector v6 (`MDV6-yolov10-e`) as a standalone model choice in Biowatch — a fast blank-filter that detects `animal` / `person` / `vehicle` worldwide, no species classification. + +**Architecture:** New Python LitServe server (`run_megadetector_server.py`) modeled on `run_deepfaune_server.py` minus the classifier; new tarball hosted at `huggingface.co/earthtoolsmaker/megadetector/main/6.0.tar.gz`; new `detectionOnly: true` flag in `mlmodels.js` that suppresses the species panel in two UI components. MD's three labels (`animal` / `person` / `vehicle` + implicit `blank`) flow through Biowatch's existing species-centric data model as pseudo-species, so no changes to observations / annotations / exports are needed. + +**Tech Stack:** Electron + React (renderer), Node.js (main), Python 3.11 + LitServe + `ultralytics.YOLO` (model server), Drizzle ORM (SQLite), `node:test` (JS test runner), pytest (Python e2e tests). + +**Reference spec:** `docs/specs/2026-05-16-megadetector-model-choice-design.md`. + +--- + +## File Structure + +**Create:** +- `scripts/build-megadetector-tarball.py` — packaging script (downloads weights, builds tarball) +- `python-environments/common/run_megadetector_server.py` — LitServe HTTP server +- `python-environments/common/tests/test_megadetector_server.py` — e2e tests +- `src/shared/commonNames/sources/megadetector.json` — 3-entry common-name source + +**Modify:** +- `src/shared/mlmodels.js` — add `megadetector` entry with `detectionOnly: true` +- `src/main/services/ml/server.ts` — add `startMegaDetectorHTTPServer()` + new `case 'megadetector'` +- `src/main/utils/bbox.js` — add `megadetector` to xywhn case + `detectModelType()` branch +- `src/renderer/src/models/ModelCard.jsx` — gate species panel on `!detectionOnly` +- `src/renderer/src/models/ModelSelect.jsx` — show "Detection only" when `detectionOnly` +- `scripts/build-common-names-dict.js` — add `megadetector.json` to merge order +- `scripts/lib/aliases.js` — add megadetector source to imports + sources list +- `src/shared/commonNames/dictionary.json` — regenerated by `npm run dict:build` (do not edit by hand) +- `python-environments/common/scripts/download_model.py` — add megadetector entry +- `python-environments/common/Makefile` — add megadetector env var + download command +- `docs/http-servers.md` — add MegaDetector to supported-models table + project-structure listing +- `test/shared/mlmodels.test.js` — extend coverage for the new entry +- `test/main/utils/bbox.test.js` (create if absent) — covers the new megadetector branch +- `test/shared/commonNames/resolver.test.js` — covers `animal`/`person`/`vehicle` resolution + +**User-provided (NOT in this plan):** +- Run the build script and upload `6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector/main`. +- Provide back the final tarball size in MB for `size_in_MB` in `mlmodels.js`. +- Drop a `megadetector.png` logo into `src/renderer/src/assets/logos/`. + +--- + +## Task 1: Build script (`scripts/build-megadetector-tarball.py`) + +**Files:** +- Create: `scripts/build-megadetector-tarball.py` + +- [ ] **Step 1: Verify the upstream weights URL** + +```bash +curl -I -L "https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10-e.pt" +``` +Expected: HTTP 200/302 (release exists). If the filename differs in the release manifest, use the actual filename returned by `gh release view v6.0 --repo agentmorris/MegaDetector --json assets` instead, and update every occurrence of `MDV6-yolov10-e.pt` in this plan to match. + +- [ ] **Step 2: Write the build script** + +Mirror `scripts/build-speciesnet-tarball.py` exactly — stdlib only (`urllib`, `tarfile`, `hashlib`, `json`), same `download()` / `sha256_of()` helpers, same `--version` / `--output-dir` / `--work-dir` CLI. + +```python +#!/usr/bin/env python3 +"""Build a MegaDetector model tarball ready to upload to huggingface.co/earthtoolsmaker/megadetector. + +Downloads the official MDv6 release weights and the upstream LICENSE so AGPL attribution +travels with the binary, then tar-gzips them into a directory named for the version. + +Usage: + python scripts/build-megadetector-tarball.py --version 6.0 + +Produces dist/.tar.gz alongside a summary of size + SHA256 + file list. +""" + +import argparse +import hashlib +import shutil +import sys +import tarfile +import tempfile +import urllib.request +from pathlib import Path + +WEIGHTS_URL = "https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10-e.pt" +WEIGHTS_FILENAME = "MDV6-yolov10-e.pt" +WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN" # paste actual hash printed at end of first run +LICENSE_URL = "https://raw.githubusercontent.com/agentmorris/MegaDetector/main/LICENSE" +LICENSE_FILENAME = "LICENSE" + + +def download(url: str, dest: Path) -> None: + if dest.exists(): + print(f"[skip] {dest.name} already present ({dest.stat().st_size / 1e6:.0f} MB)") + return + print(f"[download] {url}") + is_tty = sys.stdout.isatty() + last_milestone = -1 + + def report(block_num, block_size, total_size): + nonlocal last_milestone + if total_size <= 0: + return + downloaded = block_num * block_size + pct = min(100, downloaded * 100 // total_size) + if is_tty: + sys.stdout.write(f"\r {pct:3d}% {downloaded / 1e6:7.1f} / {total_size / 1e6:.0f} MB") + sys.stdout.flush() + else: + milestone = pct // 10 + if milestone > last_milestone: + last_milestone = milestone + print(f" {pct:3d}% {downloaded / 1e6:7.1f} / {total_size / 1e6:.0f} MB") + + urllib.request.urlretrieve(url, dest, reporthook=report) + if is_tty: + sys.stdout.write("\n") + + +def sha256_of(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as fp: + for chunk in iter(lambda: fp.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def build(version: str, output_dir: Path, work_dir: Path) -> Path: + work_dir.mkdir(parents=True, exist_ok=True) + output_dir.mkdir(parents=True, exist_ok=True) + + model_dir = work_dir / version + if model_dir.exists(): + shutil.rmtree(model_dir) + model_dir.mkdir() + + weights_path = model_dir / WEIGHTS_FILENAME + download(WEIGHTS_URL, weights_path) + + actual_hash = sha256_of(weights_path) + if WEIGHTS_SHA256 != "TBD_LOCK_ON_FIRST_RUN" and actual_hash != WEIGHTS_SHA256: + raise RuntimeError(f"Weights SHA256 mismatch:\n expected {WEIGHTS_SHA256}\n got {actual_hash}") + print(f"[verify] {WEIGHTS_FILENAME} SHA256 = {actual_hash}") + + download(LICENSE_URL, model_dir / LICENSE_FILENAME) + + tarball = output_dir / f"{version}.tar.gz" + if tarball.exists(): + tarball.unlink() + print(f"[tar] {tarball}") + with tarfile.open(tarball, "w:gz") as tar: + tar.add(model_dir, arcname=version) + return tarball + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--version", required=True, help="MegaDetector version, e.g. 6.0") + parser.add_argument("--output-dir", type=Path, default=Path("dist")) + parser.add_argument( + "--work-dir", + type=Path, + default=Path(tempfile.gettempdir()) / "megadetector-build", + help="Working directory for downloads (retained between runs)", + ) + args = parser.parse_args() + + tarball = build(args.version, args.output_dir, args.work_dir) + + size_mb = tarball.stat().st_size / 1e6 + digest = sha256_of(tarball) + with tarfile.open(tarball) as tar: + members = sorted(m.name for m in tar.getmembers() if not m.isdir()) + + print() + print("=" * 60) + print(f" Tarball: {tarball}") + print(f" Size: {size_mb:.1f} MB") + print(f" SHA256: {digest}") + print(" Contents:") + for name in members: + print(f" {name}") + print("=" * 60) + + +if __name__ == "__main__": + main() +``` + +- [ ] **Step 3: Make it executable and run it once locally** + +```bash +chmod +x scripts/build-megadetector-tarball.py +python3 scripts/build-megadetector-tarball.py --version 6.0 +``` +Expected output (last 8 lines): +``` +============================================================ + Tarball: dist/6.0.tar.gz + Size: ~30 MB + SHA256: <64 hex chars> + Contents: + 6.0/LICENSE + 6.0/MDV6-yolov10-e.pt +============================================================ +``` + +- [ ] **Step 4: Lock the SHA256** + +Replace the `WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN"` line with the actual hash printed by Step 3 (the per-file hash of `MDV6-yolov10-e.pt`, not the tarball hash — re-run the script if you only kept the tarball hash; the per-file hash is printed on the line starting with `[verify]`). + +- [ ] **Step 5: Run it again to confirm SHA verification passes** + +```bash +rm -rf /tmp/megadetector-build dist/6.0.tar.gz +python3 scripts/build-megadetector-tarball.py --version 6.0 +``` +Expected: same output, and the `[verify]` line shows the SHA matches the constant (no `Weights SHA256 mismatch` error). + +- [ ] **Step 6: Commit** + +```bash +git add scripts/build-megadetector-tarball.py +git commit -m "tools(megadetector): script to build MD v6 tarball for HF upload" +``` + +**>>> EXTERNAL DEPENDENCY: After this task, the maintainer (Arthur) uploads `dist/6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector/main` and shares back the exact tarball size in MB. Subsequent tasks can proceed in parallel — only Task 5 (Python server e2e tests) and Task 9 (Python tooling) depend on the upload landing first.** + +--- + +## Task 2: Common-names source + dictionary rebuild + +**Files:** +- Create: `src/shared/commonNames/sources/megadetector.json` +- Modify: `scripts/build-common-names-dict.js` +- Modify: `scripts/lib/aliases.js` +- Modify: `src/shared/commonNames/dictionary.json` (regenerated, do not edit by hand) +- Modify: `test/shared/commonNames/resolver.test.js` + +- [ ] **Step 1: Create the megadetector source file** + +`src/shared/commonNames/sources/megadetector.json`: + +```json +{ + "modelId": "megadetector", + "modelVersion": "6.0", + "source": "MegaDetector v6 categories (animal / person / vehicle)", + "entries": [ + { + "scientificName": null, + "label": "animal", + "commonName": "Animal" + }, + { + "scientificName": null, + "label": "person", + "commonName": "Person" + }, + { + "scientificName": null, + "label": "vehicle", + "commonName": "Vehicle" + } + ] +} +``` + +This matches the manas.json schema exactly: `scientificName: null` for non-binomial labels. + +- [ ] **Step 2: Add the source to the dictionary-build pipeline** + +In `scripts/build-common-names-dict.js`, modify the `order` array at line ~97: + +```js +const order = ['speciesnet.json', 'deepfaune.json', 'manas.json', 'megadetector.json'] +``` + +- [ ] **Step 3: Add the source to the alias-map builder** + +In `scripts/lib/aliases.js`, modify lines 1–17: + +```js +import speciesnetSource from '../../src/shared/commonNames/sources/speciesnet.json' with { type: 'json' } +import deepfauneSource from '../../src/shared/commonNames/sources/deepfaune.json' with { type: 'json' } +import manasSource from '../../src/shared/commonNames/sources/manas.json' with { type: 'json' } +import megadetectorSource from '../../src/shared/commonNames/sources/megadetector.json' with { type: 'json' } +import extras from '../../src/shared/commonNames/extras.json' with { type: 'json' } +import { normalizeScientificName } from '../../src/shared/commonNames/normalize.js' + +export function buildAliasMap() { + const aliases = new Map() + const sources = [speciesnetSource, deepfauneSource, manasSource, megadetectorSource, extras] + // ...rest unchanged +} +``` + +(`megadetectorSource` entries have `scientificName: null`, so the alias loop skips them per the existing `if (!entry.scientificName || !entry.label) continue` guard — adding it to the list is a no-op for aliases but keeps the imports symmetric with the dict builder.) + +- [ ] **Step 4: Add the failing resolver test** + +Append to `test/shared/commonNames/resolver.test.js`: + +```js +test('resolveCommonName returns Animal / Person / Vehicle for MegaDetector labels', () => { + assert.equal(resolveCommonName('animal'), 'animal') + assert.equal(resolveCommonName('person'), 'person') + assert.equal(resolveCommonName('vehicle'), 'vehicle') +}) +``` + +Note: dictionary values are lowercased by `mergeEntries` (build-common-names-dict.js:72), so `"Animal"` becomes `"animal"` in the dictionary — assertions match the stored value. + +- [ ] **Step 5: Run the test to confirm it fails (entries not yet in dictionary)** + +```bash +node --test test/shared/commonNames/resolver.test.js +``` +Expected: the new assertions fail (resolver returns `null` for `animal`/`person`/`vehicle` because the dictionary hasn't been rebuilt yet). + +- [ ] **Step 6: Rebuild the dictionary** + +```bash +npm run dict:build +``` +Expected stdout: +``` +Wrote entries to .../dictionary.json +Wrote label aliases to .../labelAliases.json +``` +(N+3 reflects the three new keys.) + +- [ ] **Step 7: Verify the dictionary contains the new keys** + +```bash +grep -E '"(animal|person|vehicle)":' src/shared/commonNames/dictionary.json +``` +Expected: three lines, each with the lowercased common name. (Some of `animal`/`person`/`vehicle` may already exist as keys from other sources — that's fine; the merge order puts megadetector last among the model sources but `extras.json` still wins after it, so existing curated values are preserved. Confirm none of the three points to something unexpected.) + +- [ ] **Step 8: Run the resolver test again to confirm it passes** + +```bash +node --test test/shared/commonNames/resolver.test.js +``` +Expected: all tests pass, including the three new assertions. + +- [ ] **Step 9: Commit** + +```bash +git add src/shared/commonNames/sources/megadetector.json \ + scripts/build-common-names-dict.js \ + scripts/lib/aliases.js \ + src/shared/commonNames/dictionary.json \ + test/shared/commonNames/resolver.test.js +git commit -m "feat(models): common-name source for MegaDetector labels" +``` + +--- + +## Task 3: Model zoo entry (`src/shared/mlmodels.js`) + +**Files:** +- Modify: `src/shared/mlmodels.js` +- Modify: `test/shared/mlmodels.test.js` + +- [ ] **Step 1: Write the failing test** + +Append to the `describe('findModel', ...)` block in `test/shared/mlmodels.test.js`: + +```js +test('should find megadetector entry with detectionOnly flag', () => { + const md = findModel({ id: 'megadetector', version: '6.0' }) + assert(md, 'megadetector entry should exist') + assert.equal(md.detectionOnly, true, 'megadetector should be marked detectionOnly') + assert.equal(md.region, 'worldwide', 'megadetector should be worldwide') + assert.equal(md.species_count, 3, 'megadetector should report 3 categories') + assert(md.downloadURL.includes('earthtoolsmaker/megadetector'), 'should download from ETM HF') +}) +``` + +- [ ] **Step 2: Run the test to verify it fails** + +```bash +node --test test/shared/mlmodels.test.js +``` +Expected: FAIL — `megadetector entry should exist` (`findModel` returns null). + +- [ ] **Step 3: Add the megadetector entry** + +In `src/shared/mlmodels.js`, append a fourth object to the `modelZoo` array (after the Manas entry, before the closing `]`): + +```js +, +{ + reference: { id: 'megadetector', version: '6.0' }, + pythonEnvironment: { id: 'common', version: '0.1.4' }, + name: 'MegaDetector', + size_in_MB: 30, // PLACEHOLDER — replace with exact size from build-megadetector-tarball.py output + files: 2, + downloadURL: + 'https://huggingface.co/earthtoolsmaker/megadetector/resolve/main/6.0.tar.gz?download=true', + description: + "MegaDetector is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, without identifying species. Useful as a fast blank-filter before manual species annotation.", + website: 'https://github.com/agentmorris/MegaDetector', + logo: 'megadetector', + detectionConfidenceThreshold: 0.2, + region: 'worldwide', + species_count: 3, + detectionOnly: true +} +``` + +(`size_in_MB` is a placeholder — once the maintainer uploads the tarball and reports the size, update this line in a follow-up commit. The structural tests pass on any positive number.) + +- [ ] **Step 4: Run the test to verify it passes** + +```bash +node --test test/shared/mlmodels.test.js +``` +Expected: all tests pass, including all existing structural validations applied to the new entry. + +- [ ] **Step 5: Commit** + +```bash +git add src/shared/mlmodels.js test/shared/mlmodels.test.js +git commit -m "feat(models): MegaDetector entry in model zoo" +``` + +--- + +## Task 4: `bbox.js` — megadetector routing + +**Files:** +- Modify: `src/main/utils/bbox.js` +- Create or modify: `test/main/utils/bbox.test.js` + +- [ ] **Step 1: Check if a bbox test file exists** + +```bash +ls test/main/utils/bbox.test.js 2>/dev/null || echo "no test file" +``` +If "no test file", create `test/main/utils/bbox.test.js` with this scaffolding: + +```js +import { test, describe } from 'node:test' +import assert from 'node:assert/strict' +import { transformBboxToCamtrapDP, detectModelType } from '../../../src/main/utils/bbox.js' +``` + +- [ ] **Step 2: Write the failing tests** + +Append to `test/main/utils/bbox.test.js`: + +```js +describe('MegaDetector bbox handling', () => { + test('transformBboxToCamtrapDP routes megadetector through xywhn → top-left', () => { + const detection = { xywhn: [0.5, 0.5, 0.4, 0.6] } + const result = transformBboxToCamtrapDP(detection, 'megadetector') + assert.deepEqual(result, { + bboxX: 0.3, // 0.5 - 0.4/2 + bboxY: 0.2, // 0.5 - 0.6/2 + bboxWidth: 0.4, + bboxHeight: 0.6 + }) + }) + + test('detectModelType identifies megadetector by version 6.0 + xywhn', () => { + const prediction = { + model_version: '6.0', + detections: [{ xywhn: [0.5, 0.5, 0.4, 0.6], conf: 0.9, label: 'animal' }] + } + assert.equal(detectModelType(prediction), 'megadetector') + }) +}) +``` + +- [ ] **Step 3: Run the tests to verify they fail** + +```bash +node --test test/main/utils/bbox.test.js +``` +Expected: FAIL — `transformBboxToCamtrapDP` logs `Unknown model type for bbox transformation: megadetector` and returns `null`; `detectModelType` likely returns `'unknown'` or routes through a different branch. + +- [ ] **Step 4: Add megadetector to both functions in `src/main/utils/bbox.js`** + +In the `transformBboxToCamtrapDP` switch (around line 45), change the existing `case 'manas': case 'deepfaune':` block to include megadetector — these three all use `xywhn`: + +```js +case 'manas': +case 'megadetector': +case 'deepfaune': { + // existing xywhn → top-left conversion, unchanged + ... +} +``` + +In `detectModelType` (around line 85), add an explicit megadetector branch *before* the generic xywhn fallback at the bottom: + +```js +// MegaDetector v6.0 uses xywhn (ultralytics YOLO output) and ships model_version '6.0' +if (prediction.model_version === '6.0' && prediction.detections?.[0]?.xywhn) { + return 'megadetector' +} +``` + +Place this branch directly after the `if (version === '1.0' && prediction.detections?.[0]?.xywhn) return 'manas'` line. Order matters because the generic xywhn fallback below would otherwise misroute it. + +- [ ] **Step 5: Run the tests to verify they pass** + +```bash +node --test test/main/utils/bbox.test.js +``` +Expected: PASS. + +- [ ] **Step 6: Commit** + +```bash +git add src/main/utils/bbox.js test/main/utils/bbox.test.js +git commit -m "feat(models): bbox + model-type detection for MegaDetector" +``` + +--- + +## Task 5: Python HTTP server (`run_megadetector_server.py` + tests) + +**Files:** +- Create: `python-environments/common/run_megadetector_server.py` +- Create: `python-environments/common/tests/test_megadetector_server.py` + +**Depends on:** The maintainer has uploaded the tarball to HF (Task 1's external dependency). If not yet uploaded, you can still write the code; tests will skip via `pytest.skipif` until weights are downloadable. + +- [ ] **Step 1: Write the e2e test file** + +Create `python-environments/common/tests/test_megadetector_server.py`: + +```python +"""End-to-end tests for MegaDetector ML server.""" + +import os + +import httpx +import pytest + +from tests.conftest import ServerProcess, find_free_port, parse_streaming_response + +MEGADETECTOR_DETECTOR_WEIGHTS = os.environ.get("MEGADETECTOR_DETECTOR_WEIGHTS", "") + +pytestmark = pytest.mark.skipif( + not MEGADETECTOR_DETECTOR_WEIGHTS, + reason="MegaDetector weights not available. Set MEGADETECTOR_DETECTOR_WEIGHTS.", +) + + +@pytest.fixture(scope="module") +def megadetector_server(): + """Start MegaDetector server for the test module.""" + port = find_free_port() + command = [ + "uv", + "run", + "python", + "run_megadetector_server.py", + "--port", + str(port), + "--timeout", + "60", + "--filepath-detector-weights", + MEGADETECTOR_DETECTOR_WEIGHTS, + "--detection-confidence-threshold", + "0.2", + ] + with ServerProcess(command, port, startup_timeout=300) as server: + yield server + + +class TestMegaDetectorServer: + """Tests for MegaDetector server endpoints.""" + + def test_health_endpoint(self, megadetector_server): + resp = httpx.get(f"{megadetector_server.base_url}/health", timeout=10.0) + assert resp.status_code == 200 + assert resp.text == "ok" + + def test_info_endpoint(self, megadetector_server): + resp = httpx.get(f"{megadetector_server.base_url}/info", timeout=10.0) + assert resp.status_code == 200 + data = resp.json() + assert "model" in data + assert data["model"]["type"] == "megadetector" + assert "server" in data + + def test_predict_animal_image(self, megadetector_server, test_images): + """A clearly-animal image should produce prediction='animal'.""" + payload = {"instances": [{"filepath": str(test_images["chamois"])}]} + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=60.0, + ) as resp: + assert resp.status_code == 200 + results = parse_streaming_response(resp) + + assert len(results) == 1 + pred = results[0]["output"]["predictions"][0] + assert pred["filepath"] == str(test_images["chamois"]) + assert pred["prediction"] == "animal" + assert pred["prediction_score"] >= 0.2 + assert pred["classifications"] == {} + assert len(pred["detections"]) >= 1 + assert pred["detections"][0]["label"] in {"animal", "person", "vehicle"} + assert "xywhn" in pred["detections"][0] + + def test_predict_human_image(self, megadetector_server, test_images): + """A human image should produce prediction='person'.""" + payload = {"instances": [{"filepath": str(test_images["human"])}]} + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=60.0, + ) as resp: + results = parse_streaming_response(resp) + pred = results[0]["output"]["predictions"][0] + assert pred["prediction"] == "person" + + def test_predict_empty_image(self, megadetector_server, test_images): + """An empty (no subject) image should produce prediction='blank'.""" + payload = {"instances": [{"filepath": str(test_images["empty"])}]} + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=60.0, + ) as resp: + results = parse_streaming_response(resp) + pred = results[0]["output"]["predictions"][0] + assert pred["prediction"] == "blank" + assert pred["prediction_score"] is None + + def test_predict_streaming(self, megadetector_server, test_images): + """Streaming returns one chunk per image, all classifications empty.""" + payload = { + "instances": [ + {"filepath": str(test_images["badger"])}, + {"filepath": str(test_images["fox"])}, + {"filepath": str(test_images["empty"])}, + ] + } + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=120.0, + ) as resp: + results = parse_streaming_response(resp) + assert len(results) == 3 + for r in results: + pred = r["output"]["predictions"][0] + assert pred["classifications"] == {} + assert pred["model_version"] == "6.0" + + def test_predict_invalid_filepath(self, megadetector_server): + payload = {"instances": [{"filepath": "/nonexistent/image.jpg"}]} + resp = httpx.post( + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=30.0, + ) + assert resp.status_code in (200, 400) + + def test_docs_endpoint(self, megadetector_server): + resp = httpx.get(f"{megadetector_server.base_url}/docs", timeout=10.0) + assert resp.status_code == 200 + assert "swagger" in resp.text.lower() or "openapi" in resp.text.lower() +``` + +- [ ] **Step 2: Write the server file** + +Create `python-environments/common/run_megadetector_server.py`: + +```python +""" +CLI script to run MegaDetector v6 as a LitServer. + +MegaDetector v6 is a YOLO-based detector (animals / people / vehicles only — no +species classification). Used in Biowatch as a fast blank-filter before manual +species annotation. + +Start the server: + +``` +run_megadetector_server.py \ + --port 8000 \ + --filepath-detector-weights ./path/to/MDV6-yolov10-e.pt \ + --detection-confidence-threshold 0.2 +``` + +A Swagger API documentation is served at localhost:${port}/docs + +health: + +``` +$ curl http://localhost:${port}/health +"ok" +``` + +predict (streaming): + +``` +$ curl -X POST http://localhost:${port}/predict \ + -H "Content-Type: application/json" \ + -d '{"instances": [{"filepath": "/path/to/your/image"}]}' +``` + +Output per image: + +```json +{ + "output": { + "predictions": [{ + "filepath": "/path/to/image", + "classifications": {}, + "detections": [ + {"label": "animal", "conf": 0.94, "xywhn": [...], "xyxy": [...]} + ], + "prediction": "animal", + "prediction_score": 0.94, + "model_version": "6.0" + }] + } +} +``` +""" + +import logging +from pathlib import Path + +import litserve as ls +from absl import app, flags +from fastapi import HTTPException +from ultralytics import YOLO + +from detection_utils import propagate_extra_fields, to_detection_record +from utils import VideoCapableLitAPI, is_video_file, safe_imread + +logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s") +logger = logging.getLogger(__name__) + +MODEL_VERSION = "6.0" + +_PORT = flags.DEFINE_integer("port", 8000, "Port to run the server on.") +_API_PATH = flags.DEFINE_string("api_path", "/predict", "URL path for the server endpoint.") +_WORKERS_PER_DEVICE = flags.DEFINE_integer("workers_per_device", 1, "Number of server replicas per device.") +_TIMEOUT = flags.DEFINE_integer("timeout", 30, "Timeout (in seconds) for requests.") +_BACKLOG = flags.DEFINE_integer("backlog", 2048, "Maximum number of connections to hold in backlog.") +_FILEPATH_DETECTOR_WEIGHTS = flags.DEFINE_string( + name="filepath-detector-weights", + default=None, + help="filepath for the weights of the MegaDetector detector", + required=True, +) +_DETECTION_CONFIDENCE_THRESHOLD = flags.DEFINE_float( + "detection-confidence-threshold", + 0.2, + "Confidence threshold below which a detection is ignored when computing the top prediction. " + "Detections themselves are always returned; this only controls the 'prediction'/'prediction_score' fields.", +) +_EXTRA_FIELDS = flags.DEFINE_list( + "extra_fields", + None, + "Comma-separated list of extra fields to propagate from request to response.", +) + + +def predict_one( + detector: YOLO, + filepath: Path, + confidence_threshold: float, + model_version: str = MODEL_VERSION, +) -> dict: + """Run MegaDetector on a single image and produce a Biowatch-compatible prediction dict. + + The 'prediction' field is the label of the highest-confidence detection + whose conf >= confidence_threshold. When no detection passes, prediction + is 'blank' and prediction_score is None. + """ + imagecv = safe_imread(filepath) + ultralytics_results = detector(imagecv, verbose=False) + yolo_out = ultralytics_results[0] + bboxes = yolo_out.boxes + # class_names comes from the .pt file's embedded `names` attribute — never hard-code. + class_names = yolo_out.names + + detection_records = [ + to_detection_record( + conf=conf, + class_instance=class_instance, + xywhn=xywhn, + xyxy=xyxy, + class_label_mapping=class_names, + ) + for conf, class_instance, xywhn, xyxy in zip( + bboxes.conf.cpu().tolist(), + bboxes.cls.cpu().numpy().astype(int).tolist(), + bboxes.xywhn.cpu().numpy().tolist(), + bboxes.xyxy.cpu().numpy().tolist(), + strict=True, + ) + ] + + # MD detects across animal/person/vehicle — pick top-confidence across ALL classes + # (cannot use select_best_animal_detection, which filters for the animal class). + above = [d for d in detection_records if d["conf"] >= confidence_threshold] + if not above: + return { + "predictions": [ + { + "filepath": str(filepath), + "classifications": {}, + "detections": detection_records, + "prediction": "blank", + "prediction_score": None, + "model_version": model_version, + } + ], + } + + top = max(above, key=lambda d: d["conf"]) + return { + "predictions": [ + { + "filepath": str(filepath), + "classifications": {}, + "detections": detection_records, + "prediction": top["label"], + "prediction_score": top["conf"], + "model_version": model_version, + } + ], + } + + +class MegaDetectorLitAPI(ls.LitAPI, VideoCapableLitAPI): + """MegaDetector API server with video support.""" + + def __init__( + self, + filepath_detector_weights: Path, + detection_confidence_threshold: float, + extra_fields: list[str] | None = None, + *args, + **kwargs, + ) -> None: + super().__init__(*args, **kwargs) + self.filepath_detector_weights = filepath_detector_weights + self.detection_confidence_threshold = detection_confidence_threshold + self.extra_fields = extra_fields or [] + + def setup(self, device): + del device # Unused. + self.detector = YOLO(self.filepath_detector_weights) + + def decode_request(self, request, **kwargs): + for instance in request["instances"]: + filepath = instance["filepath"] + if not is_video_file(filepath) and not Path(filepath).exists(): + raise HTTPException(400, f"Cannot access filepath: `{filepath}`") + return request + + def _predict_single_image(self, filepath: str, **kwargs) -> dict: + single_instances_dict = {"instances": [{"filepath": filepath}]} + single_predictions_dict = predict_one( + detector=self.detector, + filepath=Path(filepath), + confidence_threshold=self.detection_confidence_threshold, + ) + return propagate_extra_fields(self.extra_fields, single_instances_dict, single_predictions_dict) + + def predict(self, x, **kwargs): + instances = x.get("instances", []) + logger.info(f"[MegaDetector] Processing {len(instances)} instances") + try: + yield from self.predict_with_video_support(x, **kwargs) + except Exception as e: + logger.error(f"[MegaDetector] Prediction failed: {e}", exc_info=True) + raise + + def encode_response(self, output, **kwargs): + for out in output: + yield {"output": out} + + +def main(argv: list[str]) -> None: + del argv # Unused. + print("[STARTUP] Starting MegaDetector LitServer...") + api = MegaDetectorLitAPI( + filepath_detector_weights=Path(_FILEPATH_DETECTOR_WEIGHTS.value), + detection_confidence_threshold=_DETECTION_CONFIDENCE_THRESHOLD.value, + extra_fields=_EXTRA_FIELDS.value, + api_path=_API_PATH.value, + stream=True, + ) + model_metadata = {"version": MODEL_VERSION, "type": "megadetector"} + server = ls.LitServer( + api, + accelerator="auto", + devices="auto", + workers_per_device=_WORKERS_PER_DEVICE.value, + model_metadata=model_metadata, + timeout=_TIMEOUT.value, + enable_shutdown_api=True, + ) + server.run( + port=_PORT.value, + generate_client_file=False, + backlog=_BACKLOG.value, + ) + + +if __name__ == "__main__": + app.run(main) +``` + +- [ ] **Step 3: Lint + format the Python files** + +```bash +cd python-environments/common +make lint +make format +``` +Expected: zero errors. If lint flags anything, fix it. If format reformats files, accept the changes. + +- [ ] **Step 4: Run the Python tests (requires weights downloaded)** + +Skip if the HF upload hasn't landed yet — the tests `pytest.skipif` themselves. + +```bash +cd python-environments/common +# Once the tarball is on HF, this will work (the env var is set by the make target +# added in Task 9, but you can also export it manually for this one-shot run): +export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e.pt +make test +``` +Expected: the new `TestMegaDetectorServer` class runs and passes all assertions. If `MEGADETECTOR_DETECTOR_WEIGHTS` is unset, the class is skipped — that's acceptable for landing the PR, with a follow-up to wire up Task 9 after the upload. + +- [ ] **Step 5: Commit** + +```bash +git add python-environments/common/run_megadetector_server.py \ + python-environments/common/tests/test_megadetector_server.py +git commit -m "feat(megadetector): LitServe HTTP server + e2e tests" +``` + +--- + +## Task 6: `server.ts` — server launcher + switch case + +**Files:** +- Modify: `src/main/services/ml/server.ts` + +- [ ] **Step 1: Add the `startMegaDetectorHTTPServer()` helper** + +In `src/main/services/ml/server.ts`, immediately after `startManasHTTPServer()` (around line 510, just before the `// ============= Server Lifecycle Management` section), add: + +```ts +interface MegaDetectorServerOptions { + port: number + detectorWeightsFilepath: string + detectionConfidenceThreshold: number + timeout: number + pythonEnvironment: { + reference: { id: string; version: string } + } +} + +/** + * Starts the MegaDetector HTTP server. + * + * @async + * @param {MegaDetectorServerOptions} options + * @returns {Promise<{process: ChildProcess, shutdownApiKey: string}>} + */ +export async function startMegaDetectorHTTPServer({ + port, + detectorWeightsFilepath, + detectionConfidenceThreshold, + timeout, + pythonEnvironment +}: MegaDetectorServerOptions): Promise<{ process: ChildProcess; shutdownApiKey: string }> { + log.info('startMegaDetectorHTTPServer initiated') + log.info(pythonEnvironment) + const localInstalRootDirPythonEnvironment = join( + getMLModelEnvironmentLocalInstallPath({ + ...pythonEnvironment.reference + }), + pythonEnvironment.reference.id + ) + log.info('Local Python Environment root dir is', localInstalRootDirPythonEnvironment) + const scriptPath = is.dev + ? join(__dirname, '../../python-environments/common/run_megadetector_server.py') + : join(process.resourcesPath, 'python-environments', 'common', 'run_megadetector_server.py') + const pythonInterpreter = is.dev + ? join(__dirname, '../../python-environments/common/.venv/bin/python') + : os.platform() === 'win32' + ? join(localInstalRootDirPythonEnvironment, 'python.exe') + : join(localInstalRootDirPythonEnvironment, 'bin', 'python') + log.info('Python Interpreter found in', pythonInterpreter) + log.info('Script path is', scriptPath) + const scriptArgs = [ + '--port', + String(port), + '--filepath-detector-weights', + detectorWeightsFilepath, + '--detection-confidence-threshold', + String(detectionConfidenceThreshold), + '--timeout', + String(timeout) + ] + log.info('Script args: ', scriptArgs) + + const shutdownApiKey = crypto.randomUUID() + log.info('Generated shutdown API key for MegaDetector server') + + const pythonProcess = await startAndWaitTillServerHealty({ + pythonInterpreter, + scriptPath, + scriptArgs, + healthEndpoint: `http://localhost:${port}/health`, + env: { LIT_SHUTDOWN_API_KEY: shutdownApiKey } + }) + + return { process: pythonProcess, shutdownApiKey } +} +``` + +- [ ] **Step 2: Import `findModel` at the top of the file if not already imported** + +Add this near the existing imports if it's not already there: + +```ts +import { findModel } from '../../../shared/mlmodels.js' +``` + +(`findModel` is needed in the switch case to read `detectionConfidenceThreshold` from the model definition rather than hard-coding it. Check the existing imports first — if `findModel` is already imported elsewhere in this file, skip this step.) + +- [ ] **Step 3: Add the `case 'megadetector'` in `startMLModelHTTPServer()`** + +In the `switch (modelReference.id)` block (around line 640), add a new case after `case 'manas'`: + +```ts +case 'megadetector': { + const port = await resolveServerPort(is.dev ? 8003 : null) + const localInstallPath = getMLModelLocalInstallPath({ ...modelReference }) + log.info(`Local ML Model install path ${localInstallPath}`) + const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10-e.pt') + const model = findModel({ ...modelReference }) + const { process: pythonProcess, shutdownApiKey } = await startMegaDetectorHTTPServer({ + port, + detectorWeightsFilepath, + detectionConfidenceThreshold: model?.detectionConfidenceThreshold ?? 0.2, + timeout: 30, + pythonEnvironment: pythonEnvironment + }) + log.info(`pythonProcess: ${JSON.stringify(pythonProcess)}`) + registerActiveServer({ + pid: pythonProcess.pid as number, + port, + shutdownApiKey, + modelId: modelReference.id + }) + return { port: port, process: pythonProcess, shutdownApiKey } +} +``` + +- [ ] **Step 4: Run the existing JS test suite to confirm no regressions** + +```bash +npm test +``` +Expected: all tests pass. (Server.ts isn't directly unit-tested but its consumers are, and TypeScript will fail to compile the renderer bundle if the new code has type errors.) + +- [ ] **Step 5: Commit** + +```bash +git add src/main/services/ml/server.ts +git commit -m "feat(megadetector): main-process server launcher + switch case" +``` + +--- + +## Task 7: `ModelCard.jsx` — `detectionOnly` conditional + +**Files:** +- Modify: `src/renderer/src/models/ModelCard.jsx` + +- [ ] **Step 1: Update the species_count summary line** + +In `src/renderer/src/models/ModelCard.jsx`, locate the block at lines 180–183 that renders the version/size/species line: + +```jsx +
+ v{model.reference.version} · {formatSize(model.size_in_MB)} ·{' '} + {model.species_count} species +
+``` + +Replace it with: + +```jsx +
+ v{model.reference.version} · {formatSize(model.size_in_MB)} ·{' '} + + {model.detectionOnly + ? `Detection only · ${model.species_count} categories` + : `${model.species_count} species`} + +
+``` + +- [ ] **Step 2: Hide the "View species" toggle and panel when detectionOnly** + +In the same file, locate the block at lines 203–213 that renders the species-toggle div and `speciesPanel`: + +```jsx +
{ + e.stopPropagation() + onToggleSpecies?.(model.reference.id) + }} +> + {speciesOpen ? '▾ Hide species' : `▸ View ${model.species_count} species`} +
+ +{speciesOpen && speciesPanel} +``` + +Wrap both elements in `!model.detectionOnly && (...)`: + +```jsx +{!model.detectionOnly && ( + <> +
{ + e.stopPropagation() + onToggleSpecies?.(model.reference.id) + }} + > + {speciesOpen ? '▾ Hide species' : `▸ View ${model.species_count} species`} +
+ + {speciesOpen && speciesPanel} + +)} +``` + +- [ ] **Step 3: Manual UI smoke test** + +```bash +npm run dev +``` +- Open the AI Models tab. +- Scroll to the MegaDetector card. +- Confirm the summary line reads: `v6.0 · 50 MB · Detection only · 3 categories` (size depends on the placeholder). +- Confirm there is NO "▸ View species" link below the description. +- Confirm the Download / Delete buttons still render and work. +- Confirm other model cards (SpeciesNet, DeepFaune, Manas) still show their species count and "View species" link. + +If anything is off, fix it and re-test. + +- [ ] **Step 4: Commit** + +```bash +git add src/renderer/src/models/ModelCard.jsx +git commit -m "feat(models): hide species panel for detectionOnly models" +``` + +--- + +## Task 8: `ModelSelect.jsx` — `detectionOnly` conditional + +**Files:** +- Modify: `src/renderer/src/models/ModelSelect.jsx` + +- [ ] **Step 1: Update the species_count line in the dropdown row** + +In `src/renderer/src/models/ModelSelect.jsx`, locate line 64: + +```jsx +{model.species_count} species +``` + +Replace with: + +```jsx + + {model.detectionOnly + ? `Detection only · ${model.species_count} categories` + : `${model.species_count} species`} + +``` + +- [ ] **Step 2: Manual UI smoke test** + +```bash +npm run dev +``` +- Open the Add Source modal (Sources tab → Add Source) or the Import flow. +- Click the model dropdown. +- Confirm MegaDetector appears in the list as a rich card with "Detection only · 3 categories" instead of "3 species". +- Confirm other models still display species counts correctly. +- Confirm selecting MegaDetector works (selection persists, dropdown closes). + +- [ ] **Step 3: Commit** + +```bash +git add src/renderer/src/models/ModelSelect.jsx +git commit -m "feat(models): \"detection only\" label in model picker dropdown" +``` + +--- + +## Task 9: Python tooling (`download_model.py` + `Makefile`) + +**Files:** +- Modify: `python-environments/common/scripts/download_model.py` +- Modify: `python-environments/common/Makefile` + +**Depends on:** The maintainer has uploaded the tarball to HF. + +- [ ] **Step 1: Add megadetector to `download_model.py`** + +In `python-environments/common/scripts/download_model.py`, modify the `MODELS` dict (around line 17): + +```python +MODELS = { + "speciesnet": { + "repo_id": "earthtoolsmaker/speciesnet", + "filename": "4.0.2a.tar.gz", + }, + "deepfaune": { + "repo_id": "earthtoolsmaker/deepfaune", + "filename": "1.3.tar.gz", + }, + "manas": { + "repo_id": "earthtoolsmaker/manas", + "filename": "1.0.tar.gz", + }, + "megadetector": { + "repo_id": "earthtoolsmaker/megadetector", + "filename": "6.0.tar.gz", + }, +} +``` + +Also update the `--model` help text on the argparse line (around line 60): + +```python +help="Model to download (speciesnet, deepfaune, manas, megadetector)", +``` + +- [ ] **Step 2: Add megadetector env var + download command to `Makefile`** + +In `python-environments/common/Makefile`, add a new env var after the existing exports: + +```makefile +export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e.pt +``` + +Append a new line to the `download-models:` target: + +```makefile +download-models: + uv run python scripts/download_model.py --model speciesnet --output /tmp/models/speciesnet + uv run python scripts/download_model.py --model deepfaune --output /tmp/models/deepfaune + uv run python scripts/download_model.py --model manas --output /tmp/models/manas + uv run python scripts/download_model.py --model megadetector --output /tmp/models/megadetector +``` + +- [ ] **Step 3: Run `make download-models` to verify the tarball pulls cleanly** + +```bash +cd python-environments/common +make download-models +``` +Expected: prints "Downloading megadetector from earthtoolsmaker/megadetector..." then "Done! Model extracted to /tmp/models/megadetector". Verify the weights file lands at the expected path: + +```bash +ls -la /tmp/models/megadetector/6.0/MDV6-yolov10-e.pt +``` +Expected: file exists, size matches Task 1's tarball-content size. + +- [ ] **Step 4: Run the full Python test suite** + +```bash +cd python-environments/common +make test +``` +Expected: all tests pass, including the new `TestMegaDetectorServer` class. The animal/person/blank assertions from Task 5 should now exercise real weights. + +- [ ] **Step 5: Commit** + +```bash +git add python-environments/common/scripts/download_model.py python-environments/common/Makefile +git commit -m "tools(megadetector): downloader + e2e test wiring" +``` + +--- + +## Task 10: Documentation (`docs/http-servers.md`) + +**Files:** +- Modify: `docs/http-servers.md` + +- [ ] **Step 1: Add MegaDetector to the Supported Models table** + +In `docs/http-servers.md`, locate the "Supported Models" table (around line 16). Add a fourth row: + +```markdown +| Model | Focus | Species Coverage | +| --------------------------- | ---------------------------------- | ----------------------------------------- | +| **SpeciesNet** (Google) | Global wildlife | 2,000+ species worldwide | +| **DeepFaune** (CNRS) | European fauna | 34 European species | +| **Manas** (OSI-Panthera) | Central Asian fauna | Snow leopard and 11 regional species | +| **MegaDetector** (Microsoft AI for Earth) | Blank filter / worldwide detection | 3 categories (animal, person, vehicle) | +``` + +- [ ] **Step 2: Add `run_megadetector_server.py` to the project structure listing** + +In the same file, locate the project structure block (around line 98–102) and add the new script: + +``` +python-environments/common/ +├── run_speciesnet_server.py # SpeciesNet LitServe implementation +├── run_deepfaune_server.py # DeepFaune LitServe implementation +├── run_manas_server.py # Manas LitServe implementation +└── run_megadetector_server.py # MegaDetector LitServe implementation +``` + +- [ ] **Step 3: Commit** + +```bash +git add docs/http-servers.md +git commit -m "docs(megadetector): document MD model in http-servers.md" +``` + +--- + +## Task 11: End-to-end manual smoke test + +This is a human-driven verification — no automated steps. Run after Tasks 1–10 have landed and the maintainer has provided the final `size_in_MB`. + +- [ ] **Step 1: Bump `size_in_MB` in `mlmodels.js`** + +Replace the placeholder `size_in_MB: 30` from Task 3 with the actual size reported by the maintainer. + +```bash +git add src/shared/mlmodels.js +git commit -m "feat(megadetector): set actual tarball size in model zoo" +``` + +- [ ] **Step 2: Add the logo asset** + +Drop `megadetector.png` into `src/renderer/src/assets/logos/` (provided by the maintainer). + +```bash +git add src/renderer/src/assets/logos/megadetector.png +git commit -m "assets(megadetector): add logo" +``` + +- [ ] **Step 3: Fresh-install smoke test** + +1. Wipe local app data: delete `~/Library/Application Support/biowatch` (macOS) / `~/.config/biowatch` (Linux) / `%APPDATA%/biowatch` (Windows). +2. `npm run dev`. +3. AI Models tab → MegaDetector card → click Download. +4. Confirm: card shows the new size, downloads complete, button switches to Delete. +5. The species panel does NOT appear (no "View species" link). + +- [ ] **Step 4: Inference smoke test** + +1. Sources tab → Add Source. +2. Pick a folder with ~20 mixed images (animals, blanks, ideally a human shot or vehicle). +3. In the model dropdown, confirm MegaDetector shows "Detection only · 3 categories"; select it. +4. Start the import. +5. Wait for processing to finish; the toast says "Processing complete". + +- [ ] **Step 5: Verify predictions landed correctly** + +1. Sources tab → click the new source. +2. The species panel shows entries for `animal`, optionally `person` / `vehicle`, and a "Blank" bucket. +3. Open Media tab → filter `species = animal` → confirm only frames with subjects appear. +4. Open Media tab → filter `species = blank` → confirm empty frames appear. +5. Open one `animal` frame → use the existing annotation UI to assign a real species (e.g. "Vulpes vulpes" / "red fox"). +6. Confirm the annotation persists across a page reload. + +- [ ] **Step 6: Multi-model coexistence check** + +1. Open a second study that uses SpeciesNet. +2. Import a small folder into it while the MegaDetector import is still queued / running. +3. Confirm: both servers run on different ports (`8000` SpeciesNet, `8003` MegaDetector in dev); both finish without interfering with each other. + +- [ ] **Step 7: Regression check — bbox rendering** + +1. On any `animal` frame, open the media-detail view with bounding boxes overlaid. +2. Confirm the bbox is correctly positioned around the subject (xywhn → top-left conversion works). +3. Open a frame imported earlier with SpeciesNet or DeepFaune; confirm those bboxes also still render correctly (no regression in `transformBboxToCamtrapDP`). + +--- + +## Notes for the executor + +- **Frequent commits**: each task ends in a commit. Don't bundle tasks unless explicitly told to. +- **No Claude attribution**: commits and PR descriptions must NOT include `Co-Authored-By: Claude …` lines. +- **gh CLI**: prefix every `gh` invocation with `GH_TOKEN=""` (e.g. `GH_TOKEN="" gh pr view`). +- **External dependency ordering**: Task 5 running real e2e tests, Task 9 (Python tooling), and Task 11 (manual smoke) all require the maintainer to have uploaded `6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector`. Tasks 2, 3, 4, 6, 7, 8, 10 do not — they can land first. The Python server code itself (Task 5) can also be written and committed pre-upload because its tests `skipif` on the env var. +- **Release notes for the PR description** (per spec): when opening the PR, include three call-outs: + 1. New model: MegaDetector v6 — worldwide animal/person/vehicle detector, useful as a fast blank-filter before manual species annotation. + 2. MegaDetector does not identify species; it only detects whether a frame contains an animal, a person, or a vehicle. + 3. ~30 MB download. Reuses the existing `common` Python environment, so no extra environment install if you already have SpeciesNet/DeepFaune/Manas. +- **Documentation maintenance** (per repository `CLAUDE.md`): when modifying files in `src/main/services/import/*.js`, `src/main/services/export/exporter.js`, `src/shared/mlmodels.js`, or `src/main/database/models.js`, also update the corresponding `docs/*.md` file. This plan covers `docs/http-servers.md` in Task 10; no other docs need editing. From ce74fa93b66913d29bb1ca73e690f5104e6c5e9e Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:06:14 +0200 Subject: [PATCH 03/24] docs(megadetector): correct upstream weights URL to Zenodo + filename suffix --- ...26-05-16-megadetector-model-choice-plan.md | 24 +++++++------- ...-05-16-megadetector-model-choice-design.md | 33 ++++++++++--------- 2 files changed, 29 insertions(+), 28 deletions(-) diff --git a/docs/plans/2026-05-16-megadetector-model-choice-plan.md b/docs/plans/2026-05-16-megadetector-model-choice-plan.md index e9ca577b..01e260c3 100644 --- a/docs/plans/2026-05-16-megadetector-model-choice-plan.md +++ b/docs/plans/2026-05-16-megadetector-model-choice-plan.md @@ -51,9 +51,9 @@ - [ ] **Step 1: Verify the upstream weights URL** ```bash -curl -I -L "https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10-e.pt" +curl -sI -L "https://zenodo.org/records/15398270/files/MDV6-yolov10-e-1280.pt?download=1" | head -3 ``` -Expected: HTTP 200/302 (release exists). If the filename differs in the release manifest, use the actual filename returned by `gh release view v6.0 --repo agentmorris/MegaDetector --json assets` instead, and update every occurrence of `MDV6-yolov10-e.pt` in this plan to match. +Expected: HTTP 200 (Zenodo serves the file). The canonical source for MDv6 weights is Zenodo record `15398270`, linked from `https://microsoft.github.io/Biodiversity/model_zoo/megadetector/`. If Zenodo is down or the record id has changed, fall back to the model zoo page to find the current direct-download URL. - [ ] **Step 2: Write the build script** @@ -81,10 +81,10 @@ import tempfile import urllib.request from pathlib import Path -WEIGHTS_URL = "https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10-e.pt" -WEIGHTS_FILENAME = "MDV6-yolov10-e.pt" +WEIGHTS_URL = "https://zenodo.org/records/15398270/files/MDV6-yolov10-e-1280.pt?download=1" +WEIGHTS_FILENAME = "MDV6-yolov10-e-1280.pt" WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN" # paste actual hash printed at end of first run -LICENSE_URL = "https://raw.githubusercontent.com/agentmorris/MegaDetector/main/LICENSE" +LICENSE_URL = "https://raw.githubusercontent.com/microsoft/MegaDetector/main/LICENSE" LICENSE_FILENAME = "LICENSE" @@ -200,13 +200,13 @@ Expected output (last 8 lines): SHA256: <64 hex chars> Contents: 6.0/LICENSE - 6.0/MDV6-yolov10-e.pt + 6.0/MDV6-yolov10-e-1280.pt ============================================================ ``` - [ ] **Step 4: Lock the SHA256** -Replace the `WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN"` line with the actual hash printed by Step 3 (the per-file hash of `MDV6-yolov10-e.pt`, not the tarball hash — re-run the script if you only kept the tarball hash; the per-file hash is printed on the line starting with `[verify]`). +Replace the `WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN"` line with the actual hash printed by Step 3 (the per-file hash of `MDV6-yolov10-e-1280.pt`, not the tarball hash — re-run the script if you only kept the tarball hash; the per-file hash is printed on the line starting with `[verify]`). - [ ] **Step 5: Run it again to confirm SHA verification passes** @@ -691,7 +691,7 @@ Start the server: ``` run_megadetector_server.py \ --port 8000 \ - --filepath-detector-weights ./path/to/MDV6-yolov10-e.pt \ + --filepath-detector-weights ./path/to/MDV6-yolov10-e-1280.pt \ --detection-confidence-threshold 0.2 ``` @@ -938,7 +938,7 @@ Skip if the HF upload hasn't landed yet — the tests `pytest.skipif` themselves cd python-environments/common # Once the tarball is on HF, this will work (the env var is set by the make target # added in Task 9, but you can also export it manually for this one-shot run): -export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e.pt +export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt make test ``` Expected: the new `TestMegaDetectorServer` class runs and passes all assertions. If `MEGADETECTOR_DETECTOR_WEIGHTS` is unset, the class is skipped — that's acceptable for landing the PR, with a follow-up to wire up Task 9 after the upload. @@ -1052,7 +1052,7 @@ case 'megadetector': { const port = await resolveServerPort(is.dev ? 8003 : null) const localInstallPath = getMLModelLocalInstallPath({ ...modelReference }) log.info(`Local ML Model install path ${localInstallPath}`) - const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10-e.pt') + const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10-e-1280.pt') const model = findModel({ ...modelReference }) const { process: pythonProcess, shutdownApiKey } = await startMegaDetectorHTTPServer({ port, @@ -1265,7 +1265,7 @@ help="Model to download (speciesnet, deepfaune, manas, megadetector)", In `python-environments/common/Makefile`, add a new env var after the existing exports: ```makefile -export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e.pt +export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt ``` Append a new line to the `download-models:` target: @@ -1287,7 +1287,7 @@ make download-models Expected: prints "Downloading megadetector from earthtoolsmaker/megadetector..." then "Done! Model extracted to /tmp/models/megadetector". Verify the weights file lands at the expected path: ```bash -ls -la /tmp/models/megadetector/6.0/MDV6-yolov10-e.pt +ls -la /tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt ``` Expected: file exists, size matches Task 1's tarball-content size. diff --git a/docs/specs/2026-05-16-megadetector-model-choice-design.md b/docs/specs/2026-05-16-megadetector-model-choice-design.md index ffcb5d69..0e258d27 100644 --- a/docs/specs/2026-05-16-megadetector-model-choice-design.md +++ b/docs/specs/2026-05-16-megadetector-model-choice-design.md @@ -6,11 +6,11 @@ Biowatch currently offers three model choices in the AI Models tab: SpeciesNet ( [MegaDetector](https://github.com/agentmorris/MegaDetector) is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, with **no species classification**. Inside Biowatch it would fill a different role: a fast blank-filter that lets a user separate frames with subjects from empty frames, then manually annotate species on the non-blank frames using Biowatch's existing annotation UI. -MegaDetector v6 (`MDV6-yolov10x.pt`) is already bundled inside the DeepFaune and Manas tarballs and loaded via `ultralytics.YOLO` in `run_deepfaune_server.py`. The Python loading code is therefore already proven inside our `common` conda environment. +MegaDetector v6 weights are loaded via `ultralytics.YOLO` in `run_deepfaune_server.py` (the DeepFaune tarball bundles an earlier MDv6 build, `MDV6-yolov10x.pt`). The Python loading code is therefore already proven inside our `common` conda environment. The standalone MD model entry ships the current upstream variant **`MDV6-yolov10-e`** — best accuracy in the MDv6 lineup (82.8% animal recall, 92.8% mAP50) at ~30 MB. ## Goals -1. Ship MegaDetector v6 (YOLOv10x) as a standalone model choice in the AI Models tab. +1. Ship MegaDetector v6 (`MDV6-yolov10-e`) as a standalone model choice in the AI Models tab. 2. Match the established model-integration pattern (one Python server script, one `switch` case in `server.ts`, one HF tarball, one `mlmodels.js` entry). 3. Map MD's detector labels (`animal`/`person`/`vehicle` + implicit `blank`) into Biowatch's species-centric data model with **zero changes** to the observation/annotation/export pipelines — by treating the three labels as pseudo-species. 4. Suppress UI affordances that only make sense for true species classifiers (the "View species" panel) via an opt-out flag, not a special-case in every consumer. @@ -20,22 +20,22 @@ MegaDetector v6 (`MDV6-yolov10x.pt`) is already bundled inside the DeepFaune and - No conda env bump. MD reuses `common/0.1.4` — `ultralytics` and `torch` are already pinned for DeepFaune/Manas. - No detector-weight sharing across models. MegaDetector ships its own tarball; the copy bundled inside DeepFaune/Manas is untouched. The marginal ~110 MB is acceptable in exchange for the model-isolation property the codebase keeps today. - No "MD-then-classifier" workflow primitive. MD is a standalone model; chaining is out of scope. -- No MDv5a packaging. Only MDv6 (YOLOv10x) ships. If a user needs v5 for reproducibility, that becomes a separate model entry later. +- No MDv5a packaging. Only MDv6 (`MDV6-yolov10-e`) ships. If a user needs another MDv6 variant or MDv5 for reproducibility, that becomes a separate model entry later. +- License compatibility (MDv6 is AGPL-3.0; Biowatch is CC BY-NC 4.0) is acknowledged but out of scope for this change. Prior art: DeepFaune and Manas tarballs already bundle an MDv6 build. - No automated reclassification of existing observations. ## Source-of-truth artifact -Upstream weights: `https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10x.pt`. +Upstream weights: `MDV6-yolov10-e-1280.pt` from the MegaDetector v6 model zoo. Downloaded from Zenodo at `https://zenodo.org/records/15398270/files/MDV6-yolov10-e-1280.pt?download=1` (the canonical hosting location linked from `https://microsoft.github.io/Biodiversity/model_zoo/megadetector/`). The `-1280` suffix denotes the model's input image size. The build script (see `scripts/build-megadetector-tarball.py` below) downloads from this URL and verifies the SHA256. -We rehost this single file on `huggingface.co/earthtoolsmaker/megadetector` (repo already created by the maintainer), as a tarball `6.0.tar.gz` containing a top-level `6.0/` directory with `MDV6-yolov10x.pt` inside it. This matches the layout convention of the other three model tarballs and gives us a stable, offline-deterministic download URL. - -Re-hosting is permitted: MegaDetector is MIT-licensed. +We rehost this single file on `huggingface.co/earthtoolsmaker/megadetector` (repo already created by the maintainer), as a tarball `6.0.tar.gz` containing a top-level `6.0/` directory with the weights file inside it. This matches the layout convention of the other three model tarballs and gives us a stable, offline-deterministic download URL. The upstream `LICENSE` (AGPL-3.0) is included in the tarball next to the weights so AGPL attribution travels with the binary. ## Tarball layout ``` 6.0/ - MDV6-yolov10x.pt + MDV6-yolov10-e-1280.pt + LICENSE ``` Top-level directory name (`6.0/`) matches the model version. No `README.md` shipped (consistent with how `build-speciesnet-tarball.py` strips Kaggle's README). @@ -137,7 +137,7 @@ Add a new `case 'megadetector'` in the `switch` inside `startMLModelHTTPServer() case 'megadetector': { const port = await resolveServerPort(is.dev ? 8003 : null) const localInstallPath = getMLModelLocalInstallPath({ ...modelReference }) - const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10x.pt') + const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10-e-1280.pt') const model = findModel({ ...modelReference }) const { process: pythonProcess, shutdownApiKey } = await startMegaDetectorHTTPServer({ port, @@ -216,17 +216,18 @@ Add `src/renderer/src/assets/logos/megadetector.png` (maintainer provides). The `scripts/build-megadetector-tarball.py` — same shape as `build-speciesnet-tarball.py`, using only the Python standard library: -1. Download `MDV6-yolov10x.pt` from `https://github.com/agentmorris/MegaDetector/releases/download/v6.0/MDV6-yolov10x.pt`. -2. Verify SHA256 against the hash of the copy already shipped in the DeepFaune tarball (locked at build time after first run — encoded as a constant in the script the same way `MEGADETECTOR_SHA256` is in `build-speciesnet-tarball.py`). -3. Place the weights at `6.0/MDV6-yolov10x.pt`. -4. `tar -czf dist/6.0.tar.gz 6.0/`. -5. Print final size + SHA256 + file list for the `mlmodels.js` update. +1. Download `MDV6-yolov10-e-1280.pt` from the MegaDetector v6 release page on GitHub (exact URL filled in once the script's author confirms it from the upstream release manifest). +2. Verify SHA256 against a constant encoded in the script (locked after the first successful build), matching the pattern `MEGADETECTOR_SHA256` follows in `build-speciesnet-tarball.py`. +3. Download `LICENSE` (AGPL-3.0) from the MegaDetector repo at the same release tag so the attribution travels with the weights. +4. Place both files at `6.0/MDV6-yolov10-e-1280.pt` and `6.0/LICENSE`. +5. `tar -czf dist/6.0.tar.gz 6.0/`. +6. Print final size + SHA256 + file list for the `mlmodels.js` update. The upload step (`huggingface-cli upload …` to `earthtoolsmaker/megadetector`) stays manual — credentials are user-scoped. Maintainer (Arthur) uploads. ## Verification -1. **Build script self-check**: tarball contains exactly one file at `6.0/MDV6-yolov10x.pt`; SHA256 of inner file matches the hash also shipped in the DeepFaune tarball. +1. **Build script self-check**: tarball contains exactly two files at `6.0/MDV6-yolov10-e-1280.pt` and `6.0/LICENSE`; SHA256 of the weights file matches the constant locked into the build script. 2. **`npm test`**: covers `mlmodels.test.js` (new entry valid), `bbox.test.js` (new megadetector case routes to xywhn), `downloadState.test.js`, common-names resolver tests (megadetector source loads). 3. **`cd python-environments/common && make lint && make format && make test`**: includes a new `tests/test_megadetector_server.py`. Test cases: - Start the server with the bundled weights. @@ -241,7 +242,7 @@ User-facing changelog should call out: 1. **New model: MegaDetector v6.** Worldwide animal/person/vehicle detector. Useful as a fast blank-filter before manual species annotation. 2. MegaDetector does not identify species — it only detects whether a frame contains an animal, a person, or a vehicle. Use it to triage large folders, then annotate species manually inside Biowatch. -3. ~110 MB download. Reuses the existing `common` Python environment, so no extra environment install if you already have SpeciesNet/DeepFaune/Manas. +3. ~30 MB download. Reuses the existing `common` Python environment, so no extra environment install if you already have SpeciesNet/DeepFaune/Manas. ## Documentation updates From 7fdae5a7ffbb3e0f5ab06e84aa1d286bc679c1ba Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:10:22 +0200 Subject: [PATCH 04/24] tools(megadetector): script to build MD v6 tarball for HF upload --- scripts/build-megadetector-tarball.py | 124 ++++++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100755 scripts/build-megadetector-tarball.py diff --git a/scripts/build-megadetector-tarball.py b/scripts/build-megadetector-tarball.py new file mode 100755 index 00000000..26be91d5 --- /dev/null +++ b/scripts/build-megadetector-tarball.py @@ -0,0 +1,124 @@ +#!/usr/bin/env python3 +"""Build a MegaDetector model tarball ready to upload to huggingface.co/earthtoolsmaker/megadetector. + +Downloads the official MDv6 release weights and the upstream LICENSE so AGPL attribution +travels with the binary, then tar-gzips them into a directory named for the version. + +Usage: + python scripts/build-megadetector-tarball.py --version 6.0 + +Produces dist/.tar.gz alongside a summary of size + SHA256 + file list. +""" + +import argparse +import hashlib +import shutil +import sys +import tarfile +import tempfile +import urllib.request +from pathlib import Path + +WEIGHTS_URL = "https://zenodo.org/records/15398270/files/MDV6-yolov10-e-1280.pt?download=1" +WEIGHTS_FILENAME = "MDV6-yolov10-e-1280.pt" +WEIGHTS_SHA256 = "4a3a3d380ce7e151b2a8b991ab5d86f329ccd7b0b33e5d3ba0593a1166d55109" +LICENSE_URL = "https://raw.githubusercontent.com/microsoft/MegaDetector/main/LICENSE" +LICENSE_FILENAME = "LICENSE" + + +def download(url: str, dest: Path) -> None: + if dest.exists(): + print(f"[skip] {dest.name} already present ({dest.stat().st_size / 1e6:.0f} MB)") + return + print(f"[download] {url}") + is_tty = sys.stdout.isatty() + last_milestone = -1 + + def report(block_num: int, block_size: int, total_size: int) -> None: + nonlocal last_milestone + if total_size <= 0: + return + downloaded = block_num * block_size + pct = min(100, downloaded * 100 // total_size) + if is_tty: + sys.stdout.write(f"\r {pct:3d}% {downloaded / 1e6:7.1f} / {total_size / 1e6:.0f} MB") + sys.stdout.flush() + else: + milestone = pct // 10 + if milestone > last_milestone: + last_milestone = milestone + print(f" {pct:3d}% {downloaded / 1e6:7.1f} / {total_size / 1e6:.0f} MB") + + urllib.request.urlretrieve(url, dest, reporthook=report) + if is_tty: + sys.stdout.write("\n") + + +def sha256_of(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as fp: + for chunk in iter(lambda: fp.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def build(version: str, output_dir: Path, work_dir: Path) -> Path: + work_dir.mkdir(parents=True, exist_ok=True) + output_dir.mkdir(parents=True, exist_ok=True) + + model_dir = work_dir / version + if model_dir.exists(): + shutil.rmtree(model_dir) + model_dir.mkdir() + + weights_path = model_dir / WEIGHTS_FILENAME + download(WEIGHTS_URL, weights_path) + + actual_hash = sha256_of(weights_path) + if WEIGHTS_SHA256 != "TBD_LOCK_ON_FIRST_RUN" and actual_hash != WEIGHTS_SHA256: + raise RuntimeError(f"Weights SHA256 mismatch:\n expected {WEIGHTS_SHA256}\n got {actual_hash}") + print(f"[verify] {WEIGHTS_FILENAME} SHA256 = {actual_hash}") + + download(LICENSE_URL, model_dir / LICENSE_FILENAME) + + tarball = output_dir / f"{version}.tar.gz" + if tarball.exists(): + tarball.unlink() + print(f"[tar] {tarball}") + with tarfile.open(tarball, "w:gz") as tar: + tar.add(model_dir, arcname=version) + return tarball + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--version", required=True, help="MegaDetector version, e.g. 6.0") + parser.add_argument("--output-dir", type=Path, default=Path("dist")) + parser.add_argument( + "--work-dir", + type=Path, + default=Path(tempfile.gettempdir()) / "megadetector-build", + help="Working directory for downloads (retained between runs)", + ) + args = parser.parse_args() + + tarball = build(args.version, args.output_dir, args.work_dir) + + size_mb = tarball.stat().st_size / 1e6 + digest = sha256_of(tarball) + with tarfile.open(tarball) as tar: + members = sorted(m.name for m in tar.getmembers() if not m.isdir()) + + print() + print("=" * 60) + print(f" Tarball: {tarball}") + print(f" Size: {size_mb:.1f} MB") + print(f" SHA256: {digest}") + print(" Contents:") + for name in members: + print(f" {name}") + print("=" * 60) + + +if __name__ == "__main__": + main() From dec50e2240b686381eef9021ec081feb5ac5c40a Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:15:17 +0200 Subject: [PATCH 05/24] feat(megadetector): common-name source for animal/person/vehicle labels --- scripts/build-common-names-dict.js | 2 +- scripts/lib/aliases.js | 3 ++- src/shared/commonNames/dictionary.json | 2 ++ .../commonNames/sources/megadetector.json | 22 +++++++++++++++++++ test/shared/commonNames/resolver.test.js | 6 +++++ 5 files changed, 33 insertions(+), 2 deletions(-) create mode 100644 src/shared/commonNames/sources/megadetector.json diff --git a/scripts/build-common-names-dict.js b/scripts/build-common-names-dict.js index 88ced495..74e494fe 100644 --- a/scripts/build-common-names-dict.js +++ b/scripts/build-common-names-dict.js @@ -94,7 +94,7 @@ function main() { const dictionary = {} const labelAliases = {} - const order = ['speciesnet.json', 'deepfaune.json', 'manas.json'] + const order = ['speciesnet.json', 'deepfaune.json', 'manas.json', 'megadetector.json'] for (const filename of order) { const snapshot = loadJson(path.join(SOURCES_DIR, filename)) mergeEntries(dictionary, snapshot.entries) diff --git a/scripts/lib/aliases.js b/scripts/lib/aliases.js index 82dbe808..d432ab20 100644 --- a/scripts/lib/aliases.js +++ b/scripts/lib/aliases.js @@ -1,6 +1,7 @@ import speciesnetSource from '../../src/shared/commonNames/sources/speciesnet.json' with { type: 'json' } import deepfauneSource from '../../src/shared/commonNames/sources/deepfaune.json' with { type: 'json' } import manasSource from '../../src/shared/commonNames/sources/manas.json' with { type: 'json' } +import megadetectorSource from '../../src/shared/commonNames/sources/megadetector.json' with { type: 'json' } import extras from '../../src/shared/commonNames/extras.json' with { type: 'json' } import { normalizeScientificName } from '../../src/shared/commonNames/normalize.js' @@ -14,7 +15,7 @@ import { normalizeScientificName } from '../../src/shared/commonNames/normalize. */ export function buildAliasMap() { const aliases = new Map() - const sources = [speciesnetSource, deepfauneSource, manasSource, extras] + const sources = [speciesnetSource, deepfauneSource, manasSource, megadetectorSource, extras] for (const src of sources) { for (const entry of src.entries || []) { if (!entry.scientificName || !entry.label) continue diff --git a/src/shared/commonNames/dictionary.json b/src/shared/commonNames/dictionary.json index 97f53af5..85069d19 100644 --- a/src/shared/commonNames/dictionary.json +++ b/src/shared/commonNames/dictionary.json @@ -132,6 +132,7 @@ "aneides lugubris": "arboreal salamander", "anhima cornuta": "horned screamer", "anhinga anhinga": "anhinga", + "animal": "animal", "annamite_striped_rabbit": "annamite striped rabbit", "anolis carolinensis": "green anole", "anolis sagrei": "cuban brown anole", @@ -2289,6 +2290,7 @@ "peromyscus leucopus": "white-footed mouse", "peromyscus maniculatus": "north american deermouse", "peromyscus species": "peromyscus species", + "person": "person", "petaurista elegans": "spotted giant flying squirrel", "petaurista petaurista": "red giant flying squirrel", "petaurista philippensis": "indian giant flying squirrel", diff --git a/src/shared/commonNames/sources/megadetector.json b/src/shared/commonNames/sources/megadetector.json new file mode 100644 index 00000000..568c7b31 --- /dev/null +++ b/src/shared/commonNames/sources/megadetector.json @@ -0,0 +1,22 @@ +{ + "modelId": "megadetector", + "modelVersion": "6.0", + "source": "MegaDetector v6 categories (animal / person / vehicle)", + "entries": [ + { + "scientificName": null, + "label": "animal", + "commonName": "Animal" + }, + { + "scientificName": null, + "label": "person", + "commonName": "Person" + }, + { + "scientificName": null, + "label": "vehicle", + "commonName": "Vehicle" + } + ] +} diff --git a/test/shared/commonNames/resolver.test.js b/test/shared/commonNames/resolver.test.js index c15f3f3a..29d4274e 100644 --- a/test/shared/commonNames/resolver.test.js +++ b/test/shared/commonNames/resolver.test.js @@ -32,4 +32,10 @@ describe('resolveCommonName', () => { test('returns null for unknown scientific name', () => { assert.equal(resolveCommonName('Foobar nonexistentium'), null) }) + + test('resolves MegaDetector pseudo-species labels (animal/person/vehicle)', () => { + assert.equal(resolveCommonName('animal'), 'animal') + assert.equal(resolveCommonName('person'), 'person') + assert.equal(resolveCommonName('vehicle'), 'vehicle') + }) }) From 44ca493c9c9a43525418ef7552238c453633d003 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:23:13 +0200 Subject: [PATCH 06/24] feat(megadetector): map 'person' label to homo sapiens binomial --- src/shared/commonNames/dictionary.json | 2 +- src/shared/commonNames/labelAliases.json | 1 + src/shared/commonNames/sources/megadetector.json | 4 ++-- test/shared/commonNames/resolver.test.js | 11 +++++++++-- 4 files changed, 13 insertions(+), 5 deletions(-) diff --git a/src/shared/commonNames/dictionary.json b/src/shared/commonNames/dictionary.json index 85069d19..dde6d6f5 100644 --- a/src/shared/commonNames/dictionary.json +++ b/src/shared/commonNames/dictionary.json @@ -2290,7 +2290,7 @@ "peromyscus leucopus": "white-footed mouse", "peromyscus maniculatus": "north american deermouse", "peromyscus species": "peromyscus species", - "person": "person", + "person": "human", "petaurista elegans": "spotted giant flying squirrel", "petaurista petaurista": "red giant flying squirrel", "petaurista philippensis": "indian giant flying squirrel", diff --git a/src/shared/commonNames/labelAliases.json b/src/shared/commonNames/labelAliases.json index a15ab76e..9ea0f0ff 100644 --- a/src/shared/commonNames/labelAliases.json +++ b/src/shared/commonNames/labelAliases.json @@ -293,6 +293,7 @@ "pacific wren": "troglodytes pacificus", "painted_turtle": "chrysemys picta", "panthera_uncia": "panthera uncia", + "person": "homo sapiens", "phrynosomatidae family": "phrynosomatidae", "pig": "sus scrofa", "pig_tailed_macaque": "macaca nemestrina", diff --git a/src/shared/commonNames/sources/megadetector.json b/src/shared/commonNames/sources/megadetector.json index 568c7b31..61c65a0a 100644 --- a/src/shared/commonNames/sources/megadetector.json +++ b/src/shared/commonNames/sources/megadetector.json @@ -9,9 +9,9 @@ "commonName": "Animal" }, { - "scientificName": null, + "scientificName": "homo sapiens", "label": "person", - "commonName": "Person" + "commonName": "Human" }, { "scientificName": null, diff --git a/test/shared/commonNames/resolver.test.js b/test/shared/commonNames/resolver.test.js index 29d4274e..4f41f530 100644 --- a/test/shared/commonNames/resolver.test.js +++ b/test/shared/commonNames/resolver.test.js @@ -33,9 +33,16 @@ describe('resolveCommonName', () => { assert.equal(resolveCommonName('Foobar nonexistentium'), null) }) - test('resolves MegaDetector pseudo-species labels (animal/person/vehicle)', () => { + test('resolves MegaDetector pseudo-species labels (animal/vehicle)', () => { assert.equal(resolveCommonName('animal'), 'animal') - assert.equal(resolveCommonName('person'), 'person') assert.equal(resolveCommonName('vehicle'), 'vehicle') }) + + test("resolves MegaDetector's 'person' label and its 'homo sapiens' binomial to 'human'", () => { + // MD's raw label is 'person', but the Python server emits 'homo sapiens' + // as the prediction. Both keys resolve to the same common name so either + // route (alias-translated or directly emitted) renders consistently. + assert.equal(resolveCommonName('person'), 'human') + assert.equal(resolveCommonName('homo sapiens'), 'human') + }) }) From 96c8952b6a06778d232e5999ec58b55bf3f45e2c Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:24:34 +0200 Subject: [PATCH 07/24] docs(megadetector): person->homo sapiens mapping in spec + plan --- ...26-05-16-megadetector-model-choice-plan.md | 38 ++++++++++++++----- ...-05-16-megadetector-model-choice-design.md | 26 +++++++------ 2 files changed, 44 insertions(+), 20 deletions(-) diff --git a/docs/plans/2026-05-16-megadetector-model-choice-plan.md b/docs/plans/2026-05-16-megadetector-model-choice-plan.md index 01e260c3..65ac2227 100644 --- a/docs/plans/2026-05-16-megadetector-model-choice-plan.md +++ b/docs/plans/2026-05-16-megadetector-model-choice-plan.md @@ -252,9 +252,9 @@ git commit -m "tools(megadetector): script to build MD v6 tarball for HF upload" "commonName": "Animal" }, { - "scientificName": null, + "scientificName": "homo sapiens", "label": "person", - "commonName": "Person" + "commonName": "Human" }, { "scientificName": null, @@ -265,7 +265,7 @@ git commit -m "tools(megadetector): script to build MD v6 tarball for HF upload" } ``` -This matches the manas.json schema exactly: `scientificName: null` for non-binomial labels. +Schema matches `manas.json`: `scientificName: null` for non-binomial labels (`animal`, `vehicle`); a real binomial (`"homo sapiens"`) for `person` so the dict-build pipeline produces a `homo sapiens → human` dictionary entry plus a `person → homo sapiens` alias. - [ ] **Step 2: Add the source to the dictionary-build pipeline** @@ -296,16 +296,23 @@ export function buildAliasMap() { (`megadetectorSource` entries have `scientificName: null`, so the alias loop skips them per the existing `if (!entry.scientificName || !entry.label) continue` guard — adding it to the list is a no-op for aliases but keeps the imports symmetric with the dict builder.) -- [ ] **Step 4: Add the failing resolver test** +- [ ] **Step 4: Add the failing resolver tests** Append to `test/shared/commonNames/resolver.test.js`: ```js -test('resolveCommonName returns Animal / Person / Vehicle for MegaDetector labels', () => { +test('resolves MegaDetector pseudo-species labels (animal/vehicle)', () => { assert.equal(resolveCommonName('animal'), 'animal') - assert.equal(resolveCommonName('person'), 'person') assert.equal(resolveCommonName('vehicle'), 'vehicle') }) + +test("resolves MegaDetector's 'person' label and its 'homo sapiens' binomial to 'human'", () => { + // MD's raw label is 'person', but the Python server emits 'homo sapiens' + // as the prediction. Both keys resolve to the same common name so either + // route (alias-translated or directly emitted) renders consistently. + assert.equal(resolveCommonName('person'), 'human') + assert.equal(resolveCommonName('homo sapiens'), 'human') +}) ``` Note: dictionary values are lowercased by `mergeEntries` (build-common-names-dict.js:72), so `"Animal"` becomes `"animal"` in the dictionary — assertions match the stored value. @@ -611,7 +618,7 @@ class TestMegaDetectorServer: assert "xywhn" in pred["detections"][0] def test_predict_human_image(self, megadetector_server, test_images): - """A human image should produce prediction='person'.""" + """A human image should produce prediction='homo sapiens' (translated from MD's 'person' label).""" payload = {"instances": [{"filepath": str(test_images["human"])}]} with httpx.stream( "POST", @@ -621,7 +628,10 @@ class TestMegaDetectorServer: ) as resp: results = parse_streaming_response(resp) pred = results[0]["output"]["predictions"][0] - assert pred["prediction"] == "person" + assert pred["prediction"] == "homo sapiens" + # Raw bbox labels stay 'person' so MD's native output is preserved per-bbox. + person_dets = [d for d in pred["detections"] if d["label"] == "person"] + assert len(person_dets) >= 1, "raw 'person' label must be preserved per-detection" def test_predict_empty_image(self, megadetector_server, test_images): """An empty (no subject) image should produce prediction='blank'.""" @@ -748,6 +758,15 @@ logger = logging.getLogger(__name__) MODEL_VERSION = "6.0" +# Map MegaDetector detection labels to Biowatch-canonical scientific names +# for the top-level `prediction` field. Per-bbox `detections[].label` stays raw. +# `person` is genuinely a species — emit the binomial so it integrates with +# Biowatch's species tooltips and IUCN lookups. `animal` and `vehicle` are not +# species and pass through unchanged. +LABEL_TO_PREDICTION = { + "person": "homo sapiens", +} + _PORT = flags.DEFINE_integer("port", 8000, "Port to run the server on.") _API_PATH = flags.DEFINE_string("api_path", "/predict", "URL path for the server endpoint.") _WORKERS_PER_DEVICE = flags.DEFINE_integer("workers_per_device", 1, "Number of server replicas per device.") @@ -826,13 +845,14 @@ def predict_one( } top = max(above, key=lambda d: d["conf"]) + prediction_label = LABEL_TO_PREDICTION.get(top["label"], top["label"]) return { "predictions": [ { "filepath": str(filepath), "classifications": {}, "detections": detection_records, - "prediction": top["label"], + "prediction": prediction_label, "prediction_score": top["conf"], "model_version": model_version, } diff --git a/docs/specs/2026-05-16-megadetector-model-choice-design.md b/docs/specs/2026-05-16-megadetector-model-choice-design.md index 0e258d27..7904f42d 100644 --- a/docs/specs/2026-05-16-megadetector-model-choice-design.md +++ b/docs/specs/2026-05-16-megadetector-model-choice-design.md @@ -65,14 +65,16 @@ The MD server emits one prediction object per image, matching the shape `inferen The three MD labels are stored as if they were species: -| `prediction` field | Treated as scientific name | Common name | -|---|---|---| -| `animal` | `animal` | `Animal` | -| `person` | `person` | `Person` | -| `vehicle` | `vehicle` | `Vehicle` | -| `blank` | `blank` | (existing blank handling) | +| `prediction` field | Treated as scientific name | Common name | +|---------------------------|----------------------------|--------------------------| +| `animal` | `animal` | `Animal` | +| `homo sapiens` (mapped from MD's `person` label) | `homo sapiens` | `Human` | +| `vehicle` | `vehicle` | `Vehicle` | +| `blank` | `blank` | (existing blank handling) | -A new common-names source file resolves these: +The `person` label is the only MD category that is genuinely a species; the Python server translates `person → homo sapiens` for the top-level `prediction` field so it integrates with Biowatch's species tooltips and IUCN lookups. Per-bbox `detections[].label` stays raw (`"person"`). `animal` and `vehicle` are not species and stay as plain labels. + +A new common-names source file resolves these (schema matches `manas.json`): `src/shared/commonNames/sources/megadetector.json`: @@ -82,14 +84,16 @@ A new common-names source file resolves these: "modelVersion": "6.0", "source": "MegaDetector v6 categories (animal / person / vehicle)", "entries": [ - { "scientific": "animal", "common": "Animal" }, - { "scientific": "person", "common": "Person" }, - { "scientific": "vehicle", "common": "Vehicle" } + { "scientificName": null, "label": "animal", "commonName": "Animal" }, + { "scientificName": "homo sapiens", "label": "person", "commonName": "Human" }, + { "scientificName": null, "label": "vehicle", "commonName": "Vehicle" } ] } ``` -User-facing consequence: in the Media tab the user filters `species = animal`, finds the non-blank frames, opens each one, and assigns the real species via the existing annotation UI. The MD prediction stays as the model's raw output in `model_outputs`; the user's annotation lives on the observation, same as any model. +The `person` entry has a non-null `scientificName` so the dict-build pipeline (a) registers `homo sapiens → human` in `dictionary.json` and (b) registers `person → homo sapiens` in `labelAliases.json`. The Python server emits `homo sapiens` directly for human detections; the alias map exists for downstream consistency with the LILA-style file importers that translate label → binomial at insert time. + +User-facing consequence: in the Media tab the user filters `species = animal`, finds the non-blank frames, opens each one, and assigns the real species via the existing annotation UI. Human frames already land with `scientificName = "homo sapiens"` so the species tooltip fires the same way it would for any other Homo-sapiens prediction in the system. The raw MD output stays in `model_outputs`; the user's annotation lives on the observation, same as any model. ## Code changes From 7216cd49d20033f184479b1ac538c2f11781f37d Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:25:24 +0200 Subject: [PATCH 08/24] feat(megadetector): MD v6 entry in model zoo with detectionOnly flag --- src/shared/mlmodels.js | 17 +++++++++++++++++ test/shared/mlmodels.test.js | 12 ++++++++++++ 2 files changed, 29 insertions(+) diff --git a/src/shared/mlmodels.js b/src/shared/mlmodels.js index 02790a5c..8d0e11cf 100644 --- a/src/shared/mlmodels.js +++ b/src/shared/mlmodels.js @@ -205,6 +205,23 @@ export const modelZoo = [ region: 'himalayas', species_count: 11, species_data: 'manas' + }, + { + reference: { id: 'megadetector', version: '6.0' }, + pythonEnvironment: { id: 'common', version: '0.1.4' }, + name: 'MegaDetector', + size_in_MB: 118, // tarball ~117.8 MB, weights file ~128 MB — replace with exact size once HF upload lands + files: 2, + downloadURL: + 'https://huggingface.co/earthtoolsmaker/megadetector/resolve/main/6.0.tar.gz?download=true', + description: + "MegaDetector is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, without identifying species. Useful as a fast blank-filter before manual species annotation.", + website: 'https://github.com/microsoft/MegaDetector', + logo: 'megadetector', + detectionConfidenceThreshold: 0.2, + region: 'worldwide', + species_count: 3, + detectionOnly: true } ] diff --git a/test/shared/mlmodels.test.js b/test/shared/mlmodels.test.js index 82217021..ba79d35c 100644 --- a/test/shared/mlmodels.test.js +++ b/test/shared/mlmodels.test.js @@ -187,6 +187,18 @@ describe('ML Model Management Tests', () => { assert.equal(result, null, 'Should return null for wrong version') }) + + test('should find megadetector entry with detectionOnly flag', () => { + const md = findModel({ id: 'megadetector', version: '6.0' }) + assert(md, 'megadetector entry should exist') + assert.equal(md.detectionOnly, true, 'megadetector should be marked detectionOnly') + assert.equal(md.region, 'worldwide', 'megadetector should be worldwide') + assert.equal(md.species_count, 3, 'megadetector should report 3 categories') + assert( + md.downloadURL.includes('earthtoolsmaker/megadetector'), + 'should download from ETM HF repo' + ) + }) }) describe('platformToKey', () => { From 8dc5feb8aaad6be2c40bd1b03bccc07c7495c34d Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 16:26:17 +0200 Subject: [PATCH 09/24] feat(megadetector): bbox routing + detectModelType branch --- src/main/utils/bbox.js | 7 +++++++ test/main/utils/bbox.test.js | 24 ++++++++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 test/main/utils/bbox.test.js diff --git a/src/main/utils/bbox.js b/src/main/utils/bbox.js index d675dc20..9ffef372 100644 --- a/src/main/utils/bbox.js +++ b/src/main/utils/bbox.js @@ -43,6 +43,7 @@ export function transformBboxToCamtrapDP(detection, modelType) { } case 'manas': + case 'megadetector': case 'deepfaune': { // DeepFaune/Manas xywhn format: [x_center, y_center, width, height] (normalized 0-1, center format) if (!detection.xywhn || !Array.isArray(detection.xywhn) || detection.xywhn.length < 4) { @@ -94,6 +95,12 @@ export function detectModelType(prediction) { return 'manas' } + // MegaDetector v6.0 also uses xywhn (ultralytics YOLO output) and ships model_version '6.0'. + // Branch before the generic xywhn fallback so it doesn't get misrouted as 'deepfaune'. + if (version === '6.0' && prediction.detections?.[0]?.xywhn) { + return 'megadetector' + } + // DeepFaune versions typically look like "1.3" if (version.match(/^\d+\.\d+$/) && prediction.detections?.[0]?.xywhn) { return 'deepfaune' diff --git a/test/main/utils/bbox.test.js b/test/main/utils/bbox.test.js new file mode 100644 index 00000000..35608623 --- /dev/null +++ b/test/main/utils/bbox.test.js @@ -0,0 +1,24 @@ +import { test, describe } from 'node:test' +import assert from 'node:assert/strict' +import { transformBboxToCamtrapDP, detectModelType } from '../../../src/main/utils/bbox.js' + +describe('MegaDetector bbox handling', () => { + test('transformBboxToCamtrapDP routes megadetector through xywhn → top-left', () => { + const detection = { xywhn: [0.5, 0.5, 0.4, 0.6] } + const result = transformBboxToCamtrapDP(detection, 'megadetector') + assert.deepEqual(result, { + bboxX: 0.3, // 0.5 - 0.4/2 + bboxY: 0.2, // 0.5 - 0.6/2 + bboxWidth: 0.4, + bboxHeight: 0.6 + }) + }) + + test('detectModelType identifies megadetector by version 6.0 + xywhn', () => { + const prediction = { + model_version: '6.0', + detections: [{ xywhn: [0.5, 0.5, 0.4, 0.6], conf: 0.9, label: 'animal' }] + } + assert.equal(detectModelType(prediction), 'megadetector') + }) +}) From d2b86e86d413cfb0b8720ee362359cd998cc90cb Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 21:25:56 +0200 Subject: [PATCH 10/24] feat(megadetector): LitServe HTTP server + e2e tests --- .../common/run_megadetector_server.py | 266 ++++++++++++++++++ .../common/tests/test_megadetector_server.py | 142 ++++++++++ 2 files changed, 408 insertions(+) create mode 100644 python-environments/common/run_megadetector_server.py create mode 100644 python-environments/common/tests/test_megadetector_server.py diff --git a/python-environments/common/run_megadetector_server.py b/python-environments/common/run_megadetector_server.py new file mode 100644 index 00000000..8f69ac07 --- /dev/null +++ b/python-environments/common/run_megadetector_server.py @@ -0,0 +1,266 @@ +""" +CLI script to run MegaDetector v6 as a LitServer. + +MegaDetector v6 is a YOLO-based detector (animals / people / vehicles only — no +species classification). Used in Biowatch as a fast blank-filter before manual +species annotation. + +Start the server: + +``` +run_megadetector_server.py \\ + --port 8000 \\ + --filepath-detector-weights ./path/to/MDV6-yolov10-e-1280.pt \\ + --detection-confidence-threshold 0.2 +``` + +A Swagger API documentation is served at localhost:${port}/docs + +health: + +``` +$ curl http://localhost:${port}/health +"ok" +``` + +info: + +``` +$ curl http://localhost:${port}/info +{ + "model": {"type": "megadetector", "version": "6.0"}, + "server": {...} +} +``` + +predict (streaming): + +``` +$ curl -X POST http://localhost:${port}/predict \\ + -H "Content-Type: application/json" \\ + -d '{"instances": [{"filepath": "/path/to/your/image"}]}' +``` + +Output per image: + +```json +{ + "output": { + "predictions": [{ + "filepath": "/path/to/image", + "classifications": {}, + "detections": [ + {"label": "animal", "conf": 0.94, "xywhn": [...], "xyxy": [...]} + ], + "prediction": "animal", + "prediction_score": 0.94, + "model_version": "6.0" + }] + } +} +``` + +The top-level `prediction` field translates MD's `"person"` label to the +binomial `"homo sapiens"` (the only MD category that is genuinely a species), +so it integrates with Biowatch's species tooltips and IUCN lookups. Per-bbox +`detections[].label` stays raw. +""" + +import logging +from pathlib import Path + +import litserve as ls +from absl import app, flags +from fastapi import HTTPException +from ultralytics import YOLO + +from detection_utils import propagate_extra_fields, to_detection_record +from utils import VideoCapableLitAPI, is_video_file, safe_imread + +logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s") +logger = logging.getLogger(__name__) + +MODEL_VERSION = "6.0" + +# Map MegaDetector detection labels to Biowatch-canonical scientific names +# for the top-level `prediction` field. Per-bbox `detections[].label` stays raw. +# `person` is genuinely a species — emit the binomial so it integrates with +# Biowatch's species tooltips and IUCN lookups. `animal` and `vehicle` are not +# species and pass through unchanged. +LABEL_TO_PREDICTION = { + "person": "homo sapiens", +} + +_PORT = flags.DEFINE_integer("port", 8000, "Port to run the server on.") +_API_PATH = flags.DEFINE_string("api_path", "/predict", "URL path for the server endpoint.") +_WORKERS_PER_DEVICE = flags.DEFINE_integer("workers_per_device", 1, "Number of server replicas per device.") +_TIMEOUT = flags.DEFINE_integer("timeout", 30, "Timeout (in seconds) for requests.") +_BACKLOG = flags.DEFINE_integer("backlog", 2048, "Maximum number of connections to hold in backlog.") +_FILEPATH_DETECTOR_WEIGHTS = flags.DEFINE_string( + name="filepath-detector-weights", + default=None, + help="filepath for the weights of the MegaDetector detector", + required=True, +) +_DETECTION_CONFIDENCE_THRESHOLD = flags.DEFINE_float( + "detection-confidence-threshold", + 0.2, + "Confidence threshold below which a detection is ignored when computing the top prediction. " + "Detections themselves are always returned; this only controls the 'prediction'/'prediction_score' fields.", +) +_EXTRA_FIELDS = flags.DEFINE_list( + "extra_fields", + None, + "Comma-separated list of extra fields to propagate from request to response.", +) + + +def predict_one( + detector: YOLO, + filepath: Path, + confidence_threshold: float, + model_version: str = MODEL_VERSION, +) -> dict: + """Run MegaDetector on a single image and produce a Biowatch-compatible prediction dict. + + The 'prediction' field is the label of the highest-confidence detection + whose conf >= confidence_threshold, translated via LABEL_TO_PREDICTION. + When no detection passes the threshold, prediction is 'blank' and + prediction_score is None. + """ + imagecv = safe_imread(filepath) + ultralytics_results = detector(imagecv, verbose=False) + yolo_out = ultralytics_results[0] + bboxes = yolo_out.boxes + # class_names comes from the .pt file's embedded `names` attribute — never hard-code. + class_names = yolo_out.names + + detection_records = [ + to_detection_record( + conf=conf, + class_instance=class_instance, + xywhn=xywhn, + xyxy=xyxy, + class_label_mapping=class_names, + ) + for conf, class_instance, xywhn, xyxy in zip( + bboxes.conf.cpu().tolist(), + bboxes.cls.cpu().numpy().astype(int).tolist(), + bboxes.xywhn.cpu().numpy().tolist(), + bboxes.xyxy.cpu().numpy().tolist(), + strict=True, + ) + ] + + # MD detects across animal/person/vehicle — pick top-confidence across ALL classes. + # (We don't reuse select_best_animal_detection, which filters for the animal class.) + above = [d for d in detection_records if d["conf"] >= confidence_threshold] + if not above: + return { + "predictions": [ + { + "filepath": str(filepath), + "classifications": {}, + "detections": detection_records, + "prediction": "blank", + "prediction_score": None, + "model_version": model_version, + } + ], + } + + top = max(above, key=lambda d: d["conf"]) + prediction_label = LABEL_TO_PREDICTION.get(top["label"], top["label"]) + return { + "predictions": [ + { + "filepath": str(filepath), + "classifications": {}, + "detections": detection_records, + "prediction": prediction_label, + "prediction_score": top["conf"], + "model_version": model_version, + } + ], + } + + +class MegaDetectorLitAPI(ls.LitAPI, VideoCapableLitAPI): + """MegaDetector API server with video support.""" + + def __init__( + self, + filepath_detector_weights: Path, + detection_confidence_threshold: float, + extra_fields: list[str] | None = None, + *args, + **kwargs, + ) -> None: + super().__init__(*args, **kwargs) + self.filepath_detector_weights = filepath_detector_weights + self.detection_confidence_threshold = detection_confidence_threshold + self.extra_fields = extra_fields or [] + + def setup(self, device): + del device # Unused. + self.detector = YOLO(self.filepath_detector_weights) + + def decode_request(self, request, **kwargs): + for instance in request["instances"]: + filepath = instance["filepath"] + if not is_video_file(filepath) and not Path(filepath).exists(): + raise HTTPException(400, f"Cannot access filepath: `{filepath}`") + return request + + def _predict_single_image(self, filepath: str, **kwargs) -> dict: + single_instances_dict = {"instances": [{"filepath": filepath}]} + single_predictions_dict = predict_one( + detector=self.detector, + filepath=Path(filepath), + confidence_threshold=self.detection_confidence_threshold, + ) + return propagate_extra_fields(self.extra_fields, single_instances_dict, single_predictions_dict) + + def predict(self, x, **kwargs): + instances = x.get("instances", []) + logger.info(f"[MegaDetector] Processing {len(instances)} instances") + try: + yield from self.predict_with_video_support(x, **kwargs) + except Exception as e: + logger.error(f"[MegaDetector] Prediction failed: {e}", exc_info=True) + raise + + def encode_response(self, output, **kwargs): + for out in output: + yield {"output": out} + + +def main(argv: list[str]) -> None: + del argv # Unused. + print("[STARTUP] Starting MegaDetector LitServer...") + api = MegaDetectorLitAPI( + filepath_detector_weights=Path(_FILEPATH_DETECTOR_WEIGHTS.value), + detection_confidence_threshold=_DETECTION_CONFIDENCE_THRESHOLD.value, + extra_fields=_EXTRA_FIELDS.value, + api_path=_API_PATH.value, + stream=True, + ) + model_metadata = {"version": MODEL_VERSION, "type": "megadetector"} + server = ls.LitServer( + api, + accelerator="auto", + devices="auto", + workers_per_device=_WORKERS_PER_DEVICE.value, + model_metadata=model_metadata, + timeout=_TIMEOUT.value, + enable_shutdown_api=True, + ) + server.run( + port=_PORT.value, + generate_client_file=False, + backlog=_BACKLOG.value, + ) + + +if __name__ == "__main__": + app.run(main) diff --git a/python-environments/common/tests/test_megadetector_server.py b/python-environments/common/tests/test_megadetector_server.py new file mode 100644 index 00000000..e6429a33 --- /dev/null +++ b/python-environments/common/tests/test_megadetector_server.py @@ -0,0 +1,142 @@ +"""End-to-end tests for MegaDetector ML server.""" + +import os + +import httpx +import pytest + +from tests.conftest import ServerProcess, find_free_port, parse_streaming_response + +MEGADETECTOR_DETECTOR_WEIGHTS = os.environ.get("MEGADETECTOR_DETECTOR_WEIGHTS", "") + +pytestmark = pytest.mark.skipif( + not MEGADETECTOR_DETECTOR_WEIGHTS, + reason="MegaDetector weights not available. Set MEGADETECTOR_DETECTOR_WEIGHTS.", +) + + +@pytest.fixture(scope="module") +def megadetector_server(): + """Start MegaDetector server for the test module.""" + port = find_free_port() + command = [ + "uv", + "run", + "python", + "run_megadetector_server.py", + "--port", + str(port), + "--timeout", + "60", + "--filepath-detector-weights", + MEGADETECTOR_DETECTOR_WEIGHTS, + "--detection-confidence-threshold", + "0.2", + ] + with ServerProcess(command, port, startup_timeout=300) as server: + yield server + + +class TestMegaDetectorServer: + """Tests for MegaDetector server endpoints.""" + + def test_health_endpoint(self, megadetector_server): + resp = httpx.get(f"{megadetector_server.base_url}/health", timeout=10.0) + assert resp.status_code == 200 + assert resp.text == "ok" + + def test_info_endpoint(self, megadetector_server): + resp = httpx.get(f"{megadetector_server.base_url}/info", timeout=10.0) + assert resp.status_code == 200 + data = resp.json() + assert "model" in data + assert data["model"]["type"] == "megadetector" + assert "server" in data + + def test_predict_animal_image(self, megadetector_server, test_images): + """A clearly-animal image should produce prediction='animal'.""" + payload = {"instances": [{"filepath": str(test_images["chamois"])}]} + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=60.0, + ) as resp: + assert resp.status_code == 200 + results = parse_streaming_response(resp) + + assert len(results) == 1 + pred = results[0]["output"]["predictions"][0] + assert pred["filepath"] == str(test_images["chamois"]) + assert pred["prediction"] == "animal" + assert pred["prediction_score"] >= 0.2 + assert pred["classifications"] == {} + assert len(pred["detections"]) >= 1 + assert pred["detections"][0]["label"] in {"animal", "person", "vehicle"} + assert "xywhn" in pred["detections"][0] + + def test_predict_human_image(self, megadetector_server, test_images): + """A human image's top prediction should be 'homo sapiens' (translated from MD's 'person' label).""" + payload = {"instances": [{"filepath": str(test_images["human"])}]} + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=60.0, + ) as resp: + results = parse_streaming_response(resp) + pred = results[0]["output"]["predictions"][0] + assert pred["prediction"] == "homo sapiens" + # Raw bbox labels stay 'person' — MD's native output is preserved per-bbox. + person_dets = [d for d in pred["detections"] if d["label"] == "person"] + assert len(person_dets) >= 1, "raw 'person' label must be preserved per-detection" + + def test_predict_empty_image(self, megadetector_server, test_images): + """An empty (no subject) image should produce prediction='blank'.""" + payload = {"instances": [{"filepath": str(test_images["empty"])}]} + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=60.0, + ) as resp: + results = parse_streaming_response(resp) + pred = results[0]["output"]["predictions"][0] + assert pred["prediction"] == "blank" + assert pred["prediction_score"] is None + + def test_predict_streaming(self, megadetector_server, test_images): + """Streaming returns one chunk per image, all classifications empty.""" + payload = { + "instances": [ + {"filepath": str(test_images["badger"])}, + {"filepath": str(test_images["fox"])}, + {"filepath": str(test_images["empty"])}, + ] + } + with httpx.stream( + "POST", + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=120.0, + ) as resp: + results = parse_streaming_response(resp) + assert len(results) == 3 + for r in results: + pred = r["output"]["predictions"][0] + assert pred["classifications"] == {} + assert pred["model_version"] == "6.0" + + def test_predict_invalid_filepath(self, megadetector_server): + payload = {"instances": [{"filepath": "/nonexistent/image.jpg"}]} + resp = httpx.post( + f"{megadetector_server.base_url}/predict", + json=payload, + timeout=30.0, + ) + assert resp.status_code in (200, 400) + + def test_docs_endpoint(self, megadetector_server): + resp = httpx.get(f"{megadetector_server.base_url}/docs", timeout=10.0) + assert resp.status_code == 200 + assert "swagger" in resp.text.lower() or "openapi" in resp.text.lower() From bb20bf3bdf7313a0d4ed9f5b0973c2e54e3af070 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 21:26:48 +0200 Subject: [PATCH 11/24] feat(megadetector): main-process server launcher + switch case --- src/main/services/ml/server.ts | 92 ++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/src/main/services/ml/server.ts b/src/main/services/ml/server.ts index f96c87f4..1bc357cd 100644 --- a/src/main/services/ml/server.ts +++ b/src/main/services/ml/server.ts @@ -14,6 +14,7 @@ import crypto from 'crypto' import os from 'node:os' import { getMLModelLocalInstallPath, getMLModelEnvironmentLocalInstallPath } from './paths.js' +import { findModel } from '../../../shared/mlmodels.js' // ============================================================================ // Active Server Registry @@ -509,6 +510,75 @@ export async function startManasHTTPServer({ return { process: pythonProcess, shutdownApiKey } } +interface MegaDetectorServerOptions { + port: number + detectorWeightsFilepath: string + detectionConfidenceThreshold: number + timeout: number + pythonEnvironment: { + reference: { id: string; version: string } + } +} + +/** + * Starts the MegaDetector HTTP server using a specified Python environment and configuration. + * + * @async + * @param {MegaDetectorServerOptions} options + * @returns {Promise<{process: ChildProcess, shutdownApiKey: string}>} + */ +export async function startMegaDetectorHTTPServer({ + port, + detectorWeightsFilepath, + detectionConfidenceThreshold, + timeout, + pythonEnvironment +}: MegaDetectorServerOptions): Promise<{ process: ChildProcess; shutdownApiKey: string }> { + log.info('startMegaDetectorHTTPServer initiated') + log.info(pythonEnvironment) + const localInstalRootDirPythonEnvironment = join( + getMLModelEnvironmentLocalInstallPath({ + ...pythonEnvironment.reference + }), + pythonEnvironment.reference.id + ) + log.info('Local Python Environment root dir is', localInstalRootDirPythonEnvironment) + const scriptPath = is.dev + ? join(__dirname, '../../python-environments/common/run_megadetector_server.py') + : join(process.resourcesPath, 'python-environments', 'common', 'run_megadetector_server.py') + const pythonInterpreter = is.dev + ? join(__dirname, '../../python-environments/common/.venv/bin/python') + : os.platform() === 'win32' + ? join(localInstalRootDirPythonEnvironment, 'python.exe') + : join(localInstalRootDirPythonEnvironment, 'bin', 'python') + log.info('Python Interpreter found in', pythonInterpreter) + log.info('Script path is', scriptPath) + const scriptArgs = [ + '--port', + String(port), + '--filepath-detector-weights', + detectorWeightsFilepath, + '--detection-confidence-threshold', + String(detectionConfidenceThreshold), + '--timeout', + String(timeout) + ] + log.info('Script args: ', scriptArgs) + + const shutdownApiKey = crypto.randomUUID() + log.info('Generated shutdown API key for MegaDetector server') + + const pythonProcess = await startAndWaitTillServerHealty({ + pythonInterpreter, + scriptPath, + scriptArgs, + healthEndpoint: `http://localhost:${port}/health`, + env: { LIT_SHUTDOWN_API_KEY: shutdownApiKey } + }) + + return { process: pythonProcess, shutdownApiKey } +} + // ============================================================================ // Server Lifecycle Management // ============================================================================ @@ -711,6 +781,28 @@ export async function startMLModelHTTPServer({ }) return { port: port, process: pythonProcess, shutdownApiKey } } + case 'megadetector': { + const port = await resolveServerPort(is.dev ? 8003 : null) + const localInstallPath = getMLModelLocalInstallPath({ ...modelReference }) + log.info(`Local ML Model install path ${localInstallPath}`) + const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10-e-1280.pt') + const model = findModel({ ...modelReference }) + const { process: pythonProcess, shutdownApiKey } = await startMegaDetectorHTTPServer({ + port, + detectorWeightsFilepath, + detectionConfidenceThreshold: model?.detectionConfidenceThreshold ?? 0.2, + timeout: 30, + pythonEnvironment: pythonEnvironment + }) + log.info(`pythonProcess: ${JSON.stringify(pythonProcess)}`) + registerActiveServer({ + pid: pythonProcess.pid as number, + port, + shutdownApiKey, + modelId: modelReference.id + }) + return { port: port, process: pythonProcess, shutdownApiKey } + } default: { log.warn( `startMLModelHTTPServer: Not implemented for ${modelReference.id} version ${modelReference.version}` From 01c5cb37dcffa7b4a12fa3093980a0df30c9c563 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 21:27:21 +0200 Subject: [PATCH 12/24] feat(megadetector): hide species panel + show 'detection only' on MD card --- src/renderer/src/models/ModelCard.jsx | 30 +++++++++++++++++---------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/src/renderer/src/models/ModelCard.jsx b/src/renderer/src/models/ModelCard.jsx index a64f461c..fa86dffe 100644 --- a/src/renderer/src/models/ModelCard.jsx +++ b/src/renderer/src/models/ModelCard.jsx @@ -179,7 +179,11 @@ export default function ModelCard({
v{model.reference.version} · {formatSize(model.size_in_MB)} ·{' '} - {model.species_count} species + + {model.detectionOnly + ? `Detection only · ${model.species_count} categories` + : `${model.species_count} species`} +
{!isDownloading && ( @@ -200,17 +204,21 @@ export default function ModelCard({ )} -
{ - e.stopPropagation() - onToggleSpecies?.(model.reference.id) - }} - > - {speciesOpen ? '▾ Hide species' : `▸ View ${model.species_count} species`} -
+ {!model.detectionOnly && ( + <> +
{ + e.stopPropagation() + onToggleSpecies?.(model.reference.id) + }} + > + {speciesOpen ? '▾ Hide species' : `▸ View ${model.species_count} species`} +
- {speciesOpen && speciesPanel} + {speciesOpen && speciesPanel} + + )} ) } From 8baa18b3026ef692317bd8248fc918c9919148b2 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Sat, 16 May 2026 21:27:35 +0200 Subject: [PATCH 13/24] feat(megadetector): 'detection only' label in model picker dropdown --- src/renderer/src/models/ModelSelect.jsx | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/renderer/src/models/ModelSelect.jsx b/src/renderer/src/models/ModelSelect.jsx index c5f4c0ec..bc8dd2f4 100644 --- a/src/renderer/src/models/ModelSelect.jsx +++ b/src/renderer/src/models/ModelSelect.jsx @@ -61,7 +61,11 @@ function ModelRow({ model, status }) {
- {model.species_count} species + + {model.detectionOnly + ? `Detection only · ${model.species_count} categories` + : `${model.species_count} species`} +

{model.description}

From dc8c4284f382e6fbf874971d7f8bb9b332f1de87 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 09:59:01 +0200 Subject: [PATCH 14/24] docs(megadetector): add MD to http-servers supported-models table --- docs/http-servers.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/http-servers.md b/docs/http-servers.md index 68d88343..d902a132 100644 --- a/docs/http-servers.md +++ b/docs/http-servers.md @@ -13,11 +13,12 @@ Biowatch uses an **HTTP-based ML model serving architecture** where each machine ### Supported Models -| Model | Focus | Species Coverage | -| ------------------------ | ------------------- | ------------------------------------ | -| **SpeciesNet** (Google) | Global wildlife | 2,000+ species worldwide | -| **DeepFaune** (CNRS) | European fauna | 34 European species | -| **Manas** (OSI-Panthera) | Central Asian fauna | Snow leopard and 11 regional species | +| Model | Focus | Species Coverage | +| ----------------------------------------- | ---------------------------------- | --------------------------------------------- | +| **SpeciesNet** (Google) | Global wildlife | 2,000+ species worldwide | +| **DeepFaune** (CNRS) | European fauna | 34 European species | +| **Manas** (OSI-Panthera) | Central Asian fauna | Snow leopard and 11 regional species | +| **MegaDetector** (Microsoft AI for Earth) | Blank filter / worldwide detection | 3 categories (animal, person, vehicle) | ### Technology Stack @@ -98,7 +99,8 @@ src/ python-environments/common/ ├── run_speciesnet_server.py # SpeciesNet LitServe implementation ├── run_deepfaune_server.py # DeepFaune LitServe implementation -└── run_manas_server.py # Manas LitServe implementation +├── run_manas_server.py # Manas LitServe implementation +└── run_megadetector_server.py # MegaDetector LitServe implementation (detection-only) ``` ## How HTTP Servers Work From 904cb24afdc25707699bc0da12c22426401b993a Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:07:07 +0200 Subject: [PATCH 15/24] tools(megadetector): downloader entry + Makefile env var for MD --- python-environments/common/Makefile | 2 ++ python-environments/common/scripts/download_model.py | 7 ++++++- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/python-environments/common/Makefile b/python-environments/common/Makefile index f69609fb..f46227f4 100644 --- a/python-environments/common/Makefile +++ b/python-environments/common/Makefile @@ -27,6 +27,7 @@ export DEEPFAUNE_CLASSIFIER_WEIGHTS=/tmp/models/deepfaune/1.3/deepfaune-vit_larg export MANAS_DETECTOR_WEIGHTS=/tmp/models/manas/1.0/MDV6-yolov10x.pt export MANAS_CLASSIFIER_WEIGHTS=/tmp/models/manas/1.0/best_model_Fri_Sep__1_18_50_55_2023.pt export MANAS_CLASSES=/tmp/models/manas/1.0/classes_Fri_Sep__1_18_50_55_2023.pickle +export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt # Run all e2e tests (downloads models if not present) test: download-models @@ -41,3 +42,4 @@ download-models: uv run python scripts/download_model.py --model speciesnet --output /tmp/models/speciesnet uv run python scripts/download_model.py --model deepfaune --output /tmp/models/deepfaune uv run python scripts/download_model.py --model manas --output /tmp/models/manas + uv run python scripts/download_model.py --model megadetector --output /tmp/models/megadetector diff --git a/python-environments/common/scripts/download_model.py b/python-environments/common/scripts/download_model.py index 44481900..574113c0 100644 --- a/python-environments/common/scripts/download_model.py +++ b/python-environments/common/scripts/download_model.py @@ -5,6 +5,7 @@ uv run python scripts/download_model.py --model speciesnet --output /tmp/models/speciesnet uv run python scripts/download_model.py --model deepfaune --output /tmp/models/deepfaune uv run python scripts/download_model.py --model manas --output /tmp/models/manas + uv run python scripts/download_model.py --model megadetector --output /tmp/models/megadetector """ import argparse @@ -26,6 +27,10 @@ "repo_id": "earthtoolsmaker/manas", "filename": "1.0.tar.gz", }, + "megadetector": { + "repo_id": "earthtoolsmaker/megadetector", + "filename": "6.0.tar.gz", + }, } @@ -57,7 +62,7 @@ def main(): "--model", required=True, choices=list(MODELS.keys()), - help="Model to download (speciesnet, deepfaune, manas)", + help="Model to download (speciesnet, deepfaune, manas, megadetector)", ) parser.add_argument( "--output", From 5f97777d01bd48de01e57659cc99fdcbd5abae68 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:12:59 +0200 Subject: [PATCH 16/24] fix(megadetector): parseScientificName branch for MD simple labels --- src/main/services/prediction.js | 8 +-- .../services/predictionCommonName.test.js | 49 +++++++++++++++++++ 2 files changed, 54 insertions(+), 3 deletions(-) diff --git a/src/main/services/prediction.js b/src/main/services/prediction.js index 0136c0c2..d4b90367 100644 --- a/src/main/services/prediction.js +++ b/src/main/services/prediction.js @@ -226,12 +226,14 @@ export async function getMedia(db, filepath) { /** * Parse scientific name from prediction based on model type * @param {Object} prediction - Model prediction output - * @param {string} modelType - 'speciesnet' | 'deepfaune' | 'manas' + * @param {string} modelType - 'speciesnet' | 'deepfaune' | 'manas' | 'megadetector' * @returns {string|null} Scientific name or null for blank predictions */ function parseScientificName(prediction, modelType) { - if (modelType === 'deepfaune' || modelType === 'manas') { - // DeepFaune/Manas: Simple label like "chamois", "panthera_uncia", "blank", "empty", "vide" + if (modelType === 'deepfaune' || modelType === 'manas' || modelType === 'megadetector') { + // DeepFaune/Manas/MegaDetector: Simple label like "chamois", "panthera_uncia", + // "animal", "vehicle", "homo sapiens" (MD's translation of "person"), or + // "blank"/"empty"/"vide"/"error" for no-detection cases. const label = prediction.prediction if (!label || label === 'blank' || label === 'empty' || label === 'vide' || label === 'error') { return null diff --git a/test/main/services/predictionCommonName.test.js b/test/main/services/predictionCommonName.test.js index d72a1f36..1424e930 100644 --- a/test/main/services/predictionCommonName.test.js +++ b/test/main/services/predictionCommonName.test.js @@ -118,4 +118,53 @@ describe('insertPrediction populates commonName via dictionary', () => { assert.equal(insertedRows[0].scientificName, null) assert.equal(insertedRows[0].commonName, null) }) + + test('MegaDetector "animal" label stores as simple label with resolved common name', async () => { + const { db, insertedRows } = makeFakeDb({ mediaRecord: baseMedia() }) + const prediction = { + filepath: '/fake/img.jpg', + prediction: 'animal', + prediction_score: 0.7, + detections: [] + } + + await insertPrediction(db, prediction, { modelID: 'megadetector' }) + + assert.equal(insertedRows.length, 1) + // Must NOT be parsed as a SpeciesNet binomial — that produced "undefined undefined" in production. + assert.equal(insertedRows[0].scientificName, 'animal') + assert.equal(insertedRows[0].commonName, 'animal') + }) + + test('MegaDetector "homo sapiens" translation lands with Human common name', async () => { + const { db, insertedRows } = makeFakeDb({ mediaRecord: baseMedia() }) + const prediction = { + filepath: '/fake/img.jpg', + prediction: 'homo sapiens', + prediction_score: 0.93, + detections: [] + } + + await insertPrediction(db, prediction, { modelID: 'megadetector' }) + + assert.equal(insertedRows.length, 1) + assert.equal(insertedRows[0].scientificName, 'homo sapiens') + assert.equal(insertedRows[0].commonName, 'human') + }) + + test('MegaDetector blank prediction leaves both scientificName and commonName null', async () => { + const { db, insertedRows } = makeFakeDb({ mediaRecord: baseMedia() }) + const prediction = { + filepath: '/fake/img.jpg', + prediction: 'blank', + // prediction_score deliberately omitted — server omits it for blanks + detections: [] + } + + await insertPrediction(db, prediction, { modelID: 'megadetector' }) + + assert.equal(insertedRows.length, 1) + assert.equal(insertedRows[0].scientificName, null) + assert.equal(insertedRows[0].commonName, null) + }) }) From 48bea5e6a695d3b5cbf4f8973b2f86f2c7f2ac1e Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:13:02 +0200 Subject: [PATCH 17/24] fix(megadetector): omit prediction_score on blank predictions (was null, rejected by Zod) --- python-environments/common/run_megadetector_server.py | 7 ++++--- .../common/tests/test_megadetector_server.py | 8 ++++++-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/python-environments/common/run_megadetector_server.py b/python-environments/common/run_megadetector_server.py index 8f69ac07..89908257 100644 --- a/python-environments/common/run_megadetector_server.py +++ b/python-environments/common/run_megadetector_server.py @@ -125,8 +125,10 @@ def predict_one( The 'prediction' field is the label of the highest-confidence detection whose conf >= confidence_threshold, translated via LABEL_TO_PREDICTION. - When no detection passes the threshold, prediction is 'blank' and - prediction_score is None. + When no detection passes the threshold, 'prediction' is 'blank' and + the 'prediction_score' key is omitted entirely (matches DeepFaune's + behavior and the JS-side Zod schema, which accepts an absent + prediction_score but rejects null). """ imagecv = safe_imread(filepath) ultralytics_results = detector(imagecv, verbose=False) @@ -163,7 +165,6 @@ def predict_one( "classifications": {}, "detections": detection_records, "prediction": "blank", - "prediction_score": None, "model_version": model_version, } ], diff --git a/python-environments/common/tests/test_megadetector_server.py b/python-environments/common/tests/test_megadetector_server.py index e6429a33..53b84a67 100644 --- a/python-environments/common/tests/test_megadetector_server.py +++ b/python-environments/common/tests/test_megadetector_server.py @@ -92,7 +92,11 @@ def test_predict_human_image(self, megadetector_server, test_images): assert len(person_dets) >= 1, "raw 'person' label must be preserved per-detection" def test_predict_empty_image(self, megadetector_server, test_images): - """An empty (no subject) image should produce prediction='blank'.""" + """An empty (no subject) image should produce prediction='blank' with no prediction_score key. + + The JS-side Zod validator accepts an absent prediction_score but rejects null, + so the server omits the key entirely on blanks (matches DeepFaune's behavior). + """ payload = {"instances": [{"filepath": str(test_images["empty"])}]} with httpx.stream( "POST", @@ -103,7 +107,7 @@ def test_predict_empty_image(self, megadetector_server, test_images): results = parse_streaming_response(resp) pred = results[0]["output"]["predictions"][0] assert pred["prediction"] == "blank" - assert pred["prediction_score"] is None + assert "prediction_score" not in pred, "blank predictions must omit prediction_score, not emit null" def test_predict_streaming(self, megadetector_server, test_images): """Streaming returns one chunk per image, all classifications empty.""" From 28bddcac9a711f3de978755c21658574d4a5650c Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:18:18 +0200 Subject: [PATCH 18/24] feat(models): move MegaDetector to second position in model zoo --- src/shared/mlmodels.js | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/src/shared/mlmodels.js b/src/shared/mlmodels.js index 8d0e11cf..11ac0903 100644 --- a/src/shared/mlmodels.js +++ b/src/shared/mlmodels.js @@ -172,6 +172,23 @@ export const modelZoo = [ species_count: '2,000+', species_data: 'speciesnet' }, + { + reference: { id: 'megadetector', version: '6.0' }, + pythonEnvironment: { id: 'common', version: '0.1.4' }, + name: 'MegaDetector', + size_in_MB: 118, // tarball ~117.8 MB, weights file ~128 MB — replace with exact size once HF upload lands + files: 2, + downloadURL: + 'https://huggingface.co/earthtoolsmaker/megadetector/resolve/main/6.0.tar.gz?download=true', + description: + "MegaDetector is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, without identifying species. Useful as a fast blank-filter before manual species annotation.", + website: 'https://github.com/microsoft/MegaDetector', + logo: 'megadetector', + detectionConfidenceThreshold: 0.2, + region: 'worldwide', + species_count: 3, + detectionOnly: true + }, { reference: { id: 'deepfaune', version: '1.3' }, pythonEnvironment: { id: 'common', version: '0.1.4' }, @@ -205,23 +222,6 @@ export const modelZoo = [ region: 'himalayas', species_count: 11, species_data: 'manas' - }, - { - reference: { id: 'megadetector', version: '6.0' }, - pythonEnvironment: { id: 'common', version: '0.1.4' }, - name: 'MegaDetector', - size_in_MB: 118, // tarball ~117.8 MB, weights file ~128 MB — replace with exact size once HF upload lands - files: 2, - downloadURL: - 'https://huggingface.co/earthtoolsmaker/megadetector/resolve/main/6.0.tar.gz?download=true', - description: - "MegaDetector is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, without identifying species. Useful as a fast blank-filter before manual species annotation.", - website: 'https://github.com/microsoft/MegaDetector', - logo: 'megadetector', - detectionConfidenceThreshold: 0.2, - region: 'worldwide', - species_count: 3, - detectionOnly: true } ] From 3f0bc9492b1cba422c7db554be6e361b40930c41 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:25:40 +0200 Subject: [PATCH 19/24] fix(ml-server): reject corrupt video metadata so false-positive 'animal' tags don't slip through --- .../common/tests/test_utils.py | 89 ++++++++++++++++++- python-environments/common/utils.py | 15 ++++ 2 files changed, 103 insertions(+), 1 deletion(-) diff --git a/python-environments/common/tests/test_utils.py b/python-environments/common/tests/test_utils.py index a3c7ec04..59b5c353 100644 --- a/python-environments/common/tests/test_utils.py +++ b/python-environments/common/tests/test_utils.py @@ -1,6 +1,10 @@ """Unit tests for shared ML server utilities.""" -from utils import VideoCapableLitAPI +from unittest.mock import MagicMock, patch + +import pytest + +from utils import VideoCapableLitAPI, get_video_metadata class TestNormalizeFailedPredictions: @@ -96,3 +100,86 @@ def test_missing_predictions_key(self): result = {} normalized = VideoCapableLitAPI._normalize_failed_predictions(result, "/x.jpg") assert normalized == {"predictions": []} + + +class TestGetVideoMetadataSanity: + """Tests that get_video_metadata rejects corrupt-file metadata. + + OpenCV can return huge negative frame_count values on broken containers + (uint64 underflow cast to int), producing absurd durations and letting a + single garbage frame slip into inference. The metadata helper rejects + these so the consumer's error-handling pipeline can flag the file. + """ + + def _mocked_capture(self, fps, frame_count, opened=True): + cap = MagicMock() + cap.isOpened.return_value = opened + cap.get.side_effect = lambda prop: { + 5: fps, # cv2.CAP_PROP_FPS + 7: frame_count, # cv2.CAP_PROP_FRAME_COUNT + }.get(prop, 0) + return cap + + def test_rejects_negative_frame_count(self): + """A corrupt MP4 with negative frame_count must raise, not return junk.""" + with ( + patch( + "utils.cv2.VideoCapture", + return_value=self._mocked_capture(fps=30, frame_count=-922337203685477), + ), + pytest.raises(ValueError, match="Invalid video metadata"), + ): + get_video_metadata("/corrupt.mp4") + + def test_rejects_zero_frame_count(self): + """A file that reports zero frames is not a usable video.""" + with ( + patch( + "utils.cv2.VideoCapture", + return_value=self._mocked_capture(fps=30, frame_count=0), + ), + pytest.raises(ValueError, match="Invalid video metadata"), + ): + get_video_metadata("/empty.mp4") + + def test_rejects_zero_fps(self): + """fps=0 means duration is meaningless.""" + with ( + patch( + "utils.cv2.VideoCapture", + return_value=self._mocked_capture(fps=0, frame_count=100), + ), + pytest.raises(ValueError, match="Invalid video metadata"), + ): + get_video_metadata("/no-fps.mp4") + + def test_rejects_excessive_duration(self): + """Duration above the 24h sanity ceiling is rejected.""" + # 30 fps × 30 hours of frames → 30h duration → over the 24h limit. + excessive_frame_count = 30 * 60 * 60 * 30 + with ( + patch( + "utils.cv2.VideoCapture", + return_value=self._mocked_capture(fps=30, frame_count=excessive_frame_count), + ), + pytest.raises(ValueError, match="Invalid video metadata"), + ): + get_video_metadata("/huge.mp4") + + def test_rejects_unopenable_video(self): + """Cannot open returns the existing 'Cannot open video' error.""" + with ( + patch( + "utils.cv2.VideoCapture", + return_value=self._mocked_capture(fps=30, frame_count=100, opened=False), + ), + pytest.raises(ValueError, match="Cannot open video"), + ): + get_video_metadata("/missing.mp4") + + def test_accepts_normal_video(self): + """Sane metadata round-trips through.""" + with patch("utils.cv2.VideoCapture", return_value=self._mocked_capture(fps=30, frame_count=300)): + metadata = get_video_metadata("/ok.mp4") + assert metadata["fps"] == 30 + assert metadata["duration"] == pytest.approx(10.0) # 300 frames / 30 fps diff --git a/python-environments/common/utils.py b/python-environments/common/utils.py index b1d96807..c02864d9 100644 --- a/python-environments/common/utils.py +++ b/python-environments/common/utils.py @@ -70,6 +70,13 @@ def get_video_metadata(video_path: str) -> dict[str, float]: Returns: Dictionary with 'fps' and 'duration' keys + + Raises: + ValueError: if the video cannot be opened, or if OpenCV reports + implausible metadata (corrupt files often surface as a hugely + negative frame_count from a uint64 underflow, which yields a + nonsense duration and would otherwise let a single garbage + frame leak through to inference). """ cap = cv2.VideoCapture(video_path) if not cap.isOpened(): @@ -80,6 +87,14 @@ def get_video_metadata(video_path: str) -> dict[str, float]: duration = frame_count / fps if fps > 0 else 0 cap.release() + # 24h ceiling is well above any realistic camera-trap clip but well below + # the absurd values OpenCV surfaces on corrupt containers. + MAX_REASONABLE_DURATION_S = 24 * 60 * 60 + if fps <= 0 or frame_count <= 0 or not (0 < duration <= MAX_REASONABLE_DURATION_S): + raise ValueError( + f"Invalid video metadata (fps={fps}, frame_count={frame_count}, duration={duration}): {video_path}" + ) + return {"fps": fps, "duration": duration} From 0754c7280f6b2e5319ae4804a0d509813181732c Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:35:34 +0200 Subject: [PATCH 20/24] fix(megadetector): per-detection classificationProbability for MD (no classifier) --- src/main/services/prediction.js | 10 ++- .../services/predictionCommonName.test.js | 61 +++++++++++++++++++ 2 files changed, 69 insertions(+), 2 deletions(-) diff --git a/src/main/services/prediction.js b/src/main/services/prediction.js index d4b90367..00680e2b 100644 --- a/src/main/services/prediction.js +++ b/src/main/services/prediction.js @@ -329,7 +329,12 @@ export async function insertPrediction(db, prediction, modelInfo = {}) { // Combine: best + filtered additional const validDetections = [bestDetection, ...additionalDetections] - // Create one observation per valid detection + // Create one observation per valid detection. For detection-only models + // (MegaDetector) there is no whole-image classifier, so each bbox's + // detection confidence IS the classification probability — write it + // per-row instead of copying the top-level prediction_score across all + // rows (which would render as the same number on every box in the UI). + const isDetectionOnly = modelType === 'megadetector' for (const detection of validDetections) { const bbox = transformBboxToCamtrapDP(detection, modelType) const observationData = { @@ -339,7 +344,8 @@ export async function insertPrediction(db, prediction, modelInfo = {}) { bboxY: bbox?.bboxY ?? null, bboxWidth: bbox?.bboxWidth ?? null, bboxHeight: bbox?.bboxHeight ?? null, - detectionConfidence: detection.conf + detectionConfidence: detection.conf, + ...(isDetectionOnly && { classificationProbability: detection.conf }) } await db.insert(observations).values(observationData) } diff --git a/test/main/services/predictionCommonName.test.js b/test/main/services/predictionCommonName.test.js index 1424e930..efd155c7 100644 --- a/test/main/services/predictionCommonName.test.js +++ b/test/main/services/predictionCommonName.test.js @@ -167,4 +167,65 @@ describe('insertPrediction populates commonName via dictionary', () => { assert.equal(insertedRows[0].scientificName, null) assert.equal(insertedRows[0].commonName, null) }) + + test('MegaDetector classificationProbability is per-detection (not the top-level prediction_score)', async () => { + // MD has no classifier — every bbox is a separate detection. Each + // observation row must carry that detection's own confidence, not the + // top-level prediction_score copied across all rows. Regression test for + // the production bug where 6 detections all showed conf=0.70 in the UI. + const { db, insertedRows } = makeFakeDb({ mediaRecord: baseMedia() }) + const prediction = { + filepath: '/fake/img.jpg', + prediction: 'animal', + prediction_score: 0.699, + detections: [ + { class: 0, label: 'animal', conf: 0.699, xywhn: [0.1, 0.5, 0.1, 0.2], xyxy: [0, 0, 0, 0] }, + { class: 0, label: 'animal', conf: 0.446, xywhn: [0.2, 0.6, 0.1, 0.1], xyxy: [0, 0, 0, 0] }, + { class: 0, label: 'animal', conf: 0.293, xywhn: [0.3, 0.7, 0.1, 0.1], xyxy: [0, 0, 0, 0] } + ] + } + + await insertPrediction(db, prediction, { + modelID: 'megadetector', + detectionConfidenceThreshold: 0.2 + }) + + assert.equal(insertedRows.length, 3) + // Sorted by conf desc per insertPrediction's existing behavior + assert.equal(insertedRows[0].classificationProbability, 0.699) + assert.equal(insertedRows[1].classificationProbability, 0.446) + assert.equal(insertedRows[2].classificationProbability, 0.293) + // detectionConfidence must match — for MD they are the same value. + assert.equal(insertedRows[0].detectionConfidence, 0.699) + assert.equal(insertedRows[1].detectionConfidence, 0.446) + assert.equal(insertedRows[2].detectionConfidence, 0.293) + }) + + test('SpeciesNet classificationProbability remains the top-level prediction_score across detections', async () => { + // For a true classifier model the classification probability IS the + // whole-image classifier output — same across all bboxes by design. + // Locks the existing behavior so the MD-specific change doesn't regress it. + const { db, insertedRows } = makeFakeDb({ mediaRecord: baseMedia() }) + const prediction = { + filepath: '/fake/img.jpg', + prediction: '00000000-0000-0000-0000-000000000001;mammalia;rodentia;sciuridae;sciurus;vulgaris;eurasian red squirrel', + prediction_score: 0.95, + detections: [ + { category: '1', label: 'animal', conf: 0.99, bbox: [0.1, 0.1, 0.1, 0.1] }, + { category: '1', label: 'animal', conf: 0.7, bbox: [0.3, 0.3, 0.1, 0.1] } + ] + } + + await insertPrediction(db, prediction, { + modelID: 'speciesnet', + detectionConfidenceThreshold: 0.5 + }) + + assert.equal(insertedRows.length, 2) + assert.equal(insertedRows[0].classificationProbability, 0.95) + assert.equal(insertedRows[1].classificationProbability, 0.95) + // detectionConfidence remains per-bbox + assert.equal(insertedRows[0].detectionConfidence, 0.99) + assert.equal(insertedRows[1].detectionConfidence, 0.7) + }) }) From 6fd5b5c5b6337fd5f85ebb49409c09a44dcbcb99 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:37:27 +0200 Subject: [PATCH 21/24] fix(megadetector): mean-conf weighted vote for MD video winner selection --- src/main/services/ml/classification.js | 45 +++++++---- src/main/services/prediction.js | 8 +- test/main/services/ml/classification.test.js | 79 ++++++++++++++++++++ 3 files changed, 115 insertions(+), 17 deletions(-) diff --git a/src/main/services/ml/classification.js b/src/main/services/ml/classification.js index 376f131b..b24189ed 100644 --- a/src/main/services/ml/classification.js +++ b/src/main/services/ml/classification.js @@ -4,12 +4,21 @@ */ /** - * Select winner species from aggregated frame data using majority voting. + * Select winner species from aggregated frame data. * - * Algorithm: - * 1. Primary criteria: Species with the highest frame count wins - * 2. Tiebreaker: If multiple species have the same frame count, the one with - * the highest average confidence wins + * Two modes, controlled by the `weightedVote` option: + * + * Default (frame-count primary) — for true classifiers (SpeciesNet, + * DeepFaune, Manas) where every frame is an independent classification of + * the same scene. The species detected in the most frames wins; ties are + * broken by average confidence. + * + * `weightedVote: true` (mean-confidence primary) — for detection-only + * models (MegaDetector). MD's labels are coarse (animal/person/vehicle) + * and a video can legitimately contain multiple labels in different + * frames, so a noisy stream of low-confidence "animal" detections must + * not drown out a few high-confidence "person" detections. The label + * with the highest *mean* confidence wins; frame count is the tiebreaker. * * @param {Map} speciesMap * Map of species names to their frame data: @@ -18,31 +27,37 @@ * - firstFrame: First frame number where species appeared * - lastFrame: Last frame number where species appeared * + * @param {Object} [options] + * @param {boolean} [options.weightedVote=false] Use mean-confidence as the + * primary criterion (with frame count as tiebreaker) instead of the default + * frame-count-primary logic. + * * @returns {{ winner: string|null, winnerData: object|null }} * - winner: The winning species name, or null if no species detected * - winnerData: The winning species data with avgConfidence added, or null */ -export function selectVideoClassificationWinner(speciesMap) { +export function selectVideoClassificationWinner(speciesMap, options = {}) { if (!speciesMap || speciesMap.size === 0) { return { winner: null, winnerData: null } } + const { weightedVote = false } = options + let winner = null let winnerData = null - let maxFrameCount = 0 - let maxAvgConfidence = 0 + let bestPrimary = -Infinity + let bestSecondary = -Infinity for (const [species, data] of speciesMap) { const frameCount = data.frames.length const avgConfidence = data.scores.reduce((sum, s) => sum + s, 0) / data.scores.length - // Primary: highest frame count; Tiebreaker: highest average confidence - if ( - frameCount > maxFrameCount || - (frameCount === maxFrameCount && avgConfidence > maxAvgConfidence) - ) { - maxFrameCount = frameCount - maxAvgConfidence = avgConfidence + const primary = weightedVote ? avgConfidence : frameCount + const secondary = weightedVote ? frameCount : avgConfidence + + if (primary > bestPrimary || (primary === bestPrimary && secondary > bestSecondary)) { + bestPrimary = primary + bestSecondary = secondary winner = species winnerData = { ...data, avgConfidence } } diff --git a/src/main/services/prediction.js b/src/main/services/prediction.js index 00680e2b..0291fe8b 100644 --- a/src/main/services/prediction.js +++ b/src/main/services/prediction.js @@ -459,8 +459,12 @@ export async function insertVideoPredictions(db, predictions, mediaRecord, model entry.lastFrame = Math.max(entry.lastFrame, pred.frame_number) } - // 4. Select winner using majority voting with average confidence tiebreaker - const { winner, winnerData } = selectVideoClassificationWinner(speciesMap) + // 4. Select winner. Detection-only models (MegaDetector) use mean-confidence + // voting so a noisy stream of low-conf "animal" frames can't drown out a + // few high-conf "person" frames. Classifiers keep the frame-count-primary + // majority vote, which suits per-frame classification of the same scene. + const weightedVote = modelType === 'megadetector' + const { winner, winnerData } = selectVideoClassificationWinner(speciesMap, { weightedVote }) // 5. Create exactly ONE observation (winner or blank) const fps = mediaRecord.exifData?.fps || 1 diff --git a/test/main/services/ml/classification.test.js b/test/main/services/ml/classification.test.js index 452e2754..0baf85e3 100644 --- a/test/main/services/ml/classification.test.js +++ b/test/main/services/ml/classification.test.js @@ -174,3 +174,82 @@ describe('selectVideoClassificationWinner', () => { assert.deepEqual(winnerData.scores, [0.8, 0.9, 0.85]) }) }) + +describe('selectVideoClassificationWinner with weightedVote=true (mean-confidence primary)', () => { + test('DSCF0028.MP4 regression — strong 2-frame person beats weak 7-frame animal', () => { + // Real per-frame data from a Biowatch import that exposed the bug. + // Pure frame-count voting picked "animal" (7 frames > 2 frames), but the + // animal detections are noisy (avg 0.36) while the person detections are + // unambiguous (avg 0.83). Weighted vote correctly picks "person". + const speciesMap = new Map([ + [ + 'homo sapiens', + { frames: [0, 1], scores: [0.945, 0.708], firstFrame: 0, lastFrame: 1 } + ], + [ + 'animal', + { + frames: [2, 3, 4, 5, 6, 8, 9], + scores: [0.492, 0.301, 0.350, 0.360, 0.323, 0.374, 0.319], + firstFrame: 2, + lastFrame: 9 + } + ] + ]) + const { winner, winnerData } = selectVideoClassificationWinner(speciesMap, { weightedVote: true }) + assert.equal(winner, 'homo sapiens') + assert.ok(Math.abs(winnerData.avgConfidence - 0.8265) < 0.001) + }) + + test('with weightedVote=false (default), frame count wins (existing behavior)', () => { + // Same input, default mode. Locks the existing classifier-friendly logic. + const speciesMap = new Map([ + ['homo sapiens', { frames: [0, 1], scores: [0.945, 0.708], firstFrame: 0, lastFrame: 1 }], + [ + 'animal', + { + frames: [2, 3, 4, 5, 6, 8, 9], + scores: [0.492, 0.301, 0.350, 0.360, 0.323, 0.374, 0.319], + firstFrame: 2, + lastFrame: 9 + } + ] + ]) + const { winner } = selectVideoClassificationWinner(speciesMap) + assert.equal(winner, 'animal') + }) + + test('frame count is the tiebreaker when mean confidences match', () => { + const speciesMap = new Map([ + ['A', { frames: [1], scores: [0.8], firstFrame: 1, lastFrame: 1 }], + ['B', { frames: [2, 3, 4], scores: [0.8, 0.8, 0.8], firstFrame: 2, lastFrame: 4 }] + ]) + const { winner } = selectVideoClassificationWinner(speciesMap, { weightedVote: true }) + assert.equal(winner, 'B') // same mean (0.8), more frames → wins + }) + + test('higher mean confidence wins regardless of frame count', () => { + const speciesMap = new Map([ + ['high-conf-few', { frames: [1, 2], scores: [0.95, 0.93], firstFrame: 1, lastFrame: 2 }], + [ + 'low-conf-many', + { + frames: [3, 4, 5, 6, 7, 8], + scores: [0.4, 0.4, 0.4, 0.4, 0.4, 0.4], + firstFrame: 3, + lastFrame: 8 + } + ] + ]) + const { winner } = selectVideoClassificationWinner(speciesMap, { weightedVote: true }) + assert.equal(winner, 'high-conf-few') + }) + + test('single-species video still wins regardless of mode', () => { + const speciesMap = new Map([ + ['Vulpes vulpes', { frames: [1, 2, 3], scores: [0.7, 0.7, 0.7], firstFrame: 1, lastFrame: 3 }] + ]) + const { winner } = selectVideoClassificationWinner(speciesMap, { weightedVote: true }) + assert.equal(winner, 'Vulpes vulpes') + }) +}) From bc24a2da4059ff15266281e4149f2b28f2e1e450 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 10:42:29 +0200 Subject: [PATCH 22/24] chore: prettier reformat MD test files --- test/main/services/ml/classification.test.js | 13 ++++++------- test/main/services/predictionCommonName.test.js | 3 ++- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/test/main/services/ml/classification.test.js b/test/main/services/ml/classification.test.js index 0baf85e3..49519b24 100644 --- a/test/main/services/ml/classification.test.js +++ b/test/main/services/ml/classification.test.js @@ -182,21 +182,20 @@ describe('selectVideoClassificationWinner with weightedVote=true (mean-confidenc // animal detections are noisy (avg 0.36) while the person detections are // unambiguous (avg 0.83). Weighted vote correctly picks "person". const speciesMap = new Map([ - [ - 'homo sapiens', - { frames: [0, 1], scores: [0.945, 0.708], firstFrame: 0, lastFrame: 1 } - ], + ['homo sapiens', { frames: [0, 1], scores: [0.945, 0.708], firstFrame: 0, lastFrame: 1 }], [ 'animal', { frames: [2, 3, 4, 5, 6, 8, 9], - scores: [0.492, 0.301, 0.350, 0.360, 0.323, 0.374, 0.319], + scores: [0.492, 0.301, 0.35, 0.36, 0.323, 0.374, 0.319], firstFrame: 2, lastFrame: 9 } ] ]) - const { winner, winnerData } = selectVideoClassificationWinner(speciesMap, { weightedVote: true }) + const { winner, winnerData } = selectVideoClassificationWinner(speciesMap, { + weightedVote: true + }) assert.equal(winner, 'homo sapiens') assert.ok(Math.abs(winnerData.avgConfidence - 0.8265) < 0.001) }) @@ -209,7 +208,7 @@ describe('selectVideoClassificationWinner with weightedVote=true (mean-confidenc 'animal', { frames: [2, 3, 4, 5, 6, 8, 9], - scores: [0.492, 0.301, 0.350, 0.360, 0.323, 0.374, 0.319], + scores: [0.492, 0.301, 0.35, 0.36, 0.323, 0.374, 0.319], firstFrame: 2, lastFrame: 9 } diff --git a/test/main/services/predictionCommonName.test.js b/test/main/services/predictionCommonName.test.js index efd155c7..9ee66297 100644 --- a/test/main/services/predictionCommonName.test.js +++ b/test/main/services/predictionCommonName.test.js @@ -208,7 +208,8 @@ describe('insertPrediction populates commonName via dictionary', () => { const { db, insertedRows } = makeFakeDb({ mediaRecord: baseMedia() }) const prediction = { filepath: '/fake/img.jpg', - prediction: '00000000-0000-0000-0000-000000000001;mammalia;rodentia;sciuridae;sciurus;vulgaris;eurasian red squirrel', + prediction: + '00000000-0000-0000-0000-000000000001;mammalia;rodentia;sciuridae;sciurus;vulgaris;eurasian red squirrel', prediction_score: 0.95, detections: [ { category: '1', label: 'animal', conf: 0.99, bbox: [0.1, 0.1, 0.1, 0.1] }, From 08f970afbc7a1966f5806c6c551e166e98f5dffa Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 11:01:55 +0200 Subject: [PATCH 23/24] fix(export): MD pseudo-species map to correct Camtrap DP observationType Previously the early-return 'if (scientificName) return animal' in mapObservationType shadowed MegaDetector's 'homo sapiens' and 'vehicle' scientificName labels, exporting both as 'animal' instead of the correct Camtrap DP 'human' and 'vehicle' values. Extract mapObservationType into a pure module so the MD-specific routing can be unit-tested without transitively loading download.ts. --- src/main/services/export/exporter.js | 18 ++------ src/main/services/export/observationType.js | 35 ++++++++++++++++ .../services/export/observationType.test.js | 42 +++++++++++++++++++ 3 files changed, 80 insertions(+), 15 deletions(-) create mode 100644 src/main/services/export/observationType.js create mode 100644 test/main/services/export/observationType.test.js diff --git a/src/main/services/export/exporter.js b/src/main/services/export/exporter.js index bcb75892..84a79133 100644 --- a/src/main/services/export/exporter.js +++ b/src/main/services/export/exporter.js @@ -25,6 +25,7 @@ import { sanitizeDatapackage, CAMTRAP_DP_PROFILE_URL } from './sanitizers.js' +import { mapObservationType } from './observationType.js' function getStudyDatabasePath(userDataPath, studyId) { return join(getStudyPath(userDataPath, studyId), 'study.db') @@ -553,21 +554,8 @@ function inferMimeType(filePath) { return MIME_TYPES[ext] || 'application/octet-stream' } -/** - * Map internal observationType to Camtrap DP vocabulary - * Valid values: animal, human, vehicle, blank, unknown, unclassified - */ -function mapObservationType(dbType, scientificName) { - // If scientificName is present, it's an animal observation - if (scientificName) return 'animal' - if (!dbType || dbType === 'blank') return 'blank' - if (dbType === 'machine') return 'animal' - if (dbType === 'animal') return 'animal' - if (dbType === 'human') return 'human' - if (dbType === 'vehicle') return 'vehicle' - if (dbType === 'unclassified') return 'unclassified' - return 'unknown' -} +// mapObservationType is imported from ./observationType.js — see that module +// for the Camtrap DP vocabulary mapping and MegaDetector-specific routing. /** * Group observations into sequences based on deployment and timestamp gap. diff --git a/src/main/services/export/observationType.js b/src/main/services/export/observationType.js new file mode 100644 index 00000000..f96ba2bf --- /dev/null +++ b/src/main/services/export/observationType.js @@ -0,0 +1,35 @@ +/** + * Map internal observationType + scientificName to Camtrap DP vocabulary. + * + * Valid output values: `animal`, `human`, `vehicle`, `blank`, `unknown`, `unclassified`. + * + * MegaDetector's three pseudo-species labels (`animal`, `homo sapiens`, + * `vehicle`) map directly to Camtrap DP's vocabulary. We special-case + * `homo sapiens` → `human` and `vehicle` → `vehicle` BEFORE the generic + * "any scientificName → animal" fallback, otherwise the fallback would + * shadow them. The string matches also catch the same values coming from + * any other source (e.g. a SpeciesNet "homo sapiens" classification), + * which is semantically correct. + * + * Extracted from `exporter.js` into a pure module so it can be unit-tested + * without transitively loading the rest of the export pipeline (which pulls + * in `download.ts` and friends). + * + * @param {string|null} dbType - The `observationType` column value (`'machine'`, + * `'animal'`, `'human'`, `'vehicle'`, `'blank'`, `'unclassified'`, or null). + * @param {string|null} scientificName - The observation's scientific name. + * @returns {'animal'|'human'|'vehicle'|'blank'|'unknown'|'unclassified'} + */ +export function mapObservationType(dbType, scientificName) { + if (scientificName === 'homo sapiens') return 'human' + if (scientificName === 'vehicle') return 'vehicle' + // Any other scientificName (binomials, MD's literal "animal", ...) → animal. + if (scientificName) return 'animal' + if (!dbType || dbType === 'blank') return 'blank' + if (dbType === 'machine') return 'animal' + if (dbType === 'animal') return 'animal' + if (dbType === 'human') return 'human' + if (dbType === 'vehicle') return 'vehicle' + if (dbType === 'unclassified') return 'unclassified' + return 'unknown' +} diff --git a/test/main/services/export/observationType.test.js b/test/main/services/export/observationType.test.js new file mode 100644 index 00000000..f3bc75d9 --- /dev/null +++ b/test/main/services/export/observationType.test.js @@ -0,0 +1,42 @@ +import { test, describe } from 'node:test' +import assert from 'node:assert/strict' + +import { mapObservationType } from '../../../../src/main/services/export/observationType.js' + +describe('mapObservationType — Camtrap DP vocabulary', () => { + test('MegaDetector "homo sapiens" exports as "human"', () => { + // Regression test: previously the early-return `if (scientificName) return 'animal'` + // shadowed MD's homo sapiens label, exporting humans as animals. + assert.equal(mapObservationType('machine', 'homo sapiens'), 'human') + assert.equal(mapObservationType(null, 'homo sapiens'), 'human') + }) + + test('MegaDetector "vehicle" exports as "vehicle"', () => { + // Same regression as homo sapiens — vehicles were silently mapped to animal. + assert.equal(mapObservationType('machine', 'vehicle'), 'vehicle') + assert.equal(mapObservationType(null, 'vehicle'), 'vehicle') + }) + + test('MegaDetector "animal" exports as "animal"', () => { + // MD's literal animal label still maps to animal (was coincidentally correct). + assert.equal(mapObservationType('machine', 'animal'), 'animal') + assert.equal(mapObservationType(null, 'animal'), 'animal') + }) + + test('real binomial scientific names still export as "animal"', () => { + // Locks pre-existing behavior for true classifier output. + assert.equal(mapObservationType('machine', 'Vulpes vulpes'), 'animal') + assert.equal(mapObservationType('machine', 'Panthera leo'), 'animal') + assert.equal(mapObservationType('machine', 'sciurus vulgaris'), 'animal') + }) + + test('null scientificName falls through to dbType branches', () => { + assert.equal(mapObservationType(null, null), 'blank') + assert.equal(mapObservationType('blank', null), 'blank') + assert.equal(mapObservationType('machine', null), 'animal') + assert.equal(mapObservationType('human', null), 'human') + assert.equal(mapObservationType('vehicle', null), 'vehicle') + assert.equal(mapObservationType('unclassified', null), 'unclassified') + assert.equal(mapObservationType('something-unexpected', null), 'unknown') + }) +}) From 5ce050ef1625cebf7a11e503627051a786035922 Mon Sep 17 00:00:00 2001 From: Chouffe Date: Mon, 18 May 2026 11:17:44 +0200 Subject: [PATCH 24/24] chore: drop one-shot megadetector implementation plan --- ...26-05-16-megadetector-model-choice-plan.md | 1441 ----------------- 1 file changed, 1441 deletions(-) delete mode 100644 docs/plans/2026-05-16-megadetector-model-choice-plan.md diff --git a/docs/plans/2026-05-16-megadetector-model-choice-plan.md b/docs/plans/2026-05-16-megadetector-model-choice-plan.md deleted file mode 100644 index 65ac2227..00000000 --- a/docs/plans/2026-05-16-megadetector-model-choice-plan.md +++ /dev/null @@ -1,1441 +0,0 @@ -# MegaDetector Model Choice Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Ship MegaDetector v6 (`MDV6-yolov10-e`) as a standalone model choice in Biowatch — a fast blank-filter that detects `animal` / `person` / `vehicle` worldwide, no species classification. - -**Architecture:** New Python LitServe server (`run_megadetector_server.py`) modeled on `run_deepfaune_server.py` minus the classifier; new tarball hosted at `huggingface.co/earthtoolsmaker/megadetector/main/6.0.tar.gz`; new `detectionOnly: true` flag in `mlmodels.js` that suppresses the species panel in two UI components. MD's three labels (`animal` / `person` / `vehicle` + implicit `blank`) flow through Biowatch's existing species-centric data model as pseudo-species, so no changes to observations / annotations / exports are needed. - -**Tech Stack:** Electron + React (renderer), Node.js (main), Python 3.11 + LitServe + `ultralytics.YOLO` (model server), Drizzle ORM (SQLite), `node:test` (JS test runner), pytest (Python e2e tests). - -**Reference spec:** `docs/specs/2026-05-16-megadetector-model-choice-design.md`. - ---- - -## File Structure - -**Create:** -- `scripts/build-megadetector-tarball.py` — packaging script (downloads weights, builds tarball) -- `python-environments/common/run_megadetector_server.py` — LitServe HTTP server -- `python-environments/common/tests/test_megadetector_server.py` — e2e tests -- `src/shared/commonNames/sources/megadetector.json` — 3-entry common-name source - -**Modify:** -- `src/shared/mlmodels.js` — add `megadetector` entry with `detectionOnly: true` -- `src/main/services/ml/server.ts` — add `startMegaDetectorHTTPServer()` + new `case 'megadetector'` -- `src/main/utils/bbox.js` — add `megadetector` to xywhn case + `detectModelType()` branch -- `src/renderer/src/models/ModelCard.jsx` — gate species panel on `!detectionOnly` -- `src/renderer/src/models/ModelSelect.jsx` — show "Detection only" when `detectionOnly` -- `scripts/build-common-names-dict.js` — add `megadetector.json` to merge order -- `scripts/lib/aliases.js` — add megadetector source to imports + sources list -- `src/shared/commonNames/dictionary.json` — regenerated by `npm run dict:build` (do not edit by hand) -- `python-environments/common/scripts/download_model.py` — add megadetector entry -- `python-environments/common/Makefile` — add megadetector env var + download command -- `docs/http-servers.md` — add MegaDetector to supported-models table + project-structure listing -- `test/shared/mlmodels.test.js` — extend coverage for the new entry -- `test/main/utils/bbox.test.js` (create if absent) — covers the new megadetector branch -- `test/shared/commonNames/resolver.test.js` — covers `animal`/`person`/`vehicle` resolution - -**User-provided (NOT in this plan):** -- Run the build script and upload `6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector/main`. -- Provide back the final tarball size in MB for `size_in_MB` in `mlmodels.js`. -- Drop a `megadetector.png` logo into `src/renderer/src/assets/logos/`. - ---- - -## Task 1: Build script (`scripts/build-megadetector-tarball.py`) - -**Files:** -- Create: `scripts/build-megadetector-tarball.py` - -- [ ] **Step 1: Verify the upstream weights URL** - -```bash -curl -sI -L "https://zenodo.org/records/15398270/files/MDV6-yolov10-e-1280.pt?download=1" | head -3 -``` -Expected: HTTP 200 (Zenodo serves the file). The canonical source for MDv6 weights is Zenodo record `15398270`, linked from `https://microsoft.github.io/Biodiversity/model_zoo/megadetector/`. If Zenodo is down or the record id has changed, fall back to the model zoo page to find the current direct-download URL. - -- [ ] **Step 2: Write the build script** - -Mirror `scripts/build-speciesnet-tarball.py` exactly — stdlib only (`urllib`, `tarfile`, `hashlib`, `json`), same `download()` / `sha256_of()` helpers, same `--version` / `--output-dir` / `--work-dir` CLI. - -```python -#!/usr/bin/env python3 -"""Build a MegaDetector model tarball ready to upload to huggingface.co/earthtoolsmaker/megadetector. - -Downloads the official MDv6 release weights and the upstream LICENSE so AGPL attribution -travels with the binary, then tar-gzips them into a directory named for the version. - -Usage: - python scripts/build-megadetector-tarball.py --version 6.0 - -Produces dist/.tar.gz alongside a summary of size + SHA256 + file list. -""" - -import argparse -import hashlib -import shutil -import sys -import tarfile -import tempfile -import urllib.request -from pathlib import Path - -WEIGHTS_URL = "https://zenodo.org/records/15398270/files/MDV6-yolov10-e-1280.pt?download=1" -WEIGHTS_FILENAME = "MDV6-yolov10-e-1280.pt" -WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN" # paste actual hash printed at end of first run -LICENSE_URL = "https://raw.githubusercontent.com/microsoft/MegaDetector/main/LICENSE" -LICENSE_FILENAME = "LICENSE" - - -def download(url: str, dest: Path) -> None: - if dest.exists(): - print(f"[skip] {dest.name} already present ({dest.stat().st_size / 1e6:.0f} MB)") - return - print(f"[download] {url}") - is_tty = sys.stdout.isatty() - last_milestone = -1 - - def report(block_num, block_size, total_size): - nonlocal last_milestone - if total_size <= 0: - return - downloaded = block_num * block_size - pct = min(100, downloaded * 100 // total_size) - if is_tty: - sys.stdout.write(f"\r {pct:3d}% {downloaded / 1e6:7.1f} / {total_size / 1e6:.0f} MB") - sys.stdout.flush() - else: - milestone = pct // 10 - if milestone > last_milestone: - last_milestone = milestone - print(f" {pct:3d}% {downloaded / 1e6:7.1f} / {total_size / 1e6:.0f} MB") - - urllib.request.urlretrieve(url, dest, reporthook=report) - if is_tty: - sys.stdout.write("\n") - - -def sha256_of(path: Path) -> str: - digest = hashlib.sha256() - with path.open("rb") as fp: - for chunk in iter(lambda: fp.read(1024 * 1024), b""): - digest.update(chunk) - return digest.hexdigest() - - -def build(version: str, output_dir: Path, work_dir: Path) -> Path: - work_dir.mkdir(parents=True, exist_ok=True) - output_dir.mkdir(parents=True, exist_ok=True) - - model_dir = work_dir / version - if model_dir.exists(): - shutil.rmtree(model_dir) - model_dir.mkdir() - - weights_path = model_dir / WEIGHTS_FILENAME - download(WEIGHTS_URL, weights_path) - - actual_hash = sha256_of(weights_path) - if WEIGHTS_SHA256 != "TBD_LOCK_ON_FIRST_RUN" and actual_hash != WEIGHTS_SHA256: - raise RuntimeError(f"Weights SHA256 mismatch:\n expected {WEIGHTS_SHA256}\n got {actual_hash}") - print(f"[verify] {WEIGHTS_FILENAME} SHA256 = {actual_hash}") - - download(LICENSE_URL, model_dir / LICENSE_FILENAME) - - tarball = output_dir / f"{version}.tar.gz" - if tarball.exists(): - tarball.unlink() - print(f"[tar] {tarball}") - with tarfile.open(tarball, "w:gz") as tar: - tar.add(model_dir, arcname=version) - return tarball - - -def main() -> None: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--version", required=True, help="MegaDetector version, e.g. 6.0") - parser.add_argument("--output-dir", type=Path, default=Path("dist")) - parser.add_argument( - "--work-dir", - type=Path, - default=Path(tempfile.gettempdir()) / "megadetector-build", - help="Working directory for downloads (retained between runs)", - ) - args = parser.parse_args() - - tarball = build(args.version, args.output_dir, args.work_dir) - - size_mb = tarball.stat().st_size / 1e6 - digest = sha256_of(tarball) - with tarfile.open(tarball) as tar: - members = sorted(m.name for m in tar.getmembers() if not m.isdir()) - - print() - print("=" * 60) - print(f" Tarball: {tarball}") - print(f" Size: {size_mb:.1f} MB") - print(f" SHA256: {digest}") - print(" Contents:") - for name in members: - print(f" {name}") - print("=" * 60) - - -if __name__ == "__main__": - main() -``` - -- [ ] **Step 3: Make it executable and run it once locally** - -```bash -chmod +x scripts/build-megadetector-tarball.py -python3 scripts/build-megadetector-tarball.py --version 6.0 -``` -Expected output (last 8 lines): -``` -============================================================ - Tarball: dist/6.0.tar.gz - Size: ~30 MB - SHA256: <64 hex chars> - Contents: - 6.0/LICENSE - 6.0/MDV6-yolov10-e-1280.pt -============================================================ -``` - -- [ ] **Step 4: Lock the SHA256** - -Replace the `WEIGHTS_SHA256 = "TBD_LOCK_ON_FIRST_RUN"` line with the actual hash printed by Step 3 (the per-file hash of `MDV6-yolov10-e-1280.pt`, not the tarball hash — re-run the script if you only kept the tarball hash; the per-file hash is printed on the line starting with `[verify]`). - -- [ ] **Step 5: Run it again to confirm SHA verification passes** - -```bash -rm -rf /tmp/megadetector-build dist/6.0.tar.gz -python3 scripts/build-megadetector-tarball.py --version 6.0 -``` -Expected: same output, and the `[verify]` line shows the SHA matches the constant (no `Weights SHA256 mismatch` error). - -- [ ] **Step 6: Commit** - -```bash -git add scripts/build-megadetector-tarball.py -git commit -m "tools(megadetector): script to build MD v6 tarball for HF upload" -``` - -**>>> EXTERNAL DEPENDENCY: After this task, the maintainer (Arthur) uploads `dist/6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector/main` and shares back the exact tarball size in MB. Subsequent tasks can proceed in parallel — only Task 5 (Python server e2e tests) and Task 9 (Python tooling) depend on the upload landing first.** - ---- - -## Task 2: Common-names source + dictionary rebuild - -**Files:** -- Create: `src/shared/commonNames/sources/megadetector.json` -- Modify: `scripts/build-common-names-dict.js` -- Modify: `scripts/lib/aliases.js` -- Modify: `src/shared/commonNames/dictionary.json` (regenerated, do not edit by hand) -- Modify: `test/shared/commonNames/resolver.test.js` - -- [ ] **Step 1: Create the megadetector source file** - -`src/shared/commonNames/sources/megadetector.json`: - -```json -{ - "modelId": "megadetector", - "modelVersion": "6.0", - "source": "MegaDetector v6 categories (animal / person / vehicle)", - "entries": [ - { - "scientificName": null, - "label": "animal", - "commonName": "Animal" - }, - { - "scientificName": "homo sapiens", - "label": "person", - "commonName": "Human" - }, - { - "scientificName": null, - "label": "vehicle", - "commonName": "Vehicle" - } - ] -} -``` - -Schema matches `manas.json`: `scientificName: null` for non-binomial labels (`animal`, `vehicle`); a real binomial (`"homo sapiens"`) for `person` so the dict-build pipeline produces a `homo sapiens → human` dictionary entry plus a `person → homo sapiens` alias. - -- [ ] **Step 2: Add the source to the dictionary-build pipeline** - -In `scripts/build-common-names-dict.js`, modify the `order` array at line ~97: - -```js -const order = ['speciesnet.json', 'deepfaune.json', 'manas.json', 'megadetector.json'] -``` - -- [ ] **Step 3: Add the source to the alias-map builder** - -In `scripts/lib/aliases.js`, modify lines 1–17: - -```js -import speciesnetSource from '../../src/shared/commonNames/sources/speciesnet.json' with { type: 'json' } -import deepfauneSource from '../../src/shared/commonNames/sources/deepfaune.json' with { type: 'json' } -import manasSource from '../../src/shared/commonNames/sources/manas.json' with { type: 'json' } -import megadetectorSource from '../../src/shared/commonNames/sources/megadetector.json' with { type: 'json' } -import extras from '../../src/shared/commonNames/extras.json' with { type: 'json' } -import { normalizeScientificName } from '../../src/shared/commonNames/normalize.js' - -export function buildAliasMap() { - const aliases = new Map() - const sources = [speciesnetSource, deepfauneSource, manasSource, megadetectorSource, extras] - // ...rest unchanged -} -``` - -(`megadetectorSource` entries have `scientificName: null`, so the alias loop skips them per the existing `if (!entry.scientificName || !entry.label) continue` guard — adding it to the list is a no-op for aliases but keeps the imports symmetric with the dict builder.) - -- [ ] **Step 4: Add the failing resolver tests** - -Append to `test/shared/commonNames/resolver.test.js`: - -```js -test('resolves MegaDetector pseudo-species labels (animal/vehicle)', () => { - assert.equal(resolveCommonName('animal'), 'animal') - assert.equal(resolveCommonName('vehicle'), 'vehicle') -}) - -test("resolves MegaDetector's 'person' label and its 'homo sapiens' binomial to 'human'", () => { - // MD's raw label is 'person', but the Python server emits 'homo sapiens' - // as the prediction. Both keys resolve to the same common name so either - // route (alias-translated or directly emitted) renders consistently. - assert.equal(resolveCommonName('person'), 'human') - assert.equal(resolveCommonName('homo sapiens'), 'human') -}) -``` - -Note: dictionary values are lowercased by `mergeEntries` (build-common-names-dict.js:72), so `"Animal"` becomes `"animal"` in the dictionary — assertions match the stored value. - -- [ ] **Step 5: Run the test to confirm it fails (entries not yet in dictionary)** - -```bash -node --test test/shared/commonNames/resolver.test.js -``` -Expected: the new assertions fail (resolver returns `null` for `animal`/`person`/`vehicle` because the dictionary hasn't been rebuilt yet). - -- [ ] **Step 6: Rebuild the dictionary** - -```bash -npm run dict:build -``` -Expected stdout: -``` -Wrote entries to .../dictionary.json -Wrote label aliases to .../labelAliases.json -``` -(N+3 reflects the three new keys.) - -- [ ] **Step 7: Verify the dictionary contains the new keys** - -```bash -grep -E '"(animal|person|vehicle)":' src/shared/commonNames/dictionary.json -``` -Expected: three lines, each with the lowercased common name. (Some of `animal`/`person`/`vehicle` may already exist as keys from other sources — that's fine; the merge order puts megadetector last among the model sources but `extras.json` still wins after it, so existing curated values are preserved. Confirm none of the three points to something unexpected.) - -- [ ] **Step 8: Run the resolver test again to confirm it passes** - -```bash -node --test test/shared/commonNames/resolver.test.js -``` -Expected: all tests pass, including the three new assertions. - -- [ ] **Step 9: Commit** - -```bash -git add src/shared/commonNames/sources/megadetector.json \ - scripts/build-common-names-dict.js \ - scripts/lib/aliases.js \ - src/shared/commonNames/dictionary.json \ - test/shared/commonNames/resolver.test.js -git commit -m "feat(models): common-name source for MegaDetector labels" -``` - ---- - -## Task 3: Model zoo entry (`src/shared/mlmodels.js`) - -**Files:** -- Modify: `src/shared/mlmodels.js` -- Modify: `test/shared/mlmodels.test.js` - -- [ ] **Step 1: Write the failing test** - -Append to the `describe('findModel', ...)` block in `test/shared/mlmodels.test.js`: - -```js -test('should find megadetector entry with detectionOnly flag', () => { - const md = findModel({ id: 'megadetector', version: '6.0' }) - assert(md, 'megadetector entry should exist') - assert.equal(md.detectionOnly, true, 'megadetector should be marked detectionOnly') - assert.equal(md.region, 'worldwide', 'megadetector should be worldwide') - assert.equal(md.species_count, 3, 'megadetector should report 3 categories') - assert(md.downloadURL.includes('earthtoolsmaker/megadetector'), 'should download from ETM HF') -}) -``` - -- [ ] **Step 2: Run the test to verify it fails** - -```bash -node --test test/shared/mlmodels.test.js -``` -Expected: FAIL — `megadetector entry should exist` (`findModel` returns null). - -- [ ] **Step 3: Add the megadetector entry** - -In `src/shared/mlmodels.js`, append a fourth object to the `modelZoo` array (after the Manas entry, before the closing `]`): - -```js -, -{ - reference: { id: 'megadetector', version: '6.0' }, - pythonEnvironment: { id: 'common', version: '0.1.4' }, - name: 'MegaDetector', - size_in_MB: 30, // PLACEHOLDER — replace with exact size from build-megadetector-tarball.py output - files: 2, - downloadURL: - 'https://huggingface.co/earthtoolsmaker/megadetector/resolve/main/6.0.tar.gz?download=true', - description: - "MegaDetector is Microsoft AI for Earth's open-source detector for camera trap images. It locates animals, people, and vehicles in any image worldwide, without identifying species. Useful as a fast blank-filter before manual species annotation.", - website: 'https://github.com/agentmorris/MegaDetector', - logo: 'megadetector', - detectionConfidenceThreshold: 0.2, - region: 'worldwide', - species_count: 3, - detectionOnly: true -} -``` - -(`size_in_MB` is a placeholder — once the maintainer uploads the tarball and reports the size, update this line in a follow-up commit. The structural tests pass on any positive number.) - -- [ ] **Step 4: Run the test to verify it passes** - -```bash -node --test test/shared/mlmodels.test.js -``` -Expected: all tests pass, including all existing structural validations applied to the new entry. - -- [ ] **Step 5: Commit** - -```bash -git add src/shared/mlmodels.js test/shared/mlmodels.test.js -git commit -m "feat(models): MegaDetector entry in model zoo" -``` - ---- - -## Task 4: `bbox.js` — megadetector routing - -**Files:** -- Modify: `src/main/utils/bbox.js` -- Create or modify: `test/main/utils/bbox.test.js` - -- [ ] **Step 1: Check if a bbox test file exists** - -```bash -ls test/main/utils/bbox.test.js 2>/dev/null || echo "no test file" -``` -If "no test file", create `test/main/utils/bbox.test.js` with this scaffolding: - -```js -import { test, describe } from 'node:test' -import assert from 'node:assert/strict' -import { transformBboxToCamtrapDP, detectModelType } from '../../../src/main/utils/bbox.js' -``` - -- [ ] **Step 2: Write the failing tests** - -Append to `test/main/utils/bbox.test.js`: - -```js -describe('MegaDetector bbox handling', () => { - test('transformBboxToCamtrapDP routes megadetector through xywhn → top-left', () => { - const detection = { xywhn: [0.5, 0.5, 0.4, 0.6] } - const result = transformBboxToCamtrapDP(detection, 'megadetector') - assert.deepEqual(result, { - bboxX: 0.3, // 0.5 - 0.4/2 - bboxY: 0.2, // 0.5 - 0.6/2 - bboxWidth: 0.4, - bboxHeight: 0.6 - }) - }) - - test('detectModelType identifies megadetector by version 6.0 + xywhn', () => { - const prediction = { - model_version: '6.0', - detections: [{ xywhn: [0.5, 0.5, 0.4, 0.6], conf: 0.9, label: 'animal' }] - } - assert.equal(detectModelType(prediction), 'megadetector') - }) -}) -``` - -- [ ] **Step 3: Run the tests to verify they fail** - -```bash -node --test test/main/utils/bbox.test.js -``` -Expected: FAIL — `transformBboxToCamtrapDP` logs `Unknown model type for bbox transformation: megadetector` and returns `null`; `detectModelType` likely returns `'unknown'` or routes through a different branch. - -- [ ] **Step 4: Add megadetector to both functions in `src/main/utils/bbox.js`** - -In the `transformBboxToCamtrapDP` switch (around line 45), change the existing `case 'manas': case 'deepfaune':` block to include megadetector — these three all use `xywhn`: - -```js -case 'manas': -case 'megadetector': -case 'deepfaune': { - // existing xywhn → top-left conversion, unchanged - ... -} -``` - -In `detectModelType` (around line 85), add an explicit megadetector branch *before* the generic xywhn fallback at the bottom: - -```js -// MegaDetector v6.0 uses xywhn (ultralytics YOLO output) and ships model_version '6.0' -if (prediction.model_version === '6.0' && prediction.detections?.[0]?.xywhn) { - return 'megadetector' -} -``` - -Place this branch directly after the `if (version === '1.0' && prediction.detections?.[0]?.xywhn) return 'manas'` line. Order matters because the generic xywhn fallback below would otherwise misroute it. - -- [ ] **Step 5: Run the tests to verify they pass** - -```bash -node --test test/main/utils/bbox.test.js -``` -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add src/main/utils/bbox.js test/main/utils/bbox.test.js -git commit -m "feat(models): bbox + model-type detection for MegaDetector" -``` - ---- - -## Task 5: Python HTTP server (`run_megadetector_server.py` + tests) - -**Files:** -- Create: `python-environments/common/run_megadetector_server.py` -- Create: `python-environments/common/tests/test_megadetector_server.py` - -**Depends on:** The maintainer has uploaded the tarball to HF (Task 1's external dependency). If not yet uploaded, you can still write the code; tests will skip via `pytest.skipif` until weights are downloadable. - -- [ ] **Step 1: Write the e2e test file** - -Create `python-environments/common/tests/test_megadetector_server.py`: - -```python -"""End-to-end tests for MegaDetector ML server.""" - -import os - -import httpx -import pytest - -from tests.conftest import ServerProcess, find_free_port, parse_streaming_response - -MEGADETECTOR_DETECTOR_WEIGHTS = os.environ.get("MEGADETECTOR_DETECTOR_WEIGHTS", "") - -pytestmark = pytest.mark.skipif( - not MEGADETECTOR_DETECTOR_WEIGHTS, - reason="MegaDetector weights not available. Set MEGADETECTOR_DETECTOR_WEIGHTS.", -) - - -@pytest.fixture(scope="module") -def megadetector_server(): - """Start MegaDetector server for the test module.""" - port = find_free_port() - command = [ - "uv", - "run", - "python", - "run_megadetector_server.py", - "--port", - str(port), - "--timeout", - "60", - "--filepath-detector-weights", - MEGADETECTOR_DETECTOR_WEIGHTS, - "--detection-confidence-threshold", - "0.2", - ] - with ServerProcess(command, port, startup_timeout=300) as server: - yield server - - -class TestMegaDetectorServer: - """Tests for MegaDetector server endpoints.""" - - def test_health_endpoint(self, megadetector_server): - resp = httpx.get(f"{megadetector_server.base_url}/health", timeout=10.0) - assert resp.status_code == 200 - assert resp.text == "ok" - - def test_info_endpoint(self, megadetector_server): - resp = httpx.get(f"{megadetector_server.base_url}/info", timeout=10.0) - assert resp.status_code == 200 - data = resp.json() - assert "model" in data - assert data["model"]["type"] == "megadetector" - assert "server" in data - - def test_predict_animal_image(self, megadetector_server, test_images): - """A clearly-animal image should produce prediction='animal'.""" - payload = {"instances": [{"filepath": str(test_images["chamois"])}]} - with httpx.stream( - "POST", - f"{megadetector_server.base_url}/predict", - json=payload, - timeout=60.0, - ) as resp: - assert resp.status_code == 200 - results = parse_streaming_response(resp) - - assert len(results) == 1 - pred = results[0]["output"]["predictions"][0] - assert pred["filepath"] == str(test_images["chamois"]) - assert pred["prediction"] == "animal" - assert pred["prediction_score"] >= 0.2 - assert pred["classifications"] == {} - assert len(pred["detections"]) >= 1 - assert pred["detections"][0]["label"] in {"animal", "person", "vehicle"} - assert "xywhn" in pred["detections"][0] - - def test_predict_human_image(self, megadetector_server, test_images): - """A human image should produce prediction='homo sapiens' (translated from MD's 'person' label).""" - payload = {"instances": [{"filepath": str(test_images["human"])}]} - with httpx.stream( - "POST", - f"{megadetector_server.base_url}/predict", - json=payload, - timeout=60.0, - ) as resp: - results = parse_streaming_response(resp) - pred = results[0]["output"]["predictions"][0] - assert pred["prediction"] == "homo sapiens" - # Raw bbox labels stay 'person' so MD's native output is preserved per-bbox. - person_dets = [d for d in pred["detections"] if d["label"] == "person"] - assert len(person_dets) >= 1, "raw 'person' label must be preserved per-detection" - - def test_predict_empty_image(self, megadetector_server, test_images): - """An empty (no subject) image should produce prediction='blank'.""" - payload = {"instances": [{"filepath": str(test_images["empty"])}]} - with httpx.stream( - "POST", - f"{megadetector_server.base_url}/predict", - json=payload, - timeout=60.0, - ) as resp: - results = parse_streaming_response(resp) - pred = results[0]["output"]["predictions"][0] - assert pred["prediction"] == "blank" - assert pred["prediction_score"] is None - - def test_predict_streaming(self, megadetector_server, test_images): - """Streaming returns one chunk per image, all classifications empty.""" - payload = { - "instances": [ - {"filepath": str(test_images["badger"])}, - {"filepath": str(test_images["fox"])}, - {"filepath": str(test_images["empty"])}, - ] - } - with httpx.stream( - "POST", - f"{megadetector_server.base_url}/predict", - json=payload, - timeout=120.0, - ) as resp: - results = parse_streaming_response(resp) - assert len(results) == 3 - for r in results: - pred = r["output"]["predictions"][0] - assert pred["classifications"] == {} - assert pred["model_version"] == "6.0" - - def test_predict_invalid_filepath(self, megadetector_server): - payload = {"instances": [{"filepath": "/nonexistent/image.jpg"}]} - resp = httpx.post( - f"{megadetector_server.base_url}/predict", - json=payload, - timeout=30.0, - ) - assert resp.status_code in (200, 400) - - def test_docs_endpoint(self, megadetector_server): - resp = httpx.get(f"{megadetector_server.base_url}/docs", timeout=10.0) - assert resp.status_code == 200 - assert "swagger" in resp.text.lower() or "openapi" in resp.text.lower() -``` - -- [ ] **Step 2: Write the server file** - -Create `python-environments/common/run_megadetector_server.py`: - -```python -""" -CLI script to run MegaDetector v6 as a LitServer. - -MegaDetector v6 is a YOLO-based detector (animals / people / vehicles only — no -species classification). Used in Biowatch as a fast blank-filter before manual -species annotation. - -Start the server: - -``` -run_megadetector_server.py \ - --port 8000 \ - --filepath-detector-weights ./path/to/MDV6-yolov10-e-1280.pt \ - --detection-confidence-threshold 0.2 -``` - -A Swagger API documentation is served at localhost:${port}/docs - -health: - -``` -$ curl http://localhost:${port}/health -"ok" -``` - -predict (streaming): - -``` -$ curl -X POST http://localhost:${port}/predict \ - -H "Content-Type: application/json" \ - -d '{"instances": [{"filepath": "/path/to/your/image"}]}' -``` - -Output per image: - -```json -{ - "output": { - "predictions": [{ - "filepath": "/path/to/image", - "classifications": {}, - "detections": [ - {"label": "animal", "conf": 0.94, "xywhn": [...], "xyxy": [...]} - ], - "prediction": "animal", - "prediction_score": 0.94, - "model_version": "6.0" - }] - } -} -``` -""" - -import logging -from pathlib import Path - -import litserve as ls -from absl import app, flags -from fastapi import HTTPException -from ultralytics import YOLO - -from detection_utils import propagate_extra_fields, to_detection_record -from utils import VideoCapableLitAPI, is_video_file, safe_imread - -logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s") -logger = logging.getLogger(__name__) - -MODEL_VERSION = "6.0" - -# Map MegaDetector detection labels to Biowatch-canonical scientific names -# for the top-level `prediction` field. Per-bbox `detections[].label` stays raw. -# `person` is genuinely a species — emit the binomial so it integrates with -# Biowatch's species tooltips and IUCN lookups. `animal` and `vehicle` are not -# species and pass through unchanged. -LABEL_TO_PREDICTION = { - "person": "homo sapiens", -} - -_PORT = flags.DEFINE_integer("port", 8000, "Port to run the server on.") -_API_PATH = flags.DEFINE_string("api_path", "/predict", "URL path for the server endpoint.") -_WORKERS_PER_DEVICE = flags.DEFINE_integer("workers_per_device", 1, "Number of server replicas per device.") -_TIMEOUT = flags.DEFINE_integer("timeout", 30, "Timeout (in seconds) for requests.") -_BACKLOG = flags.DEFINE_integer("backlog", 2048, "Maximum number of connections to hold in backlog.") -_FILEPATH_DETECTOR_WEIGHTS = flags.DEFINE_string( - name="filepath-detector-weights", - default=None, - help="filepath for the weights of the MegaDetector detector", - required=True, -) -_DETECTION_CONFIDENCE_THRESHOLD = flags.DEFINE_float( - "detection-confidence-threshold", - 0.2, - "Confidence threshold below which a detection is ignored when computing the top prediction. " - "Detections themselves are always returned; this only controls the 'prediction'/'prediction_score' fields.", -) -_EXTRA_FIELDS = flags.DEFINE_list( - "extra_fields", - None, - "Comma-separated list of extra fields to propagate from request to response.", -) - - -def predict_one( - detector: YOLO, - filepath: Path, - confidence_threshold: float, - model_version: str = MODEL_VERSION, -) -> dict: - """Run MegaDetector on a single image and produce a Biowatch-compatible prediction dict. - - The 'prediction' field is the label of the highest-confidence detection - whose conf >= confidence_threshold. When no detection passes, prediction - is 'blank' and prediction_score is None. - """ - imagecv = safe_imread(filepath) - ultralytics_results = detector(imagecv, verbose=False) - yolo_out = ultralytics_results[0] - bboxes = yolo_out.boxes - # class_names comes from the .pt file's embedded `names` attribute — never hard-code. - class_names = yolo_out.names - - detection_records = [ - to_detection_record( - conf=conf, - class_instance=class_instance, - xywhn=xywhn, - xyxy=xyxy, - class_label_mapping=class_names, - ) - for conf, class_instance, xywhn, xyxy in zip( - bboxes.conf.cpu().tolist(), - bboxes.cls.cpu().numpy().astype(int).tolist(), - bboxes.xywhn.cpu().numpy().tolist(), - bboxes.xyxy.cpu().numpy().tolist(), - strict=True, - ) - ] - - # MD detects across animal/person/vehicle — pick top-confidence across ALL classes - # (cannot use select_best_animal_detection, which filters for the animal class). - above = [d for d in detection_records if d["conf"] >= confidence_threshold] - if not above: - return { - "predictions": [ - { - "filepath": str(filepath), - "classifications": {}, - "detections": detection_records, - "prediction": "blank", - "prediction_score": None, - "model_version": model_version, - } - ], - } - - top = max(above, key=lambda d: d["conf"]) - prediction_label = LABEL_TO_PREDICTION.get(top["label"], top["label"]) - return { - "predictions": [ - { - "filepath": str(filepath), - "classifications": {}, - "detections": detection_records, - "prediction": prediction_label, - "prediction_score": top["conf"], - "model_version": model_version, - } - ], - } - - -class MegaDetectorLitAPI(ls.LitAPI, VideoCapableLitAPI): - """MegaDetector API server with video support.""" - - def __init__( - self, - filepath_detector_weights: Path, - detection_confidence_threshold: float, - extra_fields: list[str] | None = None, - *args, - **kwargs, - ) -> None: - super().__init__(*args, **kwargs) - self.filepath_detector_weights = filepath_detector_weights - self.detection_confidence_threshold = detection_confidence_threshold - self.extra_fields = extra_fields or [] - - def setup(self, device): - del device # Unused. - self.detector = YOLO(self.filepath_detector_weights) - - def decode_request(self, request, **kwargs): - for instance in request["instances"]: - filepath = instance["filepath"] - if not is_video_file(filepath) and not Path(filepath).exists(): - raise HTTPException(400, f"Cannot access filepath: `{filepath}`") - return request - - def _predict_single_image(self, filepath: str, **kwargs) -> dict: - single_instances_dict = {"instances": [{"filepath": filepath}]} - single_predictions_dict = predict_one( - detector=self.detector, - filepath=Path(filepath), - confidence_threshold=self.detection_confidence_threshold, - ) - return propagate_extra_fields(self.extra_fields, single_instances_dict, single_predictions_dict) - - def predict(self, x, **kwargs): - instances = x.get("instances", []) - logger.info(f"[MegaDetector] Processing {len(instances)} instances") - try: - yield from self.predict_with_video_support(x, **kwargs) - except Exception as e: - logger.error(f"[MegaDetector] Prediction failed: {e}", exc_info=True) - raise - - def encode_response(self, output, **kwargs): - for out in output: - yield {"output": out} - - -def main(argv: list[str]) -> None: - del argv # Unused. - print("[STARTUP] Starting MegaDetector LitServer...") - api = MegaDetectorLitAPI( - filepath_detector_weights=Path(_FILEPATH_DETECTOR_WEIGHTS.value), - detection_confidence_threshold=_DETECTION_CONFIDENCE_THRESHOLD.value, - extra_fields=_EXTRA_FIELDS.value, - api_path=_API_PATH.value, - stream=True, - ) - model_metadata = {"version": MODEL_VERSION, "type": "megadetector"} - server = ls.LitServer( - api, - accelerator="auto", - devices="auto", - workers_per_device=_WORKERS_PER_DEVICE.value, - model_metadata=model_metadata, - timeout=_TIMEOUT.value, - enable_shutdown_api=True, - ) - server.run( - port=_PORT.value, - generate_client_file=False, - backlog=_BACKLOG.value, - ) - - -if __name__ == "__main__": - app.run(main) -``` - -- [ ] **Step 3: Lint + format the Python files** - -```bash -cd python-environments/common -make lint -make format -``` -Expected: zero errors. If lint flags anything, fix it. If format reformats files, accept the changes. - -- [ ] **Step 4: Run the Python tests (requires weights downloaded)** - -Skip if the HF upload hasn't landed yet — the tests `pytest.skipif` themselves. - -```bash -cd python-environments/common -# Once the tarball is on HF, this will work (the env var is set by the make target -# added in Task 9, but you can also export it manually for this one-shot run): -export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt -make test -``` -Expected: the new `TestMegaDetectorServer` class runs and passes all assertions. If `MEGADETECTOR_DETECTOR_WEIGHTS` is unset, the class is skipped — that's acceptable for landing the PR, with a follow-up to wire up Task 9 after the upload. - -- [ ] **Step 5: Commit** - -```bash -git add python-environments/common/run_megadetector_server.py \ - python-environments/common/tests/test_megadetector_server.py -git commit -m "feat(megadetector): LitServe HTTP server + e2e tests" -``` - ---- - -## Task 6: `server.ts` — server launcher + switch case - -**Files:** -- Modify: `src/main/services/ml/server.ts` - -- [ ] **Step 1: Add the `startMegaDetectorHTTPServer()` helper** - -In `src/main/services/ml/server.ts`, immediately after `startManasHTTPServer()` (around line 510, just before the `// ============= Server Lifecycle Management` section), add: - -```ts -interface MegaDetectorServerOptions { - port: number - detectorWeightsFilepath: string - detectionConfidenceThreshold: number - timeout: number - pythonEnvironment: { - reference: { id: string; version: string } - } -} - -/** - * Starts the MegaDetector HTTP server. - * - * @async - * @param {MegaDetectorServerOptions} options - * @returns {Promise<{process: ChildProcess, shutdownApiKey: string}>} - */ -export async function startMegaDetectorHTTPServer({ - port, - detectorWeightsFilepath, - detectionConfidenceThreshold, - timeout, - pythonEnvironment -}: MegaDetectorServerOptions): Promise<{ process: ChildProcess; shutdownApiKey: string }> { - log.info('startMegaDetectorHTTPServer initiated') - log.info(pythonEnvironment) - const localInstalRootDirPythonEnvironment = join( - getMLModelEnvironmentLocalInstallPath({ - ...pythonEnvironment.reference - }), - pythonEnvironment.reference.id - ) - log.info('Local Python Environment root dir is', localInstalRootDirPythonEnvironment) - const scriptPath = is.dev - ? join(__dirname, '../../python-environments/common/run_megadetector_server.py') - : join(process.resourcesPath, 'python-environments', 'common', 'run_megadetector_server.py') - const pythonInterpreter = is.dev - ? join(__dirname, '../../python-environments/common/.venv/bin/python') - : os.platform() === 'win32' - ? join(localInstalRootDirPythonEnvironment, 'python.exe') - : join(localInstalRootDirPythonEnvironment, 'bin', 'python') - log.info('Python Interpreter found in', pythonInterpreter) - log.info('Script path is', scriptPath) - const scriptArgs = [ - '--port', - String(port), - '--filepath-detector-weights', - detectorWeightsFilepath, - '--detection-confidence-threshold', - String(detectionConfidenceThreshold), - '--timeout', - String(timeout) - ] - log.info('Script args: ', scriptArgs) - - const shutdownApiKey = crypto.randomUUID() - log.info('Generated shutdown API key for MegaDetector server') - - const pythonProcess = await startAndWaitTillServerHealty({ - pythonInterpreter, - scriptPath, - scriptArgs, - healthEndpoint: `http://localhost:${port}/health`, - env: { LIT_SHUTDOWN_API_KEY: shutdownApiKey } - }) - - return { process: pythonProcess, shutdownApiKey } -} -``` - -- [ ] **Step 2: Import `findModel` at the top of the file if not already imported** - -Add this near the existing imports if it's not already there: - -```ts -import { findModel } from '../../../shared/mlmodels.js' -``` - -(`findModel` is needed in the switch case to read `detectionConfidenceThreshold` from the model definition rather than hard-coding it. Check the existing imports first — if `findModel` is already imported elsewhere in this file, skip this step.) - -- [ ] **Step 3: Add the `case 'megadetector'` in `startMLModelHTTPServer()`** - -In the `switch (modelReference.id)` block (around line 640), add a new case after `case 'manas'`: - -```ts -case 'megadetector': { - const port = await resolveServerPort(is.dev ? 8003 : null) - const localInstallPath = getMLModelLocalInstallPath({ ...modelReference }) - log.info(`Local ML Model install path ${localInstallPath}`) - const detectorWeightsFilepath = join(localInstallPath, 'MDV6-yolov10-e-1280.pt') - const model = findModel({ ...modelReference }) - const { process: pythonProcess, shutdownApiKey } = await startMegaDetectorHTTPServer({ - port, - detectorWeightsFilepath, - detectionConfidenceThreshold: model?.detectionConfidenceThreshold ?? 0.2, - timeout: 30, - pythonEnvironment: pythonEnvironment - }) - log.info(`pythonProcess: ${JSON.stringify(pythonProcess)}`) - registerActiveServer({ - pid: pythonProcess.pid as number, - port, - shutdownApiKey, - modelId: modelReference.id - }) - return { port: port, process: pythonProcess, shutdownApiKey } -} -``` - -- [ ] **Step 4: Run the existing JS test suite to confirm no regressions** - -```bash -npm test -``` -Expected: all tests pass. (Server.ts isn't directly unit-tested but its consumers are, and TypeScript will fail to compile the renderer bundle if the new code has type errors.) - -- [ ] **Step 5: Commit** - -```bash -git add src/main/services/ml/server.ts -git commit -m "feat(megadetector): main-process server launcher + switch case" -``` - ---- - -## Task 7: `ModelCard.jsx` — `detectionOnly` conditional - -**Files:** -- Modify: `src/renderer/src/models/ModelCard.jsx` - -- [ ] **Step 1: Update the species_count summary line** - -In `src/renderer/src/models/ModelCard.jsx`, locate the block at lines 180–183 that renders the version/size/species line: - -```jsx -
- v{model.reference.version} · {formatSize(model.size_in_MB)} ·{' '} - {model.species_count} species -
-``` - -Replace it with: - -```jsx -
- v{model.reference.version} · {formatSize(model.size_in_MB)} ·{' '} - - {model.detectionOnly - ? `Detection only · ${model.species_count} categories` - : `${model.species_count} species`} - -
-``` - -- [ ] **Step 2: Hide the "View species" toggle and panel when detectionOnly** - -In the same file, locate the block at lines 203–213 that renders the species-toggle div and `speciesPanel`: - -```jsx -
{ - e.stopPropagation() - onToggleSpecies?.(model.reference.id) - }} -> - {speciesOpen ? '▾ Hide species' : `▸ View ${model.species_count} species`} -
- -{speciesOpen && speciesPanel} -``` - -Wrap both elements in `!model.detectionOnly && (...)`: - -```jsx -{!model.detectionOnly && ( - <> -
{ - e.stopPropagation() - onToggleSpecies?.(model.reference.id) - }} - > - {speciesOpen ? '▾ Hide species' : `▸ View ${model.species_count} species`} -
- - {speciesOpen && speciesPanel} - -)} -``` - -- [ ] **Step 3: Manual UI smoke test** - -```bash -npm run dev -``` -- Open the AI Models tab. -- Scroll to the MegaDetector card. -- Confirm the summary line reads: `v6.0 · 50 MB · Detection only · 3 categories` (size depends on the placeholder). -- Confirm there is NO "▸ View species" link below the description. -- Confirm the Download / Delete buttons still render and work. -- Confirm other model cards (SpeciesNet, DeepFaune, Manas) still show their species count and "View species" link. - -If anything is off, fix it and re-test. - -- [ ] **Step 4: Commit** - -```bash -git add src/renderer/src/models/ModelCard.jsx -git commit -m "feat(models): hide species panel for detectionOnly models" -``` - ---- - -## Task 8: `ModelSelect.jsx` — `detectionOnly` conditional - -**Files:** -- Modify: `src/renderer/src/models/ModelSelect.jsx` - -- [ ] **Step 1: Update the species_count line in the dropdown row** - -In `src/renderer/src/models/ModelSelect.jsx`, locate line 64: - -```jsx -{model.species_count} species -``` - -Replace with: - -```jsx - - {model.detectionOnly - ? `Detection only · ${model.species_count} categories` - : `${model.species_count} species`} - -``` - -- [ ] **Step 2: Manual UI smoke test** - -```bash -npm run dev -``` -- Open the Add Source modal (Sources tab → Add Source) or the Import flow. -- Click the model dropdown. -- Confirm MegaDetector appears in the list as a rich card with "Detection only · 3 categories" instead of "3 species". -- Confirm other models still display species counts correctly. -- Confirm selecting MegaDetector works (selection persists, dropdown closes). - -- [ ] **Step 3: Commit** - -```bash -git add src/renderer/src/models/ModelSelect.jsx -git commit -m "feat(models): \"detection only\" label in model picker dropdown" -``` - ---- - -## Task 9: Python tooling (`download_model.py` + `Makefile`) - -**Files:** -- Modify: `python-environments/common/scripts/download_model.py` -- Modify: `python-environments/common/Makefile` - -**Depends on:** The maintainer has uploaded the tarball to HF. - -- [ ] **Step 1: Add megadetector to `download_model.py`** - -In `python-environments/common/scripts/download_model.py`, modify the `MODELS` dict (around line 17): - -```python -MODELS = { - "speciesnet": { - "repo_id": "earthtoolsmaker/speciesnet", - "filename": "4.0.2a.tar.gz", - }, - "deepfaune": { - "repo_id": "earthtoolsmaker/deepfaune", - "filename": "1.3.tar.gz", - }, - "manas": { - "repo_id": "earthtoolsmaker/manas", - "filename": "1.0.tar.gz", - }, - "megadetector": { - "repo_id": "earthtoolsmaker/megadetector", - "filename": "6.0.tar.gz", - }, -} -``` - -Also update the `--model` help text on the argparse line (around line 60): - -```python -help="Model to download (speciesnet, deepfaune, manas, megadetector)", -``` - -- [ ] **Step 2: Add megadetector env var + download command to `Makefile`** - -In `python-environments/common/Makefile`, add a new env var after the existing exports: - -```makefile -export MEGADETECTOR_DETECTOR_WEIGHTS=/tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt -``` - -Append a new line to the `download-models:` target: - -```makefile -download-models: - uv run python scripts/download_model.py --model speciesnet --output /tmp/models/speciesnet - uv run python scripts/download_model.py --model deepfaune --output /tmp/models/deepfaune - uv run python scripts/download_model.py --model manas --output /tmp/models/manas - uv run python scripts/download_model.py --model megadetector --output /tmp/models/megadetector -``` - -- [ ] **Step 3: Run `make download-models` to verify the tarball pulls cleanly** - -```bash -cd python-environments/common -make download-models -``` -Expected: prints "Downloading megadetector from earthtoolsmaker/megadetector..." then "Done! Model extracted to /tmp/models/megadetector". Verify the weights file lands at the expected path: - -```bash -ls -la /tmp/models/megadetector/6.0/MDV6-yolov10-e-1280.pt -``` -Expected: file exists, size matches Task 1's tarball-content size. - -- [ ] **Step 4: Run the full Python test suite** - -```bash -cd python-environments/common -make test -``` -Expected: all tests pass, including the new `TestMegaDetectorServer` class. The animal/person/blank assertions from Task 5 should now exercise real weights. - -- [ ] **Step 5: Commit** - -```bash -git add python-environments/common/scripts/download_model.py python-environments/common/Makefile -git commit -m "tools(megadetector): downloader + e2e test wiring" -``` - ---- - -## Task 10: Documentation (`docs/http-servers.md`) - -**Files:** -- Modify: `docs/http-servers.md` - -- [ ] **Step 1: Add MegaDetector to the Supported Models table** - -In `docs/http-servers.md`, locate the "Supported Models" table (around line 16). Add a fourth row: - -```markdown -| Model | Focus | Species Coverage | -| --------------------------- | ---------------------------------- | ----------------------------------------- | -| **SpeciesNet** (Google) | Global wildlife | 2,000+ species worldwide | -| **DeepFaune** (CNRS) | European fauna | 34 European species | -| **Manas** (OSI-Panthera) | Central Asian fauna | Snow leopard and 11 regional species | -| **MegaDetector** (Microsoft AI for Earth) | Blank filter / worldwide detection | 3 categories (animal, person, vehicle) | -``` - -- [ ] **Step 2: Add `run_megadetector_server.py` to the project structure listing** - -In the same file, locate the project structure block (around line 98–102) and add the new script: - -``` -python-environments/common/ -├── run_speciesnet_server.py # SpeciesNet LitServe implementation -├── run_deepfaune_server.py # DeepFaune LitServe implementation -├── run_manas_server.py # Manas LitServe implementation -└── run_megadetector_server.py # MegaDetector LitServe implementation -``` - -- [ ] **Step 3: Commit** - -```bash -git add docs/http-servers.md -git commit -m "docs(megadetector): document MD model in http-servers.md" -``` - ---- - -## Task 11: End-to-end manual smoke test - -This is a human-driven verification — no automated steps. Run after Tasks 1–10 have landed and the maintainer has provided the final `size_in_MB`. - -- [ ] **Step 1: Bump `size_in_MB` in `mlmodels.js`** - -Replace the placeholder `size_in_MB: 30` from Task 3 with the actual size reported by the maintainer. - -```bash -git add src/shared/mlmodels.js -git commit -m "feat(megadetector): set actual tarball size in model zoo" -``` - -- [ ] **Step 2: Add the logo asset** - -Drop `megadetector.png` into `src/renderer/src/assets/logos/` (provided by the maintainer). - -```bash -git add src/renderer/src/assets/logos/megadetector.png -git commit -m "assets(megadetector): add logo" -``` - -- [ ] **Step 3: Fresh-install smoke test** - -1. Wipe local app data: delete `~/Library/Application Support/biowatch` (macOS) / `~/.config/biowatch` (Linux) / `%APPDATA%/biowatch` (Windows). -2. `npm run dev`. -3. AI Models tab → MegaDetector card → click Download. -4. Confirm: card shows the new size, downloads complete, button switches to Delete. -5. The species panel does NOT appear (no "View species" link). - -- [ ] **Step 4: Inference smoke test** - -1. Sources tab → Add Source. -2. Pick a folder with ~20 mixed images (animals, blanks, ideally a human shot or vehicle). -3. In the model dropdown, confirm MegaDetector shows "Detection only · 3 categories"; select it. -4. Start the import. -5. Wait for processing to finish; the toast says "Processing complete". - -- [ ] **Step 5: Verify predictions landed correctly** - -1. Sources tab → click the new source. -2. The species panel shows entries for `animal`, optionally `person` / `vehicle`, and a "Blank" bucket. -3. Open Media tab → filter `species = animal` → confirm only frames with subjects appear. -4. Open Media tab → filter `species = blank` → confirm empty frames appear. -5. Open one `animal` frame → use the existing annotation UI to assign a real species (e.g. "Vulpes vulpes" / "red fox"). -6. Confirm the annotation persists across a page reload. - -- [ ] **Step 6: Multi-model coexistence check** - -1. Open a second study that uses SpeciesNet. -2. Import a small folder into it while the MegaDetector import is still queued / running. -3. Confirm: both servers run on different ports (`8000` SpeciesNet, `8003` MegaDetector in dev); both finish without interfering with each other. - -- [ ] **Step 7: Regression check — bbox rendering** - -1. On any `animal` frame, open the media-detail view with bounding boxes overlaid. -2. Confirm the bbox is correctly positioned around the subject (xywhn → top-left conversion works). -3. Open a frame imported earlier with SpeciesNet or DeepFaune; confirm those bboxes also still render correctly (no regression in `transformBboxToCamtrapDP`). - ---- - -## Notes for the executor - -- **Frequent commits**: each task ends in a commit. Don't bundle tasks unless explicitly told to. -- **No Claude attribution**: commits and PR descriptions must NOT include `Co-Authored-By: Claude …` lines. -- **gh CLI**: prefix every `gh` invocation with `GH_TOKEN=""` (e.g. `GH_TOKEN="" gh pr view`). -- **External dependency ordering**: Task 5 running real e2e tests, Task 9 (Python tooling), and Task 11 (manual smoke) all require the maintainer to have uploaded `6.0.tar.gz` to `huggingface.co/earthtoolsmaker/megadetector`. Tasks 2, 3, 4, 6, 7, 8, 10 do not — they can land first. The Python server code itself (Task 5) can also be written and committed pre-upload because its tests `skipif` on the env var. -- **Release notes for the PR description** (per spec): when opening the PR, include three call-outs: - 1. New model: MegaDetector v6 — worldwide animal/person/vehicle detector, useful as a fast blank-filter before manual species annotation. - 2. MegaDetector does not identify species; it only detects whether a frame contains an animal, a person, or a vehicle. - 3. ~30 MB download. Reuses the existing `common` Python environment, so no extra environment install if you already have SpeciesNet/DeepFaune/Manas. -- **Documentation maintenance** (per repository `CLAUDE.md`): when modifying files in `src/main/services/import/*.js`, `src/main/services/export/exporter.js`, `src/shared/mlmodels.js`, or `src/main/database/models.js`, also update the corresponding `docs/*.md` file. This plan covers `docs/http-servers.md` in Task 10; no other docs need editing.