where-is-this

Figure out where a photo — or a reel — was taken. A coarse-to-fine geolocation pipeline packaged as two Claude Code skills: a structured GeoGuessr-style cold read that narrows the world to a region, then an optional retrieval + geometric-verification stage that snaps to a GPS pin where open street-level imagery has coverage.

It is built on one uncomfortable fact:

MegaLoc is a matcher, not a geolocator. It turns an image into an 8448-d descriptor and finds near-duplicates. It cannot locate a blind photo, because there is no global reference database you can hold — and building one is not a weekend project.

So the reference set is fetched on demand, for one region at a time. Something has to name that region first. In 2026 the best downloadable thing for that job is a frontier VLM's own reasoning — which is why stage 1 here is a playbook, not a model.

Architecture

  photo / reel
       │
       ▼
  ┌─────────────────────────────────────────┐
  │ 0. FREE ANSWER                          │  EXIF GPS · platform geotag
  │    geoguess.py exif · reel.py           │  caption · spoken place-names
  └─────────────────────────────────────────┘  → often just done
       │ (usually stripped)
       ▼
  ┌─────────────────────────────────────────┐
  │ 1. COARSE — cold read (the VLM itself)  │  driving side · road lines
  │    SKILL.md meta-playbook, zero deps    │  bollards · plates · script
  └─────────────────────────────────────────┘  flora · soil · sun · poles
       │ region + confidence                    → country / region / locality
       ▼
  ┌─────────────────────────────────────────┐
  │ 2. FETCH — open imagery for THAT region │  Mapillary Graph API (CC-BY-SA)
  │    mapillary_fetch.py                   │  tiled <0.009° bboxes
  └─────────────────────────────────────────┘  ↳ Panoramax fallback (no token)
       │ geo-tagged reference set
       ▼
  ┌─────────────────────────────────────────┐
  │ 3. FINE — retrieve                      │  MegaLoc 8448-d descriptors
  │    pinpoint.py + megaloc.py             │  exact cosine NN (numpy)
  └─────────────────────────────────────────┘
       │ top-k candidates
       ▼
  ┌─────────────────────────────────────────┐
  │ 4. VERIFY — geometric                   │  SuperPoint + LightGlue
  │    verify.py                            │  cv2 RANSAC inlier count
  └─────────────────────────────────────────┘
       │
       ▼   GPS pin  ·  or an honest `region_only`

Step 4 is the part that matters most. Retrieval will always return a nearest neighbour, look-alike or not. Geometric verification is what lets the pipeline say "I don't know" instead of confidently planting a wrong pin.

What's here

File	Role
skills/geolocate/SKILL.md	The meta-playbook — cue checklist, answer format, confidence tiers. The engine for stage 1.
skills/geolocate/geoguess.py	EXIF/GPS extractor. Pillow only.
skills/geolocate/reel.py	Reel/video ingest: yt-dlp download → ffmpeg keyframes → Whisper transcript → caption/geotag. Writes bundle.json.
skills/geolocate/mapillary_fetch.py	Mapillary Graph API fetch, auto-tiled under the Jan-2026 bbox cap. Stdlib only.
skills/geolocate/panoramax_fetch.py	Panoramax (OSM-community) STAC fetch — the fallback when Mapillary has no coverage. No token, stdlib only.
skills/geolocate/pinpoint.py	Orchestrates embed → search → verify → GPS (or region_only). Tries Mapillary then Panoramax (--source auto).
skills/geolocate/verify.py	SuperPoint + LightGlue + RANSAC fundamental-matrix inliers.
skills/megaloc/megaloc.py	MegaLoc wrapper — info, describe, compare, search, where, pairs. Usable standalone as a place-recognition CLI.
docs/global-geolocation-landscape.md	The landscape report behind every adopt/extend/build call, with licenses verified against primary sources.

Install

Copy the two skill directories into your Claude Code skills folder — they resolve each other as siblings, so keep them together:

git clone https://github.com/LamaSu/where-is-this
cp -r where-is-this/skills/geolocate where-is-this/skills/megaloc ~/.claude/skills/

Then, by stage:

# Stage 1 (reasoning + EXIF) — this is ~90% of the accuracy
pip install pillow

# Stage 2 (pinpoint)
pip install torch torchvision huggingface_hub numpy opencv-python
pip install "git+https://github.com/cvg/LightGlue.git"

# Reel/video ingest (also needs ffmpeg + ffprobe on PATH)
pip install yt-dlp openai-whisper

# Mapillary token — free, read-only client token
export MAPILLARY_TOKEN="MLY|..."     # https://www.mapillary.com/dashboard/developers

Runs on CPU. No GPU required (and none was used to verify it).

Use

A photo. Ask Claude where it is; the skill runs the playbook and reports cues, a guess, and a calibrated confidence tier. Or drive the pieces directly:

python ~/.claude/skills/geolocate/geoguess.py exif photo.jpg

python ~/.claude/skills/geolocate/pinpoint.py \
    --image photo.jpg --lat 52.3730 --lon 4.8930 --radius-km 0.3

A reel or short video. More signal than a photo — distinct shots, spoken place-names, on-screen text, a caption, sometimes a geotag:

python ~/.claude/skills/geolocate/reel.py "<url-or-file>" --out ./out

A travel reel usually shows several places. Each distinct shot gets located separately and reported separately — they are not averaged into one blurred guess.

Standalone place recognition, no geolocation involved — ask which of your own images show the same place. --database takes a folder, a glob, or a .npy built by describe:

python ~/.claude/skills/megaloc/megaloc.py search --query query.jpg --database photos/

What was actually measured

Verified end-to-end on CPU against live data, not simulated:

• 6.0 m error on a held-out query against 12 real Mapillary images fetched live for Amsterdam's Dam Square.
• LightGlue verification behaves as a gate: identical pair → 1024 inliers; unrelated scenes → 0; a real held-out neighbour → 168 inliers at ~3 m; an out-of-region query → 0 inliers and a region_only refusal. That last case is the important one — it declined to invent a pin.
• Reel ingest on a public clip: 8 keyframes, correct transcript, and the platform geotag recovered from metadata.

Small-N by construction. These are existence proofs that the chain works, not benchmark numbers.

Honest limits

• Accuracy is capped by what the frame shows. Strong on outdoor scenes with roads, signage and vegetation; weak-to-useless on interiors, open ocean, tight crops and generic scenes.
• "Global" means coverage-bounded. Mapillary is dense in cities and along roads and sparse elsewhere; no coverage means stage 1's region is your answer.
• The coarse guess is the whole ballgame. A wrong or too-broad region fetches the wrong imagery, and no amount of fine matching recovers from that.
• Search is exact numpy cosine, which is fine at region scale. FAISS is the documented city-scale upgrade and needs a separate process on Windows (torch and faiss both load OpenMP → OMP Error #15).

Licenses and attribution

This repository is MIT. It vendors no models, weights or imagery — everything heavy is fetched from upstream at runtime, under upstream's terms:

Component	License	Verified
MegaLoc (gmberton)	MIT	✅ primary source
hloc	Apache-2.0	✅ primary source
FAISS	MIT	✅ primary source
Mapillary imagery	CC-BY-SA — attribution + share-alike	✅ primary source
Panoramax imagery (OSM-community)	CC-BY-SA / open (etalab)	✅ landscape report
LightGlue / SuperPoint	⚠️ check upstream	❌ not verified here

Two flags worth taking seriously before any commercial use:

• SuperPoint weights have historically carried a non-commercial research license, separate from LightGlue's own repo license. Verify upstream yourself.
• Mapillary's commercial and derived-database terms are unverified here — the terms page returned 403 during research. The CC-BY-SA imagery license is confirmed; the platform terms around building a derived database are not.

docs/global-geolocation-landscape.md records why other options were rejected — notably PIGEON/PIGEOTTO (weights withheld, cannot self-host) and StreetCLIP (CC-BY-NC, non-commercial only).

Intended use

Locating your own photos, travel content, and open-source-intelligence work on public imagery — plus playing a better game of GeoGuessr. Don't point it at people: inferring where someone lives, works or currently is, from images they didn't publish for that purpose, is exactly the use this wasn't built for.