Image Retrieval System

A local multimodal image search engine powered by CLIP embeddings and Qdrant vector database. Search your personal photo library using natural language queries or by uploading a reference image — all running entirely on your machine, with no data sent to the cloud.

---

Features

• Text-to-image search — describe what you're looking for in plain language ("sunset over the ocean", "birthday party with cake")
• Image-to-image search — upload a photo to find visually similar images in your library
• Local & offline — CLIP model runs locally via PyTorch with Apple Silicon GPU acceleration (MPS), no internet required after setup
• Fast vector search — Qdrant stores and retrieves embeddings efficiently, even across large collections
• Web UI — clean, dark-mode interface accessible from any browser
• Drag & drop — drop images directly onto the browser window to start a search
• Metadata extraction — indexes EXIF data including timestamps, camera model, and GPS coordinates
• Thumbnail generation — 300×300px JPEG thumbnails for fast grid previews

---

How It Works

1. Ingest — the system scans a directory, extracts metadata, generates thumbnails, and encodes each image into a 768-dimensional CLIP embedding
2. Store — embeddings and metadata are stored in a local Qdrant vector database (running in Docker)
3. Search — at query time, your text or image is encoded with the same CLIP model and the nearest embeddings are retrieved via cosine similarity

---

Requirements

• Python 3.10+
• Docker (for Qdrant)
• Node.js (optional, not required at runtime)
• Apple Silicon Mac recommended (MPS acceleration); CUDA GPU or CPU also supported

---

Installation

1. Clone the repository

git clone <repo-url>
cd image-retrieval-system

2. Create and activate a virtual environment

python -m venv .venv
source .venv/bin/activate

3. Install Python dependencies

pip install -r requirements.txt

4. Download the CLIP model

The model is stored locally so it can run offline:

python download_model.py

This downloads clip-vit-large-patch14 (~900MB) to data/models/clip-vit-large-patch14/.

5. Start Qdrant

docker-compose up -d

Qdrant will be available at http://localhost:6333.

---

Usage

Start the application

./start.sh

Or manually:

source .venv/bin/activate
python app.py

Then open http://localhost:8000 in your browser.

The API documentation is available at http://localhost:8000/docs.

Index a directory

1. Click Manage in the top-right corner
2. Enter the full path to your image directory (e.g. /Users/yourname/Pictures)
3. Click Ingest New to index only new images, or Re-index Directory to reprocess everything

Alternatively, use the API directly:

# Index new images only
curl -X POST http://localhost:8000/ingest/new \
  -H "Content-Type: application/json" \
  -d '{"path": "/Users/yourname/Pictures"}'

# Re-index all images in a directory
curl -X POST http://localhost:8000/ingest \
  -H "Content-Type: application/json" \
  -d '{"path": "/Users/yourname/Pictures"}'

Search

Text search — type a description in the search box and press Enter or click the search button.

Image search — switch to Image Search mode, then either click to select an image or drag and drop one anywhere on the page.

Results are ranked by cosine similarity. Each result card shows a thumbnail, similarity score, file path, and buttons to open the file in Finder or copy its path.

---

Project Structure

.
├── app.py                    # FastAPI backend
├── download_model.py         # One-time model download script
├── start.sh                  # Startup script
├── docker-compose.yml        # Qdrant service
├── requirements.txt
├── frontend/
│   ├── index.html
│   └── static/
│       ├── app.js
│       └── styles.css
├── src/
│   ├── search.py             # Text and image search logic
│   ├── ingest.py             # Directory ingestion and re-ingestion
│   ├── database/
│   │   └── qdrant_db.py      # Qdrant client wrapper
│   ├── models/
│   │   └── clip_model.py     # CLIP model wrapper (encode text & images)
│   └── utils/
│       ├── metadata.py       # EXIF metadata extraction
│       ├── scanner.py        # Recursive directory scanner
│       └── thumbnails.py     # Thumbnail generation
└── data/
    ├── models/               # Local CLIP model (created by download_model.py)
    ├── qdrant_data/          # Qdrant persistent storage (created by Docker)
    └── thumbnails/           # Generated thumbnails

---

API Reference

Method	Endpoint	Description
GET	/	Serves the web UI
GET	/thumbnail?path=...	Serves an image or thumbnail
POST	/search_text	Search by text query
POST	/search_image	Search by uploaded image
POST	/ingest/new	Index new images in a directory
POST	/ingest	Re-index all images in a directory
POST	/open	Reveal a file in Finder (macOS)
GET	/health	Health check

Search request body

{ "query": "a dog on the beach" }

Search response

[
  {
    "id": 123456789,
    "score": 0.312,
    "metadata": {
      "path": "/Users/yourname/Pictures/holiday.jpg",
      "thumbnail_path": "data/thumbnails/abc123.jpg",
      "width": 4032,
      "height": 3024,
      "camera_make": "Apple",
      "camera_model": "iPhone 14 Pro",
      "timestamp": "2023-08-15T14:22:01",
      "gps_coordinates": [44.4, 8.9]
    },
    "path_exists": true
  }
]

---

Configuration

Result limit — search results are capped at 10% of the total indexed collection (minimum 5). This scales automatically as your library grows.

Score filtering — results below the mean similarity score are filtered out by default, keeping only the most relevant matches.

Supported image formats — .jpg, .jpeg, .png, .heic, .heif, .webp, .bmp, .tiff

Thumbnail size — 300×300px JPEG at quality 85, stored in data/thumbnails/

---

Troubleshooting

"Search failed. Is the backend running?" — make sure python app.py is running and Qdrant is up (docker ps | grep qdrant).

"Model directory not found" — run python download_model.py to download the CLIP model before starting the app.

Slow first search — the CLIP model is loaded into memory on the first query. Subsequent searches reuse the cached model and are much faster.

Images show as "Missing" — the file was indexed but has since been moved or deleted. Use Re-index Directory to refresh the database, or run db.cleanup_missing_images() from Python.

Qdrant not starting — ensure Docker Desktop is running and port 6333 is not in use by another service.

---

License

This project is for personal, local use. CLIP model weights are subject to OpenAI's model license. Qdrant is Apache 2.0 licensed.