PickPic Search API & Map Frontend

A unified system for multi-modal (Text/Image) search with geographic filtering, powered by CLIP, Qdrant, MinIO, and FastAPI.

Preview

Overview

This project consists of a Python backend and a React-based frontend. It allows users to search for products or images using natural language queries and filter results based on their location on a map.

Key Components

• FastAPI (app.py): The main web server that provides the search API and serves the compiled frontend.
• CLIP & Qdrant (main.py):
  • CLIP (OpenAI): Encodes both images and text into the same 512-dimensional vector space.
  • Qdrant: A vector database used to store embeddings and perform efficient similarity searches with geo-spatial filtering.
• MinIO: Object storage used for hosting images. During ingestion, images can be uploaded to MinIO to generate publicly accessible URLs stored in Qdrant payloads.
• React Frontend (map_search): An interactive map interface using Leaflet. Users can drop a pin, define a search radius, and see matching products as markers on the map.

---

Getting Started

Prerequisites

• Python 3.12+
• Node.js & npm (for frontend)
• Docker (to run Qdrant and MinIO)

Installation & Environment Setup

1. Clone the repository and navigate to the search API folder:

   cd search_api

2. Configure environment variables:

   cp .env.example .env
   # Edit .env if you need to change ports or credentials

3. Install Python dependencies using uv:

   uv sync

4. Install Frontend dependencies:

   cd map_search
   npm install
   cd ..

---

Data Ingestion & Setup

The ingestion process is handled by search_api/main.py. It uses CLIPQdrantManager to process images.

Ingestion Flow (main.py)

1. Connect to Qdrant: Initializes the client and creates a collection with 512-dimensional vectors (Cosine similarity).
2. Load Images: Loads raw images (e.g., from a dataset or local folder).
3. Embed & Upload:
   • Encodes images into vectors using the sentence-transformers/clip-ViT-B-32 model.
   • (Optional) Uploads images to MinIO and retrieves their permanent URLs.
   • (Optional) Attaches random or specific geo-coordinates to the metadata.
4. Upsert to Qdrant: Stores the vectors and metadata (image URL, location, source) in the Qdrant collection.

To run the demo ingestion:

uv run python search_api/main.py

---

Running the Application

1. Production Mode (Single Server)

In production, FastAPI serves both the API and the React frontend from the dist folder.

1. Build the frontend:

   cd search_api/map_search
   npm run build

2. Start the FastAPI server:

   cd search_api
   uv run uvicorn app:app --host 0.0.0.0 --port 8100

   Access the app at http://localhost:8100/.

2. Development Mode (Hot Reload)

1. Start the Backend:

   cd search_api
   uv run uvicorn app:app --reload --port 8100

2. Start the Frontend (Vite):

   cd search_api/map_search
   npm run dev

   Access the frontend at http://localhost:5173/ (Vite proxies API requests to port 8100).

3. Docker Mode (Containerized)

You can also run the entire stack using Docker Compose.

1. Build the frontend:

   cd search_api/map_search
   npm install
   npm run build
   cd ../..

2. Start the stack:

   docker compose up --build

   Access the app at http://localhost:8100/.

---

Data Ingestion via Docker

To run the ingestion script (main.py) inside the running container:

docker compose exec api uv run python main.py

API Reference

POST /api/search

Search for products using text and a geo-radius.

Request Body:

{
  "query_text": "blue summer dress",
  "lat": 40.7128,
  "lon": -74.0060,
  "radius_km": 50,
  "limit": 20
}

Response: Returns a list of matching products including image_url, lat, lon, and score.

---

Configuration

Environment variables can be set in a .env file in the search_api directory. Copy .env.example to get started:

cp .env.example .env

Environment Variables Reference

Variable	Description	Default
QDRANT_URL	Qdrant server address	localhost:6333
QDRANT_API_KEY	Optional API key for Qdrant	-
MINIO_ENDPOINT	MinIO server address	localhost:9000
MINIO_ACCESS_KEY	MinIO access key	minio
MINIO_SECRET_KEY	MinIO secret key	minio_secret_change_me
MINIO_BUCKET	Bucket name for images	pickpic
MINIO_PUBLIC_URL	Publicly accessible URL for MinIO	http://localhost:9000
SEARCH_COLLECTION	Qdrant collection name	hm_ecommerce_clip
SEARCH_WARMUP	Preload CLIP model on startup	true
MINIO_UPLOAD	Upload images to MinIO during ingestion	true
EMBED_MIN_SHORT_EDGE	Minimum pixels for upscaling small images	512
EMBED_MAX_SHORT_EDGE	Maximum pixels for capping huge images	2048