Find which photos contain the person shown in a reference portrait. The project runs locally and uses DeepFace with ArcFace embeddings, RetinaFace detection, face alignment, and cosine distance. Every face in a candidate group photo is checked.
Install uv if it is not already available, then run:
uv python install 3.11
uv syncThe first real match downloads the ArcFace and RetinaFace model weights, so it takes longer and requires network access. Later runs use the local model cache.
The reference image must contain exactly one detectable face.
uv run face-match reference.jpg --photos vacation-1.jpg vacation-2.jpg
uv run face-match reference.jpg --folder ./photos --recursive
uv run face-match reference.jpg --folder ./photos --json
uv run face-match reference.jpg --folder ./photos --matches-only
uv run face-match reference.jpg --folder ./photos --rank-quality
uv run face-match reference.jpg --folder ./photos --rank-quality --top 3
uv run face-match reference.jpg --folder ./photos --rank-quality --top 3 --portraits-dir ./best
uv run face-match reference.jpg --folder ./photos --rank-quality --top 3 --cutouts-dir ./best-cutouts
uv run face-match reference.jpg --folder ./photos --rank-quality --top 3 --headshots-dir ./best --headshot-background blurred_original
uv run face-match reference.jpg --folder ./photos --rank-quality --top 3 --headshots-dir ./best --headshot-background solid --headshot-background-color '#e8eef7'
uv run face-match reference.jpg --folder ./photos --rank-quality --json
uv run face-match reference.jpg --folder ./photos --portraits-dir ./portraits
uv run face-match reference.jpg --folder ./photos --cutouts-dir ./cutouts
uv run face-match reference.jpg --folder ./photos --cutouts-dir ./cutouts --debug-overlaysSupported candidate extensions are .jpg, .jpeg, .png, and .webp. Folder results are sorted deterministically. Use --threshold NUMBER to override DeepFace's default ArcFace cosine threshold. A smaller threshold is stricter; calibrate it with representative photos before relying on it.
Pass --rank-quality to order matching photos by descending portrait quality after
identity matching completes. Ranking reuses the matched face metadata and does not
rerun DeepFace, RetinaFace, ArcFace, YOLO, or SAM. Stable ties retain candidate input
order, and a photo that cannot be loaded or scored is reported after successful
scores without stopping the collection.
Use --top N with --rank-quality to return only the first N ranked results. N
must be a positive integer. --json returns the serialized original match, detailed
quality scores, scoring status and error, and original candidate index.
When --portraits-dir, --cutouts-dir, or --headshots-dir is combined with ranking, face matching
finishes first, then quality ranking and --top selection are applied before any
artifacts are generated. Only successfully scored selected matches are processed.
The existing matched face metadata is reused without rerunning DeepFace, RetinaFace,
or ArcFace; cutout and segmented-headshot processors are each loaded once and YOLO
and SAM run only for the selected batch. Headshots are collision-safe PNGs using the
same head-and-shoulders framing as extract-face-portraits. Their background is
transparent by default; choose blurred_original, blurred_and_dimmed, or solid
with --headshot-background, and set a solid color with
--headshot-background-color (white by default). Headshot paths or isolated errors
appear in both ranked table and JSON output. Headshot options are ranked-only.
Pass --portraits-dir DIR to save a portrait for the best face in each image that passes the match threshold. Without this flag, the matcher never creates portrait files or directories. Source photos are always preserved.
Portraits are EXIF-oriented, optionally straightened using RetinaFace eye landmarks, framed to include hair, neck, and part of the shoulders, and saved as 512×512 quality-95 JPEGs. The default padding is 50% of the face width on each side, 75% of the face height above, and 100% below. If reliable eye landmarks are unavailable, cropping continues without rotation.
Names use the source stem and RetinaFace index, such as group-photo_face-2.jpg. Faces smaller than 48 pixels and crops requiring more than 4× enlargement are skipped and reported without changing the successful match result. Existing output names are not overwritten; collision-safe suffixes are added instead.
Pass --cutouts-dir DIR to detect the person containing the best-matching face and save:
- a tightly bounded transparent PNG (
*_cutout.png), - a full-source-resolution binary mask (
*_mask.png), and - with
--debug-overlays, a source-resolution JPEG showing the face, person, crop, and mask (*_overlay.jpg).
YOLO detects people in the EXIF-oriented source image. The selected person box is expanded by 15%, clamped to the image, and only that crop is passed to box-prompted SAM 2.1. SAM is never run on the complete source image. A failed detection or segmentation is reported in cutout_error without invalidating the face match or changing the source image.
Choose a speed/quality tradeoff with --cutout-model-profile:
fast: YOLO11n and SAM 2.1 tinybalanced(default): YOLO11s and SAM 2.1 basequality: YOLO11m and SAM 2.1 large
Models download automatically on first use. Device selection prefers Apple MPS, then CUDA, and falls back to CPU. Ultralytics software and downloaded model weights have their own license terms; review them before commercial deployment.
The Python workflows are thin orchestration layers over shared public modules. Domain types and geometry, lazy model lifecycle, detection adapters, SAM segmentation and mask processing, transparent composition, and collision-safe atomic output handling are separated so the same in-memory operations can support future interfaces without changing the current CLI or convenience functions. Legacy imports remain available from their original modules.
Use the separate command when there is no reference identity and you want one transparent portrait for every person with a visible face:
uv run extract-person-portraits group_photo.jpg --output-dir ./person_portraits
uv run extract-person-portraits group_photo.jpg \
--output-dir ./person_portraits \
--model-profile balanced \
--output-size 1024x1280 \
--debug
uv run extract-person-portraits group_photo.jpg \
--output-dir ./person_portraits \
--background blurred_and_dimmed \
--blur-radius 24 \
--dim-strength 0.3Faces are ordered left-to-right, associated with YOLO person boxes, deduplicated, and segmented independently on expanded crops. Successful files use names such as group_photo_person_01.png. Output width is fixed and height is capped by --output-size; each PNG ends at the final foreground row without artificial bottom space. Use --fixed-height for the legacy fixed canvas. Person padding is 15%, subject padding is 8%, and unmatched faces are skipped with a structured reason. Background composition reuses the completed segmentation and always writes PNG output. Use --fallback estimate_upper_body to attempt an approximate torso crop instead.
Debug mode additionally writes the raw crop mask, source detection overlay, and SAM input crop. People without a visible eligible face are intentionally omitted. A failure for one person does not stop later candidates.
Python API:
from pathlib import Path
from face_matcher import PortraitExtractionConfig, extract_all_person_portraits
results = extract_all_person_portraits(
Path("group_photo.jpg"),
Path("person_portraits"),
PortraitExtractionConfig(output_width=1024, output_height=1280),
)Use the face-driven command when full-body cutouts look incomplete because a person is partly hidden by furniture, another person, or the edge of the photo:
uv run extract-face-portraits group_photo.jpg --output-dir ./face_portraits
uv run extract-face-portraits group_photo.jpg \
--output-dir ./face_portraits \
--model-profile quality \
--output-size 1024x1280 \
--debug
uv run extract-face-portraits group_photo.jpg \
--output-dir ./face_portraits \
--background solid \
--background-color '#e8eef7'RetinaFace finds each visible face and defines the portrait framing. YOLO is used only
to give SAM a more precise target-person prompt when a matching body box is available;
if an occluded person is not detected—or YOLO loading or inference fails—the command
falls back to a face-derived prompt as long as SAM remains available.
SAM 2.1 runs only on a local upper-body crop, never on the full source image. The
default SAM crop is four face widths wide, extends 75% of a face height above the
detected face, and extends 250% below it so hair, shoulders, nearby hands, and the
upper torso are available to segmentation. When YOLO finds the matching person, its
upper-body width also expands this crop. After SAM runs once, the visible mask and
face landmarks determine the final framing: roughly 6% headroom, eyes near the upper
third, and a mid-chest lower boundary. Adjust the initial crop with --shoulder-width,
--space-above, and --space-below.
Face portrait width is fixed by --output-size; its height is a maximum. Shorter
portraits are cropped to the final foreground row instead of retaining artificial
background space below the person.
Portraits are ordered left-to-right and saved as 1024×1280 PNGs named like
group_photo_headshot_01.png; transparency remains the default. Existing files are never overwritten. --debug also
writes the SAM input crop, raw mask, and a detection image showing the face, framing,
prompt, and crop boxes. A failed face does not stop later faces.
Python API:
from pathlib import Path
from face_matcher import FacePortraitConfig, extract_face_portraits
results = extract_face_portraits(
Path("group_photo.jpg"),
Path("face_portraits"),
FacePortraitConfig(shoulder_width=3.0, space_below=1.0),
)This is still rectangular crop-based segmentation rather than generative completion: it will not invent missing shoulders. It limits the output to the visually useful upper portion so lower-body occlusion does not dominate the portrait.
renamer.py can shorten unwieldy filenames by renaming every file directly inside a folder to 1, 2, 3, and so on while preserving each extension:
uv run python renamer.py ./photosFiles are numbered in case-insensitive filename order. The helper is non-recursive and renames all file types in the selected folder, so review the folder before running it. Renaming is destructive; the original filenames are only printed to the terminal and are not stored for automatic recovery.
from pathlib import Path
from face_matcher import CutoutConfig, PortraitConfig, match_photos
results = match_photos(
"reference.jpg",
["photos/group.jpg", "photos/portrait.png"],
portrait_config=PortraitConfig(Path("portraits")),
cutout_config=CutoutConfig(Path("cutouts")),
)
matching_paths = [result.path for result in results if result.matched]
for result in results:
print(result.to_dict())Each result includes status, matched, the best cosine distance, the effective threshold, typed face metadata, portrait fields, cutout_artifacts or cutout_error, and a processing error when applicable. Candidate, portrait, and cutout errors do not abort the batch. Invalid or ambiguous reference images do.
Portrait candidates can be scored and ranked entirely in memory, without loading a detector, segmentation model, or vision-language model:
from face_matcher import rank_portrait_candidates, score_portrait_candidate
score = score_portrait_candidate(image, detected_face)
ranked = rank_portrait_candidates(image, detected_faces)Every public numeric score is normalized to the inclusive 0–1 range, where higher is
better. The total is a normalized weighted average; weights and normalization targets
are configurable with PortraitQualityOptions. If frontal pose is unavailable, its
weight is omitted and the remaining weights are renormalized.
- Sharpness is normalized Laplacian variance in a padded grayscale face region. Texture, noise, and compression artifacts can raise it even when a portrait does not look sharp.
- Exposure uses mean grayscale luminance. A preferred interval receives full credit, with strong penalties beyond severe dark and bright thresholds. It does not model HDR, skin tone, artistic lighting, or color-channel clipping.
- Contrast is local grayscale standard deviation divided by a configurable target. It cannot distinguish useful subject detail from noise or a busy background.
- Face resolution uses the shorter detected face-box dimension relative to a configurable target. It measures available pixels, not genuine optical detail.
- Face visibility combines the fraction of the detected box inside the image with clearance from image boundaries. It identifies clipping risk, not occlusion by hair, hands, glasses, or other objects.
- Frontal pose uses only reliable eye-and-nose geometry, optionally strengthened by mouth-center symmetry. It is unavailable when those landmarks are absent or geometrically unreliable and is only a rough 2D proxy for head pose.
Scoring is deterministic and local and never modifies or saves the image. Detection confidence is not part of the score. These heuristics do not detect closed eyes, smiles, identity, attractiveness, age, gender, or emotion.
uv run pytestThe default suite mocks model inference. To run the opt-in real-model smoke test:
FACE_MATCH_REFERENCE=reference.jpg \
FACE_MATCH_CANDIDATE=candidate.jpg \
uv run pytest -m integrationImages and embeddings are processed locally by this application, but model weights are downloaded on first use. Face recognition is probabilistic: lighting, pose, age, occlusion, image quality, and the selected threshold can cause false matches or missed matches. Do not use the output as the sole basis for consequential decisions. This project does not perform liveness or anti-spoofing checks and does not infer demographic attributes.