Face-Labeler-Pilot-Workflow

Tag people shown in images

Face Labeler Pilot is an interactive human-in-the-loop Python-based workflow tool for photographers, digital asset managers, and anyone needing to tag people shown in photographs.

The tool is designed to assist in tagging photos from events, portrait sessions, or similar photo shoots that show the same people multiple times across many images. The tool performs best on images of people with their faces turned directly towards the camera, like in group shots and portraits.

The tool does not rely on any database of known faces or identities, rather it builds a list of known faces from the current session only and the names entered by the user. In its essence, a user need only enter the name of an individual once, thereafter, all representations of the individual is recognized if a match is found from previously labeled faces. This makes it a great tool for labeling faces that have never been seen before and may never be seen again by the photographer or editor. It only relies on the data from the current session and learns as it iterates through the session's images.

Workflow Steps

1. Detect Faces: Simply add a folder of images to the watch_folder directory at the root of the project and select it when prompted in the workflow. Supported file types are JPEG, PNG, TIFF, CR2, DNG, and NEF. Click 'Detect Faces' to let the face detection model iterate over the images and get all the face locations and face encodings.

2. Label Faces: After all the faces are found, the workflow will begin iterating over all the found faces and will prompt you to label them. Start by typing in the names for new unknown faces. Once a face has been labeled it will be added to a list of known faces. When there are known faces in the list, the face recognition algorithm will check for matches with the known faces and ask you to confirm its prediction if a match is found, otherwise you'll be prompted to enter a new name or choose a name from the list of names that have already been entered.

3. Save Metadata: Once all the faces have been labeled, click the 'Write Metadata' button to save/embed the face locations (bounding boxes) and names of the person(s) shown in the image's metadata. Face locations along with names are saved in the MGW Regions List uri. Names are also saved in the XMP:EXT4 PersonInImage field. Click the 'Export Metadata' button to export the bounding boxes and names to a csv file alongside the original images.

Installation & Setup

Clone the repository

git clone https://github.com/peterjakubowski/Face-Labeler-Pilot-Workflow.git
cd Face-Labeler-Pilot-Workflow

Before configuring your environment, you should have Python 3.10+ installed.

It's highly recommended that you create a virtual environment, either using Python's built-in virtual environment or Conda virtual environment.

Python Virtual Environment

1. Create a virtual environment

   Navigate to the project directory and execute the built-in Python venv module:

   python3 -m venv .venv
   

2. Activate the environment

   You must activate the environment to ensure your terminal uses this isolated instance of Python and pip

• macOS / Linux

  source .venv/bin/activate
  

• Windows (Command Prompt)

  .venv\Scripts\activate.bat
  

• Windows (PowerShell)

  .venv\Scripts\Activate.ps1
  

  Once activated, your terminal prompt will show (.venv) at the beginning of the line.

3. Install the Requirements

   Make sure your terminal is in the same directory as your requirements.txt file, then run the following command to install all dependencies:

   pip install -r requirements.txt
   

Conda Virtual Environment

Alternatively, if you're using Conda, create a new Conda virtual environment from the environment.yml file using the following command in your command prompt. This will create a new environment and install all dependencies:

conda env update --file environment.yml --prune

Python Requirements

Dependency	Category	Primary Function & Usage in Repository
streamlit	UI Framework	Interactive Web Application: Serves as the web GUI framework in app.py. Powers the interactive dashboard, rendering image previews, labeling controls, and metadata export buttons.
face-fecognition	Deep Learning / AI	Face Detection & Embedding Extraction: Interface built on dlib (used in utils/image_processing.py and app.py) that locates faces in images and extracts 128-dimensional facial embedding vectors for identity matching.
dlib	Machine Learning Algorithms	Underlying Vision Engine: Core C++ computer vision engine supporting face-recognition. Provides pre-trained facial landmark predictors and HOG/CNN face detectors for precise facial alignment and detection.
opencv-python	Image Processing	Image Preprocessing & Manipulation: Utilized within utils/image_processing.py for reading images from disk, converting color spaces (BGR to RGB), cropping face bounding boxes $(x, y, w, h)$, and drawing visual box overlays.
image_utils	Image Processing Utilities	Image Path Helper: Used in the file workflow specifically to search, filter, and extract list sequences of image file paths across input directories.
numpy	Scientific Computing	Image Data Structure & Math: Essential matrix backend required by opencv-python (cv2), as images are loaded and manipulated directly as multi-dimensional NumPy arrays throughout the application pipeline.
pandas	Data Engineering	Interactive Table Display: Used within the Streamlit user interface to format, organize, and display structured DataFrames.
PyExifTool	Metadata	EXIF & XMP Metadata Writing: Python wrapper around ExifTool in utils/exiftool.py used to extract image metadata and embed standardized Person Shown (PersonInImage) tags and MWG Regions (Metadata Working Group face bounding box regions and identity names) directly into image files. Note: PyExifTool requires Exiftool, which must be installed separately.
rawpy	RAW Image Processing	RAW Image Processing: Python wrapper for libraw used to postprocess raw image files and extract their thumbnails.

Additional system requirements

Exiftool

Additionally, ExifTool must be installed on your system in order to read, write and edit image metadata using PyExifTool. Installation instructions can be found on the Exiftool website here.

ExifTool is used at the third step of the workflow to embed image metadata. It embeds face regions and names (XMP-mwg-rs:Region and XMP:PersonInImage) directly in the original file. Note: ExifTool is configured by default to create a copy of the original file when embedding metadata. Original files are renamed with "_orginal" appended to their extension.

ExifTool is used on the viewer page to extract image metadata from image files. It reads face regions from the embedded image metadata.

Launching the tool

To launch the Streamlit server within the virtual environment, run the following command while the virtual environment is activated (in your command prompt, you should see (.venv) if using a Python virtual environment and (facelabelerpilot_env) if using conda):

streamlit run app.py

Then open http://localhost:8501

Pages

Workflow

The 3-step Face Labeler Pilot post-production workflow tool that uses face recognition to tag people shown in photographs.

Viewer

A simple image viewer to view labeled faces. It opens all images in a folder (up to 50) and displays them with annotated face regions.

Settings

Controls how the system recognizes faces.

• Top-k: Controls how many close matches the system reviews before applying a name tag.
  • Lower values (1–3) work best if you have very few reference photos of a person.
  • Higher values require a larger consensus among your existing collection, which improves tagging accuracy but requires you to have already tagged multiple photos of that person.
• Threshold: Controls how aggressively the system applies automatic tags.
  • Lower (0.3 – 0.45): High accuracy. Prevents wrong tags, but forces you to manually tag photos the system wasn't 100% sure about.
  • Higher (0.65 – 0.8): The system will aggressively guess and tag more faces, but you may occasionally have to fix a mistagged photo.
• Load Library Faces: Loads labeled faces from images in the library folder if the face classifier is uninitialized. Scans the library folder to extract existing face region metadata and names from labeled images. It runs each face through a face recognition model to generate face embeddings to be used in the face classifier.
• Reset Face Classifier: Clears all loaded name metadata and face embedding from the application's memory, resetting the face classifier to an uninitialized state. Your physical image files on disk are not altered.

Project Structure

Face-Labeler-Pilot-Workflow/
├── .github/
│   └── workflows/ 
│       └── ci.yaml                # Github actions: ruff, mypy, and unit tests
├── .streamlit/
│   └── config.toml                # Streamlit configuration options 
├── library/                       # Face classifier library folder
│   └── your-labeled-images.jpg    # Place labeled images here to load into face classifier
├── models/                        
│   └── face.py                    # Python class definitions
├── pages/                         # Streamlit pages
│   ├── settings.py                # Streamlit settings page
│   ├── viewer.py                  # Streamlit viewer page
│   └── workflow.py                # Streamlit workflow page
├── tests/                         # Pytest suite
│   ├── conftest.py                # Test fixtures
│   ├── test_app.py                # Streamlit AppTest app tests
│   ├── test_settings.py           # Streamlit AppTest settings page tests
│   ├── test_viewer.py             # Streamlit AppTest viewer page tests
│   └── test_workflow.py           # Streamlit AppTest workflow page tests
├── utils/
│   ├── csv.py                     # Exports metadata to csv files
│   ├── exiftool.py                # Python functions that use ExifTool (PyExifTool)
│   ├── face_classifier.py         # Streamlit BaseConnection face classifier
│   ├── helpers.py                 # Streamlit helper functions
│   ├── image_processing.py        # Image processing with OpenCV
│   ├── image_readers.py           # Python tools for reading images
│   └── library.py                 # Python tool to load library images for the face classifier
├── watchfolder/                   # Place unlabeled images here to send through workflow
│   └── your-folder-of-images/ 
│       └── your-images.jpg 
├── app.py/                        # The main Streamlit app entrypoint
├── config.py/                     # Face recognition and image settings
├── environment.yml/               # Conda environment dependencies
├── README.md                      # This file
├── requirements.txt               # Python Project dependencies
└── requirements-dev.txt           # Development and testing dependencies

Future Improvements and Features

• Add additional face detection algorithms (or replace the current with another) that can better detect faces turned to the side or in profile. The current algorithm in use does best at detected faces that are turned towards the camera.

• Try using an object detection model that first detects people (not faces) in an image before checking for faces with the face detection algorithm. This could help increase the probability of finding a person in the image that needs to be tagged. A person without a face would always need to be manually tagged using this method since there would be no way to get a face encoding to compare to the list of known faces.

• Create some kind of option to add additional tags based on the person shown in the image. This would require the user to supply keys and values for data lookup to take place. A possible use case could be e-commerce on-figure photography where a particular model is associated with products and skus that must be tagged in a shot. Images would be placed in the watch folder organized by shots (shot number) in separate subfolders. A csv file could be included to provide model names (key), shot number(key), and product/skus (values).