Skip to content

plan.md

Latest commit

 

History

History
95 lines (79 loc) · 5.36 KB

plan.md

File metadata and controls

95 lines (79 loc) · 5.36 KB
Raw

Project Plan - Faces

Background (paper & repo)

The ACLab created a new dataset of infant facial landmarks to be used for machine learning applications. The main take-aways from the paper are:

  • The ACLab created infanface dataset (train and test, with 410 images, 68 landmarks, with pose attributes)
  • They showed that existing deep learning algorithms for image -> landmark coordinates have a high error rate on the baby images (~22%)
  • They produced (primarily) 2 modified models:
    • 1 HrNet trained on the union of the adult and baby images with some rotation augmentation applied to some of the training data
    • 1 HrNet that is first pre-trained on the adult faces, and then fine-tuned on the baby faces, with some rotation augmentation also added
  • Both these models preformed comparably well with ~2% error
  • The main two steps for producing landmarks are:
    1. Generate a bounding box from the original image.
    2. Estimate landmarks from the bounding box. Existing models perform well on step 1, but fails on step 2. Wan’s paper is trying to fill the gap in step 2.

Other interesting things:

  • They increased "size" of dataset by doing augmentations (rotations) of the images and random zooming?
  • They do not "do anything" with the landmarks themselves, the paper is focused on using deep learning to do image -> landmark coordinates successfully with babies
    • However, there is a figure (fig 4) that plots a t-SNE dimensionally reduced HrNet representation of each image (adult and baby), showing that the HrNet model represents the baby and adult images differently. This figure is simply used to show the "domain gap" that the paper is trying to address. It is interesting that the model that fails accurately landmark the baby faces still does treat adult and baby faces differently?
      • t-SNE for improved HrNet model?

Project Goal

Use classical machine learning techniques to classify adult vs infant faces using facial landmark data generated by the ACLab (InAnFace and 300W data)

Sub-goals: (roles and responsibilities)

  1. Deal with the class imbalance between the adult and infant data
  • Which models are more biased towards inbalanced data? Which preformance metrics should be used per model?
  1. Transformations/preprocessing of the landmarks
  2. Test out several ML classifiers
  • Create executable (from command line) and reproducible code/results & upload to repository
  • Keep track of what choices are made, such as:
    • Learning curve (as we grow training set, how do diff models respond)?
    • Dimensional reduction (or not)
      • fwd feature selection?
    • Which hyper-parameters are chosen (and why)
    • What works/doesn’t work (and why)
      • Even if model is “bad”, still useful to know
  • Classifier ideas
    1. Logistic regression
    2. Naive bayes / discriminate analysis
    3. SVC
    4. Trees
    5. KNN
  1. Project-check in
  2. Stakeholder feedback on prototype
  3. Create simple website outlining our chosen ML methods and results

Connor: Naive bayes / discriminate analysis Jesse: preprocessing, feature selection Sophia: class imbalance/resampling, feature selection Zongyu: pca, SVC

Timeline

  • 22 Nov: project check in
    • demonstrated progress on the project goals presented with a walk-through of the project repo
    • should include a submission in canvas with the url of a "checkin" branch in the project repo
  • 2 Dec: first stakeholder feedback session confirmed
  • 4 Dec: project prototype due
    • submitted in canvas as the URL of a "prototype" branch of the project repo
  • 7 Dec: main branch of project is ready to go, feedback is implemented
  • 8 Dec: project presentations
    • 9-minute presentations with 2-minute Q&A
  • 15 Dec: final submission, feedback from presentation session is implemented
    • submitted in canvas as a "final" branch of the project repo

Obstacles and Challenges

Place to record encountered and/or anticipated obstacles and challenges.

  • The data is somewhat imbalanced in shape
  • Some of the preprocessing steps are tricky without 3D face meshes?
  • What should we use as predictors?
    • Possible predictors:
      1. All 68 landmarks
      2. Relationships between landmarks, like Geometric values. Eg. box width / box height, and min box size / interocular distance
      3. Euc. distances of landmarks
      4. Sym. diff of landmarks
      5. Turned / normalized to bounding box landmarks
  • Ideally the bounding box should be derived from the original image, but that would demand performing Retinaface, another deep learning model. Instead we use the estimated landmarks to estimate the geometric values.

HTML page

Plans for HTML page that tells "the story" concisely for a general audience.

Structure it somewhat like this.

  • Brief into explaining the project goal & concise results (which models worked and didn't work)
  • Visuals and concise description (analysis) that clearly show the success/failure of the model in classifying adult/infant based on landmark data
    • sub-visuals and concise description that demonstrate why certain hyper-parameters were chosen, validation curves, or dimensional reduction used
  • Short summary on pre-processing used to improve model accuracy
  • Short summary on how the data was generated (using info from the paper)
  • Acknowledgements