Different Face Images Have Different Rotations, How To Standardise?

[mindreader-gh]

Hello, I'm having issues aligning 1 face from 2 different images into the same view, so that I can run distance-based comparisons with their face features.

I'm aiming to plot the images straight up, as though looking from a camera at eye-level.

However, I can't seem to get the rotation matrix right. The ranges for the x,y,x coordinates for 2 different meshes might also be different. I notice this most especially with the z-axis most frequently.

Do I have to do the rotation first, then normalise using the translation vector or is it the other way around?

Could you kindly take a look and point me in the right direction?

lmk3d, mesh, pose = model.get_all_outputs(I)

# Define the original face pose (Euler angles and translation)
original_yaw = np.deg2rad(pose[0][0][0])   
original_pitch = np.deg2rad(pose[0][0][1])  
original_roll = np.deg2rad(pose[0][0][2])  
original_translation = pose[0][1]  

original_mesh_vertices = mesh[0]

# Function to rotate vertices using Euler angles
def rotate_vertices(vertices, yaw, pitch, roll):
    R_yaw = np.array([[np.cos(yaw), -np.sin(yaw), 0],
                      [np.sin(yaw), np.cos(yaw), 0],
                      [0, 0, 1]])

    R_pitch = np.array([[np.cos(pitch), 0, np.sin(pitch)],
                        [0, 1, 0],
                        [-np.sin(pitch), 0, np.cos(pitch)]])

    R_roll = np.array([[1, 0, 0],
                       [0, np.cos(roll), -np.sin(roll)],
                       [0, np.sin(roll), np.cos(roll)]])

    # Combined rotation matrix
    R = np.dot(R_yaw, np.dot(R_pitch, R_roll))

    # Rotate vertices
    rotated_vertices = np.dot(R, vertices)

    return rotated_vertices

# Function to translate vertices
def translate_vertices(vertices, translation):
    translated_vertices = vertices + np.expand_dims(translation, axis=1)
    return translated_vertices

# Normalize face pose (rotate and translate vertices)
rotated_vertices = rotate_vertices(original_mesh_vertices, -original_yaw, -original_pitch, -original_roll)
normalized_vertices = translate_vertices(rotated_vertices, -original_translation)

[ken881015]

Although I don't quite realize your question, but maybe I can give you a suggestion~
In file SingleImage.py there has a process of how author using the output of model to plot 3D mesh align with Input image, maybe there has the answer you need!