The Python module “common” contains a variety of submodules that provide utility classes and functions. The module is part of a series of tutorials on using PyTorch to create and train generative deep learning models. The code for these tutorials is available here.
common.quaternion
This module provides functions for performing calculations on quaternions that are either stored in tensors or as numpy arrays. This module has been adapted from an original implementation by the authors of the Quaternet model.
The following functions operate on quaternions stored in tensors.
def qmul(q, r) # multiply quaternion(s) q with quaternion(s) r
def qrot(q, v) # rotate vector(s) v about the rotation described by quaternion(s) q
def qeuler(q, order, epsilon=0) # Convert quaternion(s) q to Euler angles
def slerp(q0, q1, amount=0.5) # perform quaternion slerp between quaternion q0 and quaternion q1. Multiple quaternions are not supported.The following functions operate on quaternions stored in numpy arrays.
def qnormalize_np(q) # normalise a quaternion
def qmul_np(q, r) # multiply quaternion(s) q with quaternion(s) r
def qrot_np(q, v) # rotate vector(s) v about the rotation described by quaternion(s) q
def qeuler_np(q, order, epsilon=0, use_gpu=False) # Convert quaternion(s) q to Euler angles
def qfix(q): # Enforce quaternion continuity
def expmap_to_quaternion(e) # Convert axis-angle rotations (aka exponential maps) to quaternions.
def euler_to_quaternion(e, order) # Convert Euler angles to quaternions.common.utils
This module provides functions to save a history of training losses as CSV file or as graphical plot and to perform simple computations on motion capture data.
The most frequently used functions in the common.utils submodule are:
def save_loss_as_csv(loss_history, csv_file_name) # export a training loss history as CSV file
def save_loss_as_image(loss_history, image_file_name) # export a training loss history as graphical plot.
def get_skeleton_edge_list(skeleton) # compute a list of edges connecting skeleton joints.
def get_equal_mix_max_positions(poses): # compute minimum and maximum positions from a sequence of motion capture poses.common.skeleton
The Skeleton class holds information about a hierarchy of joints and positional joint offsets. An instance of the class is created by passing the positional joint offsets and a list of parent joints as arguments. Instantiation usually happens automatically when loading motion capture data from a file using the MocapDataset class.
The main use of the Skeleton class is to compute forward kinematics based on the position of a root joint and the relative rotations of all the joints. The corresponding function is named “forward_kinematics” and takes two arguments: a sequence of poses in which poses are represented as relative joint rotations (quaternions), and a sequence of positions for the root joint. Both of these arguments are provided as tensors. The first tensor has the shape: batch_size x sequence_length x joint_count x 4. The second tensor has the shape: batch_size x sequence_length x 3. The function returns a tensor of the shape: batch_size x sequence_length x joint_count x 3.
The full list of function declarations in the Skeleton class is as follows:
def __init__(self, offsets, parents) # constructor
def num_joints(self) # returns numbers of skeleton joints
def offsets(self) # returns list of joint position offsets
def parents(self) # returns list of parent joint indices
def has_children(self) # returns list of flags specifying for each joint if it has children joints or not
def children(self) # returns list of child joint indices
def remove_joints(self, joints_to_remove, dataset) # remove joints from a motion capture dataset
def forward_kinematics(self, rotations, root_positions) # compute forward kinematicscommon.mocap_dataset
This submodule defines the MocapDataset class. This class operates on motion capture data that is imported from a file containing a pickled dictionary. More information about this dictionary is available here. The class is instantiated by passing a file path and a framerate (motion capture frames per second) as arguments.
from common.mocap_dataset import MocapDataset
mocap_data_path = "../../data/Mocap/MUR_Nov_2021/MUR_Fluidity_Body_Take1_mb_proc_rh.p"
mocap_fps = 50
mocap_data = MocapDataset(mocap_data_path, fps=mocap_fps)Once motion capture data has been loaded, it is stored in the MocapDataset class instance alongside an instance of the Skeleton class. The storage format of the motion capture data follows the structure of the HumanEva dataset. Accordingly, motion capture data is hierarchically organised on two levels. The top level represents the subjects from which motion capture was recorded. The second level represents the actions a subject was performing while being recorded. The MocapDataset class provides functions to obtain a list of subject names and action names.
subjects = mocap_data.subjects()
actions = mocap_data.all_actions()
actions_of_subject = mocap_data.subject_actions(subjects[0])In the tutorial examples, the motion capture recordings contain only one subject named “S1” and one action for this subject named “A1”.
If the motion capture data is used for machine learning and the machine learning model runs on the GPU, then the MotionCapture class has to be told to move its data to the GPU. This is done by calling the function “cuda” without any arguments.
The MocapDataset provides a function named “compute_positions” for calculating absolute positions of joints from relative positions. This function takes no arguments.
mocap_data.compute_positions()The MocapDataset owns an instance of the Skeleton class. This instance is defined once the MocapDataset class has been initiated by importing a motion capture file. The Skeleton instance can be obtained by using the “skeleton” function.
skeleton = mocap_data.skeleton()The full list of function declarations in the MocapDataset class is as follows:
def __init__(self, path, fps) # constructor
def cuda(self) # move internal tensors to the GPU
def downsample(self, factor, keep_strides=True) # downsample the motion capture data to a lower frame rate.
def compute_euler_angles(self, order) # calculate joint rotations in Euler angles from rotations represented by quaternions.
def compute_positions(self): # calculate absolute joint positions from relative joint positions
def compute_standardized_values(self, value_key) # compute standardised values for the specified motion capture data.
def subjects(self): return the list of subjects from which motion capture data was recorded
def subject_actions(self, subject): return the list of recorded actions of a motion capture subject.
def all_actions(self): return a concatenated list of recorded actions of all motion capture subjects.
def fps(self): return the motion capture frame rate
def skeleton(self): return an instance of the Skeleton class created from the motion capture data.common.pose_renderer
The pose_renderer module provides the PoseRenderer class which can be used to create simple skeleton animations from motion capture data. The PoseRenderer is instantiated by passing joint connectivity information of a skeleton as argument. This information can be obtained using the “get_skeleton_edge_list” function of the common.utils module.
The PoseRenderer class provides two functions for creating either single images or multiple images from motion capture poses. These images show a simple graphical depiction of a skeleton. Both functions take as arguments either a single pose or a list of poses, the minimum and maximum range of joint positions, camera angles as elevation and azimuth, the line width with which edges of the skeleton are drawn, and the width and height of the created image in inches. The PoseRenderer also provides a simple convenience function to arrange multiple images into a grid layout.
def create_pose_image(self, pose, axis_min, axis_max, rot_elev, rot_azi, line_width, image_xinch, image_yinch) # create a single image containing graphical depiction of a single pose.
def create_pose_images(self, poses, axis_min, axis_max, rot_elev, rot_azi, line_width, image_xinch, image_yinch) # create a list of images, one for each pose.
def create_grid_image(self, images, grid) # arrange a list of images in a grid layout