{"id":1650,"date":"2022-08-15T10:49:33","date_gmt":"2022-08-15T09:49:33","guid":{"rendered":"https:\/\/wp.coventry.domains\/e2edu\/?page_id=1650"},"modified":"2022-08-23T19:07:32","modified_gmt":"2022-08-23T18:07:32","slug":"motion-capture-data","status":"publish","type":"page","link":"https:\/\/wp.coventry.domains\/e2edu\/motion-capture-data\/","title":{"rendered":"Motion Capture Data"},"content":{"rendered":"\n

Several machine learning examples provided here use data obtained through motion capture. Some example motion capture recordings are available here<\/a>. For these recordings, a marker based optical motion capture system (Qualisys, 12 cameras) provided by the dance company Cie Gilles Jobin<\/a> was used. The recordings are of a solo dancer who was improvising according to various movement qualities. Some information about the role and types of movement qualities used in contemporary dance is available here<\/a>.<\/p>\n\n\n\n

The motion capture data is based on a skeleton representation of the dancer that contains 64 joints, out of which 40 joints are used to represent the dancer’s hands. Working with this full set of joints has turned out to be challenging for machine learning since such a large number of hand joints dominate the loss function at the expense of the remaining joints. For this reason, the dataset created from motion capture contains a smaller number of joints with all hand joints except those of the middle finger removed. The reduced skeleton contains 35 joints.<\/p>\n\n\n

\n
\"\"
Skeleton Representations of a Dancer Recorded through Motion Capture. The skeleton on the left contains all joints that were tracked by the motion capture system. The skeleton on the left has most of the hand joints removed. This latter version has been used in all the machine learning examples provided here.<\/figcaption><\/figure>\n<\/div>\n\n\n

The reduced skeleton contains the following joints. <\/p>\n\n\n\n

joint 0 : Hips\njoint 1 : Spine\njoint 2 : Spine1\njoint 3 : Spine2\njoint 4 : Neck\njoint 5 : Head\njoint 6 : Head_Nub\njoint 7 : LeftShoulder\njoint 8 : LeftArm\njoint 9 : LeftForeArm\njoint 10 : LeftForeArmRoll\njoint 11 : LeftHand\njoint 12 : LeftInHandMiddle\njoint 13 : LeftHandMiddle1\njoint 14 : LeftHandMiddle2\njoint 15 : LeftHandMiddle2_Nub\njoint 16 : RightShoulder\njoint 17 : RightArm\njoint 18 : RightForeArm\njoint 19 : RightForeArmRoll\njoint 20 : RightHand\njoint 21 : RightInHandMiddle\njoint 22 : RightHandMiddle1\njoint 23 : RightHandMiddle2\njoint 24 : RightHandMiddle2_Nub\njoint 25 : LeftUpLeg\njoint 26 : LeftLeg\njoint 27 : LeftFoot\njoint 28 : LeftToeBase\njoint 29 : LeftToeBase_Nub\njoint 30 : RightUpLeg\njoint 31 : RightLeg\njoint 32 : RightFoot\njoint 33 : RightToeBase\njoint 34 : RightToeBase_Nub<\/code><\/pre>\n\n\n\n
The topology of the skeleton is as follows:<\/p>\n\n\n\n
joint 0 children: 1 25 30\njoint 1 children: 2\njoint 2 children: 3\njoint 3 children: 4 7 16\njoint 4 children: 5\njoint 5 children: 6\njoint 6 children:\njoint 7 children: 8\njoint 8 children: 9\njoint 9 children: 10\njoint 10 children: 11\njoint 11 children: 12\njoint 12 children: 13\njoint 13 children: 14\njoint 14 children: 15\njoint 15 children:\njoint 16 children: 17\njoint 17 children: 18\njoint 18 children: 19\njoint 19 children: 20\njoint 20 children: 21\njoint 21 children: 22\njoint 22 children: 23\njoint 23 children: 24\njoint 24 children:\njoint 25 children: 26\njoint 26 children: 27\njoint 27 children: 28\njoint 28 children: 29\njoint 29 children:\njoint 30 children: 31\njoint 31 children: 32\njoint 32 children: 33\njoint 33 children: 34\njoint 34 children:<\/code><\/pre>\n\n\n\n
The original mocap recordings are stored in the Autodesk Filmbox Format<\/a> (FBX) and the Biovision Hierarchy Animation File Format<\/a> (BVH). The data used for machine learning is in a custom format. This format is a “pickled<\/a>” serialised Python dictionary. The dictionary stores motion capture data as follows:<\/p>\n\n\n
\n
\"\"
Python Dictionary for Storing Motion Capture Data<\/figcaption><\/figure>\n<\/div>\n\n\n
The top level dictionary contains one key-value pair per dancer. Since there is only one dancer in all recordings, there is also only one key-value pair. The key used here is “S1” which is an abbreviation for subject 1. The value is another dictionary. This second dictionary stores joint names, the parent joint indices, parent children relationships among joints, the joint offsets, local joint coordinates, world joint coordinates, local joint rotations as quaternions, world joint rotations as quaternions. <\/p>\n\n\n\n
Such a dictionary can be created from any motion capture recording that is stored in BVH format. A small python tool has been developed for converting from BVH format into the dictionary format. This tool is available here<\/a>. To perform a conversion, the python script has to be run from the command line as follows:<\/p>\n\n\n\n
python main_bvhconv.py --input <mocap file in bvh format> --output <mocap file in pickled dictionary format><\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
Several machine learning examples provided here use data obtained through motion capture. Some example motion capture recordings are available here. For these recordings, a marker based optical motion capture system (Qualisys, 12 cameras) provided by the dance company Cie Gilles Jobin was used. The recordings are of a solo dancer who was improvising according to […]<\/p>\n","protected":false},"author":2154,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","footnotes":""},"class_list":["post-1650","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages\/1650","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/users\/2154"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/comments?post=1650"}],"version-history":[{"count":28,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages\/1650\/revisions"}],"predecessor-version":[{"id":3165,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages\/1650\/revisions\/3165"}],"wp:attachment":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/media?parent=1650"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}