Summary
Expressive Aliens is a proof of concept system that combines a data driven method with a physics simulation for the purpose of synthesizing expressive movements for computer generated characters with arbitrary morphologies. A core component of
the system is a reinforcement learning algorithm that employs reward functions based on Laban Effort Factors. This system has been tested by training three different non-anthropomorphic morphologies on different combinations of these reward functions.
Morphologies
Three agent morphologies have been designed. The morphologies consist of rigid body parts that are connected via revolute joints, and their extremities end in rounded stubs rather than articulated hands or feet. The morphologies vary with regards to the number and
shape of body parts, the number of joints, and the assignment of foot or hand functionality. These differences impact the the type and level of stability that each morphology exhibits during simulation.
Quadruped, Legless. The body parts rendered in dark blue have been assigned foot
or hand status.
Physics Simulation
The agent’s shape and articulation is simulated using the rigid body
dynamics functionality of the PyBullet game physics engine.
Reinforcement Learning Algorithm
The Reinforcement Learning (RL) system chosen is based on the Soft Actor Critic (SAC) algorithm. SAC is a model-free off-policy algorithm that operates on continuous action and state spaces. A unique feature of SAC is its use of entropy regularisation. This regularisation maximizes entropy instead of long term reward to promote exploration. The RL algorithm is implemented using the PyTorch deep-learning framework and is based on the reference implementation provided by the OpenAI Spinning Up educational
resource.
Rewards
The reward is calculated from a weighted combination of individual rewards: alive reward, collision reward, move distance reward, Flow Effort reward, Space Effort reward, Time Effort reward, and Weight Effort reward.