Stage M2R: Intelligent Character Posing Through Observers’ Feedback

Context

Motion retargeting is the task of automatically applying an existing animation or pose to a new character. It can be used for example to transfer the captured animation of a user to its Avatar in a virtual environment. An important challenge of motion retargeting is to adapt poses to different morphologies, which is non trivial for poses with close interactions between the limbs [1,4].

Recently a lot of works have applied deep learning methods to motion retargeting [2,5]. Using this kind of architecture would allow us to learn from examples how to adapt a pose to different morphologies. However very few dataset contaiifferent characters performing the exact same poses, and even less for poses with close interactions.

Scope of the internship

Determining if two characters are performing the same pose depends on contextualinformation that is difficult to translate in geometric terms. For example if a person clapping never has their hands touching, the pose will be wrong, however whether the elbows are touching the body is unimportant. Different contacts thus have different degrees of importance for defining the pose. While this contextual information is difficult to translate in geometric rules, it is easy to understand for human observers. Perceptive studies have been used to present observers with similar problems in order to better understand this contextual information (see for example [3]).

The goal of the internship is to leverage such type of information to automatically and naturally pose characters. The first step will be to create a large dataset containing different characters performing similar poses with different variations (such as the presence or absence of a specific contact). This data can be either captured with actors in Morpheo's platform Kinovis (https://kinovis.inria.fr/), or generated using motion retargeting algorithms. Then a perceptive study will ask subjects which variation of each poses are considered equivalent across different characters. An intermediate goal of the internship is therefore to obtain a dataset containing a wide variety of characters performing the same poses, validated by the perceptive study.

Tasks

• Study the literature on the topic

• Generate a dataset of various characters performing a set of similar poses with close contacts. The data will be created either by capturing actors in the Kinovis platform, or by using state of the art motion retargeting methods, or (most probably) a combination thereof.

• Design the protocol of a perceptive study to identify which similar poses in the dataset are considered equivalent by participants.

• Run the experiment on an important panel of persons

• Use the results of the experiment to label the dataset with pose information

• Use the validated dataset to train a deep learning model for motion retargeting specialised in close contact situations

• If enough time is available, write a paper presenting the dataset and the results of the model

Desired profile

• Candidates should be preparing a MSc or equivalent degree in computer science, computer vision, computer graphics or machine learning.

• Some programming skills are required, python being ideal

• Experience in perceptive studies or experiments involving human participants

• Machine learning and/or computer animation skills are a plus!

• Candidates should have a good English level, since most of the resources are in English and the internship will take place in an international environment

General informations

- Duration: 6 months

- Start of the internship: beginning of 2021, to be defined according to the candidate’s academic schedule

- Remuneration: ~600€/month

- Location: The internship will ideally take place at Inria Grenoble, but can be relocated to Inria Rennes if necessary

- Supervision: Jean Basset, PhD Student at Inria Grenoble, Ludovic Hoyet, Researcher at Inria Rennes, and Franck Multon, Leader of Mimetic team at Inria Rennes

Applications

The candidate should send a CV and cover letter, and eventually recommendation letters,

to jean.basset@inria.fr, ludovic.hoyet@inria.fr, and franck.multon@irisa.fr

Contact

Jean Basset jean.basset@inria.fr

References

[1] Basset, J., Wuhrer, S., Boyer, E., & Multon, F. (2020). Contact preserving shape transfer: Retargeting motion from one shape to another. Computers & Graphics

[2] Cosmo, L., Norelli, A., Halimi, O., Kimmel, R., & Rodolà, E. (2020). LIMP: Learning Latent Shape Representations with Metric Preservation Priors. arXiv preprint arXiv:2003.12283.

[3] Hoyet, L., McDonnell, R., & O'Sullivan, C. (2012). Push it real: Perceiving causality in virtual interactions. ACM Transactions on Graphics (TOG), 31(4), 1-9.

[4] Liu, Z., Mucherino, A., Hoyet, L., & Multon, F. (2018, November). Surface based motion retargeting by preserving spatial relationship. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games (pp. 1-11).

[5] Zhou, K., Bhatnagar, B. L., & Pons-Moll, G. (2020, August). Unsupervised Shape and Pose Disentanglement for 3D Meshes. In European Conference on Computer Vision (pp. 341-357). Springer, Cham.