Animation of Hand-drawn Faces using Machine Learning

Today's research in artificial vision has brought us new and exciting possibilities for the production and analysis of multimedia content. Pose estimation is an artificial vision technology that detects and identifies a human body's position and orientation within a picture or video. It locates key points on the bodies, and uses them to create three-dimensional models. In digital animation, pose estimation has paved the way for new visual effects and 3D renderings. By detecting human movements, it is now possible to create fluid realistic animations from still images. This bachelor thesis discusses the development of a pose estimation based program that is able to animate hand-drawn faces -- in particular the caricatured faces in Papiri di Laurea -- using machine learning and image manipulation. Working off of existing techniques for motion capture and 3D animation and making use of existing computer vision libraries like \textit{OpenCV} or \textit{dlib}, the project gave a satisfying result in the form of a short video of a hand-drawn caricatured figure that assumes the facial expressions fed to the program through an input video. The \textit{First Order Motion Model} was used to create this facial animation. It is a model based on the idea of transferring the movement detected from a source video to an image. %This model works best on close-ups of faces; the larger the background, the more the image gets distorted in the background. Possible future developments could include the creation of a website: the user loads their drawing and a video of themselves to get a gif version of their papiro. This could make for a new feature to add to portraits and caricatures, and more specifically to this thesis, a new way to celebrate graduates in Padova.