Modelling Facial Geometry using Compositional VAEs

Co-founder & General Manager USA

1 min read Published July 21, 2021

In short

This method learns non-linear face geometry representations with a variational auto-encoder using multiple levels of hidden variables. Lower layers capture global geometry while higher ones encode local deformation, producing a parameterisation that decomposes the human face into semantically meaningful levels of detail for model fitting under geometric constraints.

Key takeaways

A hierarchy of hidden variables separates global face geometry from local deformation.
The resulting parameterisation is semantically meaningful, not only statistically convenient.
Model fitting can then capture varying levels of detail depending on the constraints applied.
The work was published at CVPR and the paper is available for download.

3D face meshes generated by a compositional VAE, from smooth global geometry to fine detail such as wrinkles and a beard

We propose a method for learning non-linear face geometry representations using deep generative models. Our model is a variational auto-encoder with multiple levels of hidden variables where lower layers capture global geometry and higher ones encode more local deformations. Based on that, we propose a new parameterization of facial geometry that naturally decomposes the structure of the human face into a set of semantically meaningful levels of detail. This parameterization enables us to do model fitting while capturing varying level of detail under different types of geometrical constraints.

Download here: CVPR 2018 Publication

DEBOSH: Deep Bayesian Shape Optimization

Revolutionizing Aerodynamic Shape Optimization with DeepGeo: A Breakthrough in Neural Network Technology

HybridSDF: Combining Free Form Shapes and Geometric Primitives for effective Shape Manipulation