Separating facial features from context - a new revolution in deepfake technology

[CarderPlanet]

Israeli researchers have used CGI data augmentation (CGI - Computer Generated Images) to gain more control over deepfake images. While CGI heads still cannot be used to effectively fill the gaps in deepfake face datasets, new research may soon be unnecessary.

The creators of some of the most successful viral deepfake videos of the past few years have carefully selected the original videos for their fakes, avoiding profile, angled, or exaggerated facial images. Most often, they chose a video as a source where a person (whose real face was subsequently replaced by a deepfake) looks directly into the camera with a minimum range of facial expressions.

Israel Institute of Technology has introduced a new way of using synthetic data such as CGI heads to actually separate a face (for example, the main facial features of the fake "Tom Cruise" from all angles) from its context (for example, looking up and to the side, frowning , squinting in the dark, frowning eyebrows, closed eyes, etc.).

This is not just a deepfake puppet, suitable only for avatars and partial lip-syncing, and with limited potential for full deepfake video transformation. No, the method presented by the researchers is a huge step forward towards the fundamental separation of objects (such as a tilted head or frowning eyebrows) from the individual.

Since the inception of deepfake technology, its creators have experimented with CGI images, creating heads using 3D graphics applications such as Cinema4D and Maya. However, in the end, many decided to abandon these methods in favor of newer ones.

The Israeli researchers' method is more effective because the posture and contextual information from the CGI images is completely separated from the individual.