Analysis of the Reliability of Deepfake Facial Emotion Expression Synthesis
Advances in deep learning have made it simple to generate deepfakes, hyper-realistic videos in which photos and video snippets are processed to create fake videos that look legitimate. Although different deepfake creation methods have been proposed, it is still an open question whether synthesized f...
| Autores: | , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2024 |
| País: | España |
| Institución: | Universitat de Lleida (UdL) |
| Repositorio: | Repositori Obert UdL |
| OAI Identifier: | oai:repositori.udl.cat:10459.1/466916 |
| Acceso en línea: | https://doi.org/10.22967/HCIS.2024.14.035 https://hdl.handle.net/10459.1/466916 |
| Access Level: | acceso abierto |
| Palabra clave: | Deepfakes Emotion Recognition Image Classification |
| Sumario: | Advances in deep learning have made it simple to generate deepfakes, hyper-realistic videos in which photos and video snippets are processed to create fake videos that look legitimate. Although different deepfake creation methods have been proposed, it is still an open question whether synthesized facial expressions adequately display emotions. To address this issue, we have used one of the most typical facial deepfake creation strategies, an encoder-decoder architecture. Then, we applied this algorithm to create deepfakes from a well-known emotional database in which performers in video recordings display specified emotions. We have tested the facial emotion expression synthesis of the deepfakes comparing the emotions of the photograms in the videos with their original counterparts. Emotion analysis has been performed using two well-known emotion classifications used in the facial emotion recognition area. The results show that facial emotion expression is not successfully synthesized using the face swap algorithm. Our study provides the first empirical evidence of the degree to which different emotions are adequately synthesized in deepfakes in which facial expressions are faked. The presented work displays the limitations of current face swap algorithms to properly synthesize emotions, which have implications in both the synthesis and the detection of deepfake recordings. |
|---|