DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation
Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-...
| Authors: | , , , , |
|---|---|
| Format: | article |
| Publication Date: | 2022 |
| Country: | España |
| Institution: | Universidad Autónoma de Madrid |
| Repository: | Biblos-e Archivo. Repositorio Institucional de la UAM |
| Language: | English |
| OAI Identifier: | oai:repositorio.uam.es:10486/702775 |
| Online Access: | http://hdl.handle.net/10486/702775 https://dx.doi.org/10.1016/j.engappai.2022.104673 |
| Access Level: | Open access |
| Keyword: | Benchmark Databases DeepFakes Face manipulation Fake detection Fake news Media forensics Telecomunicaciones |
| id |
ES_a1bb129a3e1e15b94a4cf45ba321cae7 |
|---|---|
| oai_identifier_str |
oai:repositorio.uam.es:10486/702775 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluationTolosana Moranchel, RubénRomero Tapiador, SergioGonzalez SosaVera Rodríguez, RubénFiérrez Aguilar, JuliánBenchmarkDatabasesDeepFakesFace manipulationFake detectionFake newsMedia forensicsTelecomunicacionesMedia forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learningThis work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA, Spain (MINECO/FEDER RTI2018-101248-B-I00)ElsevierDepartamento de Tecnología Electrónica y de las ComunicacionesEscuela Politécnica Superior20222022-04-01research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/702775https://dx.doi.org/10.1016/j.engappai.2022.104673reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengEuropean Commission http://dx.doi.org/10.13039/501100000780 Horizon 2020 Framework Programme 860315European Commission http://dx.doi.org/10.13039/501100000780 Horizon 2020 Framework Programme 860813open accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/7027752026-06-23T12:46:27Z |
| dc.title.none.fl_str_mv |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| title |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| spellingShingle |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation Tolosana Moranchel, Rubén Benchmark Databases DeepFakes Face manipulation Fake detection Fake news Media forensics Telecomunicaciones |
| title_short |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| title_full |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| title_fullStr |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| title_full_unstemmed |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| title_sort |
DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation |
| dc.creator.none.fl_str_mv |
Tolosana Moranchel, Rubén Romero Tapiador, Sergio Gonzalez Sosa Vera Rodríguez, Rubén Fiérrez Aguilar, Julián |
| author |
Tolosana Moranchel, Rubén |
| author_facet |
Tolosana Moranchel, Rubén Romero Tapiador, Sergio Gonzalez Sosa Vera Rodríguez, Rubén Fiérrez Aguilar, Julián |
| author_role |
author |
| author2 |
Romero Tapiador, Sergio Gonzalez Sosa Vera Rodríguez, Rubén Fiérrez Aguilar, Julián |
| author2_role |
author author author author |
| dc.contributor.none.fl_str_mv |
Departamento de Tecnología Electrónica y de las Comunicaciones Escuela Politécnica Superior |
| dc.subject.none.fl_str_mv |
Benchmark Databases DeepFakes Face manipulation Fake detection Fake news Media forensics Telecomunicaciones |
| topic |
Benchmark Databases DeepFakes Face manipulation Fake detection Fake news Media forensics Telecomunicaciones |
| description |
Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning |
| publishDate |
2022 |
| dc.date.none.fl_str_mv |
2022 2022-04-01 |
| dc.type.none.fl_str_mv |
research article http://purl.org/coar/resource_type/c_2df8fbb1 VoR http://purl.org/coar/version/c_970fb48d4fbd8a85 |
| dc.type.openaire.fl_str_mv |
info:eu-repo/semantics/article |
| format |
article |
| dc.identifier.none.fl_str_mv |
http://hdl.handle.net/10486/702775 https://dx.doi.org/10.1016/j.engappai.2022.104673 |
| url |
http://hdl.handle.net/10486/702775 https://dx.doi.org/10.1016/j.engappai.2022.104673 |
| dc.language.none.fl_str_mv |
Inglés eng |
| language_invalid_str_mv |
Inglés |
| language |
eng |
| dc.relation.none.fl_str_mv |
European Commission http://dx.doi.org/10.13039/501100000780 Horizon 2020 Framework Programme 860315 European Commission http://dx.doi.org/10.13039/501100000780 Horizon 2020 Framework Programme 860813 |
| dc.rights.none.fl_str_mv |
open access http://purl.org/coar/access_right/c_abf2 |
| dc.rights.openaire.fl_str_mv |
info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
open access http://purl.org/coar/access_right/c_abf2 |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf |
| dc.publisher.none.fl_str_mv |
Elsevier |
| publisher.none.fl_str_mv |
Elsevier |
| dc.source.none.fl_str_mv |
reponame:Biblos-e Archivo. Repositorio Institucional de la UAM instname:Universidad Autónoma de Madrid |
| instname_str |
Universidad Autónoma de Madrid |
| reponame_str |
Biblos-e Archivo. Repositorio Institucional de la UAM |
| collection |
Biblos-e Archivo. Repositorio Institucional de la UAM |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869415186566217728 |
| score |
15.300724 |