Bringing order to approximate matching: Classification and attacks on similarity digest algorithms

Fuzzy hashing or similarity hashing (a.k.a. bytewise approximate matching) converts digital artifacts into an intermediate representation to allow an efficient (fast) identification of similar objects, e.g., for blacklisting. They gained a lot of popularity over the past decade with new algorithms b...

ver descrição completa

Detalhes bibliográficos
Autores: Martín-Pérez, M., Rodríguez, R.J., Breitinger, F.
Formato: artículo
Estado:Versión publicada
Fecha de publicación:2021
País:España
Recursos:Universidad de Zaragoza
Repositorio:Zaguán. Repositorio Digital de la Universidad de Zaragoza
OAI Identifier:oai:zaguan.unizar.es:151313
Acesso em linha:http://zaguan.unizar.es/record/151313
Access Level:acceso abierto
id ES_5e4c006d2cf7e5b576efbb23ae7a85c6
oai_identifier_str oai:zaguan.unizar.es:151313
network_acronym_str ES
network_name_str España
repository_id_str
spelling Bringing order to approximate matching: Classification and attacks on similarity digest algorithmsMartín-Pérez, M.Rodríguez, R.J.Breitinger, F.Fuzzy hashing or similarity hashing (a.k.a. bytewise approximate matching) converts digital artifacts into an intermediate representation to allow an efficient (fast) identification of similar objects, e.g., for blacklisting. They gained a lot of popularity over the past decade with new algorithms being developed and released to the digital forensics community. When releasing algorithms (e.g., as part of a scientific article), they are frequently compared with other algorithms to outline the benefits and sometimes also the weaknesses of the proposed approach. However, given the wide variety of algorithms and approaches, it is impossible to provide direct comparisons with all existing algorithms. In this paper, we present the first classification of approximate matching algorithms which allows an easier description and comparisons. Therefore, we first reviewed existing literature to understand the techniques various algorithms use and to familiarize ourselves with the common terminology. Our findings allowed us to develop a categorization relying heavily on the terminology proposed by NIST SP 800-168. In addition to the categorization, this article presents an abstract set of attacks against algorithms and why they are feasible. Lastly, we detail the characteristics needed to build robust algorithms to prevent attacks. We believe that this article helps newcomers, practitioners, and experts alike to better compare algorithms, understand their potential, as well as characteristics and implications they may have on forensic investigations.2021info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttp://zaguan.unizar.es/record/151313reponame:Zaguán. Repositorio Digital de la Universidad de Zaragozainstname:Universidad de ZaragozaInglésinfo:eu-repo/grantAgreement/ES/DGA/T21-20R-DISCOinfo:eu-repo/grantAgreement/ES/MICIU/Medrese-RTI2018-098543-B-I00info:eu-repo/grantAgreement/ES/MINECO-INCIBE/INCIBEC-2015-02486info:eu-repo/grantAgreement/ES/MINECO-INCIBE/INCIBEI-2015-27300info:eu-repo/semantics/openAccessoai:zaguan.unizar.es:1513132026-05-29T13:59:51Z
dc.title.none.fl_str_mv Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
title Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
spellingShingle Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
Martín-Pérez, M.
title_short Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
title_full Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
title_fullStr Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
title_full_unstemmed Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
title_sort Bringing order to approximate matching: Classification and attacks on similarity digest algorithms
dc.creator.none.fl_str_mv Martín-Pérez, M.
Rodríguez, R.J.
Breitinger, F.
author Martín-Pérez, M.
author_facet Martín-Pérez, M.
Rodríguez, R.J.
Breitinger, F.
author_role author
author2 Rodríguez, R.J.
Breitinger, F.
author2_role author
author
description Fuzzy hashing or similarity hashing (a.k.a. bytewise approximate matching) converts digital artifacts into an intermediate representation to allow an efficient (fast) identification of similar objects, e.g., for blacklisting. They gained a lot of popularity over the past decade with new algorithms being developed and released to the digital forensics community. When releasing algorithms (e.g., as part of a scientific article), they are frequently compared with other algorithms to outline the benefits and sometimes also the weaknesses of the proposed approach. However, given the wide variety of algorithms and approaches, it is impossible to provide direct comparisons with all existing algorithms. In this paper, we present the first classification of approximate matching algorithms which allows an easier description and comparisons. Therefore, we first reviewed existing literature to understand the techniques various algorithms use and to familiarize ourselves with the common terminology. Our findings allowed us to develop a categorization relying heavily on the terminology proposed by NIST SP 800-168. In addition to the categorization, this article presents an abstract set of attacks against algorithms and why they are feasible. Lastly, we detail the characteristics needed to build robust algorithms to prevent attacks. We believe that this article helps newcomers, practitioners, and experts alike to better compare algorithms, understand their potential, as well as characteristics and implications they may have on forensic investigations.
publishDate 2021
dc.date.none.fl_str_mv 2021
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://zaguan.unizar.es/record/151313
url http://zaguan.unizar.es/record/151313
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv info:eu-repo/grantAgreement/ES/DGA/T21-20R-DISCO
info:eu-repo/grantAgreement/ES/MICIU/Medrese-RTI2018-098543-B-I00
info:eu-repo/grantAgreement/ES/MINECO-INCIBE/INCIBEC-2015-02486
info:eu-repo/grantAgreement/ES/MINECO-INCIBE/INCIBEI-2015-27300
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv
publisher.none.fl_str_mv
dc.source.none.fl_str_mv reponame:Zaguán. Repositorio Digital de la Universidad de Zaragoza
instname:Universidad de Zaragoza
instname_str Universidad de Zaragoza
reponame_str Zaguán. Repositorio Digital de la Universidad de Zaragoza
collection Zaguán. Repositorio Digital de la Universidad de Zaragoza
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869409105611849728
score 15,811543

Registos relacionados