An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms

Efficient management and analysis of large volumes of digital data has emerged as a major challenge in the field of digital forensics. To quickly identify and analyze relevant artifacts within large datasets, we introduce APOTHEOSIS, an approximate similarity search system designed for scalability a...

Descripción completa

Detalles Bibliográficos
Autores: Huici, Daniel, Rodríguez, Ricardo J., Mena, Eduardo
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2025
País:España
Institución:Universidad de Zaragoza
Repositorio:Zaguán. Repositorio Digital de la Universidad de Zaragoza
OAI Identifier:oai:zaguan.unizar.es:165093
Acceso en línea:http://zaguan.unizar.es/record/165093
Access Level:acceso abierto
id ES_d4e0145d567d6b321f6d98a7cf7ea8e0
oai_identifier_str oai:zaguan.unizar.es:165093
network_acronym_str ES
network_name_str España
repository_id_str
spelling An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithmsHuici, DanielRodríguez, Ricardo J.Mena, EduardoEfficient management and analysis of large volumes of digital data has emerged as a major challenge in the field of digital forensics. To quickly identify and analyze relevant artifacts within large datasets, we introduce APOTHEOSIS, an approximate similarity search system designed for scalability and efficiency. Our system integrates approximate search techniques (which allow searching for a match on a close value) with Similarity Digest Algorithms (SDA; which capture common features between similar elements), using a space-saving radix tree and a graph-based hierarchical navigable small world structure to perform fast approximate nearest neighbor searches. We demonstrate the effectiveness and versatility of our system through two key case studies: first, in plagiarism detection, demonstrating the effectiveness of our system in identifying similar or duplicate documents within a large source code dataset; then, in memory artifact detection, showing its scalability and performance in processing large-scale forensic data collected from various versions of Microsoft Windows. Our comprehensive evaluation shows that APOTHEOSIS not only efficiently handles large datasets, but also provides a way to evaluate the performance of various SDA and their approximate similarity search in different forensic scenarios.2025info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttp://zaguan.unizar.es/record/165093reponame:Zaguán. Repositorio Digital de la Universidad de Zaragozainstname:Universidad de ZaragozaInglésinfo:eu-repo/grantAgreement/ES/AEI/PID2020-113903RB-I00info:eu-repo/grantAgreement/ES/DGA/T21-23Rinfo:eu-repo/grantAgreement/ES/DGA/T42-23Rinfo:eu-repo/grantAgreement/ES/MCIU/PID2023-151467OA-I00info:eu-repo/grantAgreement/EUR/MICINN/TED2021-131115A-I00info:eu-repo/semantics/openAccessoai:zaguan.unizar.es:1650932026-05-29T13:59:51Z
dc.title.none.fl_str_mv An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
title An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
spellingShingle An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
Huici, Daniel
title_short An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
title_full An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
title_fullStr An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
title_full_unstemmed An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
title_sort An extensible and scalable system for hash lookup and approximate similarity search with similarity digest algorithms
dc.creator.none.fl_str_mv Huici, Daniel
Rodríguez, Ricardo J.
Mena, Eduardo
author Huici, Daniel
author_facet Huici, Daniel
Rodríguez, Ricardo J.
Mena, Eduardo
author_role author
author2 Rodríguez, Ricardo J.
Mena, Eduardo
author2_role author
author
description Efficient management and analysis of large volumes of digital data has emerged as a major challenge in the field of digital forensics. To quickly identify and analyze relevant artifacts within large datasets, we introduce APOTHEOSIS, an approximate similarity search system designed for scalability and efficiency. Our system integrates approximate search techniques (which allow searching for a match on a close value) with Similarity Digest Algorithms (SDA; which capture common features between similar elements), using a space-saving radix tree and a graph-based hierarchical navigable small world structure to perform fast approximate nearest neighbor searches. We demonstrate the effectiveness and versatility of our system through two key case studies: first, in plagiarism detection, demonstrating the effectiveness of our system in identifying similar or duplicate documents within a large source code dataset; then, in memory artifact detection, showing its scalability and performance in processing large-scale forensic data collected from various versions of Microsoft Windows. Our comprehensive evaluation shows that APOTHEOSIS not only efficiently handles large datasets, but also provides a way to evaluate the performance of various SDA and their approximate similarity search in different forensic scenarios.
publishDate 2025
dc.date.none.fl_str_mv 2025
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://zaguan.unizar.es/record/165093
url http://zaguan.unizar.es/record/165093
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv info:eu-repo/grantAgreement/ES/AEI/PID2020-113903RB-I00
info:eu-repo/grantAgreement/ES/DGA/T21-23R
info:eu-repo/grantAgreement/ES/DGA/T42-23R
info:eu-repo/grantAgreement/ES/MCIU/PID2023-151467OA-I00
info:eu-repo/grantAgreement/EUR/MICINN/TED2021-131115A-I00
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv
publisher.none.fl_str_mv
dc.source.none.fl_str_mv reponame:Zaguán. Repositorio Digital de la Universidad de Zaragoza
instname:Universidad de Zaragoza
instname_str Universidad de Zaragoza
reponame_str Zaguán. Repositorio Digital de la Universidad de Zaragoza
collection Zaguán. Repositorio Digital de la Universidad de Zaragoza
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869420584696283136
score 15.811543

Similares en LA Referencia