Automatic detection of data and concept drift in ML systems for chess broadcasting

As machine learning systems evolve, they face the challenge of operating in dynamic environments where model performance can degrade over time due to model drift. Therefore, the work continues once the model is in production; continuous monitoring and adjustment are essential to maintain optimal per...

Descripción completa

Detalles Bibliográficos
Autor: Moure Alassio, Ximena Graciana
Tipo de recurso: tesis de maestría
Fecha de publicación:2024
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/419868
Acceso en línea:https://hdl.handle.net/2117/419868
Access Level:acceso abierto
Palabra clave:SQL (Computer program language)
Machine learning
deriva de dades
deriva de conceptes
codificadors automàtics
aprenentatge automàtic
SQL
NoSQL
data drift
concept drift
autoencoders
machine learning
SQL (Llenguatge de programació)
Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
id ES_ecea3aa5d5577f6854598f546ffaebee
oai_identifier_str oai:upcommons.upc.edu:2117/419868
network_acronym_str ES
network_name_str España
repository_id_str
spelling Automatic detection of data and concept drift in ML systems for chess broadcastingMoure Alassio, Ximena GracianaSQL (Computer program language)Machine learningderiva de dadesderiva de conceptescodificadors automàticsaprenentatge automàticSQLNoSQLdata driftconcept driftautoencodersmachine learningSQL (Llenguatge de programació)Aprenentatge automàticÀrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàticAs machine learning systems evolve, they face the challenge of operating in dynamic environments where model performance can degrade over time due to model drift. Therefore, the work continues once the model is in production; continuous monitoring and adjustment are essential to maintain optimal performance. As a model receives new data, it might exhibit a different probability distribution than the data used for training, necessitating strategies to detect and address drift effectively. This thesis addresses these challenges within a chess broadcasting system developed by the GESSI research group. This system leverages machine learning for real-time chess game state detection. Despite its capabilities, it is not immune to data and concept drift challenges, which can diminish its effectiveness in production environments. Moreover, this work aims to enhance the efficiency and scalability of the chess game state recognition system by optimizing both the data storage solution and the processing pipeline. These enhancements enable the system to perform consistently and reliably in production, with improved data accessibility and streamlined operations. To address these issues, this thesis presents three main contributions: (1) an enhanced pipeline for image validation that reduces human error and increases efficiency, (2) the migration of the existing data storage from CSV files to a MongoDB database, optimizing scalability and data accessibility, and (3) the integration of drift detection mechanisms, utilizing both autoencoders and production model embeddings, to monitor and mitigate drift in the system's pipeline. Through this research, we demonstrate that embedding-based drift detection is viable for high-dimensional data such as images. Furthermore, we show that the production model can serve as a reliable and resource-efficient alternative to custom autoencoders for feature extraction, offering comparable performance in drift detection tasks. Our findings highlight the value of embedding-based drift detection for unstructured data, with applications extending beyond chess broadcasting to other machine learning systems in dynamic domains.Universitat Politècnica de CatalunyaRey Juárez, Santiago delMartínez Fernández, Silverio Juan20242024-10-2320242024-12-04master thesishttp://purl.org/coar/resource_type/c_bdccNAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/masterThesisapplication/pdfhttps://hdl.handle.net/2117/419868reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/4198682026-05-27T15:37:01Z
dc.title.none.fl_str_mv Automatic detection of data and concept drift in ML systems for chess broadcasting
title Automatic detection of data and concept drift in ML systems for chess broadcasting
spellingShingle Automatic detection of data and concept drift in ML systems for chess broadcasting
Moure Alassio, Ximena Graciana
SQL (Computer program language)
Machine learning
deriva de dades
deriva de conceptes
codificadors automàtics
aprenentatge automàtic
SQL
NoSQL
data drift
concept drift
autoencoders
machine learning
SQL (Llenguatge de programació)
Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
title_short Automatic detection of data and concept drift in ML systems for chess broadcasting
title_full Automatic detection of data and concept drift in ML systems for chess broadcasting
title_fullStr Automatic detection of data and concept drift in ML systems for chess broadcasting
title_full_unstemmed Automatic detection of data and concept drift in ML systems for chess broadcasting
title_sort Automatic detection of data and concept drift in ML systems for chess broadcasting
dc.creator.none.fl_str_mv Moure Alassio, Ximena Graciana
author Moure Alassio, Ximena Graciana
author_facet Moure Alassio, Ximena Graciana
author_role author
dc.contributor.none.fl_str_mv Rey Juárez, Santiago del
Martínez Fernández, Silverio Juan
dc.subject.none.fl_str_mv SQL (Computer program language)
Machine learning
deriva de dades
deriva de conceptes
codificadors automàtics
aprenentatge automàtic
SQL
NoSQL
data drift
concept drift
autoencoders
machine learning
SQL (Llenguatge de programació)
Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
topic SQL (Computer program language)
Machine learning
deriva de dades
deriva de conceptes
codificadors automàtics
aprenentatge automàtic
SQL
NoSQL
data drift
concept drift
autoencoders
machine learning
SQL (Llenguatge de programació)
Aprenentatge automàtic
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
description As machine learning systems evolve, they face the challenge of operating in dynamic environments where model performance can degrade over time due to model drift. Therefore, the work continues once the model is in production; continuous monitoring and adjustment are essential to maintain optimal performance. As a model receives new data, it might exhibit a different probability distribution than the data used for training, necessitating strategies to detect and address drift effectively. This thesis addresses these challenges within a chess broadcasting system developed by the GESSI research group. This system leverages machine learning for real-time chess game state detection. Despite its capabilities, it is not immune to data and concept drift challenges, which can diminish its effectiveness in production environments. Moreover, this work aims to enhance the efficiency and scalability of the chess game state recognition system by optimizing both the data storage solution and the processing pipeline. These enhancements enable the system to perform consistently and reliably in production, with improved data accessibility and streamlined operations. To address these issues, this thesis presents three main contributions: (1) an enhanced pipeline for image validation that reduces human error and increases efficiency, (2) the migration of the existing data storage from CSV files to a MongoDB database, optimizing scalability and data accessibility, and (3) the integration of drift detection mechanisms, utilizing both autoencoders and production model embeddings, to monitor and mitigate drift in the system's pipeline. Through this research, we demonstrate that embedding-based drift detection is viable for high-dimensional data such as images. Furthermore, we show that the production model can serve as a reliable and resource-efficient alternative to custom autoencoders for feature extraction, offering comparable performance in drift detection tasks. Our findings highlight the value of embedding-based drift detection for unstructured data, with applications extending beyond chess broadcasting to other machine learning systems in dynamic domains.
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-10-23
2024
2024-12-04
dc.type.none.fl_str_mv master thesis
http://purl.org/coar/resource_type/c_bdcc
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/419868
url https://hdl.handle.net/2117/419868
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Universitat Politècnica de Catalunya
publisher.none.fl_str_mv Universitat Politècnica de Catalunya
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869423381526347776
score 15,81155

Similares en LA Referencia