Operationalizing and automating data governance

The ability to cross data from multiple sources represents a competitive advantage for organizations. Yet, the governance of the data lifecycle, from the data sources into valuable insights, is largely performed in an ad-hoc or manual manner. This is specifically concerning in scenarios where tens o...

Descripción completa

Detalles Bibliográficos
Autores: Nadal Francesch, Sergi|||0000-0002-8565-952X, Jovanovic, Petar|||0000-0003-4635-6646, Bilalli, Besim|||0000-0002-0575-2389, Romero Moral, Óscar|||0000-0001-6350-8328
Tipo de recurso: artículo
Fecha de publicación:2022
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/378233
Acceso en línea:https://hdl.handle.net/2117/378233
https://dx.doi.org/10.1186/s40537-022-00673-5
Access Level:acceso abierto
Palabra clave:Big data
Metadata
Data governance
Data integration
Dades massives
Metadades
Àrees temàtiques de la UPC::Informàtica::Sistemes d'informació
id ES_f4ad4bf8acd2a711f665d014ee92ee83
oai_identifier_str oai:upcommons.upc.edu:2117/378233
network_acronym_str ES
network_name_str España
repository_id_str
spelling Operationalizing and automating data governanceNadal Francesch, Sergi|||0000-0002-8565-952XJovanovic, Petar|||0000-0003-4635-6646Bilalli, Besim|||0000-0002-0575-2389Romero Moral, Óscar|||0000-0001-6350-8328Big dataMetadataData governanceData integrationDades massivesMetadadesÀrees temàtiques de la UPC::Informàtica::Sistemes d'informacióThe ability to cross data from multiple sources represents a competitive advantage for organizations. Yet, the governance of the data lifecycle, from the data sources into valuable insights, is largely performed in an ad-hoc or manual manner. This is specifically concerning in scenarios where tens or hundreds of continuously evolving data sources produce semi-structured data. To overcome this challenge, we develop a framework for operationalizing and automating data governance. For the first, we propose a zoned data lake architecture and a set of data governance processes that allow the systematic ingestion, transformation and integration of data from heterogeneous sources, in order to make them readily available for business users. For the second, we propose a set of metadata artifacts that allow the automatic execution of data governance processes, addressing a wide range of data management challenges. We showcase the usefulness of the proposed approach using a real world use case, stemming from the collaborative project with the World Health Organization for the management and analysis of data about Neglected Tropical Diseases. Overall, this work contributes on facilitating organizations the adoption of data-driven strategies into a cohesive framework operationalizing and automating data governance.This work was partly supported by the DOGO4ML project, funded by the Spanish Ministerio de Ciencia e Innovación under project PID2020-117191RB-I00/AEI/10.13039/501100011033. Sergi Nadal is partly supported by the Spanish Ministerio de Ciencia e Innovación, as well as the European Union - NextGenerationEU, under project FJC2020-045809-I/AEI/10.13039/501100011033.Peer ReviewedSpringer Nature20222022-12-1020222022-12-15journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/378233https://dx.doi.org/10.1186/s40537-022-00673-5reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)InglésengAgencia Estatal de Investigación http://doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-117191RB-I00 DESARROLLO, OPERATIVA Y GOBERNANZA DE DATOS PARA SISTEMAS SOFTWARE BASADOS EN APRENDIZAJE AUTOMATICOopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3782332026-05-27T15:37:01Z
dc.title.none.fl_str_mv Operationalizing and automating data governance
title Operationalizing and automating data governance
spellingShingle Operationalizing and automating data governance
Nadal Francesch, Sergi|||0000-0002-8565-952X
Big data
Metadata
Data governance
Data integration
Dades massives
Metadades
Àrees temàtiques de la UPC::Informàtica::Sistemes d'informació
title_short Operationalizing and automating data governance
title_full Operationalizing and automating data governance
title_fullStr Operationalizing and automating data governance
title_full_unstemmed Operationalizing and automating data governance
title_sort Operationalizing and automating data governance
dc.creator.none.fl_str_mv Nadal Francesch, Sergi|||0000-0002-8565-952X
Jovanovic, Petar|||0000-0003-4635-6646
Bilalli, Besim|||0000-0002-0575-2389
Romero Moral, Óscar|||0000-0001-6350-8328
author Nadal Francesch, Sergi|||0000-0002-8565-952X
author_facet Nadal Francesch, Sergi|||0000-0002-8565-952X
Jovanovic, Petar|||0000-0003-4635-6646
Bilalli, Besim|||0000-0002-0575-2389
Romero Moral, Óscar|||0000-0001-6350-8328
author_role author
author2 Jovanovic, Petar|||0000-0003-4635-6646
Bilalli, Besim|||0000-0002-0575-2389
Romero Moral, Óscar|||0000-0001-6350-8328
author2_role author
author
author
dc.subject.none.fl_str_mv Big data
Metadata
Data governance
Data integration
Dades massives
Metadades
Àrees temàtiques de la UPC::Informàtica::Sistemes d'informació
topic Big data
Metadata
Data governance
Data integration
Dades massives
Metadades
Àrees temàtiques de la UPC::Informàtica::Sistemes d'informació
description The ability to cross data from multiple sources represents a competitive advantage for organizations. Yet, the governance of the data lifecycle, from the data sources into valuable insights, is largely performed in an ad-hoc or manual manner. This is specifically concerning in scenarios where tens or hundreds of continuously evolving data sources produce semi-structured data. To overcome this challenge, we develop a framework for operationalizing and automating data governance. For the first, we propose a zoned data lake architecture and a set of data governance processes that allow the systematic ingestion, transformation and integration of data from heterogeneous sources, in order to make them readily available for business users. For the second, we propose a set of metadata artifacts that allow the automatic execution of data governance processes, addressing a wide range of data management challenges. We showcase the usefulness of the proposed approach using a real world use case, stemming from the collaborative project with the World Health Organization for the management and analysis of data about Neglected Tropical Diseases. Overall, this work contributes on facilitating organizations the adoption of data-driven strategies into a cohesive framework operationalizing and automating data governance.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022-12-10
2022
2022-12-15
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/378233
https://dx.doi.org/10.1186/s40537-022-00673-5
url https://hdl.handle.net/2117/378233
https://dx.doi.org/10.1186/s40537-022-00673-5
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-117191RB-I00 DESARROLLO, OPERATIVA Y GOBERNANZA DE DATOS PARA SISTEMAS SOFTWARE BASADOS EN APRENDIZAJE AUTOMATICO
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer Nature
publisher.none.fl_str_mv Springer Nature
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869424493719453696
score 15,300724

Similares en LA Referencia