Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model

This book constitutes the refereed proceedings of the 14th International Conference on the Theory and Application of Diagrams, Diagrams 2024, held in Münster, Germany, during September 27–October 1, 2024.

Detalles Bibliográficos
Autores: Schorlemmer, Marco, Ballout, Mohamad, Kühnberger, Kai-Uwe
Tipo de recurso: otro
Estado:Versión publicada
Fecha de publicación:2024
País:España
Institución:Consejo Superior de Investigaciones Científicas (CSIC)
Repositorio:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/378131
Acceso en línea:http://hdl.handle.net/10261/378131
https://api.elsevier.com/content/abstract/scopus_id/85204575273
Access Level:acceso abierto
Palabra clave:Diagram understanding
Euler diagram
Region connection calculus
Transformer-based language model
id ES_98d1869019ec9b0de0db1bc8d39f2ea3
oai_identifier_str oai:digital.csic.es:10261/378131
network_acronym_str ES
network_name_str España
repository_id_str
spelling Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language ModelSchorlemmer, MarcoBallout, MohamadKühnberger, Kai-UweDiagram understandingEuler diagramRegion connection calculusTransformer-based language modelThis book constitutes the refereed proceedings of the 14th International Conference on the Theory and Application of Diagrams, Diagrams 2024, held in Münster, Germany, during September 27–October 1, 2024.To address the task of diagram understanding we propose to distinguish between the perception of the geometric configuration of a diagram from the assignment of meaning to the geometric entities and their topological relationships. As a consequence, diagram parsing does not need to assume any particular a priori interpretations of diagrams and their constituents. Focussing on Euler diagrams, we tackle the first of these subtasks—that of identifying the geometric entities that constitute a diagram (i.e., circles, rectangles, lines, arrows, etc.) and their topological relations—as an image captioning task, using a Vision Transformer for image recognition combined with language model GPT-2 to generate qualitative spatial descriptions of Euler diagrams with an encoder-decoder model. Due to the lack of sufficient high-quality data to train the pre-trained language model for this task, we describe how we generated a synthetic dataset of Euler diagrams annotated with qualitative spatial representations based on the Region Connection Calculus (RCC8). Results showed over 95% accuracy of the transformer-based language model in the generation of meaning-carrying RCC8 specifications for given Euler diagrams.This work has been carried out during a research stay by Marco Schorlemmer at the Institute of Cognitive Science of Osnabrück University, funded with a scholarship granted by the Deutscher Akademischer Austauschdiest (DAAD) for Research Stays for University Academics and Scientists (no. 91881609). The cluster used to train the models was funded by the Deutsche Forschungsgemeinschaft (DFG, no. 456666331). The preliminary work that led to this research stay was supported by project CORPORIS (no. PID2019-109677RB-I00) funded by Spain’s Agencia Estatal de Investigación (AEI) and by a grant for Grups de recerca consolidats (no. 2021 SGR 00754) funded by Catalonia’s Agència de Gestió d’Ajuts Universitaris i de Recerca (AGAUR).Peer reviewedSpringer NatureGerman Research FoundationAgencia Estatal de Investigación (España)Generalitat de CatalunyaAgència de Gestió d'Ajuts Universitaris i de RecercaSchorlemmer, Marco [0000-0002-9591-3325]Kühnberger, Kai Uwe [0000-0003-1626-0598]Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]202520252024info:eu-repo/semantics/otherhttp://purl.org/coar/resource_type/c_3248Publisher's versioninfo:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/bookPartapplication/pdfhttp://hdl.handle.net/10261/378131https://api.elsevier.com/content/abstract/scopus_id/85204575273reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)Inglés#PLACEHOLDER_PARENT_METADATA_VALUE#info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2019-109677RB-I00Lecture Notes in Computer Science14981Lecture Notes in Artificial Intelligencehttps://doi.org/10.1007/978-3-031-71291-3_5Síinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/3781312026-05-22T06:33:51Z
dc.title.none.fl_str_mv Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
title Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
spellingShingle Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
Schorlemmer, Marco
Diagram understanding
Euler diagram
Region connection calculus
Transformer-based language model
title_short Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
title_full Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
title_fullStr Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
title_full_unstemmed Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
title_sort Generating Qualitative Descriptions of Diagrams with a Transformer-Based Language Model
dc.creator.none.fl_str_mv Schorlemmer, Marco
Ballout, Mohamad
Kühnberger, Kai-Uwe
author Schorlemmer, Marco
author_facet Schorlemmer, Marco
Ballout, Mohamad
Kühnberger, Kai-Uwe
author_role author
author2 Ballout, Mohamad
Kühnberger, Kai-Uwe
author2_role author
author
dc.contributor.none.fl_str_mv German Research Foundation
Agencia Estatal de Investigación (España)
Generalitat de Catalunya
Agència de Gestió d'Ajuts Universitaris i de Recerca
Schorlemmer, Marco [0000-0002-9591-3325]
Kühnberger, Kai Uwe [0000-0003-1626-0598]
Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]
dc.subject.none.fl_str_mv Diagram understanding
Euler diagram
Region connection calculus
Transformer-based language model
topic Diagram understanding
Euler diagram
Region connection calculus
Transformer-based language model
description This book constitutes the refereed proceedings of the 14th International Conference on the Theory and Application of Diagrams, Diagrams 2024, held in Münster, Germany, during September 27–October 1, 2024.
publishDate 2024
dc.date.none.fl_str_mv 2024
2025
2025
dc.type.none.fl_str_mv info:eu-repo/semantics/other
http://purl.org/coar/resource_type/c_3248
Publisher's version
info:eu-repo/semantics/publishedVersion
dc.type.openaire.fl_str_mv info:eu-repo/semantics/bookPart
format other
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/378131
https://api.elsevier.com/content/abstract/scopus_id/85204575273
url http://hdl.handle.net/10261/378131
https://api.elsevier.com/content/abstract/scopus_id/85204575273
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv #PLACEHOLDER_PARENT_METADATA_VALUE#
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2019-109677RB-I00
Lecture Notes in Computer Science
14981
Lecture Notes in Artificial Intelligence
https://doi.org/10.1007/978-3-031-71291-3_5
Sí
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Springer Nature
publisher.none.fl_str_mv Springer Nature
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869414223728082944
score 15,812429

Similares en LA Referencia