Template extraction for question answer generation using an image knowledge base

Deep Learning (DL) has been key to solve complex tasks in the last years. To train DL models, vast amounts of labeled data are required. Visual Question Answering (VQA) is a task in which a question in natural language about an image is asked to a system and the system has to answer the question. To...

Descripción completa

Detalles Bibliográficos
Autor: Fadic-Gutiérrez, Miguel Osvaldo
Tipo de recurso: tesis de maestría
Estado:Versión publicada
Fecha de publicación:2019
País:Chile
OAI Identifier:oai:repositorio.anid.cl:10533/236559
Acceso en línea:https://hdl.handle.net/10533/236559
Access Level:acceso abierto
Palabra clave:Ingeniería y Tecnología
id CL_2d2f97f40201fcf9e483f43bb71128a5
oai_identifier_str oai:repositorio.anid.cl:10533/236559
network_acronym_str CL
network_name_str Chile
repository_id_str
dc.title.es_CL.fl_str_mv Template extraction for question answer generation using an image knowledge base
title Template extraction for question answer generation using an image knowledge base
spellingShingle Template extraction for question answer generation using an image knowledge base
Fadic-Gutiérrez, Miguel Osvaldo
Ingeniería y Tecnología
title_short Template extraction for question answer generation using an image knowledge base
title_full Template extraction for question answer generation using an image knowledge base
title_fullStr Template extraction for question answer generation using an image knowledge base
title_full_unstemmed Template extraction for question answer generation using an image knowledge base
title_sort Template extraction for question answer generation using an image knowledge base
dc.creator.none.fl_str_mv Fadic-Gutiérrez, Miguel Osvaldo
author Fadic-Gutiérrez, Miguel Osvaldo
author_facet Fadic-Gutiérrez, Miguel Osvaldo
author_role author
dc.contributor.advisor.none.fl_str_mv Soto-Arriaza, Álvaro
Baier-Aranda, Jorge Andrés
dc.contributor.institution.es_CL.fl_str_mv PONTIFICIA UNIVERSIDAD CATOLICA DE CHILE
dc.subject.oecd1n.es_CL.fl_str_mv Ingeniería y Tecnología
topic Ingeniería y Tecnología
description Deep Learning (DL) has been key to solve complex tasks in the last years. To train DL models, vast amounts of labeled data are required. Visual Question Answering (VQA) is a task in which a question in natural language about an image is asked to a system and the system has to answer the question. To apply Deep Learning to VQA, a dataset with hundreds of thousands of images, questions about the images and their answer is needed. We propose a method to automatically obtain question-answer templates that can be used to generate questions and their answer given a knowledge base about an image with information about the objects that appears in it, their attributes and the relations between them. Our method generates two orders of magnitude bigger datasets than current human annotated ones. We find in our experiments that the most suitable strategy to use such a big dataset is to train a DL model using the generated dataset and then apply fine-tuning using the target dataset. To evaluate our generated question-answers we train models using only the training set of VQA and VQA v2 datasets and models using the fine-tune technique over our dataset. The use of our dataset improves the accuracy of What is ... and Who is .. type of questions by 2.25 and 1.02 percent points respectively in VQA and by 0.73 and 4.91 percent points respectively in VQA v2.
publishDate 2019
dc.date.accessioned.none.fl_str_mv 2019-09-16T11:59:35Z
2022-08-18T14:13:53Z
dc.date.available.none.fl_str_mv 2019-09-16T11:59:35Z
2022-08-18T14:13:53Z
dc.date.issued.es_CL.fl_str_mv 2019
dc.type.none.fl_str_mv Tesis Magíster
dc.type.driver.es_CL.fl_str_mv info:eu-repo/semantics/masterThesis
dc.type.openaire.none.fl_str_mv info:eu-repo/semantics/publishedVersion
dc.type.tesis.none.fl_str_mv Tesis
format masterThesis
status_str publishedVersion
dc.identifier.folio.es_CL.fl_str_mv 22161259
dc.identifier.uri.none.fl_str_mv https://hdl.handle.net/10533/236559
identifier_str_mv 22161259
url https://hdl.handle.net/10533/236559
dc.relation.none.fl_str_mv instname: Conicyt
reponame: Repositorio Digital RI2.0
dc.relation.projectid.es_CL.fl_str_mv info:eu-repo/grantAgreement//22161259
dc.relation.set.none.fl_str_mv info:eu-repo/semantics/dataset/hdl.handle.net/10533/93488
dc.rights.es_CL.fl_str_mv info:eu-repo/semantics/openAccess
dc.rights.*.fl_str_mv CC0 1.0 Universal
dc.rights.uri.*.fl_str_mv http://creativecommons.org/publicdomain/zero/1.0/
eu_rights_str_mv openAccess
rights_invalid_str_mv CC0 1.0 Universal
http://creativecommons.org/publicdomain/zero/1.0/
bitstream.url.fl_str_mv https://repositorio.anid.cl/bitstreams/dcf66052-8e62-468e-8483-6b7b6f6fed95/download
https://repositorio.anid.cl/bitstreams/754422f6-ed46-4b4b-800d-3aabafcd36f8/download
https://repositorio.anid.cl/bitstreams/93fde366-ab65-4757-a7cb-f527fc703742/download
https://repositorio.anid.cl/bitstreams/c4bc05b3-ec7a-4187-a9c8-d70c40495a86/download
https://repositorio.anid.cl/bitstreams/de34b531-75d4-4145-b095-7e57a6cc32db/download
bitstream.checksum.fl_str_mv 43646a1ced4f8f2af18edcb25183a239
0a703d871bf062c5fdc7850b1496693b
593a6e7305c66c56041a9f9e15a649c1
fbfd6d726782022fa80826e35db9495d
68f6657ee6f67fadf0b2b013f5488d04
bitstream.checksumAlgorithm.fl_str_mv MD5
MD5
MD5
MD5
MD5
repository.name.fl_str_mv Repositorio ANID
repository.mail.fl_str_mv aletelier@anid.cl
_version_ 1843518566717980672
spelling PONTIFICIA UNIVERSIDAD CATOLICA DE CHILEFadic-Gutiérrez, Miguel Osvaldo2019https://hdl.handle.net/10533/236559http://purl.org/coar/access_right/c_abf2Ingeniería y TecnologíaTemplate extraction for question answer generation using an image knowledge baseSoto-Arriaza, ÁlvaroBaier-Aranda, Jorge AndrésPONTIFICIA UNIVERSIDAD CATOLICA DE CHILEChileFadic-Gutiérrez, Miguel Osvaldo2019-09-16T11:59:35Z2022-08-18T14:13:53Z2019-09-16T11:59:35Z2022-08-18T14:13:53Z2019Deep Learning (DL) has been key to solve complex tasks in the last years. To train DL models, vast amounts of labeled data are required. Visual Question Answering (VQA) is a task in which a question in natural language about an image is asked to a system and the system has to answer the question. To apply Deep Learning to VQA, a dataset with hundreds of thousands of images, questions about the images and their answer is needed. We propose a method to automatically obtain question-answer templates that can be used to generate questions and their answer given a knowledge base about an image with information about the objects that appears in it, their attributes and the relations between them. Our method generates two orders of magnitude bigger datasets than current human annotated ones. We find in our experiments that the most suitable strategy to use such a big dataset is to train a DL model using the generated dataset and then apply fine-tuning using the target dataset. To evaluate our generated question-answers we train models using only the training set of VQA and VQA v2 datasets and models using the fine-tune technique over our dataset. The use of our dataset improves the accuracy of What is ... and Who is .. type of questions by 2.25 and 1.02 percent points respectively in VQA and by 0.73 and 4.91 percent points respectively in VQA v2.PFCHA-BecasPFCHA-Becas22161259https://hdl.handle.net/10533/236559instname: Conicytreponame: Repositorio Digital RI2.0info:eu-repo/grantAgreement//22161259info:eu-repo/semantics/dataset/hdl.handle.net/10533/93488info:eu-repo/semantics/openAccessCC0 1.0 Universalhttp://creativecommons.org/publicdomain/zero/1.0/Ingeniería y TecnologíaTemplate extraction for question answer generation using an image knowledge baseTesis Magísterinfo:eu-repo/semantics/masterThesisinfo:eu-repo/semantics/publishedVersionTesisTesishttps://hdl.handle.net/10533/236559PFCHA-Becasa1c4f844-bfaa-4ac4-8fe8-341deafb4a99virtual::53594-1a1c4f844-bfaa-4ac4-8fe8-341deafb4a99virtual::53594-1ORIGINALThesis Miguel Fadic.pdfThesisapplication/pdf12200497https://repositorio.anid.cl/bitstreams/dcf66052-8e62-468e-8483-6b7b6f6fed95/download43646a1ced4f8f2af18edcb25183a239MD51CC-LICENSElicense_rdfapplication/octet-stream1089https://repositorio.anid.cl/bitstreams/754422f6-ed46-4b4b-800d-3aabafcd36f8/download0a703d871bf062c5fdc7850b1496693bMD52LICENSElicense.txttext/plain1779https://repositorio.anid.cl/bitstreams/93fde366-ab65-4757-a7cb-f527fc703742/download593a6e7305c66c56041a9f9e15a649c1MD53TEXTThesis Miguel Fadic.pdf.txtExtracted texttext/plain68041https://repositorio.anid.cl/bitstreams/c4bc05b3-ec7a-4187-a9c8-d70c40495a86/downloadfbfd6d726782022fa80826e35db9495dMD54THUMBNAILThesis Miguel Fadic.pdf.jpgIM Thumbnailimage/jpeg2206https://repositorio.anid.cl/bitstreams/de34b531-75d4-4145-b095-7e57a6cc32db/download68f6657ee6f67fadf0b2b013f5488d04MD5510533/236559oai:repositorio.anid.cl:10533/2365592023-07-24 17:18:02.26http://creativecommons.org/publicdomain/zero/1.0/info:eu-repo/semantics/openAccesshttps://repositorio.anid.clRepositorio ANIDaletelier@anid.clTGljZW5jaWEgZGUgRGlzdHJpYnVjacOzbiBObyBFeGNsdXNpdmEKCkFsIGZpcm1hciB5IHByZXNlbnRhciBlc3RhIGxpY2VuY2lhICwgdXN0ZWQgKGVsIGF1dG9yKGVzKSBvIHRpdHVsYXIgZGUgZGVyZWNob3MgZGUgYXV0b3IpIGdhcmFudGl6YSBhIGxhIENvbWlzacOzbiBOYWNpb25hbCBkZSBJbnZlc3RpZ2FjacOzbiBDaWVudMOtdGljYSB5IFRlY25vbMOzZ2ljYSAoQ09OSUNZVCkgZWwgZGVyZWNobyBubyBleGNsdXNpdm8gZGUgcmVwcm9kdWNpciAsIHRyYWR1Y2lyIChjb21vIHNlIGRlZmluZSBtw6FzIGFiYWpvKSAsIHkvbyBkaXN0cmlidWlyIHN1IGRvY3VtZW50byAoaW5jbHV5ZW5kbyBlbCByZXN1bWVuKSBlbiB0b2RvIGVsIG11bmRvIGVuIGZvcm1hIGltcHJlc2EgeSBlbiBmb3JtYXRvIGVsZWN0csOzbmljbyB5IGVuIGN1YWxxdWllciBtZWRpbywgaW5jbHV5ZW5kbywgcGVybyBubyBsaW1pdGFkbyBhLCBhdWRpbyBvIHbDrWRlby4KClVzdGVkIGFjZXB0YSBxdWUgQ09OSUNZVCBwdWVkZSwgc2luIGFsdGVyYXIgc3UgY29udGVuaWRvLCBjb252ZXJ0aXJsbyBhIGN1YWxxdWllciBtZWRpbyBvIGZvcm1hdG8gcGFyYSBlbCBmaW4gZGUgbGEgY29uc2VydmFjacOzbi4KClRhbWJpw6luIGFjZXB0YSBxdWUgQ09OSUNZVCBwdWVkZSB0ZW5lciBtw6FzIGRlIHVuYSBjb3BpYSBkZSBlc3RlIGRvY3VtZW50byBwYXJhIGZpbmVzIGRlIHNlZ3VyaWRhZCwgY29waWFzIGRlIHNlZ3VyaWRhZCB5IGNvbnNlcnZhY2nDs24uCgpVc3RlZCBkZWNsYXJhIHF1ZSBlbCBkb2N1bWVudG8gZXMgdW4gdHJhYmFqbyBvcmlnaW5hbCwgeSBxdWUgdXN0ZWQgdGllbmUgZWwgZGVyZWNobyBkZSBvdG9yZ2FyIGxvcyBkZXJlY2hvcyBjb250ZW5pZG9zIGVuIGVzdGEgbGljZW5jaWEuIFRhbWJpw6luIGRlY2xhcmEgcXVlIHN1IHBldGljacOzbiBubywgYSBsbyBtZWpvciBkZSBzdSBjb25vY2ltaWVudG8sIGluZnJpbmdlIGxvcyBkZXJlY2hvcyBkZSBhdXRvciBkZSBuYWRpZS4KClNpIGVsIGRvY3VtZW50byBjb250aWVuZSBtYXRlcmlhbGVzIGRlIGxvcyBxdWUgbm8gdGllbmVuIGRlcmVjaG9zIGRlIGF1dG9yICwgdXN0ZWQgZGVjbGFyYSBxdWUgaGEgb2J0ZW5pZG8gZWwgcGVybWlzbyBzaW4gcmVzdHJpY2Npw7NuIGRlbCBwcm9waWV0YXJpbyBkZSBsb3MgZGVyZWNob3MgcmVxdWVyaWRvcyBwb3IgZXN0YSBsaWNlbmNpYSB5IHF1ZSBlc2UgbWF0ZXJpYWwgY3V5b3MgZGVyZWNob3Mgc29uIGRlIHRlcmNlcm9zIGVzdMOhIGNsYXJhbWVudGUgaWRlbnRpZmljYWRvIHkgcmVjb25vY2lkbyBlbiBlbCB0ZXh0byBvIGNvbnRlbmlkbyBkZWwgZG9jdW1lbnRvIGVudHJlZ2Fkby4KClNJIEVMIEVOVsONTyBTRSBCQVNBIEVOIEVMIFRSQUJBSk8gUVVFIEhBIFNJRE8gUEFUUk9OQURPIE8gQVBPWUFETyBQT1IgQUxHVU5BIElOU1RJVFVDScOTTiBRVUUgTk8gU0VBIENPTklDWVQsIFVTVEVEIEFDRVBUQSBRVUUgSEEgQ1VNUExJRE8gQSBDVUFMUVVJRVIgREVSRUNITyBERSBSRVZJU0nDk04gVSBPVFJBUyBPQkxJR0FDSU9ORVMgUkVRVUVSSURBUyBQT1IgRElDSE8gQ09OVFJBVE8gTyBBQ1VFUkRPLgoKQ09OSUNZVCBpZGVudGlmaWNhcsOhIGNsYXJhbWVudGUgc3Ugbm9tYnJlKGVzKSBjb21vIGVsIGF1dG9yKGVzKSBvIHByb3BpZXRhcmlvKHMpIGRlIGxhIHByZXNlbnRhY2nDs24gLCB5IG5vIGhhcsOhIG5pbmd1bmEgYWx0ZXJhY2nDs24sIGV4Y2VwdG8gc2Vnw7puIGxvIHBlcm1pdGlkbyBwb3IgbGEgbGljZW5jaWEsIHBhcmEgc3UgcHJlc2VudGFjacOzbi4K
score 15,811543

Similares en LA Referencia