Template extraction for question answer generation using an image knowledge base
Deep Learning (DL) has been key to solve complex tasks in the last years. To train DL models, vast amounts of labeled data are required. Visual Question Answering (VQA) is a task in which a question in natural language about an image is asked to a system and the system has to answer the question. To...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Estado: | Versión publicada |
| Fecha de publicación: | 2019 |
| País: | Chile |
| OAI Identifier: | oai:repositorio.anid.cl:10533/236559 |
| Acceso en línea: | https://hdl.handle.net/10533/236559 |
| Access Level: | acceso abierto |
| Palabra clave: | Ingeniería y Tecnología |
| Sumario: | Deep Learning (DL) has been key to solve complex tasks in the last years. To train DL models, vast amounts of labeled data are required. Visual Question Answering (VQA) is a task in which a question in natural language about an image is asked to a system and the system has to answer the question. To apply Deep Learning to VQA, a dataset with hundreds of thousands of images, questions about the images and their answer is needed. We propose a method to automatically obtain question-answer templates that can be used to generate questions and their answer given a knowledge base about an image with information about the objects that appears in it, their attributes and the relations between them. Our method generates two orders of magnitude bigger datasets than current human annotated ones. We find in our experiments that the most suitable strategy to use such a big dataset is to train a DL model using the generated dataset and then apply fine-tuning using the target dataset. To evaluate our generated question-answers we train models using only the training set of VQA and VQA v2 datasets and models using the fine-tune technique over our dataset. The use of our dataset improves the accuracy of What is ... and Who is .. type of questions by 2.25 and 1.02 percent points respectively in VQA and by 0.73 and 4.91 percent points respectively in VQA v2. |
|---|