Unsupervised image annotation as multimodal query expansion
This thesis deals with the unsupervised automatic image annotation (UAIA) problem, given a reference collection composed by texts and images, where not necessarily exists a direct relationship among the data, this problem is defined as the assignation of words extracted from the texts to images, loo...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis doctoral |
| Estado: | Versión aceptada para publicación |
| Fecha de publicación: | 2017 |
| País: | México |
| Institución: | Instituto Nacional de Astrofísica, Óptica y Electrónica |
| Repositorio: | Repositorio Institucional del INAOE |
| Idioma: | inglés |
| OAI Identifier: | oai:inaoe.repositorioinstitucional.mx:1009/2080 |
| Acceso en línea: | http://inaoe.repositorioinstitucional.mx/jspui/handle/1009/2080 |
| Access Level: | acceso abierto |
| Palabra clave: | info:eu-repo/classification/Inspec/Unsupervised image info:eu-repo/classification/Inspec/Annotation info:eu-repo/classification/Inspec/Automtic info:eu-repo/classification/Inspec/Query expansion info:eu-repo/classification/Inspec/Image captioning info:eu-repo/classification/Inspec/Visual prototypes. info:eu-repo/classification/cti/1 info:eu-repo/classification/cti/12 info:eu-repo/classification/cti/1203 info:eu-repo/classification/cti/120323 |
| Sumario: | This thesis deals with the unsupervised automatic image annotation (UAIA) problem, given a reference collection composed by texts and images, where not necessarily exists a direct relationship among the data, this problem is defined as the assignation of words extracted from the texts to images, looking to describe their visual content. The idea behind our work is to exploit interactions between modalities with the aim to annotate images, and taking into account complementarity and redundancy in the data. We have introduced two effective methods for UAIA in the context of a common framework inspired in the way a query is expanded throughout Automatic Query Expansion (AQE) in information retrieval. On the one hand, a local method that processes text information associated to images retrieved when using the image to annotate as query, several methods from the state of the art can be described under this formulation. On the other hand, a global method that pre-process offline the reference collection to identify visual-textual associations that are later used for annotation. Both methods are extensively evaluated in benchmarks for large-scale UAIA. |
|---|