Gaze Target Detection with Diffusion Models
Gaze target detection aims to identify where an individual is looking within an image. This task is critical for many downstream tasks, such as human-robot interaction, cognitive load estimation, and human attention estimation. This thesis addresses the task of gaze target detection, exploring exist...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2024 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/422214 |
| Acceso en línea: | https://hdl.handle.net/2117/422214 |
| Access Level: | acceso abierto |
| Palabra clave: | Image processing Deep learning (Machine learning) Diffusion models Gaze Target Detection Gaze following Gaze estimation deep learning image generation saliency map Imatges--Processament Aprenentatge profund Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Processament del senyal::Processament de la imatge i del senyal vídeo |
| Sumario: | Gaze target detection aims to identify where an individual is looking within an image. This task is critical for many downstream tasks, such as human-robot interaction, cognitive load estimation, and human attention estimation. This thesis addresses the task of gaze target detection, exploring existing models and introducing a novel approach utilizing diffusion models. Initially, an existing transformer-based model for gaze target detection has been adapted to be trained and tested on a new dataset. Subsequently, a new architecture based on diffusion models is proposed, incorporating a diffusion process conditioned by RGB image information and bounding box locations of the individuals. The objective is to generate a gaze target saliency map, indicating where individuals marked by the bounding boxes are focusing their attention. As the core section of the research, the exploration of the joint attention estimation task is performed using manually annotated bounding boxes, followed by experimentation using head bounding boxes detected by an object detection neural network, as well as the task of estimating single attention. Through extensive analysis of the model on the VideoCoAtt dataset against the baseline transformer-based model, although the metric does not meet the desired threshold for improvement, the visual outputs demonstrate promising results. This suggests potential avenues for further refinement and optimisation of diffusion models applied to the task of gaze target detection. |
|---|