Temporal copying and local hallucination for video Inpainting
Video inpainting is the task of removing objects from videos. In particular, the goal is not only to fill every frame with plausible content but also to maintain a temporal consistency so that no abrupt changes can be perceived. The current state of the art in video inpainting, which builds upon dee...
| Autores: | , |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2020 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/334882 |
| Acceso en línea: | https://hdl.handle.net/2117/334882 |
| Access Level: | acceso abierto |
| Palabra clave: | Machine learning Neural networks (Computer science) Video recording Computer vision deep learning computer vision video inpainting aprendizaje automático aprendizaje profundo redes neuronales visión por computador Aprenentatge automàtic Xarxes neuronals (Informàtica) Vídeo Visió per ordinador Àrees temàtiques de la UPC::Enginyeria de la telecomunicació |
| Sumario: | Video inpainting is the task of removing objects from videos. In particular, the goal is not only to fill every frame with plausible content but also to maintain a temporal consistency so that no abrupt changes can be perceived. The current state of the art in video inpainting, which builds upon deep neural network, suffers from the problem of handling large amounts of frames when working with decent resolution frames. In our work, we propose to tackle the problem of video inpainting by dividing it into two independent sub-tasks. The first, a Dense Flow Prediction Network (DFPN) capable of predicting the movement of the background by taking into account the movement of the object to remove. And the second, a Copy-and-Hallucinate Network (CHN) that uses the output of the previous network to copy the regions that are visible in reference frames while hallucinating those that are not. Both networks are trained independently and mixed using one of our three algorithm proposals: the Frame-by-Frame (FF) algorithm, the Inpaint-and-Propagate (IP) algorithm or the Copy-and-Propagate (CP) algorithm. We analyze our results by taking both an objective and a subjective approach in two different data sets. In both cases, we realize that our models are close to the state of the art but do not overpass it. |
|---|