Temporally-coherent video cartoonization
The automatic transformation of short background videos from real scenarios into others with a visually pleasing style, like those used in cartoons, holds application in various domains. These include animated films, video games, advertisements, and many other areas that involve visual content creat...
| Author: | |
|---|---|
| Format: | master thesis |
| Publication Date: | 2024 |
| Country: | España |
| Institution: | Universitat Politècnica de Catalunya (UPC) |
| Repository: | UPCommons. Portal del coneixement obert de la UPC |
| Language: | English |
| OAI Identifier: | oai:upcommons.upc.edu:2117/410264 |
| Online Access: | https://hdl.handle.net/2117/410264 |
| Access Level: | Open access |
| Keyword: | Video recording Artificial intelligence Video cartoonization video-to-video translation diffusion model Stable Diffusion ControlNet EbSynth Vídeo Intel·ligència artificial Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial |
| Summary: | The automatic transformation of short background videos from real scenarios into others with a visually pleasing style, like those used in cartoons, holds application in various domains. These include animated films, video games, advertisements, and many other areas that involve visual content creation. A method or tool that can perform this task, would inspire, facilitate, and streamline the work of artists and people who produce this type of content. This thesis proposes a method that integrates multiple components to translate short background videos into others that contain a particular style. We employ Stable Diffusion, a text-to-image diffusion model, along with other technologies like ControlNet to translate keyframes from the source video, ensuring content preservation. The style of the transformed keyframes is propagated to the rest of the frames using EbSynth to make the process faster and maintain the temporal coherence. We quantitatively assess content preservation and temporal coherence using CLIP-based metrics over a new dataset of videos translated into three distinct styles. The implementation of our method is publicly available at https://github.com/gustavorayo/video-to-cartoon |
|---|