LLM Multimodal Traffic Accident Forecasting
[EN] With the rise in traffic congestion in urban centers, predicting accidents has become paramount for city planning and public safety. This work comprehensively studied the efficacy of modern deep learning (DL) methods in forecasting traffic accidents and enhancing Level-4 and Level-5 (L-4 and L-...
| Authors: | , , , |
|---|---|
| Format: | article |
| Publication Date: | 2023 |
| Country: | España |
| Institution: | Universitat Politècnica de València (UPV) |
| Repository: | RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia |
| Language: | English |
| OAI Identifier: | oai:riunet.upv.es:10251/202235 |
| Online Access: | https://riunet.upv.es/handle/10251/202235 |
| Access Level: | Open access |
| Keyword: | LLM VLM LLaVA Accident forecasting Transformers Time series analysis PCA loadings ARQUITECTURA Y TECNOLOGIA DE COMPUTADORES 08.- Fomentar el crecimiento económico sostenido, inclusivo y sostenible, el empleo pleno y productivo, y el trabajo decente para todos |
| Summary: | [EN] With the rise in traffic congestion in urban centers, predicting accidents has become paramount for city planning and public safety. This work comprehensively studied the efficacy of modern deep learning (DL) methods in forecasting traffic accidents and enhancing Level-4 and Level-5 (L-4 and L-5) driving assistants with actionable visual and language cues. Using a rich dataset detailing accident occurrences, we juxtaposed the Transformer model against traditional time series models like ARIMA and the more recent Prophet model. Additionally, through detailed analysis, we delved deep into feature importance using principal component analysis (PCA) loadings, uncovering key factors contributing to accidents. We introduce the idea of using real-time interventions with large language models (LLMs) in autonomous driving with the use of lightweight compact LLMs like LLaMA-2 and Zephyr-7b-alpha. Our exploration extends to the realm of multimodality, through the use of Large Language-and-Vision Assistant (LLaVA)-a bridge between visual and linguistic cues by means of a Visual Language Model (VLM)-in conjunction with deep probabilistic reasoning, enhancing the real-time responsiveness of autonomous driving systems. In this study, we elucidate the advantages of employing large multimodal models within DL and deep probabilistic programming for enhancing the performance and usability of time series forecasting and feature weight importance, particularly in a self-driving scenario. This work paves the way for safer, smarter cities, underpinned by data-driven decision making. |
|---|