Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator

Intersections are considered one of the most complex scenarios in a self-driving framework due to the uncertainty in the behaviors of surrounding vehicles and the different types of scenarios that can be found. To deal with this problem, we provide a Deep Reinforcement Learning approach for intersec...

Full description

Bibliographic Details
Authors: Gutiérrez Moreno, Rodrigo, Barea Navarro, Rafael|||0000-0002-4179-6100, López Guillén, María Elena, Araluce Ruiz, Javier, Bergasa Pascual, Luis Miguel|||0000-0002-0087-3077
Format: article
Publication Date:2022
Country:España
Institution:Universidad de Alcalá (UAH)
Repository:e_Buah Biblioteca Digital Universidad de Alcalá
Language:English
OAI Identifier:oai:ebuah.uah.es:10017/63104
Online Access:http://hdl.handle.net/10017/63104
https://dx.doi.org/10.3390/s22218373
Access Level:Open access
Keyword:Reinforcement learning
Decision making
Autonomous driving
Electrónica
Electronics
id ES_c8dcc6afa17d8b124c1b29bb51e00c67
oai_identifier_str oai:ebuah.uah.es:10017/63104
network_acronym_str ES
network_name_str España
repository_id_str
spelling Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA SimulatorGutiérrez Moreno, RodrigoBarea Navarro, Rafael|||0000-0002-4179-6100López Guillén, María ElenaAraluce Ruiz, JavierBergasa Pascual, Luis Miguel|||0000-0002-0087-3077Reinforcement learningDecision makingAutonomous drivingElectrónicaElectronicsIntersections are considered one of the most complex scenarios in a self-driving framework due to the uncertainty in the behaviors of surrounding vehicles and the different types of scenarios that can be found. To deal with this problem, we provide a Deep Reinforcement Learning approach for intersection handling, which is combined with Curriculum Learning to improve the training process. The state space is defined by two vectors, containing adversaries and ego vehicle information. We define a features extractor module and an actor–critic approach combined with Curriculum Learning techniques, adding complexity to the environment by increasing the number of vehicles. In order to address a complete autonomous driving system, a hybrid architecture is proposed. The operative level generates the driving commands, the strategy level defines the trajectory and the tactical level executes the high-level decisions. This high-level decision system is the main goal of this research. To address realistic experiments, we set up three scenarios: intersections with traffic lights, intersections with traffic signs and uncontrolled intersections. The results of this paper show that a Proximal Policy Optimization algorithm can infer ego vehicle-desired behavior for different intersection scenarios based only on the behavior of adversarial vehicles.Agencia Estatal de InvestigaciónComunidad de MadridMDPI20222022-11-01journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10017/63104https://dx.doi.org/10.3390/s22218373reponame:e_Buah Biblioteca Digital Universidad de Alcaláinstname:Universidad de Alcalá (UAH)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2021-126623OB-I00 IMPLEMENTACION Y VALIDACION DE UNA ARQUITECTURA MODULAR BASADA EN INTELIGENCIA ARTIFICIAL PARA CONDUCCION AUTONOMAComunidad de Madrid http://dx.doi.org/10.13039/100012818 Not available P2018%2FNMT-4331 Madrid Robotics Digital Innovation Hubopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:ebuah.uah.es:10017/631042026-06-18T11:13:07Z
dc.title.none.fl_str_mv Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
title Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
spellingShingle Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
Gutiérrez Moreno, Rodrigo
Reinforcement learning
Decision making
Autonomous driving
Electrónica
Electronics
title_short Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
title_full Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
title_fullStr Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
title_full_unstemmed Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
title_sort Reinforcement Learning-Based Autonomous Driving at Intersections in CARLA Simulator
dc.creator.none.fl_str_mv Gutiérrez Moreno, Rodrigo
Barea Navarro, Rafael|||0000-0002-4179-6100
López Guillén, María Elena
Araluce Ruiz, Javier
Bergasa Pascual, Luis Miguel|||0000-0002-0087-3077
author Gutiérrez Moreno, Rodrigo
author_facet Gutiérrez Moreno, Rodrigo
Barea Navarro, Rafael|||0000-0002-4179-6100
López Guillén, María Elena
Araluce Ruiz, Javier
Bergasa Pascual, Luis Miguel|||0000-0002-0087-3077
author_role author
author2 Barea Navarro, Rafael|||0000-0002-4179-6100
López Guillén, María Elena
Araluce Ruiz, Javier
Bergasa Pascual, Luis Miguel|||0000-0002-0087-3077
author2_role author
author
author
author
dc.subject.none.fl_str_mv Reinforcement learning
Decision making
Autonomous driving
Electrónica
Electronics
topic Reinforcement learning
Decision making
Autonomous driving
Electrónica
Electronics
description Intersections are considered one of the most complex scenarios in a self-driving framework due to the uncertainty in the behaviors of surrounding vehicles and the different types of scenarios that can be found. To deal with this problem, we provide a Deep Reinforcement Learning approach for intersection handling, which is combined with Curriculum Learning to improve the training process. The state space is defined by two vectors, containing adversaries and ego vehicle information. We define a features extractor module and an actor–critic approach combined with Curriculum Learning techniques, adding complexity to the environment by increasing the number of vehicles. In order to address a complete autonomous driving system, a hybrid architecture is proposed. The operative level generates the driving commands, the strategy level defines the trajectory and the tactical level executes the high-level decisions. This high-level decision system is the main goal of this research. To address realistic experiments, we set up three scenarios: intersections with traffic lights, intersections with traffic signs and uncontrolled intersections. The results of this paper show that a Proximal Policy Optimization algorithm can infer ego vehicle-desired behavior for different intersection scenarios based only on the behavior of adversarial vehicles.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022-11-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10017/63104
https://dx.doi.org/10.3390/s22218373
url http://hdl.handle.net/10017/63104
https://dx.doi.org/10.3390/s22218373
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2021-126623OB-I00 IMPLEMENTACION Y VALIDACION DE UNA ARQUITECTURA MODULAR BASADA EN INTELIGENCIA ARTIFICIAL PARA CONDUCCION AUTONOMA
Comunidad de Madrid http://dx.doi.org/10.13039/100012818 Not available P2018%2FNMT-4331 Madrid Robotics Digital Innovation Hub
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv MDPI
publisher.none.fl_str_mv MDPI
dc.source.none.fl_str_mv reponame:e_Buah Biblioteca Digital Universidad de Alcalá
instname:Universidad de Alcalá (UAH)
instname_str Universidad de Alcalá (UAH)
reponame_str e_Buah Biblioteca Digital Universidad de Alcalá
collection e_Buah Biblioteca Digital Universidad de Alcalá
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869419322251673600
score 15.811543

Similar Items