Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach

Copyright 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.

Detalles Bibliográficos
Autores: Seck Diop, Dame, Yanes Luis, Samuel, Perales Esteve, Manuel Ángel, Toral, S. L., Gutiérrez Reina, Daniel
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2024
País:España
Institución:Universidad de Sevilla (US)
Repositorio:idUS. Depósito de Investigación de la Universidad de Sevilla
OAI Identifier:oai:idus.us.es:11441/174394
Acceso en línea:https://hdl.handle.net/11441/174394
https://doi.org/10.1109/ACCESS.2024.3403790
Access Level:acceso abierto
Palabra clave:Patrolling problem
Multi-agent deep reinforcement learning
Multi-task deep reinforcement learning environmental monitoring
Exploration and intensification phases
Partially observable Markov games
Autonomous surface vehicles
id ES_4e9ab378c77c8feacf7eebc38d3e0e1f
oai_identifier_str oai:idus.us.es:11441/174394
network_acronym_str ES
network_name_str España
repository_id_str
spelling Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning ApproachSeck Diop, DameYanes Luis, SamuelPerales Esteve, Manuel ÁngelToral, S. L.Gutiérrez Reina, DanielPatrolling problemMulti-agent deep reinforcement learningMulti-task deep reinforcement learning environmental monitoringExploration and intensification phasesPartially observable Markov gamesAutonomous surface vehiclesCopyright 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. This study proposes the use of an Autonomous Surface Vehicle (ASV) fleet with water quality sensors for efficient patrolling to monitor water resource pollution. This is formulated as a Patrolling Problem, which consists of planning and executing efficient routes to continuously monitor a given area. When patrolling Lake Ypacaraí with ASVs, the scenario transforms into a Partially Observable Markov Game (POMG) due to unknown pollution levels. Given the computational complexity, a Multi-Agent Deep Reinforcement Learning (MADRL) approach is adopted, with a common policy for homogeneous agents. A consensus algorithm assists in collision avoidance and coordination. The work introduces exploration and reinforcement phases to the patrolling problem. The Exploration Phase aims at homogeneous map coverage, while the Intensification Phase prioritizes high polluted areas. The innovative introduction of a transition variable, ν, efficiently controls the transition from exploration to intensification. Results demonstrate the superiority of the method, which outperforms a Single-Phase (trained on a single task) Deep Q-Network (DQN) by an average of 17% on the intensification task. The proposed multitask learning approach with parameter sharing, coupled with DQN training, outperforms Task-Specific DQN (two DQNs trained on separate tasks) by 6% in exploration and 13% in intensification. It also outperforms the heuristic-based Lawn Mower Path Planner (LMPP) and Random Wanderer Path Planner (RWPP) algorithms, by 35% and 20% on average respectively. Additionally, it outperforms a Particle Swarm Optimization-based Path Planner (PSOPP) by an average of 26%. The algorithm demonstrates adaptability in unforeseen scenarios, giving users flexibility in configuration.IEEEIngeniería ElectrónicaMinisterio de Ciencia e Innovación (MICIN). EspañaEuropean Union (UE)2024info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttps://hdl.handle.net/11441/174394https://doi.org/10.1109/ACCESS.2024.3403790reponame:idUS. Depósito de Investigación de la Universidad de Sevillainstname:Universidad de Sevilla (US)InglésIEEE Access, 12, 10535508.TED2021-131326B-C21https://ieeexplore.ieee.org/document/10535508info:eu-repo/semantics/openAccessoai:idus.us.es:11441/1743942026-06-17T12:51:07Z
dc.title.none.fl_str_mv Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
title Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
spellingShingle Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
Seck Diop, Dame
Patrolling problem
Multi-agent deep reinforcement learning
Multi-task deep reinforcement learning environmental monitoring
Exploration and intensification phases
Partially observable Markov games
Autonomous surface vehicles
title_short Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
title_full Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
title_fullStr Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
title_full_unstemmed Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
title_sort Decoupling Patrolling Tasks for Water Quality Monitoring: A Multi-Agent Deep Reinforcement Learning Approach
dc.creator.none.fl_str_mv Seck Diop, Dame
Yanes Luis, Samuel
Perales Esteve, Manuel Ángel
Toral, S. L.
Gutiérrez Reina, Daniel
author Seck Diop, Dame
author_facet Seck Diop, Dame
Yanes Luis, Samuel
Perales Esteve, Manuel Ángel
Toral, S. L.
Gutiérrez Reina, Daniel
author_role author
author2 Yanes Luis, Samuel
Perales Esteve, Manuel Ángel
Toral, S. L.
Gutiérrez Reina, Daniel
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Ingeniería Electrónica
Ministerio de Ciencia e Innovación (MICIN). España
European Union (UE)
dc.subject.none.fl_str_mv Patrolling problem
Multi-agent deep reinforcement learning
Multi-task deep reinforcement learning environmental monitoring
Exploration and intensification phases
Partially observable Markov games
Autonomous surface vehicles
topic Patrolling problem
Multi-agent deep reinforcement learning
Multi-task deep reinforcement learning environmental monitoring
Exploration and intensification phases
Partially observable Markov games
Autonomous surface vehicles
description Copyright 2024 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License.
publishDate 2024
dc.date.none.fl_str_mv 2024
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/11441/174394
https://doi.org/10.1109/ACCESS.2024.3403790
url https://hdl.handle.net/11441/174394
https://doi.org/10.1109/ACCESS.2024.3403790
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv IEEE Access, 12, 10535508.
TED2021-131326B-C21
https://ieeexplore.ieee.org/document/10535508
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv IEEE
publisher.none.fl_str_mv IEEE
dc.source.none.fl_str_mv reponame:idUS. Depósito de Investigación de la Universidad de Sevilla
instname:Universidad de Sevilla (US)
instname_str Universidad de Sevilla (US)
reponame_str idUS. Depósito de Investigación de la Universidad de Sevilla
collection idUS. Depósito de Investigación de la Universidad de Sevilla
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869407767340515328
score 15,811543

Similares en LA Referencia