Distributed large scale systems : a multi-agent RL-MPC architecture

This thesis describes a methodology to deal with the interaction between MPC controllers in a distributed MPC architecture. This approach combines ideas from Distributed Artificial Intelligence (DAI) and Reinforcement Learning (RL) in order to provide a controller interaction based on cooperative ag...

Descripción completa

Detalles Bibliográficos
Autor: Javalera Rincón, Valeria
Tipo de recurso: tesis doctoral
Fecha de publicación:2016
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/96332
Acceso en línea:https://hdl.handle.net/2117/96332
https://dx.doi.org/10.5821/dissertation-2117-96332
Access Level:acceso abierto
Palabra clave:Control automàtic
Àrees temàtiques de la UPC::Informàtica
id ES_eb91cdf9d7814cc7717fb6e24b3672a7
oai_identifier_str oai:upcommons.upc.edu:2117/96332
network_acronym_str ES
network_name_str España
repository_id_str
spelling Distributed large scale systems : a multi-agent RL-MPC architectureJavalera Rincón, ValeriaControl automàticÀrees temàtiques de la UPC::InformàticaThis thesis describes a methodology to deal with the interaction between MPC controllers in a distributed MPC architecture. This approach combines ideas from Distributed Artificial Intelligence (DAI) and Reinforcement Learning (RL) in order to provide a controller interaction based on cooperative agents and learning techniques. The aim of this methodology is to provide a general structure to perform optimal control in networked distributed environments, where multiple dependencies between subsystems are found. Those dependencies or connections often correspond to control variables. In that case, the distributed control has to be consistent in both subsystems. One of the main new concepts of this architecture is the negotiator agent. Negotiator agents interact with MPC agents to determine the optimal value of the shared control variables in a cooperative way using learning techniques (RL). The optimal value of those shared control variables has to accomplish a common goal, probably different from the specific goal of each agent sharing the variable. Two cases of study, in which the proposed architecture is applied and tested are considered, a small water distribution network and the Barcelona water network. The results suggest this approach is a promising strategy when centralized control is not a reasonable choice.Esta tesis describe una metodología para hacer frente a la interacción entre controladores MPC en una arquitectura MPC distribuida. Este enfoque combina las ideas de Inteligencia Artificial Distribuida (DIA) y aprendizaje por refuerzo (RL) con el fin de proporcionar una interacción entre controladores basado en agentes de cooperativos y técnicas de aprendizaje. El objetivo de esta metodología es proporcionar una estructura general para llevar a cabo un control óptimo en entornos de redes distribuidas, donde se encuentran varias dependencias entre subsistemas. Esas dependencias o conexiones corresponden a menudo a variables de control. En ese caso, el control distribuido tiene que ser coherente en ambos subsistemas. Uno de los principales conceptos novedosos de esta arquitectura es el agente negociador. Los agentes negociadores actúan junto con agentes MPC para determinar el valor óptimo de las variables de control compartidas de forma cooperativa utilizando técnicas de aprendizaje (RL). El valor óptimo de esas variables compartidas debe lograr un objetivo común, probablemente diferente de los objetivos específicos de cada agente que está compartiendo la variable. Se consideran dos casos de estudio, en el que la arquitectura propuesta se ha aplicado y probado, una pequeña red de distribución de agua y la red de agua de Barcelona. Los resultados sugieren que este enfoque es una estrategia prometedora cuando el control centralizado no es una opción razonable.Universitat Politècnica de CatalunyaPuig Cayuela, VicençMorcego Seix, Bernardo20162016-04-0720162016-11-09doctoral thesishttp://purl.org/coar/resource_type/c_db06VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/doctoralThesisapplication/pdfhttps://hdl.handle.net/2117/96332https://dx.doi.org/10.5821/dissertation-2117-96332reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/963322026-05-27T15:37:01Z
dc.title.none.fl_str_mv Distributed large scale systems : a multi-agent RL-MPC architecture
title Distributed large scale systems : a multi-agent RL-MPC architecture
spellingShingle Distributed large scale systems : a multi-agent RL-MPC architecture
Javalera Rincón, Valeria
Control automàtic
Àrees temàtiques de la UPC::Informàtica
title_short Distributed large scale systems : a multi-agent RL-MPC architecture
title_full Distributed large scale systems : a multi-agent RL-MPC architecture
title_fullStr Distributed large scale systems : a multi-agent RL-MPC architecture
title_full_unstemmed Distributed large scale systems : a multi-agent RL-MPC architecture
title_sort Distributed large scale systems : a multi-agent RL-MPC architecture
dc.creator.none.fl_str_mv Javalera Rincón, Valeria
author Javalera Rincón, Valeria
author_facet Javalera Rincón, Valeria
author_role author
dc.contributor.none.fl_str_mv Puig Cayuela, Vicenç
Morcego Seix, Bernardo
dc.subject.none.fl_str_mv Control automàtic
Àrees temàtiques de la UPC::Informàtica
topic Control automàtic
Àrees temàtiques de la UPC::Informàtica
description This thesis describes a methodology to deal with the interaction between MPC controllers in a distributed MPC architecture. This approach combines ideas from Distributed Artificial Intelligence (DAI) and Reinforcement Learning (RL) in order to provide a controller interaction based on cooperative agents and learning techniques. The aim of this methodology is to provide a general structure to perform optimal control in networked distributed environments, where multiple dependencies between subsystems are found. Those dependencies or connections often correspond to control variables. In that case, the distributed control has to be consistent in both subsystems. One of the main new concepts of this architecture is the negotiator agent. Negotiator agents interact with MPC agents to determine the optimal value of the shared control variables in a cooperative way using learning techniques (RL). The optimal value of those shared control variables has to accomplish a common goal, probably different from the specific goal of each agent sharing the variable. Two cases of study, in which the proposed architecture is applied and tested are considered, a small water distribution network and the Barcelona water network. The results suggest this approach is a promising strategy when centralized control is not a reasonable choice.
publishDate 2016
dc.date.none.fl_str_mv 2016
2016-04-07
2016
2016-11-09
dc.type.none.fl_str_mv doctoral thesis
http://purl.org/coar/resource_type/c_db06
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/doctoralThesis
format doctoralThesis
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/96332
https://dx.doi.org/10.5821/dissertation-2117-96332
url https://hdl.handle.net/2117/96332
https://dx.doi.org/10.5821/dissertation-2117-96332
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Universitat Politècnica de Catalunya
publisher.none.fl_str_mv Universitat Politècnica de Catalunya
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869423239353073664
score 15,300719

Similares en LA Referencia