Simulation of a simple E-Robot warehouse environment using reinforcement learning
Warehousing is an important part of a supply chain network. There are several types of operations that can occur in a warehouse environment; where most of the costs are associated with labor, mainly related to order-picking in warehouse building(s). Automating certain processes in a warehouse is a w...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2024 |
| País: | España |
| Institución: | Universitat Oberta de Catalunya (UOC) |
| Repositorio: | O2, repositorio institucional de la UOC |
| OAI Identifier: | oai:openaccess.uoc.edu:10609/151901 |
| Acceso en línea: | https://hdl.handle.net/10609/151901 |
| Access Level: | acceso abierto |
| Palabra clave: | reinforcement learning autonomous mobile robot deep reinforcement learning warehousing operations robot móvil autónomo aprendizaje por refuerzo aprendizaje por refuerzo profundo modelado y simulación operaciones de almacén Automation -- TFM Automatització -- TFM |
| id |
ES_b8a1395efb40edac689966c7e7c07d2d |
|---|---|
| oai_identifier_str |
oai:openaccess.uoc.edu:10609/151901 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| dc.title.none.fl_str_mv |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| title |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| spellingShingle |
Simulation of a simple E-Robot warehouse environment using reinforcement learning Salazar-Jimenez, Jose-Alberto reinforcement learning autonomous mobile robot reinforcement learning deep reinforcement learning warehousing operations robot móvil autónomo aprendizaje por refuerzo aprendizaje por refuerzo profundo modelado y simulación operaciones de almacén Automation -- TFM Automatització -- TFM |
| title_short |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| title_full |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| title_fullStr |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| title_full_unstemmed |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| title_sort |
Simulation of a simple E-Robot warehouse environment using reinforcement learning |
| dc.creator.none.fl_str_mv |
Salazar-Jimenez, Jose-Alberto |
| author |
Salazar-Jimenez, Jose-Alberto |
| author_facet |
Salazar-Jimenez, Jose-Alberto |
| author_role |
author |
| dc.contributor.none.fl_str_mv |
Juan, Angel A. Gatnau Sarret, Marta |
| dc.subject.none.fl_str_mv |
reinforcement learning autonomous mobile robot reinforcement learning deep reinforcement learning warehousing operations robot móvil autónomo aprendizaje por refuerzo aprendizaje por refuerzo profundo modelado y simulación operaciones de almacén Automation -- TFM Automatització -- TFM |
| topic |
reinforcement learning autonomous mobile robot reinforcement learning deep reinforcement learning warehousing operations robot móvil autónomo aprendizaje por refuerzo aprendizaje por refuerzo profundo modelado y simulación operaciones de almacén Automation -- TFM Automatització -- TFM |
| description |
Warehousing is an important part of a supply chain network. There are several types of operations that can occur in a warehouse environment; where most of the costs are associated with labor, mainly related to order-picking in warehouse building(s). Automating certain processes in a warehouse is a way to try to reduce labor costs. There are different types of robots that can be deployed for such purpose, depending on the task(s) they are to carry out. One kind of growing interest are Autonomous Mobile Robots (AMRs), which use hardware and software to navigate their surroundings without requiring a pre-planned routing to perform their work. Efficiently coordinating a fleet of AMRs, with a specified set of rules and goals to follow, represents a complex optimization challenge, being an active topic of research and development, due to the positive impact this has and could have, such as the reduction of costs linked to processes involved in warehousing operations. The objective of this project was to develop a reinforcement learning model environment that simulates, to certain extend, the conditions and rules involved in a warehouse, to perform the order-picking process from storage/picking locations to delivery/shipping locations, by a fleet of AMRs powered by electric batteries; using as fewer computational resources as possible, so that it can run on a personal computer, with moderate specs. Six different experiments cases were carried out, whose results demonstrate the development of a simple RL model, controlling several robots (AMRs), capable of handling the “movement” of “packages” from “storing/picking locations” to “shipping/delivery locations” and the “recharging” of their “batteries”; with a decrease in performance, with the increase in the complexity of the environment and the number of robots; within the constraints and considerations of the RL model development. |
| publishDate |
2024 |
| dc.date.none.fl_str_mv |
2024 2025 2025 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/masterThesis |
| format |
masterThesis |
| dc.identifier.none.fl_str_mv |
https://hdl.handle.net/10609/151901 |
| url |
https://hdl.handle.net/10609/151901 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.rights.none.fl_str_mv |
CC BY-NC-ND http://creativecommons.org/licenses/by-nc-nd/3.0/es/ info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
CC BY-NC-ND http://creativecommons.org/licenses/by-nc-nd/3.0/es/ |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf application/pdf |
| dc.publisher.none.fl_str_mv |
Universitat Oberta de Catalunya (UOC) |
| publisher.none.fl_str_mv |
Universitat Oberta de Catalunya (UOC) |
| dc.source.none.fl_str_mv |
reponame:O2, repositorio institucional de la UOC instname:Universitat Oberta de Catalunya (UOC) |
| instname_str |
Universitat Oberta de Catalunya (UOC) |
| reponame_str |
O2, repositorio institucional de la UOC |
| collection |
O2, repositorio institucional de la UOC |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869417666009104384 |
| spelling |
Simulation of a simple E-Robot warehouse environment using reinforcement learningSalazar-Jimenez, Jose-Albertoreinforcement learningautonomous mobile robotreinforcement learningdeep reinforcement learningwarehousing operationsrobot móvil autónomoaprendizaje por refuerzoaprendizaje por refuerzo profundomodelado y simulaciónoperaciones de almacénAutomation -- TFMAutomatització -- TFMWarehousing is an important part of a supply chain network. There are several types of operations that can occur in a warehouse environment; where most of the costs are associated with labor, mainly related to order-picking in warehouse building(s). Automating certain processes in a warehouse is a way to try to reduce labor costs. There are different types of robots that can be deployed for such purpose, depending on the task(s) they are to carry out. One kind of growing interest are Autonomous Mobile Robots (AMRs), which use hardware and software to navigate their surroundings without requiring a pre-planned routing to perform their work. Efficiently coordinating a fleet of AMRs, with a specified set of rules and goals to follow, represents a complex optimization challenge, being an active topic of research and development, due to the positive impact this has and could have, such as the reduction of costs linked to processes involved in warehousing operations. The objective of this project was to develop a reinforcement learning model environment that simulates, to certain extend, the conditions and rules involved in a warehouse, to perform the order-picking process from storage/picking locations to delivery/shipping locations, by a fleet of AMRs powered by electric batteries; using as fewer computational resources as possible, so that it can run on a personal computer, with moderate specs. Six different experiments cases were carried out, whose results demonstrate the development of a simple RL model, controlling several robots (AMRs), capable of handling the “movement” of “packages” from “storing/picking locations” to “shipping/delivery locations” and the “recharging” of their “batteries”; with a decrease in performance, with the increase in the complexity of the environment and the number of robots; within the constraints and considerations of the RL model development.El almacenamiento es una parte importante de una red de cadena de suministro. Existen varios tipos de operaciones que ocurren en un entorno de almacén; donde la mayoría de los costos se asocian a la mano de obra, principalmente relacionados con la preparación y movimiento de pedidos en los edificios del almacén. La automatización de ciertos procesos en un almacén es una forma de intentar reducir los costos de mano de obra. Existen diferentes tipos de robots que se pueden implementar para dicho propósito, según las tareas que deban llevar a cabo. Un tipo de interés creciente son los robots móviles autónomos (AMRs), que utilizan hardware y software para navegar por sus alrededores sin requerir una ruta planificada previamente para realizar su trabajo. Coordinar de manera eficiente una flota de AMRs, con un conjunto específico de objetivos y reglas a seguir, representa un desafío de optimización complejo, siendo un tema activo de investigación y desarrollo, debido al impacto positivo que esto tiene y podría tener, como la reducción de costos vinculados a los procesos involucrados en las operaciones de almacenamiento. El objetivo de este proyecto fue desarrollar un entorno de aprendizaje por refuerzo que simulara, en cierta medida, las condiciones y reglas involucradas en un almacén, para realizar la preparación de pedidos desde las ubicaciones de almacenamiento/recolección hasta las ubicaciones de entrega/envío, por una flota de robots móviles automatizados alimentados por baterías eléctricas; utilizando la menor cantidad de recurso computacional posible, pudiendo ser corrido en una computadora personal de especificaciones simples. Se llevaron a cabo seis casos experimentales diferentes, cuyos resultados demuestran el desarrollo de un modelo RL simple, controlando varios robots (AMRs), capaces de manejar el “movimiento” de “paquetes” desde las “ubicaciones de almacenamiento/recolección” hasta las “ubicaciones de envío/entrega” y la “recarga” de sus “baterías”; con una disminución en el rendimiento, con el aumento de la complejidad del entorno y el número de robots; dentro de las limitaciones y las consideraciones del desarrollo del modelo de RL.Universitat Oberta de Catalunya (UOC)Juan, Angel A.Gatnau Sarret, Marta202520252024info:eu-repo/semantics/masterThesisapplication/pdfapplication/pdfhttps://hdl.handle.net/10609/151901reponame:O2, repositorio institucional de la UOCinstname:Universitat Oberta de Catalunya (UOC)InglésCC BY-NC-NDhttp://creativecommons.org/licenses/by-nc-nd/3.0/es/info:eu-repo/semantics/openAccessoai:openaccess.uoc.edu:10609/1519012026-05-28T12:42:01Z |
| score |
15,812429 |