Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods

Precision agriculture is a growing field in the agricultural industry and it holds great potential in fruit and vegetable harvesting. In this work, we present a robust accurate method for the detection and localization of the peduncle of table grapes, with direct implementation in automatic grape ha...

Descripción completa

Detalles Bibliográficos
Autores: Coll Ribes, Gabriel, Torres Rodriguez, Ivan Jesús, Grau Saldes, Antoni|||0000-0003-4112-3325, Guerra Paradas, Edmundo|||0000-0002-6696-0982, Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
Tipo de recurso: artículo
Fecha de publicación:2023
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/397425
Acceso en línea:https://hdl.handle.net/2117/397425
https://dx.doi.org/10.1016/j.compag.2023.108362
Access Level:acceso abierto
Palabra clave:Agriculture--Automation
Image segmentation
Monocular depth
Grape bunch and peduncle detection
Grape bunch and peduncle depth estimation
Robot harvesting
Agricultura--Automatització
Àrees temàtiques de la UPC::Informàtica::Robòtica
id ES_68ccdb773ec52f6b856c3c138e9eeefc
oai_identifier_str oai:upcommons.upc.edu:2117/397425
network_acronym_str ES
network_name_str España
repository_id_str
spelling Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methodsColl Ribes, GabrielTorres Rodriguez, Ivan JesúsGrau Saldes, Antoni|||0000-0003-4112-3325Guerra Paradas, Edmundo|||0000-0002-6696-0982Sanfeliu Cortés, Alberto|||0000-0003-3868-9678Agriculture--AutomationImage segmentationMonocular depthGrape bunch and peduncle detectionGrape bunch and peduncle depth estimationRobot harvestingAgricultura--AutomatitzacióÀrees temàtiques de la UPC::Informàtica::RobòticaPrecision agriculture is a growing field in the agricultural industry and it holds great potential in fruit and vegetable harvesting. In this work, we present a robust accurate method for the detection and localization of the peduncle of table grapes, with direct implementation in automatic grape harvesting with robots. The bunch and peduncle detection methods presented in this work rely on a combination of instance segmentation and monocular depth estimation using Convolutional Neural Networks (CNN). Regarding depth estimation, we propose a combination of different depth techniques that allow precise localization of the peduncle using traditional stereo cameras, even with the particular complexity of grape peduncles. The methods proposed in this work have been tested on the WGISD (Embrapa Wine Grape Instance Segmentation) dataset, improving the results of state-of-the-art techniques. Furthermore, within the context of the EU project CANOPIES, the methods have also been tested on a dataset of 1,326 RGB-D images of table grapes, recorded at the Corsira Agricultural Cooperative Society (Aprilia, Italy), using a Realsense D435i camera located at the arm of a CANOPIES two-manipulator robot developed in the project. The detection results on the WGISD dataset show that the use of RGB-D information () leads to superior performance compared to the use of RGB data alone (). This trend is also evident in the CANOPIES Grape Bunch and Peduncle dataset, where the mAP for RGB-D images () outperforms that of RGB data (). Regarding depth estimation, our method achieves a mean squared error of 2.66 cm within a distance of 1 m in the CANOPIES dataset.Peer ReviewedElsevier20232023-12-0120232023-11-30journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/397425https://dx.doi.org/10.1016/j.compag.2023.108362reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3974252026-05-27T15:37:01Z
dc.title.none.fl_str_mv Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
spellingShingle Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
Coll Ribes, Gabriel
Agriculture--Automation
Image segmentation
Monocular depth
Grape bunch and peduncle detection
Grape bunch and peduncle depth estimation
Robot harvesting
Agricultura--Automatització
Àrees temàtiques de la UPC::Informàtica::Robòtica
title_short Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_full Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_fullStr Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_full_unstemmed Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_sort Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
dc.creator.none.fl_str_mv Coll Ribes, Gabriel
Torres Rodriguez, Ivan Jesús
Grau Saldes, Antoni|||0000-0003-4112-3325
Guerra Paradas, Edmundo|||0000-0002-6696-0982
Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
author Coll Ribes, Gabriel
author_facet Coll Ribes, Gabriel
Torres Rodriguez, Ivan Jesús
Grau Saldes, Antoni|||0000-0003-4112-3325
Guerra Paradas, Edmundo|||0000-0002-6696-0982
Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
author_role author
author2 Torres Rodriguez, Ivan Jesús
Grau Saldes, Antoni|||0000-0003-4112-3325
Guerra Paradas, Edmundo|||0000-0002-6696-0982
Sanfeliu Cortés, Alberto|||0000-0003-3868-9678
author2_role author
author
author
author
dc.subject.none.fl_str_mv Agriculture--Automation
Image segmentation
Monocular depth
Grape bunch and peduncle detection
Grape bunch and peduncle depth estimation
Robot harvesting
Agricultura--Automatització
Àrees temàtiques de la UPC::Informàtica::Robòtica
topic Agriculture--Automation
Image segmentation
Monocular depth
Grape bunch and peduncle detection
Grape bunch and peduncle depth estimation
Robot harvesting
Agricultura--Automatització
Àrees temàtiques de la UPC::Informàtica::Robòtica
description Precision agriculture is a growing field in the agricultural industry and it holds great potential in fruit and vegetable harvesting. In this work, we present a robust accurate method for the detection and localization of the peduncle of table grapes, with direct implementation in automatic grape harvesting with robots. The bunch and peduncle detection methods presented in this work rely on a combination of instance segmentation and monocular depth estimation using Convolutional Neural Networks (CNN). Regarding depth estimation, we propose a combination of different depth techniques that allow precise localization of the peduncle using traditional stereo cameras, even with the particular complexity of grape peduncles. The methods proposed in this work have been tested on the WGISD (Embrapa Wine Grape Instance Segmentation) dataset, improving the results of state-of-the-art techniques. Furthermore, within the context of the EU project CANOPIES, the methods have also been tested on a dataset of 1,326 RGB-D images of table grapes, recorded at the Corsira Agricultural Cooperative Society (Aprilia, Italy), using a Realsense D435i camera located at the arm of a CANOPIES two-manipulator robot developed in the project. The detection results on the WGISD dataset show that the use of RGB-D information () leads to superior performance compared to the use of RGB data alone (). This trend is also evident in the CANOPIES Grape Bunch and Peduncle dataset, where the mAP for RGB-D images () outperforms that of RGB data (). Regarding depth estimation, our method achieves a mean squared error of 2.66 cm within a distance of 1 m in the CANOPIES dataset.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-12-01
2023
2023-11-30
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/397425
https://dx.doi.org/10.1016/j.compag.2023.108362
url https://hdl.handle.net/2117/397425
https://dx.doi.org/10.1016/j.compag.2023.108362
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869409985736212480
score 15.300724

Similares en LA Referencia