Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods

Precision agriculture is a growing field in the agricultural industry and it holds great potential in fruit and vegetable harvesting. In this work, we present a robust accurate method for the detection and localization of the peduncle of table grapes, with direct implementation in automatic grape ha...

ver descrição completa

Detalhes bibliográficos
Autores: Coll-Ribes, Gabriel, Torres-Rodríguez, Iván J., Grau Saldes, Antoni, Guerra, Edmundo, Sanfeliu, Alberto
Formato: artículo
Estado:Versión publicada
Fecha de publicación:2023
País:España
Recursos:Consejo Superior de Investigaciones Científicas (CSIC)
Repositorio:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/351137
Acesso em linha:http://hdl.handle.net/10261/351137
https://api.elsevier.com/content/abstract/scopus_id/85175342374
Access Level:acceso abierto
Palavra-chave:Grape bunch and peduncle depth estimation
Grape bunch and peduncle detection
Image segmentation
Monocular depth
Robot harvesting
id ES_8ac3b9c37aebd62ac97e73d33a135df7
oai_identifier_str oai:digital.csic.es:10261/351137
network_acronym_str ES
network_name_str España
repository_id_str
spelling Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methodsColl-Ribes, GabrielTorres-Rodríguez, Iván J.Grau Saldes, AntoniGuerra, EdmundoSanfeliu, AlbertoGrape bunch and peduncle depth estimationGrape bunch and peduncle detectionImage segmentationMonocular depthRobot harvestingPrecision agriculture is a growing field in the agricultural industry and it holds great potential in fruit and vegetable harvesting. In this work, we present a robust accurate method for the detection and localization of the peduncle of table grapes, with direct implementation in automatic grape harvesting with robots. The bunch and peduncle detection methods presented in this work rely on a combination of instance segmentation and monocular depth estimation using Convolutional Neural Networks (CNN). Regarding depth estimation, we propose a combination of different depth techniques that allow precise localization of the peduncle using traditional stereo cameras, even with the particular complexity of grape peduncles. The methods proposed in this work have been tested on the WGISD (Embrapa Wine Grape Instance Segmentation) dataset, improving the results of state-of-the-art techniques. Furthermore, within the context of the EU project CANOPIES, the methods have also been tested on a dataset of 1,326 RGB-D images of table grapes, recorded at the Corsira Agricultural Cooperative Society (Aprilia, Italy), using a Realsense D435i camera located at the arm of a CANOPIES two-manipulator robot developed in the project. The detection results on the WGISD dataset show that the use of RGB-D information (mAP=0.949) leads to superior performance compared to the use of RGB data alone (mAP=0.891). This trend is also evident in the CANOPIES Grape Bunch and Peduncle dataset, where the mAP for RGB-D images (mAP=0.767) outperforms that of RGB data (mAP=0.725). Regarding depth estimation, our method achieves a mean squared error of 2.66 cm within a distance of 1 m in the CANOPIES dataset.Work supported under the European project CANOPIES with the grant (H2020- ICT-2020-2-101016906). The authors want to acknowledge the help and support of the Sapienza Universitá di Roma and the Universitá degli Studi Roma Tre researchers of the CANOPIES project in the data collection of the grapes images and ground truth data.Peer reviewedElsevier BVEuropean CommissionSanfeliu, Alberto [0000-0003-3868-9678]Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]202420242023info:eu-repo/semantics/articlehttp://purl.org/coar/resource_type/c_6501Publisher's versioninfo:eu-repo/semantics/publishedVersionapplication/pdfhttp://hdl.handle.net/10261/351137https://api.elsevier.com/content/abstract/scopus_id/85175342374reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)Inglés#PLACEHOLDER_PARENT_METADATA_VALUE#info:eu-repo/grantAgreement/EC/H2020/101016906The underlying dataset has been published as supplementary material of the article in the publisher platform at https://doi.org/10.1016/j.compag.2023.108362Sanfeliu, Alberto; 2024; Data for publication. CANOPIES Grape bunch and peduncle dataset [Dataset]; Version 2; Zenodo; https://doi.org/10.5281/zenodo.15013837https://doi.org/10.1016/j.compag.2023.108362Síinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/3511372026-05-22T06:33:51Z
dc.title.none.fl_str_mv Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
spellingShingle Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
Coll-Ribes, Gabriel
Grape bunch and peduncle depth estimation
Grape bunch and peduncle detection
Image segmentation
Monocular depth
Robot harvesting
title_short Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_full Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_fullStr Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_full_unstemmed Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
title_sort Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods
dc.creator.none.fl_str_mv Coll-Ribes, Gabriel
Torres-Rodríguez, Iván J.
Grau Saldes, Antoni
Guerra, Edmundo
Sanfeliu, Alberto
author Coll-Ribes, Gabriel
author_facet Coll-Ribes, Gabriel
Torres-Rodríguez, Iván J.
Grau Saldes, Antoni
Guerra, Edmundo
Sanfeliu, Alberto
author_role author
author2 Torres-Rodríguez, Iván J.
Grau Saldes, Antoni
Guerra, Edmundo
Sanfeliu, Alberto
author2_role author
author
author
author
dc.contributor.none.fl_str_mv European Commission
Sanfeliu, Alberto [0000-0003-3868-9678]
Consejo Superior de Investigaciones Científicas [https://ror.org/02gfc7t72]
dc.subject.none.fl_str_mv Grape bunch and peduncle depth estimation
Grape bunch and peduncle detection
Image segmentation
Monocular depth
Robot harvesting
topic Grape bunch and peduncle depth estimation
Grape bunch and peduncle detection
Image segmentation
Monocular depth
Robot harvesting
description Precision agriculture is a growing field in the agricultural industry and it holds great potential in fruit and vegetable harvesting. In this work, we present a robust accurate method for the detection and localization of the peduncle of table grapes, with direct implementation in automatic grape harvesting with robots. The bunch and peduncle detection methods presented in this work rely on a combination of instance segmentation and monocular depth estimation using Convolutional Neural Networks (CNN). Regarding depth estimation, we propose a combination of different depth techniques that allow precise localization of the peduncle using traditional stereo cameras, even with the particular complexity of grape peduncles. The methods proposed in this work have been tested on the WGISD (Embrapa Wine Grape Instance Segmentation) dataset, improving the results of state-of-the-art techniques. Furthermore, within the context of the EU project CANOPIES, the methods have also been tested on a dataset of 1,326 RGB-D images of table grapes, recorded at the Corsira Agricultural Cooperative Society (Aprilia, Italy), using a Realsense D435i camera located at the arm of a CANOPIES two-manipulator robot developed in the project. The detection results on the WGISD dataset show that the use of RGB-D information (mAP=0.949) leads to superior performance compared to the use of RGB data alone (mAP=0.891). This trend is also evident in the CANOPIES Grape Bunch and Peduncle dataset, where the mAP for RGB-D images (mAP=0.767) outperforms that of RGB data (mAP=0.725). Regarding depth estimation, our method achieves a mean squared error of 2.66 cm within a distance of 1 m in the CANOPIES dataset.
publishDate 2023
dc.date.none.fl_str_mv 2023
2024
2024
dc.type.none.fl_str_mv info:eu-repo/semantics/article
http://purl.org/coar/resource_type/c_6501
Publisher's version
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/351137
https://api.elsevier.com/content/abstract/scopus_id/85175342374
url http://hdl.handle.net/10261/351137
https://api.elsevier.com/content/abstract/scopus_id/85175342374
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv #PLACEHOLDER_PARENT_METADATA_VALUE#
info:eu-repo/grantAgreement/EC/H2020/101016906
The underlying dataset has been published as supplementary material of the article in the publisher platform at https://doi.org/10.1016/j.compag.2023.108362
Sanfeliu, Alberto; 2024; Data for publication. CANOPIES Grape bunch and peduncle dataset [Dataset]; Version 2; Zenodo; https://doi.org/10.5281/zenodo.15013837
https://doi.org/10.1016/j.compag.2023.108362
Sí
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier BV
publisher.none.fl_str_mv Elsevier BV
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869412741927665664
score 15,811543

Registos relacionados