In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions

In-field fruit monitoring at different growth stages provides important information for farmers. Recent advances have focused on the detection and location of fruits, although the development of accurate fruit size estimation systems is still a challenge that requires further attention. This work pr...

Descripción completa

Detalles Bibliográficos
Autores: Gené Mola, Jordi, Sanz Cortiella, Ricardo, Rosell Polo, Joan Ramon, Escolà i Agustí, Alexandre, Gregorio López, Eduard
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2021
País:España
Institución:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repositorio:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:10459.1/71933
Acceso en línea:https://doi.org/10.1016/j.compag.2021.106343
http://hdl.handle.net/10459.1/71933
Access Level:acceso abierto
Palabra clave:Structure-from-motion
fruit detection
Fruit size
Fruit visibility and occlusion
Agricultural robotics
id ES_e8ce3fbd11824fd39df76e9d0cff6c44
oai_identifier_str oai:recercat.cat:10459.1/71933
network_acronym_str ES
network_name_str España
repository_id_str
spelling In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusionsGené Mola, JordiSanz Cortiella, RicardoRosell Polo, Joan RamonEscolà i Agustí, AlexandreGregorio López, EduardStructure-from-motionfruit detectionFruit sizeFruit visibility and occlusionAgricultural roboticsIn-field fruit monitoring at different growth stages provides important information for farmers. Recent advances have focused on the detection and location of fruits, although the development of accurate fruit size estimation systems is still a challenge that requires further attention. This work proposes a novel methodology for automatic in-field apple size estimation which is based on four main steps: 1) fruit detection; 2) point cloud generation using structure-from-motion (SfM) and multi-view stereo (MVS); 3) fruit size estimation; and 4) fruit visibility estimation. Four techniques were evaluated in the fruit size estimation step. The first consisted of obtaining the fruit diameter by measuring the two most distant points of an apple detection (largest segment technique). The second and third techniques were based on fitting a sphere to apple points using least squares (LS) and M−estimator sample consensus (MSAC) algorithms, respectively. Finally, template matching (TM) was applied for fitting an apple 3D model to apple points. The best results were obtained with the LS, MSAC and TM techniques, which showed mean absolute errors of 4.5 mm, 3.7 mm and 4.2 mm, and coefficients of determination () of 0.88, 0.91 and 0.88, respectively. Besides fruit size, the proposed method also estimated the visibility percentage of apples detected. This step showed an of 0.92 with respect to the ground truth visibility. This allowed automatic identification and discrimination of the measurements of highly occluded apples. The main disadvantage of the method is the high processing time required (in this work 2760 s for 3D modelling of 6 trees), which limits its direct application in large agricultural areas. The code and the dataset have been made publicly available and a 3D visualization of results is accessible at http://www.grap.udl.cat/en/publications/apple_size_estimation_SfM.This work was partly funded by the Secretaria d’Universitats i Recerca del Departament d’Empresa i Coneixement de la Generalitat de Catalunya (grant 2017 SGR 646), the Spanish Ministry of Economy and Competitiveness (project AGL2013-48297-C2-2-R) and the Spanish Ministry of Science, Innovation and Universities (project RTI2018-094222-B-I00). The Spanish Ministry of Education is thanked for Mr. J. Gené’s pre-doctoral fellowships (FPU15/03355). We would also like to thank Nufri (especially Santiago Salamero and Oriol Morreres) for their support during data acquisition.Elsevier2021info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttps://doi.org/10.1016/j.compag.2021.106343http://hdl.handle.net/10459.1/71933reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)Inglésinfo:eu-repo/grantAgreement/MINECO//AGL2013-48297-C2-2-Rinfo:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/RTI2018-094222-B-I00Reproducció del document publicat a: https://doi.org/10.1016/j.compag.2021.106343Computers and Electronics in Agriculture, 2021, vol. 188, núm. 106343https://doi.org/10.34810/data141https://doi.org/10.34810/data2321cc-by (c) Gené et al., 2021info:eu-repo/semantics/openAccesshttp://creativecommons.org/licenses/by/4.0/oai:recercat.cat:10459.1/719332026-05-29T05:05:01Z
dc.title.none.fl_str_mv In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
title In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
spellingShingle In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
Gené Mola, Jordi
Structure-from-motion
fruit detection
Fruit size
Fruit visibility and occlusion
Agricultural robotics
title_short In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
title_full In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
title_fullStr In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
title_full_unstemmed In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
title_sort In-field apple size estimation using photogrammetry-derived 3D point clouds: Comparison of 4 different methods considering fruit occlusions
dc.creator.none.fl_str_mv Gené Mola, Jordi
Sanz Cortiella, Ricardo
Rosell Polo, Joan Ramon
Escolà i Agustí, Alexandre
Gregorio López, Eduard
author Gené Mola, Jordi
author_facet Gené Mola, Jordi
Sanz Cortiella, Ricardo
Rosell Polo, Joan Ramon
Escolà i Agustí, Alexandre
Gregorio López, Eduard
author_role author
author2 Sanz Cortiella, Ricardo
Rosell Polo, Joan Ramon
Escolà i Agustí, Alexandre
Gregorio López, Eduard
author2_role author
author
author
author
dc.subject.none.fl_str_mv Structure-from-motion
fruit detection
Fruit size
Fruit visibility and occlusion
Agricultural robotics
topic Structure-from-motion
fruit detection
Fruit size
Fruit visibility and occlusion
Agricultural robotics
description In-field fruit monitoring at different growth stages provides important information for farmers. Recent advances have focused on the detection and location of fruits, although the development of accurate fruit size estimation systems is still a challenge that requires further attention. This work proposes a novel methodology for automatic in-field apple size estimation which is based on four main steps: 1) fruit detection; 2) point cloud generation using structure-from-motion (SfM) and multi-view stereo (MVS); 3) fruit size estimation; and 4) fruit visibility estimation. Four techniques were evaluated in the fruit size estimation step. The first consisted of obtaining the fruit diameter by measuring the two most distant points of an apple detection (largest segment technique). The second and third techniques were based on fitting a sphere to apple points using least squares (LS) and M−estimator sample consensus (MSAC) algorithms, respectively. Finally, template matching (TM) was applied for fitting an apple 3D model to apple points. The best results were obtained with the LS, MSAC and TM techniques, which showed mean absolute errors of 4.5 mm, 3.7 mm and 4.2 mm, and coefficients of determination () of 0.88, 0.91 and 0.88, respectively. Besides fruit size, the proposed method also estimated the visibility percentage of apples detected. This step showed an of 0.92 with respect to the ground truth visibility. This allowed automatic identification and discrimination of the measurements of highly occluded apples. The main disadvantage of the method is the high processing time required (in this work 2760 s for 3D modelling of 6 trees), which limits its direct application in large agricultural areas. The code and the dataset have been made publicly available and a 3D visualization of results is accessible at http://www.grap.udl.cat/en/publications/apple_size_estimation_SfM.
publishDate 2021
dc.date.none.fl_str_mv 2021
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://doi.org/10.1016/j.compag.2021.106343
http://hdl.handle.net/10459.1/71933
url https://doi.org/10.1016/j.compag.2021.106343
http://hdl.handle.net/10459.1/71933
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv info:eu-repo/grantAgreement/MINECO//AGL2013-48297-C2-2-R
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/RTI2018-094222-B-I00
Reproducció del document publicat a: https://doi.org/10.1016/j.compag.2021.106343
Computers and Electronics in Agriculture, 2021, vol. 188, núm. 106343
https://doi.org/10.34810/data141
https://doi.org/10.34810/data2321
dc.rights.none.fl_str_mv cc-by (c) Gené et al., 2021
info:eu-repo/semantics/openAccess
http://creativecommons.org/licenses/by/4.0/
rights_invalid_str_mv cc-by (c) Gené et al., 2021
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869422973496066048
score 15,811543

Similares en LA Referencia