Cooktop Sensing Based on a YOLO Object Detection Algorithm

Deep Learning (DL) has provided a significant breakthrough in many areas of research and industry. The development of Convolutional Neural Networks (CNNs) has enabled the improvement of computer vision-based techniques, making the information gathered from cameras more useful. For this reason, recen...

ver descrição completa

Detalhes bibliográficos
Autores: Azurmendi Marquínez, Iker, Zulueta Guerrero, Ekaitz, López Guede, José Manuel, Azkarate, Jon, González, Manuel
Formato: artículo
Fecha de publicación:2023
País:España
Recursos:Universidad del País Vasco
Repositorio:Addi. Archivo Digital para la Docencia y la Investigación
OAI Identifier:oai:addi.ehu.eus:10810/60335
Acesso em linha:http://hdl.handle.net/10810/60335
Access Level:acceso abierto
Palavra-chave:deep learning
artificial vision
object detection
YOLO
YOLOv5
YOLOv6
YOLOv7
cooking automation
smart kitchen
image sensorization
id ES_9fa4bfe41de87c7ea2d99e9a2d783674
oai_identifier_str oai:addi.ehu.eus:10810/60335
network_acronym_str ES
network_name_str España
repository_id_str
spelling Cooktop Sensing Based on a YOLO Object Detection AlgorithmAzurmendi Marquínez, IkerZulueta Guerrero, EkaitzLópez Guede, José ManuelAzkarate, JonGonzález, Manueldeep learningartificial visionobject detectionYOLOYOLOv5YOLOv6YOLOv7cooking automationsmart kitchenimage sensorizationDeep Learning (DL) has provided a significant breakthrough in many areas of research and industry. The development of Convolutional Neural Networks (CNNs) has enabled the improvement of computer vision-based techniques, making the information gathered from cameras more useful. For this reason, recently, studies have been carried out on the use of image-based DL in some areas of people’s daily life. In this paper, an object detection-based algorithm is proposed to modify and improve the user experience in relation to the use of cooking appliances. The algorithm can sense common kitchen objects and identify interesting situations for users. Some of these situations are the detection of utensils on lit hobs, recognition of boiling, smoking and oil in kitchenware, and determination of good cookware size adjustment, among others. In addition, the authors have achieved sensor fusion by using a cooker hob with Bluetooth connectivity, so it is possible to automatically interact with it via an external device such as a computer or a mobile phone. Our main contribution focuses on supporting people when they are cooking, controlling heaters, or alerting them with different types of alarms. To the best of our knowledge, this is the first time a YOLO algorithm has been used to control the cooktop by means of visual sensorization. Moreover, this research paper provides a comparison of the detection performance among different YOLO networks. Additionally, a dataset of more than 7500 images has been generated and multiple data augmentation techniques have been compared. The results show that YOLOv5s can successfully detect common kitchen objects with high accuracy and fast speed, and it can be employed for realistic cooking environment applications. Finally, multiple examples of the identification of interesting situations and how we act on the cooktop are presented.The current study has been sponsored by the Government of the Basque Country-ELKARTEK21/10 KK-2021/00014 (“Estudio de nuevas técnicas de inteligencia artificial basadas en Deep Learning dirigidas a la optimización de procesos industriales”) and ELKARTEK23-DEEPBASK (“Creación de nuevos algoritmos de aprendizaje profundo aplicado a la industria”) research programmes.MDPI2023202320232023info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10810/60335reponame:Addi. Archivo Digital para la Docencia y la Investigacióninstname:Universidad del País VascoIngléshttps://www.mdpi.com/1424-8220/23/5/2780info:eu-repo/semantics/openAccesshttp://creativecommons.org/licenses/by/4.0/© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/ 4.0/).oai:addi.ehu.eus:10810/603352026-06-18T09:23:17Z
dc.title.none.fl_str_mv Cooktop Sensing Based on a YOLO Object Detection Algorithm
title Cooktop Sensing Based on a YOLO Object Detection Algorithm
spellingShingle Cooktop Sensing Based on a YOLO Object Detection Algorithm
Azurmendi Marquínez, Iker
deep learning
artificial vision
object detection
YOLO
YOLOv5
YOLOv6
YOLOv7
cooking automation
smart kitchen
image sensorization
title_short Cooktop Sensing Based on a YOLO Object Detection Algorithm
title_full Cooktop Sensing Based on a YOLO Object Detection Algorithm
title_fullStr Cooktop Sensing Based on a YOLO Object Detection Algorithm
title_full_unstemmed Cooktop Sensing Based on a YOLO Object Detection Algorithm
title_sort Cooktop Sensing Based on a YOLO Object Detection Algorithm
dc.creator.none.fl_str_mv Azurmendi Marquínez, Iker
Zulueta Guerrero, Ekaitz
López Guede, José Manuel
Azkarate, Jon
González, Manuel
author Azurmendi Marquínez, Iker
author_facet Azurmendi Marquínez, Iker
Zulueta Guerrero, Ekaitz
López Guede, José Manuel
Azkarate, Jon
González, Manuel
author_role author
author2 Zulueta Guerrero, Ekaitz
López Guede, José Manuel
Azkarate, Jon
González, Manuel
author2_role author
author
author
author
dc.subject.none.fl_str_mv deep learning
artificial vision
object detection
YOLO
YOLOv5
YOLOv6
YOLOv7
cooking automation
smart kitchen
image sensorization
topic deep learning
artificial vision
object detection
YOLO
YOLOv5
YOLOv6
YOLOv7
cooking automation
smart kitchen
image sensorization
description Deep Learning (DL) has provided a significant breakthrough in many areas of research and industry. The development of Convolutional Neural Networks (CNNs) has enabled the improvement of computer vision-based techniques, making the information gathered from cameras more useful. For this reason, recently, studies have been carried out on the use of image-based DL in some areas of people’s daily life. In this paper, an object detection-based algorithm is proposed to modify and improve the user experience in relation to the use of cooking appliances. The algorithm can sense common kitchen objects and identify interesting situations for users. Some of these situations are the detection of utensils on lit hobs, recognition of boiling, smoking and oil in kitchenware, and determination of good cookware size adjustment, among others. In addition, the authors have achieved sensor fusion by using a cooker hob with Bluetooth connectivity, so it is possible to automatically interact with it via an external device such as a computer or a mobile phone. Our main contribution focuses on supporting people when they are cooking, controlling heaters, or alerting them with different types of alarms. To the best of our knowledge, this is the first time a YOLO algorithm has been used to control the cooktop by means of visual sensorization. Moreover, this research paper provides a comparison of the detection performance among different YOLO networks. Additionally, a dataset of more than 7500 images has been generated and multiple data augmentation techniques have been compared. The results show that YOLOv5s can successfully detect common kitchen objects with high accuracy and fast speed, and it can be employed for realistic cooking environment applications. Finally, multiple examples of the identification of interesting situations and how we act on the cooktop are presented.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023
2023
2023
dc.type.none.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10810/60335
url http://hdl.handle.net/10810/60335
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv https://www.mdpi.com/1424-8220/23/5/2780
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by/4.0/
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv MDPI
publisher.none.fl_str_mv MDPI
dc.source.none.fl_str_mv reponame:Addi. Archivo Digital para la Docencia y la Investigación
instname:Universidad del País Vasco
instname_str Universidad del País Vasco
reponame_str Addi. Archivo Digital para la Docencia y la Investigación
collection Addi. Archivo Digital para la Docencia y la Investigación
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869414945727184896
score 15,300724

Registos relacionados