IQ tests are not for machines, yet

[EN] Complex, but specific, tasks¿such as chess or Jeopardy!¿are popularly seen as milestones for artificial intelligence (AI). However, they are not appropriate for evaluating the intelligence of machines or measuring the progress in AI. Aware of this delusion, Detterman has recently raised a chall...

Descripción completa

Detalles Bibliográficos
Autores: Dowe, David L., Hernández-Orallo, José|||0000-0001-9746-7632
Tipo de recurso: artículo
Fecha de publicación:2012
País:España
Institución:Universitat Politècnica de València (UPV)
Repositorio:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Idioma:inglés
OAI Identifier:oai:riunet.upv.es:10251/36157
Acceso en línea:https://riunet.upv.es/handle/10251/36157
Access Level:acceso abierto
Palabra clave:Machine intelligence evaluation
IQ tests
Artificial intelligence
Universal tests
Psychometrics
Task difficulty
CAPTCHA
LENGUAJES Y SISTEMAS INFORMATICOS
id ES_e265a9267dde33cae8b5b87e4ef8ef61
oai_identifier_str oai:riunet.upv.es:10251/36157
network_acronym_str ES
network_name_str España
repository_id_str
spelling IQ tests are not for machines, yetDowe, David L.Hernández-Orallo, José|||0000-0001-9746-7632Machine intelligence evaluationIQ testsArtificial intelligenceUniversal testsPsychometricsTask difficultyCAPTCHALENGUAJES Y SISTEMAS INFORMATICOS[EN] Complex, but specific, tasks¿such as chess or Jeopardy!¿are popularly seen as milestones for artificial intelligence (AI). However, they are not appropriate for evaluating the intelligence of machines or measuring the progress in AI. Aware of this delusion, Detterman has recently raised a challenge prompting AI researchers to evaluate their artefacts against IQ tests. We agree that the philosophy behind (human) IQ tests is a much better approach to machine intelligence evaluation than these specific tasks, and also more practical and informative than the Turing test. However, we have first to recall some work on machine intelligence measurement which has shown that some IQ tests can be passed by relatively simple programs. This suggests that the challenge may not be so demanding and may just work as a sophisticated CAPTCHA, since some types of tests might be easier than others for the current state of AI. Second, we show that an alternative, formal derivation of intelligence tests for machines is possible, grounded in (algorithmic) information theory. In these tests, we have a proper mathematical definition of what is being measured. Third, we re-visit some research done in the past fifteen years for effectively measuring machine intelligence¿since some assumptions about the subjects and their distribution no longer hold.This work was supported by the MEC projects EXPLORAINGENIO TIN 2009-06078-E, CONSOLIDER-INGENIO 26706 and TIN 2010-21062-C02-02, and GVA project PROMETEO/2008/051.ElsevierDepartamento de Sistemas Informáticos y ComputaciónEscuela Técnica Superior de Ingeniería InformáticaInstituto Universitario Valenciano de Investigación en Inteligencia ArtificialGeneralitat ValencianaMinisterio de Ciencia e InnovaciónRepositorio Institucional de la Universitat Politècnica de València Riunet20122012-04-01journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfapplication/pdfhttps://riunet.upv.es/handle/10251/36157reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valénciainstname:Universitat Politècnica de València (UPV)InglésengMinisterio de Ciencia e Innovación http://dx.doi.org/10.13039/501100004837 TIN2009-06078-E ANYTIME UNIVERSAL INTELLIGENCEMinisterio de Ciencia e Innovación http://dx.doi.org/10.13039/501100004837 TIN2010-21062-C02-02 SWEETLOGICS-UPVGeneralitat Valenciana https://doi.org/10.13039/501100003359 PROMETEO08%2F2008%2F051 Advances on Agreement Technologies for Computational Entities (atforce)open accesshttp://purl.org/coar/access_right/c_abf2Reserva de todos los derechoshttp://rightsstatements.org/vocab/InC/1.0/info:eu-repo/semantics/openAccessoai:riunet.upv.es:10251/361572026-06-13T07:49:27Z
dc.title.none.fl_str_mv IQ tests are not for machines, yet
title IQ tests are not for machines, yet
spellingShingle IQ tests are not for machines, yet
Dowe, David L.
Machine intelligence evaluation
IQ tests
Artificial intelligence
Universal tests
Psychometrics
Task difficulty
CAPTCHA
LENGUAJES Y SISTEMAS INFORMATICOS
title_short IQ tests are not for machines, yet
title_full IQ tests are not for machines, yet
title_fullStr IQ tests are not for machines, yet
title_full_unstemmed IQ tests are not for machines, yet
title_sort IQ tests are not for machines, yet
dc.creator.none.fl_str_mv Dowe, David L.
Hernández-Orallo, José|||0000-0001-9746-7632
author Dowe, David L.
author_facet Dowe, David L.
Hernández-Orallo, José|||0000-0001-9746-7632
author_role author
author2 Hernández-Orallo, José|||0000-0001-9746-7632
author2_role author
dc.contributor.none.fl_str_mv Departamento de Sistemas Informáticos y Computación
Escuela Técnica Superior de Ingeniería Informática
Instituto Universitario Valenciano de Investigación en Inteligencia Artificial
Generalitat Valenciana
Ministerio de Ciencia e Innovación
Repositorio Institucional de la Universitat Politècnica de València Riunet
dc.subject.none.fl_str_mv Machine intelligence evaluation
IQ tests
Artificial intelligence
Universal tests
Psychometrics
Task difficulty
CAPTCHA
LENGUAJES Y SISTEMAS INFORMATICOS
topic Machine intelligence evaluation
IQ tests
Artificial intelligence
Universal tests
Psychometrics
Task difficulty
CAPTCHA
LENGUAJES Y SISTEMAS INFORMATICOS
description [EN] Complex, but specific, tasks¿such as chess or Jeopardy!¿are popularly seen as milestones for artificial intelligence (AI). However, they are not appropriate for evaluating the intelligence of machines or measuring the progress in AI. Aware of this delusion, Detterman has recently raised a challenge prompting AI researchers to evaluate their artefacts against IQ tests. We agree that the philosophy behind (human) IQ tests is a much better approach to machine intelligence evaluation than these specific tasks, and also more practical and informative than the Turing test. However, we have first to recall some work on machine intelligence measurement which has shown that some IQ tests can be passed by relatively simple programs. This suggests that the challenge may not be so demanding and may just work as a sophisticated CAPTCHA, since some types of tests might be easier than others for the current state of AI. Second, we show that an alternative, formal derivation of intelligence tests for machines is possible, grounded in (algorithmic) information theory. In these tests, we have a proper mathematical definition of what is being measured. Third, we re-visit some research done in the past fifteen years for effectively measuring machine intelligence¿since some assumptions about the subjects and their distribution no longer hold.
publishDate 2012
dc.date.none.fl_str_mv 2012
2012-04-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://riunet.upv.es/handle/10251/36157
url https://riunet.upv.es/handle/10251/36157
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Ministerio de Ciencia e Innovación http://dx.doi.org/10.13039/501100004837 TIN2009-06078-E ANYTIME UNIVERSAL INTELLIGENCE
Ministerio de Ciencia e Innovación http://dx.doi.org/10.13039/501100004837 TIN2010-21062-C02-02 SWEETLOGICS-UPV
Generalitat Valenciana https://doi.org/10.13039/501100003359 PROMETEO08%2F2008%2F051 Advances on Agreement Technologies for Computational Entities (atforce)
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reserva de todos los derechos
http://rightsstatements.org/vocab/InC/1.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reserva de todos los derechos
http://rightsstatements.org/vocab/InC/1.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
instname:Universitat Politècnica de València (UPV)
instname_str Universitat Politècnica de València (UPV)
reponame_str RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
collection RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869422381978615808
score 15.301603

Similares en LA Referencia