On environment difficulty and discriminating power

This paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on a very general setting for tasks: interactive (possibly multiagent) environments where an agent acts upon observations and rewards. Instead of analysing the complexity of the environment,...

Descripción completa

Detalles Bibliográficos
Autor: Hernández-Orallo, José|||0000-0001-9746-7632
Tipo de recurso: artículo
Fecha de publicación:2015
País:España
Institución:Universitat Politècnica de València (UPV)
Repositorio:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Idioma:inglés
OAI Identifier:oai:riunet.upv.es:10251/64792
Acceso en línea:https://riunet.upv.es/handle/10251/64792
Access Level:acceso abierto
Palabra clave:Environment difficulty
Agent evaluation
Discriminating power
Agent policy
Algorithmic information theory
Universal psychometrics
Reinforcement learning
Elementary cellular automata
LENGUAJES Y SISTEMAS INFORMATICOS
id ES_ff1c6aaff92fdb871a68fbb73cf334bd
oai_identifier_str oai:riunet.upv.es:10251/64792
network_acronym_str ES
network_name_str España
repository_id_str
dc.title.none.fl_str_mv On environment difficulty and discriminating power
title On environment difficulty and discriminating power
spellingShingle On environment difficulty and discriminating power
Hernández-Orallo, José|||0000-0001-9746-7632
Environment difficulty
Agent evaluation
Discriminating power
Agent policy
Algorithmic information theory
Universal psychometrics
Reinforcement learning
Elementary cellular automata
LENGUAJES Y SISTEMAS INFORMATICOS
title_short On environment difficulty and discriminating power
title_full On environment difficulty and discriminating power
title_fullStr On environment difficulty and discriminating power
title_full_unstemmed On environment difficulty and discriminating power
title_sort On environment difficulty and discriminating power
dc.creator.none.fl_str_mv Hernández-Orallo, José|||0000-0001-9746-7632
author Hernández-Orallo, José|||0000-0001-9746-7632
author_facet Hernández-Orallo, José|||0000-0001-9746-7632
author_role author
dc.contributor.none.fl_str_mv Departamento de Sistemas Informáticos y Computación
Escuela Técnica Superior de Ingeniería Informática
Instituto Universitario Valenciano de Investigación en Inteligencia Artificial
Generalitat Valenciana
European Cooperation in Science and Technology
Ministerio de Educación y Ciencia
Ministerio de Ciencia e Innovación
Repositorio Institucional de la Universitat Politècnica de València Riunet
dc.subject.none.fl_str_mv Environment difficulty
Agent evaluation
Discriminating power
Agent policy
Algorithmic information theory
Universal psychometrics
Reinforcement learning
Elementary cellular automata
LENGUAJES Y SISTEMAS INFORMATICOS
topic Environment difficulty
Agent evaluation
Discriminating power
Agent policy
Algorithmic information theory
Universal psychometrics
Reinforcement learning
Elementary cellular automata
LENGUAJES Y SISTEMAS INFORMATICOS
description This paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on a very general setting for tasks: interactive (possibly multiagent) environments where an agent acts upon observations and rewards. Instead of analysing the complexity of the environment, the state space or the actions that are performed by the agent, we analyse the performance of a population of agent policies against the task, leading to a distribution that is examined in terms of policy complexity. This distribution is then sliced by the algorithmic complexity of the policy and analysed through several diagrams and indicators. The notion of environment response curve is also introduced, by inverting the performance results into an ability scale. We apply all these concepts, diagrams and indicators to two illustrative problems: a class of agent-populated elementary cellular automata, showing how the difficulty and discriminating power may vary for several environments, and a multiagent system, where agents can become predators or preys, and may need to coordinate. Finally, we discuss how these tools can be applied to characterise (interactive) tasks and (multi-agent) environments. These characterisations can then be used to get more insight about agent performance and to facilitate the development of adaptive tests for the evaluation of agent abilities.
publishDate 2015
dc.date.none.fl_str_mv 2015
2015-05-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://riunet.upv.es/handle/10251/64792
url https://riunet.upv.es/handle/10251/64792
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Ministerio de Educación y Ciencia https://doi.org/10.13039/501100008743 CSD2007-00022 Agreement Technologies
European Cooperation in Science and Technology https://doi.org/10.13039/501100000921 IC0801 Agreement Technologies
Ministerio de Ciencia e Innovación http://dx.doi.org/10.13039/501100004837 TIN2010-21062-C02-02 SWEETLOGICS-UPV
Generalitat Valenciana https://doi.org/10.13039/501100003359 PROMETEO08%2F2008%2F051 Advances on Agreement Technologies for Computational Entities (atforce)
Ministerio de Economía y Competitividad http://dx.doi.org/10.13039/501100003329 PCIN-2013-037 RETHINKING THE ESSENCE, FLEXIBILITY AND REUSABILITY OF ADVANCED MODEL EXPLOITATION
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reserva de todos los derechos
http://rightsstatements.org/vocab/InC/1.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reserva de todos los derechos
http://rightsstatements.org/vocab/InC/1.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Springer Verlag (Germany)
publisher.none.fl_str_mv Springer Verlag (Germany)
dc.source.none.fl_str_mv reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
instname:Universitat Politècnica de València (UPV)
instname_str Universitat Politècnica de València (UPV)
reponame_str RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
collection RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869425745078517760
spelling On environment difficulty and discriminating powerHernández-Orallo, José|||0000-0001-9746-7632Environment difficultyAgent evaluationDiscriminating powerAgent policyAlgorithmic information theoryUniversal psychometricsReinforcement learningElementary cellular automataLENGUAJES Y SISTEMAS INFORMATICOSThis paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on a very general setting for tasks: interactive (possibly multiagent) environments where an agent acts upon observations and rewards. Instead of analysing the complexity of the environment, the state space or the actions that are performed by the agent, we analyse the performance of a population of agent policies against the task, leading to a distribution that is examined in terms of policy complexity. This distribution is then sliced by the algorithmic complexity of the policy and analysed through several diagrams and indicators. The notion of environment response curve is also introduced, by inverting the performance results into an ability scale. We apply all these concepts, diagrams and indicators to two illustrative problems: a class of agent-populated elementary cellular automata, showing how the difficulty and discriminating power may vary for several environments, and a multiagent system, where agents can become predators or preys, and may need to coordinate. Finally, we discuss how these tools can be applied to characterise (interactive) tasks and (multi-agent) environments. These characterisations can then be used to get more insight about agent performance and to facilitate the development of adaptive tests for the evaluation of agent abilities.I thank the reviewers for their comments, especially those aiming at a clearer connection with the field of multi-agent systems and the suggestion of better approximations for the calculation of the response curves. The implementation of the elementary cellular automata used in the environments is based on the library 'CellularAutomaton' by John Hughes for R [58]. I am grateful to Fernando Soler-Toscano for letting me know about their work [65] on the complexity of 2D objects generated by elementary cellular automata. I would also like to thank David L. Dowe for his comments on a previous version of this paper. This work was supported by the MEC/MINECO projects CONSOLIDER-INGENIO CSD2007-00022 and TIN 2010-21062-C02-02, GVA project PROMETEO/2008/051, the COST - European Cooperation in the field of Scientific and Technical Research IC0801 AT, and the REFRAME project, granted by the European Coordinated Research on Long-term Challenges in Information and Communication Sciences & Technologies ERA-Net (CHIST-ERA), and funded by the Ministerio de Economia y Competitividad in Spain (PCIN-2013-037).Springer Verlag (Germany)Departamento de Sistemas Informáticos y ComputaciónEscuela Técnica Superior de Ingeniería InformáticaInstituto Universitario Valenciano de Investigación en Inteligencia ArtificialGeneralitat ValencianaEuropean Cooperation in Science and TechnologyMinisterio de Educación y CienciaMinisterio de Ciencia e InnovaciónRepositorio Institucional de la Universitat Politècnica de València Riunet20152015-05-01journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfapplication/pdfhttps://riunet.upv.es/handle/10251/64792reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valénciainstname:Universitat Politècnica de València (UPV)InglésengMinisterio de Educación y Ciencia https://doi.org/10.13039/501100008743 CSD2007-00022 Agreement TechnologiesEuropean Cooperation in Science and Technology https://doi.org/10.13039/501100000921 IC0801 Agreement TechnologiesMinisterio de Ciencia e Innovación http://dx.doi.org/10.13039/501100004837 TIN2010-21062-C02-02 SWEETLOGICS-UPVGeneralitat Valenciana https://doi.org/10.13039/501100003359 PROMETEO08%2F2008%2F051 Advances on Agreement Technologies for Computational Entities (atforce)Ministerio de Economía y Competitividad http://dx.doi.org/10.13039/501100003329 PCIN-2013-037 RETHINKING THE ESSENCE, FLEXIBILITY AND REUSABILITY OF ADVANCED MODEL EXPLOITATIONopen accesshttp://purl.org/coar/access_right/c_abf2Reserva de todos los derechoshttp://rightsstatements.org/vocab/InC/1.0/info:eu-repo/semantics/openAccessoai:riunet.upv.es:10251/647922026-06-13T07:49:27Z
score 15.300719

Similares en LA Referencia