Test Oracle Generation for REST APIs

The number and complexity of test case generation tools for REST APIs have significantly increased in recent years. These tools excel in automating input generation but are limited by their test oracles, which can only detect crashes, regressions, and violations of API specifications or design best...

Descripción completa

Detalles Bibliográficos
Autores: Alonso Valenzuela, Juan Carlos, Ernst, Michael D., Segura Rueda, Sergio, Ruiz Cortés, Antonio
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2025
País:España
Institución:Universidad de Sevilla (US)
Repositorio:idUS. Depósito de Investigación de la Universidad de Sevilla
OAI Identifier:oai:dnet:idus________::a9f9770f189f31f9f0f085fdf1a28df2
Acceso en línea:https://hdl.handle.net/11441/186407
https://doi.org/10.1145/3726524
Access Level:acceso abierto
Palabra clave:REST APIs
test oracle
invariant detection
automated testing
id ES_9ec24c8cd2c8b3724513f5279389cab4
oai_identifier_str oai:dnet:idus________::a9f9770f189f31f9f0f085fdf1a28df2
network_acronym_str ES
network_name_str España
repository_id_str
spelling Test Oracle Generation for REST APIsAlonso Valenzuela, Juan CarlosErnst, Michael D.Segura Rueda, SergioRuiz Cortés, AntonioREST APIstest oracleinvariant detectionautomated testingThe number and complexity of test case generation tools for REST APIs have significantly increased in recent years. These tools excel in automating input generation but are limited by their test oracles, which can only detect crashes, regressions, and violations of API specifications or design best practices. This article introduces AGORA+, an approach for generating test oracles for REST APIs through the detection of invariants—output properties that should always hold. AGORA+ learns the expected behavior of an API by analyzing API requests and their corresponding responses. We enhanced the Daikon tool for dynamic detection of likely invariants, adding new invariant types and creating a front-end called Beet. Beet translates any OpenAPI specification and a set of API requests and responses into Daikon inputs. AGORA+ can detect 106 different types of invariants in REST APIs. We also developed PostmanAssertify, which converts the invariants identified by AGORA+ into executable JavaScript assertions. AGORA+ achieved a precision of 80% on 25 operations from 20 industrial APIs. It also identified 48% of errors systematically seeded in the outputs of the APIs under test. AGORA+ uncovered 32 bugs in popular APIs, including Amadeus, Deutschebahn, GitHub, Marvel, NYTimesBooks, and YouTube, leading to fixes and documentation updates.Association for Computing Machinery (ACM)Lenguajes y Sistemas InformáticosTIC205: Ingeniería del Software AplicadaMinisterio de Ciencia e Innovación (MICIN). EspañaEuropean Union (UE)2025info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttps://hdl.handle.net/11441/186407https://doi.org/10.1145/3726524reponame:idUS. Depósito de Investigación de la Universidad de Sevillainstname:Universidad de Sevilla (US)InglésACM Transactions on Software Engineering and Methodology, 35, 1 p.-37 p.. PID2021-126227NB-C22PID2021-126227NB-C21TED2021-131023B-C21https://dl.acm.org/doi/10.1145/3726524info:eu-repo/semantics/openAccessoai:dnet:idus________::a9f9770f189f31f9f0f085fdf1a28df22026-06-17T12:51:07Z
dc.title.none.fl_str_mv Test Oracle Generation for REST APIs
title Test Oracle Generation for REST APIs
spellingShingle Test Oracle Generation for REST APIs
Alonso Valenzuela, Juan Carlos
REST APIs
test oracle
invariant detection
automated testing
title_short Test Oracle Generation for REST APIs
title_full Test Oracle Generation for REST APIs
title_fullStr Test Oracle Generation for REST APIs
title_full_unstemmed Test Oracle Generation for REST APIs
title_sort Test Oracle Generation for REST APIs
dc.creator.none.fl_str_mv Alonso Valenzuela, Juan Carlos
Ernst, Michael D.
Segura Rueda, Sergio
Ruiz Cortés, Antonio
author Alonso Valenzuela, Juan Carlos
author_facet Alonso Valenzuela, Juan Carlos
Ernst, Michael D.
Segura Rueda, Sergio
Ruiz Cortés, Antonio
author_role author
author2 Ernst, Michael D.
Segura Rueda, Sergio
Ruiz Cortés, Antonio
author2_role author
author
author
dc.contributor.none.fl_str_mv Lenguajes y Sistemas Informáticos
TIC205: Ingeniería del Software Aplicada
Ministerio de Ciencia e Innovación (MICIN). España
European Union (UE)
dc.subject.none.fl_str_mv REST APIs
test oracle
invariant detection
automated testing
topic REST APIs
test oracle
invariant detection
automated testing
description The number and complexity of test case generation tools for REST APIs have significantly increased in recent years. These tools excel in automating input generation but are limited by their test oracles, which can only detect crashes, regressions, and violations of API specifications or design best practices. This article introduces AGORA+, an approach for generating test oracles for REST APIs through the detection of invariants—output properties that should always hold. AGORA+ learns the expected behavior of an API by analyzing API requests and their corresponding responses. We enhanced the Daikon tool for dynamic detection of likely invariants, adding new invariant types and creating a front-end called Beet. Beet translates any OpenAPI specification and a set of API requests and responses into Daikon inputs. AGORA+ can detect 106 different types of invariants in REST APIs. We also developed PostmanAssertify, which converts the invariants identified by AGORA+ into executable JavaScript assertions. AGORA+ achieved a precision of 80% on 25 operations from 20 industrial APIs. It also identified 48% of errors systematically seeded in the outputs of the APIs under test. AGORA+ uncovered 32 bugs in popular APIs, including Amadeus, Deutschebahn, GitHub, Marvel, NYTimesBooks, and YouTube, leading to fixes and documentation updates.
publishDate 2025
dc.date.none.fl_str_mv 2025
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/11441/186407
https://doi.org/10.1145/3726524
url https://hdl.handle.net/11441/186407
https://doi.org/10.1145/3726524
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv ACM Transactions on Software Engineering and Methodology, 35, 1 p.-37 p..
PID2021-126227NB-C22
PID2021-126227NB-C21
TED2021-131023B-C21
https://dl.acm.org/doi/10.1145/3726524
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Association for Computing Machinery (ACM)
publisher.none.fl_str_mv Association for Computing Machinery (ACM)
dc.source.none.fl_str_mv reponame:idUS. Depósito de Investigación de la Universidad de Sevilla
instname:Universidad de Sevilla (US)
instname_str Universidad de Sevilla (US)
reponame_str idUS. Depósito de Investigación de la Universidad de Sevilla
collection idUS. Depósito de Investigación de la Universidad de Sevilla
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869414853223907328
score 15,812429

Similares en LA Referencia