Búsqueda avanzada

El autor Sergio Segura ha publicado 21 artículo(s):

1 - EXEMPLAR: An Experimental Information Repository for Software Engineering Research

The number and variety of experiments carried in software engineering research is growing, leading to a increasing need of replication and review. In order to support such needs the information about experiments should be provided as lab-packs comprising of: a description of the experiment, the materials used and data generated during the conduction, and the results of the analyzes performed on such data. However, this information is often scattered, poorly structured, and even unavailable, implying a tedious process of search and gathering. EXEMPLAR is an online platform for managing experimental information, that allows the uploading and publication of experimental lab packs, and an efficient search. The platform also supports the use of formal languages for providing experimental descriptions (e.g. SEDL). In so doing, EXEMPLAR enables the automated analysis of lab-packs, in order to detect common validity threats and missing information which could hinder replicability.

Autores: Jose Antonio Parejo / Sergio Segura / Pablo Fernandez / Antonio Ruiz Cortés / 
Palabras Clave: empirical research - experimental replicability - experimental repositories - experiments

2 - Towards Multi-Objective Test Case generation for Variability-Intensive Systems

Testing variability-intensive systems is a challenge due to the potentially huge number of derivable configurations. To alleviate this problem, many test case selection and prioritization techniques have been proposed with the aim of reducing the number of configurations to be tested and increasing their effectiveness. However, we found that these approaches do not exploit all available information since they are mainly driven by functional information such as the feature coverage. Furthermore, most of these works are focused on a single-objective perspective (e.g. features coverage), which could not reflect the real scenarios where several goals need to be met (e.g. features coverage and code changes coverage). In this context, we identify an important challenge, to take advantage of all available system information to guide the generation of test cases. As a first step towards a solution, we propose to study all this information with special emphasis on non-functional properties and address the test case generation as a multi-objective problem. Also, we describe some open issues to be explored that we hope have an important impact on future evaluations.

Autores: Ana B. Sánchez / Sergio Segura / Antonio Ruiz-Cortés / 
Palabras Clave: extra-functional attributes - Multi-objective test generation

3 - Automated Metamorphic Testing on the Analysis of Feature Models

Software Product Line (SPL) engineering is a reuse strategy to develop families of related systems. From common assets, different software products are assembled reducing production costs and time­to­market. Products in SPLs are defined in terms of features. A feature is an increment in product functionality. Feature models are widely used to represent all the valid combinations of features (i.e. products) of an SPL in a single model in terms of features and relations among them. The automated analysis of feature models deals with the computer­aided extraction of information from feature models. Typical operations of analysis allow determining whether a feature model is void (i.e. it represents no products), whether it contains errors (e.g. features that cannot be part of any product) or what is the number of products of the SPL represented by the model. Catalogues with up to 30 analysis operations on feature models and multiple analysis solutions have been reported.

Autores: Sergio Segura / Robert M. Hierons / David Benavides / Antonio Ruiz-Cortés / 
Palabras Clave:

4 - Automated testing on the analysis of variability-intensive artifacts: An exploratory study with SAT Solvers

The automated detection of faults on variability analysis tools is a challenging task often infeasible due to the combinatorial complexity of the analyses. In previous works, we successfully automated the generation of test data for feature model analysis tools using metamorphic testing. The positive results obtained have encouraged us to explore the applicability of this technique for the efficient detection of faults in other variability-intensive domains. In this paper, we present an automated test data generator for SAT solvers that enables the generation of random propositional formulas (inputs) and their solutions (expected output). In order to show the feasibility of our approach, we introduced 100 artificial faults (i.e. mutants) in an open source SAT solver and compared the ability of our generator and three related benchmarks to detect them. Our results are promising and encourage us to generalize the technique, which could be potentially applicable to any tool dealing with variability such as Eclipse repositories or Maven dependencies analyzers.

Autores: Ana B. Sánchez / Sergio Segura / 
Palabras Clave:

5 - BeTTy: Un Framework de Pruebas para el Análisis Automático de Modelos de Características

El análisis automático de modelos de características es un área de investigación activo que ha llamado la atencíon de numerosos investigadores durante las dos últimas décadas. Durante este tiempo, el número de herramientas y técnicas para el análisis de modelos de características se ha multiplicado y con ellas su complejidad. En este escenario, la falta de mecanismos específicos para validar y evaluar la funcionalidad y el rendimiento de las herramientas de análisis se ha convertido en un gran obstáculo dificultando el desarrollo de herramientas y afectando negativamente a su calidad y fiabilidad. En este artículo, presentamos BeTTy, un framework para la automatizacíon de pruebas en el análisis de modelos de características. Entre otras funcionalidades, BeTTy permite la deteccíon automática de errores en herramientas de análisis de modelos de características. Además, BeTTy permite generar modelos de características tanto aleatorios como computacionalmente duros, útiles para evaluar el rendimiento de las herramientas de análisis. Parte de la funcionalidad del framework es ofrecida a través de una aplicacíon Web que facilita en gran medida su uso.

Autores: Sergio Segura / José A. Galindo / David Benavides / José A. Parejo / 
Palabras Clave:

7 - A Survey on Metamorphic Testing

S. Segura, G. Fraser, A. B. Sanchez and A. Ruiz-Cortés, A Survey on Metamorphic Testing, in IEEE Transactions on Software Engineering, vol. 42, no. 9, pp. 805-824, Sept. 1 2016. Indicadores de calidad: – Revista de referencia en el área de Ingeniería del Software (CS-SE: 20/106). – Ha recibido 9 citas desde su publicación en febrero de 2016 (más otras 5-7 citas por aparecer en las actas del segundo workshop internacional de pruebas metamórficas [1]). – Hemos sido invitados a presentar el trabajo en ICSE17 como parte de la iniciativa journal-first (ver programa de la conferencia [2]). – Colaboración internacional con el profesor Gordon Fraser. [1] [2]

Autores: Sergio Segura / Gordon Fraser / Ana B. Sánchez / Antonio Ruiz-Cortés / 
Palabras Clave: metamorphic testing - oracle problem - survey

8 - Multi-Objective Test Case Prioritization in Highly Configurable Systems: A Case Study

Test case prioritization schedules test cases for execution in an order that attempts to accelerate the detection of faults. The order of test cases is determined by prioritization objectives such as covering code or critical components as rapidly as possible. The importance of this technique has been recognized in the context of Highly-Configurable Systems (HCSs), where the potentially huge number of configurations makes testing extremely challenging. However, current approaches for test case prioritization in HCSs suffer from two main limitations. First, the prioritization is usually driven by a single objective which neglects the potential benefits of combining multiple criteria to guide the detection of faults. Second, instead of using industry-strength case studies, evaluations are conducted using synthetic data, which provides no information about the effectiveness of different prioritization objectives. In this paper, we address both limitations by studying 63 combinations of up to three prioritization objectives in accelerating the detection of faults in the Drupal framework. Results show that non-functional properties such as the number of changes in the features are more effective than functional metrics extracted from the configuration model. Results also suggest that multi-objective prioritization typically results in faster fault detection than mono-objective prioritization. Indicios de calidad de la revista: Journal of Systems and Software (Elsevier) ISSN: 0164-1212 Factor de impacto 2015: 1,424 Factor de impacto a 5 años: 1,767 Indexada en dos categorías: Computer Science / Theory & Methods: 31/105 (Q2) Computer Science / Software Engineering: 24/106 (Q1) Otros datos: CiteScore: 2.93 Source Normalized Impact per Paper (SNIP): 2.415 SCImago Journal Rank (SJR): 0.897 Indicios de calidad del propio paper: Número de Citas según Google Scholar: 3 Número de lecturas según Research Gate: 73

Autores: José Antonio Parejo Maestre / Ana Belén Sánchez Jerez / Sergio Segura / Antonio Ruiz-Cortés / Roberto Erick Lopez-Herrejón / Alexander Egyed / 
Palabras Clave: optimización multi-objetivo - priorización de pruebas - sistemas altamente configurables

9 - An Approach for Debugging Model Transformations Applying Spectrum-Based Fault Localization

Model transformations play a cornerstone role in Model-Driven Engineering as they provide the essential mechanisms for manipulating and transforming models. The use of assertions for checking their correctness has been proposed in several works. However, it is still challenging and error prone to locate the faulty rules, and the situation gets more critical as the size and complexity of model transformations grow, where manual debugging is no longer possible. Spectrum-Based Fault Localization (SBFL) is a technique for software debugging that uses the results of test cases and their corresponding code coverage information to estimate the likelihood of each program component (e.g., statements) of being faulty. This paper describes a proposal for applying SBFL for locating the faulty rules in ATL model transformations. The approach aims at automatically detecting the transformation rule that makes an assertion fail.

Autores: Javier Troya / Sergio Segura / José Antonio Parejo Maestre / Antonio Ruiz-Cortés / 
Palabras Clave: Debugging - Fault Localization - Model Transformation - Spectrum

10 - SIP: Optimal Product Selection from Feature Models Using Many-Objective Evolutionary Optimization

Robert M. Hierons, Miqing Li, Xiaohui Liu, Sergio Segura, and Wei Zheng. 2016. SIP: Optimal Product Selection from Feature Models Using Many-Objective Evolutionary Optimization. ACM Trans. Softw. Eng. Methodol. 25, 2, Article 17 (April 2016), 39 pages. DOI: Indicadores de calidad: – Revista de referencia en el área de Ingeniería del Software (CS-SE: 21/106). – Colaboración internacional con los profesores Robert Hierons [1] y XiaoHui Liu [2]. – Hemos sido invitados a presentar el trabajo en FSE16 e ICSE17 como parte de la iniciativa journal-first (ver programa de la conferencia [3]). – Ha recibido 6 citas desde su publicación en abril de 2016 [4]. [1] [2] [2] [4]

Autores: Rob Hierons / Miqing Li / Xiaohui Liu Liu / Sergio Segura / Wei Zheng / 
Palabras Clave: Optimization - Search-Based Software Engineering - software product lines

11 - ARTICULO RELEVANTE: Metamorphic Testing of RESTful Web APIs

S. Segura, J. A. Parejo, J. Troya and A. Ruiz-Cortés, «Metamorphic Testing of RESTful Web APIs» in IEEE Transactions on Software Engineering, Oct 2017 (online) vol. PP, no. 99, pp. 1-1. para ser presentado en ICSE 2018 en la categoría de journal-first: lecturas en ResearchGate desde su publicación (118 en IEEE Xplore).

Autores: Sergio Segura / José Antonio Parejo Maestre / Javier Troya / Antonio Ruiz-Cortés / 
Palabras Clave: metamorphic testing - oracle problem - Web APIs

12 - ARTICULO RELEVANTE:Assessment of C++ object-oriented mutation operators: A selective mutation approach

La prueba de mutaciones está considerada como una técnicaefectiva, pero que es costosa en contrapartida. Varios estudios han pues-to de manifiesto que pueden existir mutantes redundantes y que, unavez eliminados, no suponen pérdida de la efectividad de la técnica. De lamisma manera, algunos mutantes pueden ser más efectivos que otros ala hora de guiarnos en la generación de nuevos casos de pruebas de altacalidad. En base a estos dos hallazgos, en este artículo presentamos unaevaluación de los operadores de mutación definidos para C++ en la queclasificamos dichos operadores en dos rankings diferentes tras estudiar losmutantes que cada uno de los operadores genera. La primera clasificaciónordena los operadores según el grado de redundancia de sus mutantes yla segunda en base a la calidad de las pruebas que ayudan a diseñar. Unavez establecidos ambos rankings, llevamos a cabo un proceso de mutaciónselectiva en el que seleccionamos subconjuntos de operadores desechandolos menos valorados. El objetivo es determinar qué relación existe entrela reducción que se obtendría al eliminar estos operadores y la pérdidade efectividad. Los resultados experimentales muestran de una maneraconsistente que al seleccionar los operadores que están en la parte altadel ranking podemos obtener una reducción significativa en el númerode mutantes con una mínima pérdida de efectividad. Esto se produce enambas clasificaciones a pesar de que los operadores se ordenan de for-ma distinta, lo cual valida el planteamiento de valorar los operadores demutación de una manera diferente según nuestro objetivo sea evaluar orefinar el conjunto de pruebas.P. Delgado-Pérez, S. Segura and I. Medina-Bulo, “Assessment of C++ object-oriented mutation operators: A selective mutation approach,” Software Testing,Verification and Reliability, vol. 27, num. 4–5, pp. e1630, 2017. de impacto revista: 1.588

Autores: Pedro Delgado-Pérez / Sergio Segura / Inmaculada Medina-Bulo / 
Palabras Clave: C++ - calidad de operadores de mutación - mutación selectiva - operadores de mutación de clase - prueba de mutaciones

13 - Evaluación y mejora de pruebas de rendimiento utilizando mutación del software: Un enfoque evolutivo

Los errores de rendimiento del software pueden causar una importante degradación en la experiencia de usuario y dar lugar a problemas muy costosos de detectar y resolver. Las pruebas de rendimiento persiguen detectar y reducir el impacto de estos errores. Sin embargo, no existen mecanismos para evaluar la calidad de las pruebas de rendimiento, causando en muchos casos, que estos errores pasen desapercibidos. La prueba de mutación es una técnica para evaluar y mejorar las pruebas funcionales a través de la introducción de errores artificiales en el programa bajo prueba. En este artículo, exploramos la aplicabilidad de la prueba de mutación junto con el empleo de un algoritmo evolutivo para buscar mutantes que simulen errores de rendimiento. Esta propuesta noPrueba de mutación, errores de rendimiento, pruebas de rendimiento, algoritmos evolutivos.vedosa contribuye a mejorar la confianza en las pruebas de rendimiento al mismo tiempo que reduce el coste de la prueba de mutación.

Autores: Ana B. Sánchez / Pedro Delgado-Pérez / Inmaculada Medina-Bulo / Sergio Segura / 
Palabras Clave: algoritmos evolutivos - errores de rendimiento - prueba de mutación - Pruebas de Rendimiento

14 - Evolutionary composition of QoS-aware web services: A many-objective perspective

Web service based applications often invoke services provided by third-parties in their workflow. The Quality of Service (QoS) provided by the invoked supplier can be expressed in terms of the Service Level Agreement specifying the values contracted for particular aspects like cost or throughput, among others. In this scenario, intelligent systems can support the engineer to scrutinise the service market in order to select those candidates that best fit with the expected composition focusing on different QoS aspects. This search problem, also known as QoS-aware web service composition, is characterised by the presence of many diverse QoS properties to be simultaneously optimised from a multi-objective perspective. Nevertheless, as the number of QoS properties considered during the design phase increases and a larger number of decision factors come into play, it becomes more difficult to find the most suitable candidate solutions, so more sophisticated techniques are required to explore and return diverse, competitive alternatives. With this aim, this paper explores the suitability of many objective evolutionary algorithms for addressing the binding problem of web services on the basis of a real-world benchmark with 9 QoS properties. A complete comparative study demonstrates that these techniques, never before applied to this problem, can achieve a better trade-off between all the QoS properties, or even promote specific QoS properties while keeping high values for the rest. In addition, this search process can be performed within a reasonable computational cost, enabling its adoption by intelligent and decision-support systems in the field of service oriented computation.Publicado en: Expert Systems with Applications, vol. 72, pp.357-370. 2017. DOI: 3,928 [18/133 Artificial Intelligence] (Q1).

Autores: Aurora Ramírez / José Antonio Parejo / José Raúl Romero / Sergio Segura / Antonio Ruiz-Cortés / 
Palabras Clave: many-objective evolutionary algorithms - multi-objective optimization - QoS-aware web service composition

15 - Automated inference of likely metamorphic relations for model transformations (YA PUBLICADO)

Model transformations play a cornerstone role in Model-Driven Engineering (MDE) as they provide the essential mechanisms for manipulating and transforming models. Checking whether the output of a model transformation is correct is a manual and errorprone task, referred to as the oracle problem. Metamorphic testing alleviates the oracle problem by exploiting the relations among different inputs and outputs of the program under test, so-called metamorphic relations (MRs). One of the main challenges in metamorphic testing is the automated inference of likely MRs. This paper proposes an approach to automatically infer likely MRs for ATL model transformations, where the tester does not need to have any knowledge of the transformation. The inferred MRs aim at detecting faults in model transformations in three application scenarios, namely regression testing, incremental transformations and migrations among transformation languages. In the experiments performed, the inferred likely MRs have proved to be quite accurate, with a precision of 96.4% from a total of 4101 true positives out of 4254 MRs inferred. Furthermore, they have been useful for identifying mutants in regression testing scenarios, with a mutation score of 93.3%. Finally, our approach can be used in conjunction with current approaches for the automatic generation of test cases. Artículo publicado en The Journal of Systems and Software, Vol 136, pp 188-208 (Available Online May 2017; Final Published Version February 2018) – Q1.

Autores: Javier Troya / Sergio Segura / Antonio Ruiz-Cortés / 
Palabras Clave: Automatic inference - Generic approach - Metamorphic relations - metamorphic testing - model transformations - Model-Driven Engineering

16 - Performance metamorphic testing: A Proof of concept

Sergio Segura, Javier Troya, Amador Durán, Antonio Ruiz-Cortés. «Performance Metamorphic Testing: A Proof of Concept». Information and Software Technology, 98:1-4, 2018. indicators:- JCR FI: 2.62 TOP 15% (Q1) CS/SE.- 2 citations in GScholar ( preliminary version of this paper was presented in the track of New Ideas an Emerging Results at ICSE 2017, with an acceptance rate of 16% (14 papers accepted out of 85 submissions) [1]. All four reviewers agreed on the value of the work with an overall score of 9 (out of 12), and a novelty score of 11 (out of 12).[1] Sergio Segura, Javier Troya, Amador Durán, Antonio Ruiz-Cortés. «Performance metamorphic testing: motivation and challenges». In Proceedings of the 39th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE NIER’17) Track. IEEE Press, Piscataway, NJ, USA, 7-10, 2017. [Acceptance rate: 16%. Main track ranked as Class 1 in SCIE Ranking]

Autores: Sergio Segura / Javier Troya / Amador Durán Toro / Antonio Ruiz-Cortés / 
Palabras Clave: metamorphic testing - performance testing - Search–based testing

17 - Performance mutation testing: hypothesis and open questions

Performance bugs are common, costly, and elusive. Performance tests aim to detect performance bugs by running the program with specific inputs and determining whether the observed behaviour is acceptable. There not exist mechanisms, however, to assess the effectiveness of performance tests. Mutation testing is a technique to evaluate and enhance functional test suites by seeding artificial faults in the program under test. In this new idea paper, we explore the applicability of mutation testing to assess and improve performance tests. This novel approach is motivated with examples and open questions.Ana B. Sánchez, Pedro Delgado-Pérez, Sergio Segura, Inmaculada Medina-Bulo. Performance Mutation Testing: Hypothesis and open questions. Information and Software Technology Journal, 103, 159-161, November 2018. Disponible online:ÍNDICES DE CALIDADFactor de impacto: 2,694Categoría de la revista: Q1Posición en su categoría: 16Número de autores: 4

Autores: Ana B. Sánchez / Pedro Delgado-Pérez / Sergio Segura / Inmaculada Medina-Bulo / 
Palabras Clave: Mutation testing - performance bugs - performance testing

18 - Spectrum-Based Fault Localization in Model Transformations

Model transformations play a cornerstone role in Model-Driven Engineering as they provide the essential mechanisms for manipulating and transforming models. The correctness of software built using MDE techniques greatly relies on the correctness of model transformations. However, it is challenging and error prone to debug them, and the situation gets more critical as the size and complexity of model transformations grow, where manual debugging is no longer possible.Spectrum-Based Fault Localization (SBFL) uses the results of test cases and their corresponding code coverage information to estimate the likelihood of each program component (e.g., statements) of being faulty. In this paper we present an approach to apply SBFL for locating the faulty rules in model transformations. We evaluate the feasibility and accuracy of the approach by comparing the effectiveness of 18 different state-of-the-art SBFL techniques at locating faults in model transformations. Evaluation results revealed that the best techniques, namely Kulcynski2, Mountford, Ochiai and Zoltar, lead the debugger to inspect a maximum of three rules in order to locate the bug in around 74% of the cases. Furthermore, we compare our approach with a static approach for fault localization in model transformations, observing a clear superiority of the proposed SBFL-based method.

Autores: Javier Troya / Sergio Segura / José Antonio Parejo Maestre / Antonio Ruiz-Cortés / 
Palabras Clave: Debugging - Fault Localization - Model Transformation - Spectrum-based - Testing

19 - Towards the Definition of Test Coverage Criteria for RESTful Web APIs

Web APIs following the REST architectural style (so-called RESTful Web APIs) have become the de-facto standard for software integration. As RESTful APIs gain momentum, so does the testing of them. However, there is a lack of mechanisms to assess the adequacy of testing approaches in this context, which makes it difficult to measure and compare the effectiveness of different testing techniques. In this work-in-progress paper, we take a step forward towards a framework for the assessment and comparison of testing approaches for RESTful Web APIs. To that end, we propose a preliminary catalogue of test coverage criteria. These criteria measure the adequacy of test suites based on the degree to which they exercise the different input and output elements of RESTful Web services. To the best of our knowledge, this is the first attempt to measure the adequacy of testing approaches for RESTful Web APIs.

Autores: Alberto Martin-Lopez / Sergio Segura / Antonio Ruiz-Cortés / 
Palabras Clave: coverage criteria - REST - Testing - web services

20 - Spectrum-based fault localization in software product lines

Artículo relevante publicado en 2018 en el ISTContext: Software Product Line (SPL) testing is challenging mainly due to the potentially huge number ofproducts under test. Most of the research on this field focuses on making testing affordable by selecting arepresentative subset of products to be tested. However, once the tests are executed and some failures revealed,debugging is a cumbersome and time consuming task due to difficulty to localize and isolate the faulty featuresin the SPL.Objective: This paper presents a debugging approach for the localization of bugs in SPLs.Method: The proposed approach works in two steps. First, the features of the SPL are ranked according to theirsuspiciousness (i.e., likelihood of being faulty) using spectrum-based localization techniques. Then, a novel faultisolation approach is used to generate valid products of minimum size containing the most suspicious features,helping to isolate the cause of failures.Results: For the evaluation of our approach, we compared ten suspiciousness techniques on nine SPLs of differentsizes. The results reveal that three of the techniques (Tarantula, Kulcynski2 and Ample2) stand out over the rest,showing a stable performance with different types of faults and product suite sizes. By using these metrics, faultswere localized by examining between 0.1% and 14.4% of the feature sets.Conclusion: Our results show that the proposed approach is effective at locating bugs in SPLs, serving as a helpfulcomplement for the numerous approaches for testing SPLs.

Autores: Aitor Arrieta / Sergio Segura / Urtzi Markiegi / Goiuria Sagardui / Leire Etxeberria / 
Palabras Clave: Debugging - Feature Models - software product lines - Spectrum-based fault localization

21 - Reparación de pruebas de interfaz de usuario en Android como un problema de búsqueda

Las pruebas de interfaz de usuario son una técnica muy popular gracias a su capacidad para validar el comportamiento de la aplicación tal y como lo experimentaría el usuario, y por su facilidad para generar los casos de prueba. Sin embargo, una de las limitaciones más importantes de este tipo de pruebas es su fragilidad ante los cambios de la propia interfaz de usuario, que suelen producirse durante el desarrollo del sistema. En este artículo formulamos la reparación de estas pruebas ante cambios en la intefaz o funcionalidad de la aplicación como un problema de búsqueda. Además, proponemos un algoritmo heurístico para su resolución basado en GRASP. Esta propuesta se ha implementado y validado en el dominio especifico de aplicaciones móviles para dispositivos Android. Los resultados obtenidos demuestran su aplicabilidad con varios casos de estudio para cambios de diversa envergadura.

Autores: Adrián Cantón Fernandez / José Antonio Parejo Maestre / Sergio Segura / Antonio Ruiz-Cortés / 
Palabras Clave: Android - GRASP - SBSE - test case repair - Testing