Navegación

Búsqueda

Búsqueda avanzada

Comparing manual and automated feature location in conceptual models: A Controlled experiment

Context: Maintenance activities cannot be completed without locating the set of software artifacts that realize a particular feature of a software system. Manual Feature Location (FL) is widely used in industry, but it becomes challenging (time-consuming and error prone) in large software repositories. To reduce manual efforts, automated FL techniques have been proposed. Research efforts in FL tend to make comparisons between automated FL techniques, ignoring manual FL techniques. Moreover, existing research puts the focus on code, neglecting other artifacts such as models.Objective: This paper aims to compare manual FL against automated FL in models to answer important questions about performance, productivity, and satisfaction of both treatments.Method: We run an experiment for comparing manual and automated FL on a set of 18 subjects (5 experts and 13 non-experts) in the domain of our industrial partner, BSH, manufacturer of induction hobs for more than 15 years. We measure performance (recall, precision, and F-measure), productivity (ratio between F-measure and spent time), and satisfaction (perceived ease of use, perceived usefulness, and intention to use) of both treatments, and perform statistical tests to assess whether the obtained differences are significant.Results: Regarding performance, manual FL significantly outperforms automated FL in precision and F-measure (up to 27.79+ACU and 19.05+ACU, respectively), whereas automated FL significantly outperforms manual FL in recall (up to 32.18+ACU). Regarding productivity, manual FL obtains 3.43+ACU-/min, which improves automated FL significantly. Finally, there are no significant differences in satisfaction for both treatments.Conclusions: The findings of our work can be leveraged to advance research to improve the results of manual and automated FL techniques. For instance, automated FL in industry faces issues such as low discrimination capacity. In addition, the obtained satisfaction results have implications for the usage and possible combination of manual, automated, and guided FL techniques.

Comparing business value modeling methods: A family of experiments (RELEVANTE YA PUBLICADO)

Autores: Eric Souza, Ana Moreira, JoãoAraújo, Silvia Abrahão, Emilio Insfran, Denis Silva da SilveiraRevista: Information and Software Technology, Volume 104, December 2018, Pages 179-193DOI: https://doi.org/10.1016/j.infsof.2018.08.001JCR IF 2017: 2.627 (Q1)

Effect of Domain Knowledge on Elicitation Effectiveness: An Internally Replicated Controlled Experiment

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, VOL. 42, NO. 5, MAY 2016 DOI: https://doi.org/10.1109/TSE.2015.2494588 Factor de impacto: 1.516 Posición: 20/106 (Software Engineering) – Q1

Does the level of detail of UML diagrams affect the maintainability of source code?: a family of experiments

Although the UML is considered to be the de facto standard notation with which to model software, there is still resistance to model-based development. UML modeling is perceived to be expensive and not necessarily cost-effective. It is therefore important to collect empirical evidence concerning the conditions under which the use of UML makes a practical difference. The focus of this paper is to investigate whether and how the Level of Detail (LoD) of UML diagrams impacts on the performance of maintenance tasks in a model-centric approach. A family of experiments consisting of one controlled experiment and three replications has therefore been carried out with 81 students with different abilities and levels of experience from 3 countries (The Netherlands, Spain, and Italy). The analysis of the results of the experiments indicates that there isno strong statistical evidence as to the influence of different LoDs. The analysis suggests a slight tendency toward better results when using low LoD UML diagrams, especially if used for the modification of the source code, while a high LoD would appear to be helpful in understanding the system. The participants in our study also favored low LoD diagrams because they were perceived as easier to read. Although the participants expressed a preference for low LoD diagrams, no statistically significant conclusions can be drawn from the set of experiments. One important finding attained from this family of experiments was that the participants minimized or avoided the use of UML diagrams, regardless of their LoD. This effect was probably the result of using small software systems from well-known domains as experimental materials.

On the Effectiveness, Efficiency and Perceived Utility of Architecture Evaluation Methods: A Replication Study

In this paper we describe the results of a replication study for comparing the effectiveness, efficiency and perceived utility of the quality-driven product architecture derivation and improvement method (QuaDAI), an architecture derivation and evaluation method that we presented in recent works, as opposed to the Architecture Tradeoff Analysis Method (ATAM), a well-known architectural evaluation method used in industry. The results of the original experiment (conducted with undergraduate students) showed that QuaDAI was found to be more efficient and was perceived as easier to use than ATAM. However, although QuaDAI performed better than ATAM, we could not confirm the other variables, as the differences between both methods were not statistically significant. Therefore the goal of the replication was to verify these findings with a group of more experienced students. In the replication study QuaDAI also performed better than ATAM, but as opposed to the original study, all the variables proved to be statistically significant.