The Empirical Software Engineering (ESE) community has made great progress in the last 20 years and expanded the field considerably both in scope, volume as well as quality. Nowadays, we have established conferences as well as journals focused on the area, and a majority of the papers published in the top SE conferences are empirical. However, while more established scientific fields such as Physics, Biology and Psychology have clear identities, specific schools of thought, and explicated research methods, I argue this is less so in ESE.
In this talk, I propose an updated manifesto for empirical software engineering and discuss some challenges and ways we might overcome them. This, I hope, can contribute to a more clear sense of identity and act as a vision. In particular, I discuss the negative effects of our love for novelty and how it affects publication bias and can be a challenge to uncover truths. I also summarize the ongoing debate among statisticians about how to move beyond p-values and provide some ideas for how to improve empirical studies using qualitative methods. I will conclude with concrete call-for-actions so that we can be an even stronger science in the future.
Robert Feldt is a professor of Software Engineering at Chalmers University of Technology, Sweden, and at Blekinge Institute of Technology, Sweden. He has broad research interests spanning from human factors to hardcore automation and statistics, and work on testing and quality, requirements engineering, as well as human-centred (behavioural) software engineering. Dr Feldt was an early contributor to search-based software engineering and has recently argued for increased application of psychology and social science to understand and improve software engineering. Most of his research is empirical and conducted in close collaboration with industry partners in Sweden, Europe and Asia, but he also leads more basic research. Dr Feldt received a PhD in Computer Engineering from the Chalmers University of Technology in 2002, studied Psychology at Gothenburg University in the ’90s and has also worked as an IT and software consultant for more than 30 years. He is passionate about empirical research and methods and changing organisations through technical innovation, but with the humans in focus. He is co-Editor in Chief of the EMSE journal and on the editorial boards of two other journals.