Conferencia Invitada: Prof. Dr. Carlo Ghezzi

Evaluating research (and researchers): should we care?-A software engineering perspective

Prof. Dr. Carlo Ghezzi
Politecnico di Milano, Italia
carlo.ghezzi@polimi.it

Summary

Evaluation is becoming an integral part of the professional life of researchers. Different stakeholders need to evaluate research and researchers are often evaluated as individuals. Research is evaluated by funding agencies, which need allocate funding among different areas. It is evaluated by academic institutions, who need to set priorities for their developments. Researchers are typically evaluated for promotion. How are these evaluations performed? Which criteria are used? How sound they are? Ultimately, the goal of research is to create new knowledge which can create benefits for society. Research in computing (and software engineering in particular) should also have a demonstrable societal impact. How can impact be measured?

Although these questions are hard to answer, we need to address them. Evaluations are already in place, and too often they are based on flawed criteria, which oversimplify the problem by proposing simplistic solutions. Often the pressure is on producing numbers and rankings that are used a-critically across different disciplines. What should we, as researchers, say? Do we have our own position on how to evaluate research, researchers, impact?

The talk will mostly try to clarify the questions we have to ask and will discuss why most current «solutions» are unacceptable. It will not try to propose solutions, but instead it will try to raise awareness of the problem, present some good practices, and call for a collective effort to develop possible answers.