I just attended FSE 2016, a leading academic conference on software engineering research. As is en vogue, it had a session on why so much software engineering research seems so removed from reality. One observation was that academics toil in areas of little interest to practice, publishing one incremental paper of little relevance after another. Another observation was that as empirical methods have taken hold, much research has become as rigorous as it has become irrelevant.
My answer to why so much software engineering research is irrelevant to practice is as straightforward as it is hard to change. The problem rests in the interlocking of three main forces that conspire to keep academics away from doing interesting and ultimately impactful research. These forces are:
- Academic incentive system
- Access to relevant data
- Research methods competence
Increasing competition between universities has lead many university presidents and faculty deans to manage by the key figures used to calculate university rankings. As a consequence, professors and other researchers are being driven to publish more and more. Quantity counts, not how interesting or novel or relevant the work is.
Software engineering research naturally tries to understand how software engineering works; you can’t easily study this at universities, the necessary capital investment (costs) are just too high. Thus, you have to go out and work with industry, which can be very difficult. For one, industry values solving the problem at hand, not developing a theory of it. Not many professors have the skills that industry wants and can sell research projects effectively.
Contrary to popular belief, most academics work hard, and research is only one of the many things they do. Historically, computer science education is weak on methods competence (and still is). Studying human systems like software engineering organizations requires research method skills that cannot be found easily among researchers. Often, there is little time to grow your skill set.
The interlocking of these forces is straightforward: Working with industry requires time: You have to build up a reputation, gain trust, and may still not be able to get the data you need. Building necessary methods competence also takes time and real applications that you may not have access to. The constant pressure at some universities to publish forbids veering from the established path, because it takes time that you don’t have before the hammer comes down.
Of course, the picture is not as bleak as described. Tenured professors can make time for overcoming the hurdles above. However, they need to go beyond the convenience of sticking to doing what they already know. Younger researchers are picking up new skills and are excited about what industry has to offer, but they need to be given the freedom of exploration rather than the pressure to publish or perish. All in all, not easy to achieve and it will take strength and stamina in all of us.
Leave a Reply