Sign In

Communications of the ACM

Contributed Articles

(Re)Use of Research Results (Is Rampant)

shelves of heavily used books

Credit: Getty Images

According to Popper,23 the ideas we can most trust are those that have been the most tried and tested. For that reason, many of us are involved in this process called "science," which produces trusted knowledge by sharing one's ideas and trying out and testing the ideas of others. Science and scientists form communities where people do each other the courtesy of curating, clarifying, critiquing, and improving a large pool of ideas.

Back to Top

Key Insights


According to this definition, one measure of a scientific community's health is how much it reuses results. By that measure, the software engineering research community might seem to be very unhealthy. Da Silva et al. reported that from 1994 to 2010, only 72 studies had been replicated by 96 new studies.10 In February 2022, as a double-check for da Silva's conclusion, we queried the ACM Portal for products from the International Conference on Software Engineering (ICSE), that field's premier conference. Between 2011 and 2021, only 111 out of the 8,774 ICSE research entries were labeled as 'available,' 74 as 'reusable,' 24 as 'functional,' and none as 'replicated' or 'reproduced' reuse (see Table 1 for a definition of those terms). Put another way, according to the ACM Portal, only 2.4% of the ICSE publications are explicitly associated with any kind of reuse. Worse still, according to that report, there were no replicated or reproduced results from ICSE in the last decade.


No entries found

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.
Sign In for Full Access
» Forgot Password? » Create an ACM Web Account