From the Department of “Why didn’t I think of that”: A natty idea to encourage scientific replication

An Economic Approach to Alleviate the Crises of Confidence in Science: With an Application to the Public Goods Game by Luigi Butera, John A. List – #23335 (PE)

Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings.

Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty, a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer-reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.

This entry was posted in Economics and public policy, Science, Uncategorized. Bookmark the permalink.

3 Responses to From the Department of “Why didn’t I think of that”: A natty idea to encourage scientific replication

  1. Avi Waksberg says:

    An interesting idea. However, I suspect there isn’t a strong incentive to put your paper up as a working paper. At least under the current incentive structure for academics.

Leave a Reply

Your email address will not be published. Required fields are marked *

Notify me of followup comments via e-mail. You can also subscribe without commenting.