From the Department of “Why didn’t I think of that”: A natty idea to encourage scientific replication

An Economic Approach to Alleviate the Crises of Confidence in Science: With an Application to the Public Goods Game by Luigi Butera, John A. List – #23335 (PE)

Novel empirical insights by their very nature tend to be unanticipated, and in some cases at odds with the current state of knowledge on the topic. The mechanics of statistical inference suggest that such initial findings, even when robust and statistically significant within the study, should not appreciably move priors about the phenomenon under investigation. Yet, a few well-conceived independent replications dramatically improve the reliability of novel findings.

Nevertheless, the incentives to replicate are seldom in place in the sciences, especially within the social sciences. We propose a simple incentive-compatible mechanism to promote replications, and use experimental economics to highlight our approach. We begin by reporting results from an experiment in which we investigate how cooperation in allocation games is affected by the presence of Knightian uncertainty, a pervasive and yet unexplored characteristic of most public goods. Unexpectedly, we find that adding uncertainty enhances cooperation. This surprising result serves as a test case for our mechanism: instead of sending this paper to a peer-reviewed journal, we make it available online as a working paper, but we commit never to submit it to a journal for publication. We instead offered co-authorship for a second, yet to be written, paper to other scholars willing to replicate our study. That second paper will reference this working paper, will include all replications, and will be submitted to a peer-reviewed journal for publication. Our mechanism allows mutually-beneficial gains from trade between the original investigators and other scholars, alleviates the publication bias problem that often surrounds novel experimental results, and accelerates the advancement of economic science by leveraging the mechanics of statistical inference.

This entry was posted in Economics and public policy, Science, Uncategorized. Bookmark the permalink.
Subscribe
Notify of
guest
3 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Avi Waksberg
Avi Waksberg
4 years ago

An interesting idea. However, I suspect there isn’t a strong incentive to put your paper up as a working paper. At least under the current incentive structure for academics.

Paul Frijters
Paul Frijters
4 years ago
Reply to  Avi Waksberg

agreed, but change is helped by trying different things. If nothing else, it shows someone cares about the problem.