Open Science Practices in Gambling Research Publications: A Scoping Review

Karen Amichia, BS

Cambridge Health Alliance
Open Science Practices in Gambling Research Publications: A Scoping Review

Scientific Abstract

Background: The replication crisis has stimulated researchers around the world to adopt open science research practices intended to protect against publication bias and to improve the quality of scientific research. Open science practices include research pre-registration, registered reports, open data and open access, which seek to avoid problematic tactics such as overreliance on small N WEIRD (i.e., Western, Educated, Industrialized, Rich, and Democratic) samples, HARKing, (i.e., hypothesizing after the results are known) and questionable analytic approaches (e.g., p-hacking). Although many gambling studies researchers use similar research methods as other disciplines with low replication rates, we know little about the uptake of contemporary open science research practices in gambling-related research.

Methods: In this study, we conducted a scoping review of a representative sample of recent research publications (n = 500; published between 2016 and 2019) on gambling-related topics to systematically map the existing research in this area. We charted these gambling studies on various open science research practice and study characteristics, such as open access and preprint availability, replication status, and date published.

Results: Our results showed that a minority of studies used open science practices, with certain practices including open data, open code, open materials, and open notebooks being very rare, and other practices such as open access being somewhat more common. Almost all studies were original (i.e., non-replication) studies. For the 2016-2019 period, open science practices were used more frequently in the gambling studies literature over time.

Conclusions: There is a need for more researchers in the gambling studies field to use more open science practices, which can reduce publication bias and enhance scientific rigor. We suggest several ways to enhance the uptake of open science principles and practices within gambling studies, psychology, and science more broadly, with an emphasis on teaching researchers about ways to integrate open science practices into their research agendas.

SoundCloud Transcript

Thank you so much for taking the time to learn more about my poster! My name is Karen Amichia and I am a Research Coordinator at the Division on Addiction at Cambridge Health Alliance. My poster is titled, “Open Science Practices in Gambling Research Publications: A Scoping Review.”

The replication crisis has stimulated researchers around the world to adopt open science research practices intended to protect against publication bias and to enhance the quality of scientific research. Open science practices help to improve research by avoiding problematic tactics such as over-reliance on small WEIRD samples (which refers to, Western, Educated, Industrialized, Rich, and Democratic samples), HARKing ( or hypothesizing after the results are known) and p-hacking (which is the use of analytic approaches that will favor statistically significant outcomes).

First, I will provide some definitions of various open science practices that were examined in this study and then I will describe the sections of our poster in more detail.

Pre-registration: A detailed document containing research questions, hypotheses, study methods, and plan for analysis that is written before any data collection or analysis takes place.  The document is time-stamped and unable to be changed once posted to a public repository. Any future study changes should be posted in a separate transparent changes document.

Pre-print: The posting of a non-finalized version of a study manuscript to a publicly available online archive (for example, OSF, PsyArXiv, bioRxiv)

Open access: Making a published study manuscript publicly available without paywalls or other fees to access it.

Open data/materials/notebook/code: Making datasets underlying a study, materials necessary to replicate the study (like survey questionnaires, interview scripts, experimental procedures), and/or analytic code scripts publicly available.

We know very little about the uptake and use of open science research practices in gambling-related research, despite research in other disciplines showing that such practices are uncommon. Therefore, we conducted a scoping review of a representative sample of recent gambling research publications (n = 500; published between 2016 and 2019) to better understand the recent prevalence and trends in open science research practices in this area.

Methods: We searched databases such as Medline and PsycINFO and archives of specialized gambling journals for articles that measured or focused on gambling. Our search identified 1,251 studies that met our inclusion criteria, and we took a random sample of 500 studies for our review. We charted these gambling studies on various open science research practice and study characteristics, such as open access and preprint availability, replication status, and date published. Next, we looked at the prevalence of open science practices in gambling research during the 2016 to 2019 period and how the use of these practices changed across time.

Results: Only a minority of the studies used open science practices. Open data, pre-registration, open code and open notebooks were the rarest and their prevalence ranged from 0% for open notebooks to 2% for open data. Replication studies were only slightly more prevalent at 2.4%, followed by open materials at 7%, and pre-prints at 11%. Open access was by far the most common open science practice utilized with a 38% prevalence rate. Overall, the use of most open science practices increased between 2016 and 2019.

Conclusion: There is a need for more researchers in the gambling studies field to use more open science practices, which can reduce publication bias and enhance scientific rigor. We suggest several ways to enhance the uptake of open science principles and practices within gambling studies, psychology, and science more broadly, with an emphasis on teaching researchers about ways to integrate open science practices into their research agendas. These practical recommendations include incorporating open science-related topics into undergraduate and graduate research methods courses, continuing education courses on open science best practices for early career and established researchers, and journal guidelines that encourage the use of open science practices in manuscripts.

Thank you so much for taking an interest in my poster! Please feel free to contact me if you have any questions: kamichia@challiance.org. 

And don’t forget to scan the QR code on the left side of the poster to see our study’s pre-registration on the Open Science Framework’s website!

Live Zoom Session – April 21st

research Areas

Authors

Karen Amichia, B.S., Eric R. Louderback, Ph.D., Debi A. LaPlante, Ph.D., Alessandra B. Grossman, B.S., Sally M. Gainsbury, Ph.D., & Robert M. Heirene, Ph.D.

Principal Investigator

Eric R. Louderback, Ph.D.