Open science tools

2023-11-15 Wed

Rick Gilmore

Overview

Announcements

• Friday, November 17
  • Final project survey
• Due Friday, November 17
  • Exercise 07: Data and materials sharing

Last time…

Your turn

• What do students want researchers to share?
• What are the pros and cons of active vs. post hoc data curation?

Today

Open science tools

• Discuss
  • (Kathawalla, Silverstein, & Syed, 2021)
  • [Chopik2018-wx]
  • (Optional) (Crüwell et al., 2019)
• Explore
  • (“FORRT - framework for open and reproducible research training,” n.d.)

Survey: Questions from [Chopik2018-wx]

(Kathawalla et al., 2021)

This article provides a roadmap to assist graduate students and their advisors to engage in open science practices. We suggest eight open science practices that novice graduate students could begin adopting today. The topics we cover include journal clubs, project workflow, preprints, reproducible code, data sharing, transparent writing, preregistration, and registered reports.

To address concerns about not knowing how to engage in open science practices, we provide a difficulty rating of each behavior (easy, medium, difficult), present them in order of suggested adoption, and follow the format of what, why, how, and worries.

We give graduate students ideas on how to approach conversations with their advisors/collaborators, ideas on how to integrate open science practices within the graduate school framework, and specific resources on how to engage with each behavior.

We emphasize that engaging in open science behaviors need not be an all or nothing approach, but rather graduate students can engage with any number of the behaviors outlined.

(Kathawalla et al., 2021)

• Open Science Framework (OSF) project: https://osf.io/w5mbp/
• Wiki: https://osf.io/w5mbp/wiki/home/

Figure 1 from (Kathawalla et al., 2021)

(Chopik, Bremner, Defever, & Keller, 2018)

Over the past 10 years, crises surrounding replication, fraud, and best practices in research methods have dominated discussions in the field of psychology. However, no research exists examining how to communicate these issues to undergraduates and what effect this has on their attitudes toward the field.

We developed and validated a 1-hr lecture communicating issues surrounding the replication crisis and current recommendations to increase reproducibility. Pre- and post-lecture surveys suggest that the lecture serves as an excellent pedagogical tool.

Following the lecture, students trusted psychological studies slightly less but saw greater similarities between psychology and natural science fields. We discuss challenges for instructors taking the initiative to communicate these issues to undergraduates in an evenhanded way.

(Chopik et al., 2018)

• Table 1 from (Chopik et al., 2018): https://journals.sagepub.com/doi/10.1177/0098628318762900#table1-0098628318762900
• Table 2 from (Chopik et al., 2018): https://journals.sagepub.com/doi/10.1177/0098628318762900#table2-0098628318762900

Reproducibility notes

• Survey questions in article
• OSF project for materials (Defever, Chopik, Keller, & Bremner, 2016)
• Data shared, but not in an easily reproducible form

Next time

Final project work day

• Final project survey
• Assignment Due
  • Exercise 07: Data and materials sharing

Resources

References

Chopik, W. J., Bremner, R. H., Defever, A. M., & Keller, V. N. (2018). How (and whether) to teach undergraduates about the replication crisis in psychological science. Teaching of Psychology, 45(2), 158–163. https://doi.org/10.1177/0098628318762900

Crüwell, S., Doorn, J. van, Etz, A., Makel, M. C., Moshontz, H., Niebaum, J. C., … Schulte-Mecklenbeck, M. (2019). Seven easy steps to open science. Zeitschrift für Psychologie, 227(4), 237–248. https://doi.org/10.1027/2151-2604/a000387

Defever, A. M., Chopik, W. J., Keller, V. N., & Bremner, R. (2016, August). Best practices pedagogy. Center For Open Science. Retrieved from https://osf.io/mh9pe/

FORRT - framework for open and reproducible research training. (n.d.). https://forrt.org/. Retrieved from https://forrt.org/

Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing into open science: A guide for graduate students and their advisors. Collabra. Psychology, 7(1). https://doi.org/10.1525/collabra.18684