Materials, code, & protocol sharing

2024-11-18 Mon

Rick Gilmore

Prelude

Vanderslott, Dattani, Spooner, & Roser (2024)

Vanderslott et al. (2024)

Vanderslott et al. (2024)

Overview

Announcements

• Complete (optional)
  • Questionnaire based on Chopik, Bremner, Defever, & Keller (2018). Results are here.
• Friday, November 22
  • Final project survey

In the news

https://award.einsteinfoundation.de/award-winners-finalists/recipients-2024

Today

Materials, code, & protocol sharing

• Discuss
  • Soska et al. (2021)
  • (optional) Gilroy & Kaplan (2019)
• Explore
  • protocols.io
  • Journal of Visualized Experiments (JOVE)

Quest: Sharing materials and protocols

Your mission:

• Visit protocols.io or Journal of Visualized Experiments (JOVE)
  • Find a protocol that interests you
  • Review it and be ready to report back on the contents, how easy it might be for someone else to follow, etc.

What to share

Materials

• Survey instruments
• Displays

Code

• Stimulus generation (e.g., computer task)
• Data analysis code
• SPSS syntax, R/Python/SAS/MATLAB, etc.
• Computational environments (e.g., containers)

Protocols

• Supplemental methods associate with research articles
• Examples examples:
  • PLAY Project
    • Especially video, see (Gilmore2017-wd?).
    • Survey workflow
  • Penn State Open Science Survey (Gilmore, Hillary, Lazar, & Wham, 2023)
• https://protocols.io
• Journal of Visualized Experiments (JOVE)

Where to share

Data repositories

• Open Science Framework (OSF)
• Inter-university Consortium for Political and Social Research (ICSPSR)
• Dataverse
• Qualitative Data Repository
• TalkBank
• OpenNeuro

Databrary

• https://databrary.org
• Open (not public) sharing, but restricted to authorized researchers
• Specialized for storing and sharing video
• Gilmore & Adolph (2017)

# When/how to share

Play & Learning Across a Year (PLAY) Project

• PLAY Project: https://play-project.org
• Survey data via KoBoToolbox: https://PLAY-behaviorome.github.io/KoBoToolbox

Soska et al. (2021)

Figure 2 from Soska et al. (2021)

Figure 1 from Soska et al. (2021)

Table 1 from Soska et al. (2021)

Your turn

• What do students want researchers to share?
• What are the pros and cons of active vs. post hoc data curation?

Next time

Open science tools

• Read
  • Kathawalla, Silverstein, & Syed (2021)
  • Chopik et al. (2018)
  • (Optional) Crüwell et al. (2019)
• Explore
  • “FORRT - Framework for Open and Reproducible Research Training” (n.d.)

• Complete (optional)
  • Questionnaire based on Chopik et al. (2018). Results are here.
• Class notes

Resources

References

Chopik, W. J., Bremner, R. H., Defever, A. M., & Keller, V. N. (2018). How (and whether) to teach undergraduates about the replication crisis in psychological science. Teaching of Psychology, 45(2), 158–163. https://doi.org/10.1177/0098628318762900

Crüwell, S., Doorn, J. van, Etz, A., Makel, M. C., Moshontz, H., Niebaum, J. C., … Schulte-Mecklenbeck, M. (2019). Seven easy steps to open science. Zeitschrift für Psychologie, 227(4), 237–248. https://doi.org/10.1027/2151-2604/a000387

FORRT - Framework for Open and Reproducible Research Training. (n.d.). https://forrt.org/. Retrieved from https://forrt.org/

Gilmore, R. O., & Adolph, K. E. (2017). Video can make behavioural research more reproducible. Nature Human Behavior, 1. https://doi.org/10.1038/s41562-017-0128

Gilmore, R. O., Hillary, F., Lazar, N., & Wham, B. (2023). Penn state open science survey. https://penn-state-open-science.github.io/survey-fall-2022/index.html. Retrieved from https://penn-state-open-science.github.io/survey-fall-2022/index.html

Gilroy, S. P., & Kaplan, B. A. (2019). Furthering open science in behavior analysis: An introduction and tutorial for using GitHub in research. Perspectives on Behavior Science, 42(3), 565–581. https://doi.org/10.1007/s40614-019-00202-5

Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing into open science: A guide for graduate students and their advisors. Collabra. Psychology, 7(1). https://doi.org/10.1525/collabra.18684

Soska, K. C., Xu, M., Gonzalez, S. L., Herzberg, O., Tamis-LeMonda, C. S., Gilmore, R. O., & Adolph, K. E. (2021). (Hyper)active data curation: A video case study from behavioral science. Journal of Escience Librarianship, 10(3). https://doi.org/10.7191/jeslib.2021.1208

Vanderslott, S., Dattani, S., Spooner, F., & Roser, M. (2024). Vaccination. Our World in Data. Retrieved from https://ourworldindata.org/vaccination