Materials, code, & protocol sharing

2023-11-13 Mon

Rick Gilmore

Overview

Announcements

• Friday, November 17
  • Final project survey
• Due Friday, November 17
  • Exercise 07: Data and materials sharing

Today

Materials, code, & protocol sharing

• Discuss
  • (Soska et al., 2021)
  • (optional) (Gilroy & Kaplan, 2019)
• Explore
  • protocols.io
  • Journal of Visualized Experiments (JOVE)

Quest: Sharing materials and protocols

Your mission:

• Visit protocols.io or Journal of Visualized Experiments (JOVE)
  • Find a protocol that interests you
  • Review it and be ready to report back on the contents, how easy it might be for someone else to follow, etc.

What to share

Materials

• Survey instruments
• Displays

Code

• Stimulus generation (e.g., computer task)
• Data analysis code
• SPSS syntax, R/Python/SAS/MATLAB, etc.
• Computational environments (e.g., containers)

Protocols

• Supplemental methods associate with research articles
• Examples examples:
  • PLAY Project
    • Especially video, see (Gilmore & Adolph, 2017b).
    • Survey workflow
  • Penn State Open Science Survey (Gilmore, Hillary, Lazar, & Wham, 2023)
• https://protocols.io
• Journal of Visualized Experiments (JOVE)

Where to share

Data repositories

• Open Science Framework (OSF)
• Inter-university Consortium for Political and Social Research (ICSPSR)
• Dataverse
• Qualitative Data Repository
• TalkBank
• OpenNeuro

Databrary

• https://databrary.org
• Open (not public) sharing, but restricted to authorized researchers
• Specialized for storing and sharing video
• Gilmore & Adolph (2017a)

# When/how to share

Play & Learning Across a Year (PLAY) Project

• PLAY Project: https://play-project.org
• Survey data via KoBoToolbox: https://PLAY-behaviorome.github.io/KoBoToolbox

Soska et al. (2021)

Figure 2 from Soska et al. (2021)

Figure 1 from Soska et al. (2021)

Table 1 from Soska et al. (2021)

Your turn

• What do students want researchers to share?
• What are the pros and cons of active vs. post hoc data curation?

Next time

Open science tools

• Read
  • (Kathawalla, Silverstein, & Syed, 2021)
  • (Crüwell et al., 2019)
• Explore
  • (“FORRT - framework for open and reproducible research training,” n.d.)
• Class notes

Resources

References

Crüwell, S., Doorn, J. van, Etz, A., Makel, M. C., Moshontz, H., Niebaum, J. C., … Schulte-Mecklenbeck, M. (2019). Seven easy steps to open science. Zeitschrift für Psychologie, 227(4), 237–248. https://doi.org/10.1027/2151-2604/a000387

FORRT - framework for open and reproducible research training. (n.d.). https://forrt.org/. Retrieved from https://forrt.org/

Gilmore, R. O., & Adolph, K. E. (2017a). Video can make behavioural research more reproducible. Nature Human Behavior, 1. https://doi.org/10.1038/s41562-017-0128

Gilmore, R. O., & Adolph, K. E. (2017b). Video can make behavioural science more reproducible. Nature Human Behaviour, 1(7). https://doi.org/10.1038/s41562-017-0128

Gilmore, R. O., Hillary, F., Lazar, N., & Wham, B. (2023). Penn state open science survey. https://penn-state-open-science.github.io/survey-fall-2022/index.html. Retrieved from https://penn-state-open-science.github.io/survey-fall-2022/index.html

Gilroy, S. P., & Kaplan, B. A. (2019). Furthering open science in behavior analysis: An introduction and tutorial for using GitHub in research. Perspectives on Behavior Science, 42(3), 565–581. https://doi.org/10.1007/s40614-019-00202-5

Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing into open science: A guide for graduate students and their advisors. Collabra. Psychology, 7(1). https://doi.org/10.1525/collabra.18684

Soska, K. C., Xu, M., Gonzalez, S. L., Herzberg, O., Tamis-LeMonda, C. S., Gilmore, R. O., & Adolph, K. E. (2021). (Hyper)active data curation: A video case study from behavioral science. Journal of Escience Librarianship, 10(3). https://doi.org/10.7191/jeslib.2021.1208