Data sharing

2023-11-08 Wed

Rick Gilmore

Prelude

johnlennon (2016)

Wikipedia: https://en.wikipedia.org/wiki/Data_sharing

Last time…

  • What is preregistration aiming to achieve?
  • How well does preregistration “work” according to Claesen, Gomes, Tuerlinckx, & Vanpaemel (2021)?
  • Should most studies adopt preregistration? Why or why not?

Today

Data sharing

  • Read
    • Houtkoop et al. (2018)
    • Tenopir et al. (2020)
    • (Optional) Gilmore & Adolph (2017)
    • (Optional) Meyer (2018)

Imagine…

  • Pre-registration is the norm
  • All protocols, tasks, analysis code are shared
  • All data are shared (with explicit participant permission)

Barriers to data sharing

Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V. M., Nichols, T. E. & Wagenmakers, E.-J. (2018). Data sharing in psychology: A survey on barriers and preconditions. Advances in Methods and Practices in Psychological Science, 1(1), 70–85. https://doi.org/10.1177/2515245917751886

Figure 1 from Houtkoop et al. (2018).

Figure 2 from Houtkoop et al. (2018)

Figure 3 from Houtkoop et al. (2018)

Figure 4 from Houtkoop et al. (2018)

Conclusions

• Public data sharing is not a common practice among research psychologists.

• Respondents considered data sharing to be both desirable and profitable for their particular research fields, but somewhat less desirable and profitable in the case of their own current research projects.

Houtkoop et al. (2018)

• The non-fear-related barriers to data sharing most frequently reported by respondents were that (a) sharing is not a common practice in their field, (b) they prefer to share their data only upon request, (c) they consider preparing data for sharing to be excessively time-consuming, and (d) they have never learned to share data online.

Houtkoop et al. (2018)

• Respondents believed that the largest fear-related obstacles preventing other researchers from sharing their data are the fear that alternative analyses might expose invalid conclusions and the fear of loss of control.

Houtkoop et al. (2018)

• Respondents reported that their greatest fears about sharing their own data are that the data might be misinterpreted and they might be scooped.

• In general, respondents felt that fear-related barriers affected other researchers more strongly than themselves.

Houtkoop et al. (2018)

• Mandatory data sharing (enforced by institutions, journals, or funders) and financial encouragement (i.e., increased grant amounts) are measures that would apparently be highly effective in increasing researchers’ willingness to share their data.

Houtkoop et al. (2018)

Transparency & reproducibility notes

  • Study preregistered
  • Data shared

Your turn

Your turn

  • How would you characterize the “climate” around data sharing within the psychology community sampled by Houtkoop et al. (2018)
  • Do the “fear-related barriers” make sense to you? Why or why not?
  • How might these be addressed or overcome?
  • Are mandates the best or only solution?

Data sharing at Penn State

Gilmore, R. O., Hillary, F., Lazar, N. & Wham, B. (2023). Penn State Open Science Survey. Retrieved August 1, 2023, from https://penn-state-open-science.github.io/survey-fall-2022/

Penn State researchers

Sharing with collaborators

Figure 22.10 from Gilmore et al. (2023)

Barriers to sharing

Where store data

Figure 22.9 from Gilmore et al. (2023)

Where shared

Figure 22.15 from Gilmore et al. (2023)

Importance of sharing

Figure 22.11 from Gilmore et al. (2023)

Obstacles to sharing

Figure 22.12 from Gilmore et al. (2023)

Where data are shared

Figure 22.15 from Gilmore et al. (2023)

Your turn

Your turn

  • How would you characterize the “climate” around data sharing at Penn State?
  • What advice would you give faculty, staff, and students interested in fostering open science here?

A global perspective

Tenopir, C., Rice, N. M., Allard, S., Baird, L., Borycz, J., Christian, L., Grant, B., Olendorf, R. & Sandusky, R. J. (2020). Data sharing, management, use, and reuse: Practices and perceptions of scientists worldwide. PloS One, 15(3), e0229003. https://doi.org/10.1371/journal.pone.0229003.

Fields

Table 1 from Tenopir et al. (2020)

Where data are stored

Table 2 from Tenopir et al. (2020)

Practices, perceptions, & barriers

Table 7 from Tenopir et al. (2020)

Table 8 from Tenopir et al. (2020)

Changes across time

Figure 7 from Ferguson et al. (2023)

Figure 7 from Ferguson et al. (2023)

Funder requirements

National Science Foundation (NSF)

NSF-funded investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF awards.

NSF (n.d.)

National Institute for Mental Health (NIMH)

Widespread data sharing by the research community adds significant value to research and accelerates the pace of discovery….research funded by the NIMH are required to deposit all raw and analyzed data (including, but not limited to, clinical, genomic, imaging, and phenotypic data) from studies involving human subjects into the NIMH Data Archive (NDA).

NIMH (2023)

Your turn

Should researchers share data and analysis code?

Should researchers be required to share data and analysis code?

Next time

Work Session: Final Projects

Resources

References

PSYCH 490.012 • Fall 2024

Claesen, A., Gomes, S., Tuerlinckx, F., & Vanpaemel, W. (2021). Comparing dream to reality: An assessment of adherence of the first generation of preregistered studies. Royal Society Open Science, 8(211037). https://doi.org/10.1098/rsos.211037
Ferguson, J., Littman, R., Christensen, G., Paluck, E. L., Swanson, N., Wang, Z., … Pezzuto, J.-H. (2023). Survey of open science practices and attitudes in the social sciences. Nature Communications, 14(1), 5401. https://doi.org/10.1038/s41467-023-41111-1
Gilmore, R. O., & Adolph, K. E. (2017). Video can make behavioural research more reproducible. Nature Human Behavior, 1. https://doi.org/10.1038/s41562-017-0128
Gilmore, R. O., Hillary, F., Lazar, N., & Wham, B. (2023). Penn state open science survey. https://penn-state-open-science.github.io/survey-fall-2022/index.html. Retrieved from https://penn-state-open-science.github.io/survey-fall-2022/index.html
Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V. M., Nichols, T. E., & Wagenmakers, E.-J. (2018). Data sharing in psychology: A survey on barriers and preconditions. Advances in Methods and Practices in Psychological Science, 2515245917751886. https://doi.org/10.1177/2515245917751886
johnlennon. (2016, December). IMAGINE. (Ultimate mix, 2020) - john lennon & the plastic ono band (with the flux fiddlers) HD. Youtube. Retrieved from https://www.youtube.com/watch?v=YkgkThdzX-8
Meyer, M. N. (2018). Practical tips for ethical data sharing. Advances in Methods and Practices in Psychological Science, 2515245917747656. https://doi.org/10.1177/2515245917747656
NIMH, National Institute of Mental Health. (2023). NOT-MH-23-100: Notice of data sharing policy for the national institute of mental health. https://grants.nih.gov/grants/guide/notice-files/NOT-MH-23-100.html.
NSF, National Science Foundation. (n.d.). Preparing your data management and sharing plan. https://new.nsf.gov/funding/data-management-plan.
Tenopir, C., Rice, N. M., Allard, S., Baird, L., Borycz, J., Christian, L., … Sandusky, R. J. (2020). Data sharing, management, use, and reuse: Practices and perceptions of scientists worldwide. PloS One, 15(3), e0229003. https://doi.org/10.1371/journal.pone.0229003