Penn State Resources

Psychology Department GitHub repo
- IRB template language for data sharing
Big Data Social Science IGERT
WeR User Group (HDFS)
Courses
- PSY 511 – Transparent, Open, and Reproducible Research Practices in the Social and Behavioral Sciences (Spring 2017). site.

Federal/Policy

Federal Policy for the Protection of Human Subjects. (2017, January 19). https://www.federalregister.gov/documents/2017/01/19/2017-01058/federal-policy-for-the-protection-of-human-subjects
Public Access to NSF-Funded Research Data for the Social, Behavioral, and Economic Sciences. (2016, May 17). National Science Foundation. https://www.nsf.gov/sbe/reports/Public_Access_NSF_Workshop_Report_Final_Briefs.pdf

Books

Gandrud, C. (2015). Reproducible Research with R and R Studio, Second Edition. CRC Press. https://www.crcpress.com/Reproducible-Research-with-R-and-R-Studio-Second-Edition/Gandrud/p/book/9781498715379.
Wickham, H. & Grolemund, G. (2016). R for Data Science: Import, Tidy, Transform, Visualize, and Model Data. O’Reilly Media. http://shop.oreilly.com/product/0636920034407.do.
Muenchen, R. (2009). R for SAS and SPSS Users. http://www.springer.com/us/book/9780387094182. Mentioned by Michael Hallquist at 2017-01-12 meeting.

Software, tools, training

Improving the Reproducibility of Our Research Practices. Association for Psychological Science six-part workshop. https://www.psychologicalscience.org/publications/observer/obsonline/improving-reproducibility.html
Jupyter notebooks
RStudio
rweekly.org
R-bloggers
LaTeX on the web
- ShareLaTex http://sharelatex.com
- Overleaf http://www.overleaf.com
- Authorea http://www.authorea.com
Experimental Design Assistant for planning animal studies https://eda.nc3rs.org.uk/
p-hacker Shiny app http://shinyapps.org/apps/p-hacker/
Power ANalysis for GEneral Anova designs Shiny app
Open Stats Lab. Use data published with psychology papers to teach statistics.
DataCamp
Software Carpentry
Data Carpentry

Selected peer-reviewed papers and commentaries

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Sert, N. P. du, … Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021. https://doi.org/10.1038/s41562-016-0021.
Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafò, M. R., … Yarkoni, T. (2017). Scanning the horizon: towards transparent and reproducible neuroimaging research. Nature Reviews Neuroscience, advance online publication. https://doi.org/10.1038/nrn.2016.167.- Collaboration, O. S. (2015). Estimating the reproducibility of psychological Science, 349(6251), aac4716. http://doi.org/10.1126/science.aac4716.
Schweinsberg, M., Madan, N., Vianello, M., Sommer, S. A., Jordan, J., Tierney, W., … Uhlmann, E. L. (2016). The pipeline project: Pre-publication independent replications of a single laboratory’s research pipeline. Journal of Experimental Social Psychology, 66, 55–67. https://doi.org/10.1016/j.jesp.2015.10.001
Bavel, J. J. V., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016a). Contextual sensitivity in scientific reproducibility. Proceedings of the National Academy of Sciences, 113(23), 6454–6459. https://doi.org/10.1073/pnas.1521897113
Inbar, Y. (2016). Association between contextual dependence and replicability in psychology may be spurious. Proceedings of the National Academy of Sciences, 113(34), E4933–E4934. https://doi.org/10.1073/pnas.1608676113
Bavel, J. J. V., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016b). Reply to Inbar: Contextual sensitivity helps explain the reproducibility gap between social and cognitive psychology. Proceedings of the National Academy of Sciences, 113(34), E4935–E4936. https://doi.org/10.1073/pnas.1609700113
Bishop, D. (2015, October 29). How to make biomedical research more reproducible. Nature. http://dx.doi.org/10.1038/nature.2015.18684.
Etz, A., & Vandekerckhove, J. (2016). A Bayesian Perspective on the Reproducibility Project: Psychology. PLOS ONE, 11(2), e0149794. https://doi.org/10.1371/journal.pone.0149794
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., … Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. http://doi.org/10.1126/science.aab2374. Suggests guidelines for journals.
Nuijten, M. B., Hartgerink, C. H. J., Assen, M. A. L. M. van, Epskamp, S., & Wicherts, J. M. (2015). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 1–22. http://doi.org/10.3758/s13428-015-0664-2.
Ascoli, G. A. (2006). The ups and downs of neuroscience shares. Neuroinformatics, 4(3), 213–215. http://doi.org/10.1385/NI:4:3:213
Adolph, K. E., Gilmore, R. O., Freeman, C., Sanderson, P., & Millman, D. (2012). Toward Open Behavioral Science. Psychological Inquiry, 23(3), 244–247. http://doi.org/10.1080/1047840X.2012.705133.

Organizations

The Center for Open Science
- Open Science Framework (OSF)
- Transparency and Openness Promotion (TOP) Guidelines.
Databrary.
- Open science, data sharing, analysis tool links.
- Data sharing manifesto
Society for the Improvement of Psychological Science (SIPS)
- Sponsor of PsyArXiv preprint server.
ICPSR.
ScholarSphere.
American Psychological Association (2015). Data Sharing: Principles and Considerations for Policy Development. Retrieved August 25, 2015, from http://www.apa.org/science/leadership/bsa/data-sharing-report.aspx
PubPeer

Blogs

R bloggers https://www.r-bloggers.com/
Open science in practice. https://inattentionalcoffee.wordpress.com/2017/01/03/open-science-in-practice/
Practicing what I preach: The open Ph.D. experiment. http://www.librarianinthecity.com/2016/12/practicing-what-i-preach-the-open-phd-experiment/
Jupyter, Zeppelin, Beaker: The Rise of the Notebooks. https://opendatascience.com/blog/jupyter-zeppelin-beaker-the-rise-of-the-notebooks/
Open science and free culture. http://www.simoncolumbus.com/publications-2/open-science-and-free-culture/
Collaborative Replications and Education Project (CREP).
Bohannon, J. (2015). Many psychology papers fail replication test. Science, 349(6251), 910–911. http://doi.org/10.1126/science.349.6251.910.
Barrett, L. F. (2015, September 1). Psychology Is Not in Crisis. The New York Times. Retrieved from http://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html
Caplan, A. (2015, September 8). How Reliable Are Psychology Studies? The New York Times. Retrieved from http://www.nytimes.com/2015/09/08/opinion/how-reliable-are-psychology-studies.html
Frank, M. (2015, August 31). Babies Learning Language: The slower, harder ways to increase reproducibility. Retrieved from http://babieslearninglanguage.blogspot.com/2015/08/the-slower-harder-ways-to-increase.html.
Srivastava, S. (2015, September 2). Moderator interpretations of the Reproducibility Project. Retrieved from https://hardsci.wordpress.com/2015/09/02/moderator-interpretations-of-the-reproducibility-project/
Earp, B.D. (2015, September 2). Psychology Is Not in Crisis? Depends on What You Mean by “Crisis”. Retrieved from http://www.huffingtonpost.com/brian-earp/psychology-is-not-in-crisis_b_8077522.html.
Abrams, D., Clayton, S., & Crandall, C. (n.d.). The Reproducibility Crisis: Less Than Meets the Eye. Retrieved September 20, 2015, from https://www.psychologytoday.com/blog/sound-science-sound-policy/201509/the-reproducibility-crisis-less-meets-the-eye
Bench, S. W., Rivera, G. N., Schlegel, R. J., Hicks, J. A., & Lench, H. C. (2017). Does expertise matter in replication? An examination of the reproducibility project: Psychology. Journal of Experimental Social Psychology, 68, 181–184. https://doi.org/10.1016/j.jesp.2016.07.003
Reply by Schmidt, T. (2016). Sources of false positives and false negatives in the STATCHECK algorithm: Reply to Nuijten et al. (2015). arXiv:1610.01010 [q-Bio]. Retrieved from http://arxiv.org/abs/1610.01010
Macleod, M. R., Lawson McLean, A., Kyriakopoulou, A., Serghiou, S., de Wilde, A., Sherratt, N., … Sena, E. S. (2015). Risk of Bias in Reports of In Vivo Research: A Focus for Improvement. PLoS Biol, 13(10), e1002273. http://doi.org/10.1371/journal.pbio.1002273.
Vanpaemel, W., Vermorgen, M., Deriemaecker, L., & Storms, G. (2015). Are We Wasting a Good Crisis? The Availability of Psychological Research Data after the Storm. Collabra, 1(1). http://doi.org/10.1525/collabra.13.
Shaw, D. M., & Erren, T. C. (2015). Ten Simple Rules for Protecting Research Integrity. PLoS Comput Biol, 11(10), e1004388. http://doi.org/10.1371/journal.pcbi.1004388.
Shanks DR, Vadillo MA, Riedel B, Clymo A, Govind S, Hickin N, Tamman AJ, & Puhlmann LM (2015). Romance, Risk, and Replication: Can Consumer Choices and Risk-Taking Be Primed by Mating Motives? Journal of experimental psychology. General PMID: 26501730.
Neuroskeptic. (2015, November 15). Reproducibility Crisis: The plot thickens. Retrieved November 15, 2015 from http://blogs.discovermagazine.com/neuroskeptic/2015/11/10/reproducibility-crisis-the-plot-thickens/#.VkS7loT9mzf
Gelman, A. (2015, September 14). Why aren’t people sharing their data and code? Retrieved September 20, 2015, from http://andrewgelman.com/2015/09/14/its-not-so-easy-to-share-data-and-code-and-there-are-lots-of-bureaucrats-who-spend-their-time-making-it-even-more-difficult/
Gelman, Andrew. (2015, September 16). The aching desire for regular scientific breakthroughs. Retrieved September 20, 2015, from http://andrewgelman.com/2015/09/16/harsh/
Eisenberg, N. (2015, June). Thoughts on the Future of Data Sharing - Association for Psychological Science. Retrieved from https://www.psychologicalscience.org/index.php/publications/observer/2015/may-june-15/thoughts-on-the-future-of-data-sharing.html
Bishop, Dorothy. (2015, July 11). BishopBlog: Publishing replication failures: some lessons from history. Retrieved from http://deevybee.blogspot.com/2015/07/publishing-replication-failures-some.html
Johnson, C. Y. (2015, October 15). What happened when a group of researchers tried to repeat a headline-grabbing study. The Washington Post. Retrieved from https://www.washingtonpost.com/news/wonkblog/wp/2015/10/15/what-happened-when-a-group-of-researchers-tried-to-repeat-a-headline-grabbing-study/.
Leek, J. (2015, October 1). A glass half full interpretation of the replicability of psychological science | Simply Statistics. Retrieved from http://simplystatistics.org/2015/10/01/a-glass-half-full-interpretation-of-the-replicability-of-psychological-science/.
Nagel, T. (2015, October 8). New approach could help reduce bias in research, Stanford scholar says. Retrieved November 2, 2015, from http://news.stanford.edu/news/2015/october/research-analysis-maccoun-100815.html.
Guerra, S. (2015, September 28). Reproduce or Bust?!: Bringing Reproducibility Back to Center Stage | Science in the News. Retrieved from http://sitn.hms.harvard.edu/flash/2015/reproduce-or-bust-bringing-reproducibility-back-to-center-stage/.
Earp, B. D. (2015, September 2). Psychology Is Not in Crisis? Depends on What You Mean by “Crisis”. Retrieved November 2, 2015, from http://www.huffingtonpost.com/brian-earp/psychology-is-not-in-crisis_b_8077522.html.
Makel, J. P. & M. (2015, November 4). The Painful Necessity of Replicating Research - Education Week. Education Week. Retrieved from http://www.edweek.org/ew/articles/2015/11/04/the-painful-necessity-of-replicating-research.html.
Lindsay, D. S. (2015). Replication in Psychological Science. Psychological Science, 0956797615616374. http://doi.org/10.1177/0956797615616374.
politicalsciencereplication.wordpress.com. (n.d.). Political Scientists Trying to Delay Research Transparency. Retrieved from https://politicalsciencereplication.wordpress.com/2015/11/07/political-scientists-trying-to-delay-research-transparency/
Ortman, Andreas (2015, November 2). The replication crisis has engulfed economics. Retrieved November 10, 2015, from http://theconversation.com/the-replication-crisis-has-engulfed-economics-49202.