# Overview

## Announcements

- [Due Friday]{.orange_due}
    - [Exercise 06: Replication](exercises/ex06-replication.qmd)

## Last time...

- What did we learn from watching the TED Talk about power posing?

## Today

- [Final due]{.orange_due}
    - [Exercise 05: Alpha, Power, Effect Sizes, & Sample Size](exercises/ex05-apes.qmd) write-up
- Discuss
    - [@munafo_manifesto_2017]
    - [@Begley2013-vm]

# [@munafo_manifesto_2017]

---

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Sert, N. P. du, Simonsohn, U., Wagenmakers, E.-J., Ware, J. J. & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. *Nature Human Behaviour*, 1, 0021. <https://doi.org/10.1038/s41562-016-0021>

## Abstract

>Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific process: methods, reporting and dissemination, reproducibility, evaluation and incentives. 

<p style="text-align:center;">
[@munafo_manifesto_2017]
</p>

---

>There is some evidence from both simulations and empirical studies supporting the likely effectiveness of these measures, but their broad adoption by researchers, institutions, funders and journals will require iterative evaluation and improvement. 

<p style="text-align:center;">
[@munafo_manifesto_2017]
</p>

---

>We discuss the goals of these measures, and how they can be implemented, in the hope that this will facilitate action toward improving the transparency, reproducibility and efficiency of scientific research.

<p style="text-align:center;">
[@munafo_manifesto_2017]
</p>

---

![Figure 1 from [@munafo_manifesto_2017]](https://media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41562-016-0021/MediaObjects/41562_2016_Article_BFs415620160021_Fig1_HTML.jpg?as=webp){fig-align="center"}

## Table 1

Link: <https://www.nature.com/articles/s41562-016-0021/tables/1>

## Improving methods

>Protecting against cognitive biases.

>Improving methodological training

>Implementing independent methodological support

>Encouraging collaboration and team science

## Improving reporting and dissemination

>Promoting study pre-registration

>Improving the quality of reporting

## Improving reproducibility

>Promoting transparency and open science

## Improving evaluation

>Diversifying peer review

## Changing incentives

>Rewarding open and reproducible practices

---

::: {.callout-note}

Which of these recommended practices would be at the *top* of your to-do list if you were a journal editor/publisher, funder, institution (college or university), or regulator (e.g., government entity)?

Which would be at the *bottom* of your list?

What's missing from the list?

What can students do?

:::

# [@Begley2013-vm]

---

Begley, C. G. (2013). Six red flags for suspect work. *Nature*, *497*(7450), 433–434. <https://doi.org/10.1038/497433a>

---

>Were experiments performed blinded?

>Were basic experiments repeated?

>Were all the results presented?

<p style="text-align:center;">
[@Begley2013-vm]
</p>

---

>Were there positive and negative controls? Often in the non-reproducible, high-profile papers, the crucial control experiments were excluded or mentioned as 'data not shown'.

>Were reagents validated?

>Were statistical tests appropriate?

<p style="text-align:center;">
[@Begley2013-vm]
</p>

---

>Why do we repeatedly see these poor-quality papers in basic science? In part, it is down to the fact that there is no real consequence for investigators or journals. It is also because many busy reviewers (and disappointingly, even co-authors) do not actually read the papers, and because journals are required to fill their pages with simple, complete 'stories'.

<p style="text-align:center;">
[@Begley2013-vm]
</p>

---

>And because of the apparent failure to recognize authors' competing interests — beyond direct financial interests — that may interfere with their judgement.

<p style="text-align:center;">
[@Begley2013-vm]
</p>

# Next time

*Changing journal policies*

- Read
    - [@nosek_promoting_2015]
    - [@Gilmore2020-sl]
    - [@Srcd2019-hg]
- [Class notes](wk10-2023-10-25.qmd)

# Resources

## References
