2023-08-28 Mon

How science works (or should)

Rick Gilmore

Overview

Prelude

Survey 01

  • Thanks to those of you who responded

Today

How science works (or should)

How science works (or should)

Discuss (Ritchie, 2020, Chapter 1)

Author

Stuart J. Ritchie

Chapter 1: How Science Works

  • “Science is a social construct.” (Oreskes, 2019)
    • How so?
    • What are the consequences?

What is the “process” of science?

Figure 1 from (Munafò et al., 2017)

Telling (facts/findings) AND showing

…and it has to be shown why that other theory can cannot be the true one: And until this is shown, and until we know how it is shown, we do not understand the grounds of our opinion.

John Stuart Mill quoted in (Ritchie, 2020, p. 14)

From replication to trust

…Only when certain events recur in accordance with rules or regularities, as is the case with repeatable experiments, can our observations be tested–in principle–by anyone. We do not take even our own observations quite seriously, or accept them as scientific observations, until we have repeated and test them…

Karl Popper quoted in (Ritchie, 2020, p. 23)

From replication to trust

…Only by such repetitions can we convince ourselves that we are not dealing with a mere isolated ‘coincidence’…

Karl Popper quoted in (Ritchie, 2020, p. 23)

  • In what other realm(s) do we not require someone stating an idea or finding to show us how they came to that idea or finding?
  • Are scientific experiments really repeatable by anyone?
  • In what other realm(s) do we not expect repetition or replication before trusting an idea?

Science’s social nature does come with weaknesses, however. Because scientists focus so much on trying to persuade their peers, which is the way they get those studies through peer review and oward to publication, it’s all too easy for them to disregard the real object of science: getting us closer to the truth.

(Ritchie, 2020, Chapter 1, pp. 14-15)

Questions about Ritchie

  • Do you agree that a focus on “persuading their peers” can lead researchers to lose sight of the “real object of science”?
    • Are there other examples (outside of science), where a focus on persuading others can change a person’s focus from more important goals?
  • What’s the alternative for scientists to trying to persuade their peers?
  • What if scientific research serves multiple goals?

Discuss (Nosek & Bar-Anan, 2012)

Brian Nosek

Brian Nosek

Yoav Bar-Anon

  • Social Psychology, Professor at Tel Aviv University

Scientific Utopia: I. Opening Scientific Communication

Aims & Claims

Existing norms for scientific communication are rooted in anachronistic practices of bygone eras making them needlessly inefficient. We outline a path that moves away from the existing model of scientific communication to improve the efficiency in meeting the purpose of public science—knowledge accumulation.

(Nosek & Bar-Anan, 2012)

We call for six changes: (a) full embrace of digital communication; (b) open access to all published research; (c) disentangling publication from evaluation; (d) breaking the “one article, one journal” model with a grading system for evaluation and diversified dissemination outlets; (e) publishing peer review; and (f) allowing open, continuous peer review.

(Nosek & Bar-Anan, 2012)

We address conceptual and practical barriers to change and provide examples showing how the suggested practices are being used already. The critical barriers to change are not technical or financial; they are social. Although scientists guard the status quo, they also have the power to change it.

(Nosek & Bar-Anan, 2012)

Inefficiencies in scientific communication

  • No communication
  • Slow communication
  • Incomplete communication
  • Inaccurate communication
  • Unmodifiable communication

(Nosek & Bar-Anan, 2012, , pp. 25-26)

Questions about (Nosek & Bar-Anan, 2012)

  • What are the impacts of these factors?
  • Do you agree that scientific communication should be made more efficient? Why or why not?

Questions about (Nosek & Bar-Anan, 2012)

  • Do you think all research scholarship should be open access?
    • Who should pay for the costs of publication, maintaining the data?
    • What’s wrong with enterprising people making a living by providing these services?

Questions about (Nosek & Bar-Anan, 2012)

  • What about the idea of separating publication from evaluation?
    • How can scientists discern good science from bad?
    • How can the general public?
  • What do you think reviewers would say about publishing peer review?
    • Some journals do this already, e.g. Frontiers

Common themes across Nosek & Bar-Anan (2012)

  • Goals and purposes of science
  • Utopian, idealistic aims
  • Efficiency in achieving those goals
  • High quality versus low quality science
  • Ethics, scientific integrity
  • Means of scientific communication
  • Who has access, who should
  • How are findings, articles, individuals evaluated

Next time

Scientific norms and counter-norms

Resources

Talk by Brian Nosek, (University of California Television (UCTV), 2018)

Watching the talk by Nosek is not required. But he’s a very good speaker and an inspiring person.

References

Merton, R. W. (1973). The normative structure of science. In R. K. Merton & N. W. Storer (Eds.), The Sociology of Science: Theoretical and Empirical Investigations (pp. 267–278). The University of Chicago Press.
Mitroff, I. I. (1974). Norms and counter-norms in a select group of the Apollo moon scientists: A case study of the ambivalence of scientists. American Sociological Review, 39(4), 579–595. https://doi.org/10.2307/2094423
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Sert, N. P. du, … Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021. https://doi.org/10.1038/s41562-016-0021
Nosek, B. A., & Bar-Anan, Y. (2012). Scientific utopia i: Opening scientific communication. Psychological Inquiry, 23(3), 217–243. https://doi.org/10.1080/1047840X.2012.692215
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 7(6), 615–631. https://doi.org/10.1177/1745691612459058
Oreskes, N. (2019). Why Trust Science. Princeton University Press.
Ritchie, S. (2020). Science fictions: Exposing fraud, bias, negligence and hype in science (1st ed.). Penguin Random House. Retrieved from https://www.amazon.com/Science-Fictions/dp/1847925669
University of California Television (UCTV). (2018, May). Improving openness and innovation in scholarly communication with brian nosek. Youtube. Retrieved from https://www.youtube.com/watch?v=YEwqohAjfZc