Exercise 02: The road to scientific glory

Dates

Due: Tuesday, February 21, 2023.

Acknowledgment

This exercise builds heavily on (Pownall et al. 2021).

Goals

This exercise aims to build your understanding about p-hacking, what it is, how it can occur, and its impact on research.

Materials

  • a computer, tablet, or smartphone with access to the internet.
  • a means of keeping brief notes (notebook or text/word processing document).

Background

You are a policy analyst with a not-for-profit, or as our resource says, “You’re a social scientist with a hunch: The U.S. economy is affected by whether Republicans or Democrats are in office.” Your task is to publish a short report about whether the U.S. economy does better under Republican control or Democratic control. You probably have prior ideas about this question, but your goal is to provide data that answer it.

You will make use of a dataset with information about how well the U.S. economy performs and about the degree of Republican or Democratic political power. The dataset goes back to 1948.

Your analysis

  1. Choose a party as the focus of your report (Republican or Democratic).
  2. Write down your prior hypothesis about whether the economy does better or worse under Republican or Democratic control. Do this before you visit the web site1.

[ ] Republican
[ ] Democratic

[ ] Economy does better
[ ] Economy does worse

  1. Visit the dataset site.
  2. Select the party you have chosen to focus on by pressing the button in panel 1. of the site.
  3. Write down the variables that are selected in “2. Define Terms” on the site.
    • Politicians included
      • Presidents
      • Governors
      • Senators
      • Representatives
    • Measure(s) of economic performance
      • Employment
      • Inflation
      • GDP
      • Stock prices
    • Other options
      • Factor in power
      • Exclude recessions
  4. If you know how, take a screenshot of the plot in “3. Is there a relationship” Include the screenshot in your report. In a sentence or two, describe what the plot shows.
  5. Report the result indicated in “4. Is your result significant?”. What is the p-value? What does that p-value mean in practical terms–what question was tested2? Is your result “publishable”?
  6. You may stop here and report your findings, or you may choose to explore the question further by selecting other variables (politicians included, measures of economic performance, or other options).
  7. If you explore further, for each set of variables you select, write down the variables you selected and the p-value you found:
Politicians Included Economic Measures Other factors p-value
  1. Create a report that captures the information you collected during the exercise and summarize your finding(s) in a sentence or two. You may use the simple report template as a guide.

Submit

Please save a copy of your report as a docx using the preferred file naming convention for this class: <lastname>-<firstname>-PSYCH490.002-ex02.docx, where you substitute your last name for and your first name for .

Submit your report on Canvas in the dropbox for this assignment.

Please also bring a copy to class with you on February 21, 2023.

References

Pownall, Madeleine, Flavio Azevedo, Sam Parsons, FORRT, Charlotte Rebecca Pennington, and Mahmoud Elsherif. 2021. “Lesson Plan 5: Understanding the Replication Crisis with App Activities.pdf.” http://dx.doi.org/10.17605/OSF.IO/TH254; OSF. https://doi.org/10.17605/OSF.IO/TH254.

  1. Think about why we want you to write down your prediction before you look at the dataset.↩︎

  2. Hint: The question tested has to do with the slope of the plot in 3.↩︎