## Purpose

This document summarizes an analyis of the *p*-hacking [exercise](../exercises/ex04-p-hacking.qmd).
In it, we gather data about what individual students did and try to make sense of it.

## Quantitative analysis

It often saves typing to load a set of commands into memory. In R, groups of useful commands are called 'packages'. We can load a set of useful packages into memory by issuing the following command:

```{r}
library(tidyverse)
```

::: {.callout-note}

If you are interested in a career related to data science, [`tidyverse`](https://www.tidyverse.org) is a very powerful set of tools you will want to know more about.

:::

## Data entry

Via a Google Sheet: <https://docs.google.com/spreadsheets/d/1fnSwFrUcKvgqq_agDLe4t2DHXHtHoOlmLdtLVRSemrI/edit?usp=sharing>

::: {.callout-note}

Gilmore added data validation (Format/Data Validation) to the columns. Why?

:::

---

::: {.callout-note}

These data are "long".
Each row is a unique observation.
Long data are often easier to work with.
But not always.

:::

## Data gathering

First, I authenticate (sign-in) to Google using my Gmail account. If I haven't logged in using this script recently, it will ask me to log-in again.

```{r, eval=params$update_data}
googledrive::drive_auth("rick.o.gilmore@gmail.com")
```

Then I download the Google Sheet to a directory/folder called `csv/` using the file name `p-hacking-fa23.csv`.

```{r, eval=params$update_data}
googledrive::drive_download(file = "PSYCH 490.009 2023 Fall P-hacking", path = "csv/p-hacking-fa23.csv", type = 'csv', overwrite = TRUE)
```

::: {.callout-note}

What does CSV mean?

Why are CSV files often used in data analysis?

One answer is that CSV files are **i**nter-operable and largely **r**eusable, two of the characteristics recommended for sharing data under the FAIR principles [@Wilkinson2016-qr].

:::

Next, I read the CSV file using the `read_csv()` function.

```{r}
p_hacking_fa23 <- read_csv(file = "csv/p-hacking-fa23.csv", show_col_types = FALSE)
```

::: {.callout-note}

Functions in R take inputs and deliver outputs. The inputs are `file` and `show_col_types`.

:::

The output is an object called `p_hacking`. It is a table of data that I can refer to with that name.

I like to use the 'structure' function or `str()` to see what the data look like.

::: {.callout-note}

Data is a plural noun. So, (when we don't forget this) we say 'The data are...' not 'The data is...'.

:::

```{r}
str(p_hacking_fa23)
```

## Questions to explore

- Most data analysts find that the process of exploring data is iterative.
- We start with a question. That leads to another question. That leads to yet another question.
- It is also sometimes cyclical. To answer a question requires that we modify the form of our data file.
- I like to start with thinking about "data pictures." If X was true, what would the data look like?

::: {.callout-note}

So, what are our questions?

What view of the data would help us answer them?

:::

## Visualize

### Party by prediction

```{r fig-party-prediction-fall23, fig.cap="Party chosen by prediction: PSYCH 490 Fall 2023"}
xtabs(formula = ~ party + prediction, data = p_hacking_fa23)

p_hacking_fa23 |>
  ggplot() +
  aes(x = party, fill = prediction) +
  geom_bar() +
  theme_classic() +
  ylab("n respondents")
```

```{r fig-hist-p-vals-fall23, fig.cap="Histogram of realized *p* values by party chosen: PSYCH 490 Fall 2023"}
p_hacking_fa23 |>
  ggplot() +
  aes(x = p_value, fill = party) +
  geom_histogram(bins = 10) +
  facet_grid(~ prediction)
```

```{r fig-p-vals-by-analysis-fall23, fig.cap="*p*-values by number of analyses: PSYCH 490 Fall 2023"}
p_hacking_fa23 |>
  ggplot() +
  aes(
    x = as.factor(analysis),
    y = p_value,
    color = as.factor(student),
    shape = party,
    group = as.factor(student)
  ) +
  geom_point() +
  geom_line() +
  xlab("Analysis number")
```

## How many different combinations of variables?

How many different combinations of variable choices are there? 

There are $n=4$ measures of political control; $n=4$ measures of economic performance; $n=2$ 'other' factors; $n=2$ prediction choices; and $n=2$ political parties to focus on.

We can use the `combinat` package to help us figure this out.

```{r}
combinat::combn(c('pres', 'gov', 'senate', 'house'), 1)
```

This shows us the number of ways we can pick a single political measure from among the 4 choices. 
We see that there are 4 ways.

The next function shows us the number of ways to pick two measures.

```{r}
combinat::combn(c('pres', 'gov', 'senate', 'house'), 2)
```

There are 6 columns of two, so there must be 6 different ways to pick two measures.

```{r}
combinat::combn(c('pres', 'gov', 'senate', 'house'), 3)
```

There are 4 different ways to pick 3 measures.

And there is only one way to pick 4 among 4. Make sense?

If we add these up '4 + 6 + 4 + 1' = `r 4 + 6 + 4 + 1` we get the number of different choices we can make (15) about how many combinations of political power measures are possible.

Since there are also 4 different choices of economic performance measures, we know that there are 15 ways to pick these. Now we can calculate how many different possible combinations of variables there are.

```{r}
n_combos <- 15*15*2*2*2
```

We multiply because each of the choices (political power, economic performance, party, better or worse is independent).

So, there are $n=$ `r n_combos` of variables we could have chosen.
How does this impact the conclusions we can and should draw?

## Combine with Spring 2023 data?

We did the same exercise in Spring 2023.
Let's combine our data with theirs.

```{r download-p-hacking-sp23}
googledrive::drive_download(file = "PSYCH 490.002 2023 P-hacking", path = "csv/p-hacking-sp23.csv", type = 'csv', overwrite = TRUE)
```

```{r combine-p-hacking-sp-fall}
p_hacking_sp23 <- read_csv(file = "csv/p-hacking-sp23.csv", show_col_types = FALSE)

p_hacking_sp23$semester = "sp23"
p_hacking_fa23$semester = "fa23"

p_hacking_23 <- rbind(p_hacking_fa23, p_hacking_sp23)
```

## Combined data

```{r hist-p-vals-sp-fa, out.width="80%"}
p_hacking_23 |>
  ggplot() +
  aes(x = p_value, fill = party) +
  geom_histogram(bins = 10) +
  facet_grid(~ prediction)
```

```{r p-vals-by-analysis-sp-fa, out.width="80%"}
p_hacking_23 |>
  ggplot() +
  aes(
    x = as.factor(analysis),
    y = p_value,
    color = as.factor(student),
    shape = party,
    group = as.factor(student)
  ) +
  geom_point() +
  geom_line() +
  xlab("Analysis number")
```

### Choices for 'political power'

```{r make-power-long}
power_df <- p_hacking_23 |>
  pivot_longer(cols = contains('power_'), 
                      names_to = "political_positions", 
                      values_to = "pol_pos_selected") |>
  distinct() |>
  mutate(political_positions = stringr::str_remove(string = political_positions,
                                                             pattern = "power_"))
```

```{r fig-pol-power-choices, fig.cap="Respondents' choices of political offices in their analyses"}
power_df |> 
  dplyr::group_by(party, prediction, political_positions) |>
  dplyr::summarize(n_preds = sum(as.numeric(pol_pos_selected))) |>
  dplyr::arrange(desc(n_preds)) |>
  knitr::kable(format="html") |>
  kableExtra::kable_classic()
```

### Choices for 'economy'

```{r make-econ-long}
econ_df <- p_hacking_23 |>
  pivot_longer(cols = contains('econ_'), 
                      names_to = "econ_measures", 
                      values_to = "econ_meas_selected") |>
  distinct() |>
  mutate(econ_measures = stringr::str_remove(string = econ_measures,
                                                             pattern = "econ_"))
  
```

```{r fig-econ-measure-choices, fig.cap="Respondents' choices of economic measures in their analyses"}
econ_df |> 
  dplyr::group_by(party, prediction, econ_measures) |>
  dplyr::summarize(n_preds = sum(as.numeric(econ_meas_selected))) |>
  dplyr::arrange(desc(n_preds)) |>
  knitr::kable(format="html") |>
  kableExtra::kable_classic()
```
