Module road map

Overview

In the social sciences, we often want to know whether two variables are associated with one another. By associated we mean that knowing the value of one variable tells us something about the value of the other variable. In this module, we will focus on modeling the relationship between two continuous variables. We will use a technique called simple linear regression to study this type of relationship.

Recall this example from Module 2. The Centers for Disease Control and Prevention compiled these data in order to determine if countries who adopted more COVID-19 mitigation policies had fewer deaths.

Simple linear regression (or SLR) helps us to understand the extent to which two variables are associated by estimating the straight line that relates them to one another. SLR is a statistical technique that allows the analyst to predict scores for some continuous outcome (y) using a single predictor (x).

In the plot below, there appears to be a negative relationship, more mitigation strategies were associated with fewer deaths.

To build your intuition about regression, please watch the following Crash Course: Statistics video on Regression. Please note that the video includes an overview of fitting regression models, as well as inference testing (with topics such as degrees of Freedom, test statistics, and test distributions). You don’t need to worry about the inference parts for now – we’ll revisit these techniques in Module 11.

In this module, we will use linear models to describe the relationship between two variables. Though you may not have studied or used linear models for prediction before, we often use models to predict outcomes in everyday life. In particular we often use mathematical models to solve problems or make conversions. Mathematical models are deterministic. Once the “rule” is known, the mathematical model can be used to perfectly fit the data. That is, we can perfectly predict the outcome.

Examples:

1. Perimeter of a square: \(4 \times (length\,of\,side)\)
2. Area of a square: \((length\,of\,side)^2\)
3. Convert Fahrenheit to Celsius: \(C = (F - 32)\cdot\frac{5}{9}\)

While mathematical models are extremely useful, in this course, we will focus on statistical models. Unlike mathematical models, statistical models are not deterministic. They take into account that we usually don’t have all important predictors, that we rarely perfectly measure the variables that we do have, and that people (or organizations, schools, animals, etc.) act differently.

Statistical models allow for:

1. Excluded variables
2. Measurement error
3. Individual variation

The formula for a statistical model has a residual to account for variation from these sources:

Outcome = Systematic Component + Residual

Statistical techniques allow us to explain variation in our outcome (the systematic component) in the context of what remains unexplained (the residual).

Please watch the video below for an overview of simple linear regression.

Introduction to the data

In this module we will use a linear model to predict election outcomes. The people who we elect to public office have a large influence on the health and prosperity of our population. Our elected officials and their administrations have great power to either build up or tear down programs, practices and policies that promote health and well-being for all.

As one example, we have seen great differences in how state and local elected officials have handled the COVID-19 pandemic, in terms of mandates for public health protections, aid to individuals, and the roll out of the vaccine.

Given the critical significance of government and policy on public health, understanding the motivations of voters in electing public officials is an important public health matter. In this module, we will explore one well-studied and well-respected, but relatively simple, model for forecasting election outcomes. It’s called the Bread and Peace Model, and was formulated by Douglas A. Hibbs. He describes his model in detail here, the gist of his model is described below:

Postwar US presidential elections can for the most part be interpreted as a sequence of referendums on the incumbent party’s record during its four-year mandate period. In fact aggregate two-party vote shares going to candidates of the party holding the presidency during the postwar era are well explained by just two fundamental determinants: (1) Positively by weighted-average growth of per capita real disposable personal income over the term. (2) Negatively by cumulative US military fatalities (scaled to population) owing to unprovoked, hostile deployments of American armed forces in foreign wars.

In other words, in a US presidential election, the likelihood that the incumbent party maintains power is dependent on the economic growth experienced during the prior term and the loss of military troops due to war. The former increases favor for the incumbent party, while the latter decreases favor.

We’ll start simple in this module. We will consider a single outcome (election results of US presidential elections) and a single predictor (growth in personal income over the preceding presidential term). To estimate the Bread and Peace model, we will use data compiled by Drew Thomas (2020).

The following variables are in the dataframe:

• year is the presidential election year
• vote is the percentage share of the two-party vote received by the incumbent party’s candidate
• growth is the quarter-on-quarter percentage rate of growth of per capita real disposable personal income expressed at annual rates
• fatalities denotes the cumulative number of American military fatalities per millions of US population
• wars lists the wars of the term if fatalities > 0
• inc_party_candidate is the name of the incumbent party candidate
• other_party_candidate is the name of the other party candidate
• inc_party is an indicator of the incumbent party (D = Democrat, R = Republican)

We will estimate how well growth in income of US residents during the preceding presidential term predicts the share of the vote that the incumbent party receives. That is, we will determine if growth is predictive of vote. In the equations and descriptions below, I will refer to the predictor (growth) as the x variable, and the outcome (vote) as the y variable.

Let’s begin by importing the data.

bp <- read_rds(here("data", "bread_peace.Rds"))

The glimpse() function presents the key features of the data set. Each row of the data represents a different presidential election – starting in 1952 and ending in 2016. There are 17 elections in total to consider.

bp %>% glimpse()

## Rows: 17
## Columns: 8
## $ year                  <dbl> 1952, 1956, 1960, 1964, 1968, 1972, 1976, 1980, …
## $ vote                  <dbl> 44.55, 57.75, 49.92, 61.35, 49.59, 61.79, 48.95,…
## $ growth                <dbl> 3.0325, 2.5836, 0.5768, 4.3904, 3.2644, 4.1745, …
## $ fatalities            <dbl> 206, 0, 0, 2, 174, 0, 1, 0, 0, 0, 0, 0, 0, 4, 9,…
## $ wars                  <chr> "Korean", "none", "Vietnam", "Vietnam", "Vietnam…
## $ inc_party_candidate   <chr> "Stevenson", "Eisenhower", "Nixon", "Johnson", "…
## $ other_party_candidate <chr> "Eisenhower", "Stevenson", "Kennedy", "Goldwater…
## $ inc_party             <chr> "D", "R", "R", "D", "D", "R", "R", "D", "R", "R"…

Let’s obtain some additional descriptive statistics with skim().

bp %>% skim()

Variable type: character

Variable type: numeric

Since this is a quite small dataframe, we can easily view the whole dataframe at once.

Let’s familiarize ourselves a bit more with the two variables we’ll consider here. First, the predictor (growth) ranges from .17% (in 2008, just after the 2008 financial crisis) to 4.4% (in 1964, one of the most prosperous times in US history – particularly for the wealthy). Second the outcome (vote) ranges from 44.6% (in 1952, the US was in the middle of the Korean War) and 61.8% (in 1972, riding a strong economy and moving toward the end of the Vietnam War, Richard Nixon won by a landslide).

Explore a linear relationship

Let’s take a look at the relationship between growth (our primary predictor) and vote (our primary outcome) using a scatterplot. We will map growth (the predictor) to the x-axis, and vote (the outcome) to the y-axis. I will label the datapoints with the election year.

bp %>% 
  ggplot(aes(x = growth, y = vote)) +
  geom_point() +
  ggrepel::geom_label_repel(aes(label = year),
                            color = "grey35", fill = "white", size = 2, box.padding =  0.4, 
                            label.padding = 0.1) +
  theme_bw() +
  labs(title = "Bread and Peace Voting in US Presidential Elections 1952 - 2016", 
       x = "Annualized per capita real income growth over the term (%)", 
       y = "Incumbent party share of two-party vote (%)")

We always start an analysis by examining graphs of our variables. When two continuous variables are considered, a scatterplot is an excellent tool for visualization. It’s important to determine if the relationship appears linear (as opposed to curvilinear) and to note any possible outliers or other strange occurrences in the data.

If we want to summarize the relationship between x (growth) and y (vote), one option is to draw a straight line through the data points. This is accomplished in the plot below.

This straight line does a reasonable job of describing the relationship. Many relationships can be explained with a straight line, and this is the primary purpose of a linear regression model.

When we draw a straight line through a set of data points, the line can be defined by an intercept and a slope.

The intercept is the value of y on the line when x = 0. In looking at our graph, there were actually no years during the observation period in which there was 0 growth, the closest is 2008, which has about .2% growth. By extrapolating back a bit, it looks like this straight line to relate x and y (i.e., the linear model) would predict a score of about 45 (incumbent party receives 45% of vote share) if percent growth was 0.

The slope is the rate of change of the line.

Specifically, the slope is defined as the “rise over the run.” That is, the rise (i.e., change) in vote share for a one unit increase (i.e., run) in growth. Since it’s a straight line, we can pick any one-unit increase along the x-axis.

Let’s choose a “run” from 2 to 3 – that is, going from a 2 to a 3 on the x-axis.

When growth (the value on the x-axis) is 2, the value of vote looks to be about 51.

When growth is 3, the value of vote looks to be about 54.

That means as we go from growth = 2 to growth = 3 (a run of 1), vote share tends to increase by about 3 percentage points (54 minus 51 or a rise of 3). So the rise over the run = 3/1, corresponding to a slope of about 3. That means that for each one unit increase in growth, we predict the percent vote share to increase by about 3.

You might be wondering – “How was the location of the line on this graph determined?” If I asked you to manually draw the best fit line through these points, each student in the class would likely draw a line that was close, but not exactly the same to one another, and not exactly the same line that ggplot drew onto the graph for us. It is possible to fit an infinite number of straight lines to the data points in the scatterplot. However, we want to find the best fitting line.

The method that we use to find the best fit line is called the least squares criterion. We will study this method in the next part of this module.

Please view the video below to learn more about using a straight line to summarize the relationship between two continuous variables.

Estimate the best fit line

We can imagine many reasonable lines that could be drawn through the data points to relate the predictor to the outcome. For example, each of the lines on the graph below seems reasonable.

How do we determine the best fit line for the data? For any given x and y variable that are linearly related, there is indeed one best fitting line. The best fitting line is the line that allows x (growth) to predict y (vote) with the highest amount of accuracy. Let’s take a closer look at how we can determine the accuracy of any possible line drawn through the data.

The geom_smooth() function in R, specifying a linear model (method = “lm”) for y regressed on x (y ~ x), will overlay the best fit line. This is depicted in the graph below.

bp %>% 
  ggplot(aes(x = growth, y = vote)) +
  geom_point() +
  geom_smooth(method = 'lm',formula = y ~ x,  se = FALSE) +
  ggrepel::geom_label_repel(aes(label = year),
                            color = "grey35", fill = "white", size = 2, box.padding =  0.4, 
                            label.padding = 0.1) +
  theme_bw() +
  labs(title = "Bread and Peace Voting in US Presidential Elections 1952 - 2016", 
       "Annualized per capita real income growth over the term (%)", 
       x = "Annualized per capita real income growth over the term (%)", 
       y = "Incumbent party share of two-party vote (%)")

Fit a linear model in R

We use the lm() function in R to fit a linear model. In the code below I first define the name of the R object that will store our linear model results (bp_mod1). Then, I take the data frame (bp) and feed it into the lm() function.

We want to regress the outcome on the predictor, that is, we want to regress vote on growth to determine if growth is a predictor of vote. This is coded in the lm() function by writing vote ~ growth. Note that the lm() function in R, which is short for linear model, requires a data argument. Since we are piping in the data, by indicating data = ., we remind R that it already has the data.

bp_mod1 <- bp  %>%  
  lm(vote ~ growth, data = .)

Note that an alternative way to write this code and fit a linear model without the pipe operator is as follows:

bp_mod1 <- lm(vote ~ growth, data = bp)

bp_mod1 %>% 
  get_regression_table()

bp_mod1 %>% 
  get_regression_points(ID = "year")

bp_mod1 %>% 
  get_regression_summaries()

get_rsq <- bp_mod1 %>% 
  get_regression_points(ID = "year") %>% 
  mutate(for_SSR = vote_hat - mean(vote), # calculate the difference between each predicted score and the mean of y
         for_SSR2 = for_SSR^2) %>%  # square the difference
  mutate(for_SSE2 = residual^2) # square the residual
  
get_rsq       

SSR_SSE <- get_rsq %>% 
  summarize(SSR = sum(for_SSR2),
            SSE = sum(for_SSE2))

SSR_SSE

SSR_SSE %>% 
  mutate(r_squared = SSR/(SSR + SSE))

Correlation

For two numeric variables (e.g., vote and growth in the Bread and Peace dataframe) we can also calculate the correlation coefficient. A correlation coefficient, often abbreviated as r, quantifies the strength of the linear relationship between two numerical variables using a standardized metric. A correlation coefficient ranges from -1 to 1, where -1 indicates a perfect negative relationship (as one value increases, the other decreases), and +1 indicates a perfect positive relationship (as one value increases, the other increases). A correlation coefficient of 0 indicates no relationship between the two variables.

The get_correlation() function in the moderndive package calculates the correlation coefficient.

bp %>% 
  get_correlation(vote ~ growth)

The correlation between vote and growth is .7, denoting a strong, positive association. The square of the correlation coefficient equals the \(R^2\) in a SLR (i.e., \(R^2\) = .49).

While the regression slope is in the metric of the y variable, and is dependent on the measurement scale of both the x and y variables (i.e., a one unit increase in x is associated with a \({b_1}\) unit change in y), a correlation coefficient is in the metric of standardized units.

An interesting property of linear models is that if you create z-scores of your x and y variables, and then fit a simple linear regression, the slope for the regression line will equal the correlation. Take a look at the code below, and then see the estimate for growth_z in the output. It matches the correlation coefficient. Therefore, we can also interpret the correlation coefficient as “the expected change in the z-score of y for a one standard deviation increase in x.”

bp_z <- bp %>% 
  select(vote, growth) %>% 
  mutate(vote_z = (vote - mean(vote))/sd(vote)) %>% 
  mutate(growth_z = (growth - mean(growth))/sd(growth)) 

bp_mod_z <- bp_z %>% 
  lm(vote_z ~ growth_z, data = .)

bp_mod_z %>% 
  get_regression_table()

Correlation is a statistical measure that quantifies the size and direction of a relationship between two variables. As we learned in this module, estimating the relationship between two variables can be extremely useful. However, it is critical to realize that finding a correlation between two variables does not automatically mean that one variable causes the other variable. That is, correlation does not equal causation. Causation indicates that one event is the result of the occurrence of another event – in other words, that there is a causal relationship between the two events. We’ll learn more about correlation and causation in psychological studies in Module 7. For now, please watch the following Crash Course: Statistics video that introduces these issues.

Special note for DataCamp exercises for this module

In the DataCamp course for this module, Modeling with Data in the tidyverse, the instructor represents the regression equation in a different format than presented in your text book and in this module. To represent the relationship between some variable called y and some variable called x, in this module, we represented the regression equation as:

\[{y_i} = b_0 + b_1x_i + e_i\]

In the DataCamp course, the instructor writes the equation as:

\[y=f(\vec{x})+ε\]

These two formulas are equivalent. Rather than writing the systematic part of the regression line (i.e., the intercept plus the slope times x), Modeling with Data in the tidyverse represents this quantity as \(f(\vec{x})\). Likewise, the \(e_i\) in the first equation is the same as the \(ε\) in the second equation – that is, the residual. In DataCamp, they also do not index their variables with an i subscript (to denote that each case has a score), though this is implied.

This concludes Module 5 on simple linear regression. In the next two modules, we will learn how to add categorical variables as predictors in our models (Module 6) and how to build models with multiple predictors (Module 7).