8.5 Intercept-Only Model

In logistic regression, we are interested in how various predictors are related to the probability of a specific outcome $P(Y_i = 1) = \pi_i$. In this example we are interested in the probability an individual reports being happy, in a general sense.

Making use of the logit link function, the general equation for logistic regression is
\[\mathrm{logit}(\pi_i) = \beta_{0} + \beta_{1}x_1 + ... + \beta_{q}x_q\]

Which after back transformation gives us … \[ P(Y_i = 1) = \pi_i = \frac{e^{\beta_{0} + \beta_{1}x_1 + ... + \beta_{q}x_q}}{1+e^{\beta_{0} + \beta_{1}x_1 + ... + \beta_{q}x_q}} = \frac{\mathrm{exp}(\beta_{0} + \beta_{1}x_1 + ... + \beta_{q}x_q)}{1+\mathrm{exp}(\beta_{0} + \beta_{1}x_1 + ... + \beta_{q}x_q)} \]

8.5.1 Intercept-Only Model in R

In our example the variable Happy indicates whether a subject reporting being happy. Let’s start with the simplest model for predicting Happy, the intercept-only model. More specifically, we have $ logit(_i) = b_0(1_i)$where $\pi_i = P(grad_i = 1)$.

We can use the glm() function to fit the model to the data

model9 <- glm(Happy ~ 1, 
              family = "binomial", 
              data = dunn2008, 
              na.action = na.exclude)
summary(model9)

## 
## Call:
## glm(formula = Happy ~ 1, family = "binomial", data = dunn2008, 
##     na.action = na.exclude)
## 
## Coefficients:
##             Estimate Std. Error z value Pr(>|z|)    
## (Intercept) -0.69077    0.08435   -8.19 2.62e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for binomial family taken to be 1)
## 
##     Null deviance: 805.02  on 631  degrees of freedom
## Residual deviance: 805.02  on 631  degrees of freedom
## AIC: 807.02
## 
## Number of Fisher Scoring iterations: 4

Without wanting to get to detailed we don’t need to specify the logit link here because it is the canonical link function for the binomial distribution. This essentially means there is a direct correspondence between the predicted mean and the distribution’s canonical location parameter.

8.5.2 Interpretation

8.5.2.1 Intercept Parameter

In the intercept-only model, the intercept, $b_0$, reflects

The expected log-odds ($-0.69077$) of an individual reporting they were happy.
The odds of someone reporting being happy $\mathrm{exp}(b_0)=0.5$.

exp(-0.69077)

## [1] 0.50119

The expected probability ($0.33$) of the a subject reported being happy in general.

\[ P(Happy_i = 1) = \pi_i = \frac{e^{b_0}}{1+e^{b_0}} \] or, equivalently, in R

exp(-0.69077)/(1 + exp(-0.69077))

## [1] 0.3338618

We can also confirm that the backward transformed parameter from this intercept-only logistic regression matches the expectation we get from the descriptives of the raw data.

mean(dunn2008$Happy)

## [1] 0.3338608

Note: If $\beta_j > 0$ then $\mathrm{exp}(b_j) > 1$, indicating a positive relationship between $X_{j}$ and the probability of the event occurring. If $\beta_j < 0$, the opposite relationship holds.