11.6 Critiques and Comparisons
Critiques of the Autoregressive Model
Some interpretational oddities arise from the autoregressive model that are worth considering.
Consider the following situation: We are studying a weight loss intervention where we measure weight prior to and after an intervention. The mean weight at time 1 is 250 pounds (\(\mu_{t1}=150\)) and the mean weight at time 2 is 230 pounds (\(\mu_{t1}=130\)). Now consider two people:
- Individual 1: Weight at time 1 was \(240lbs\) and weight at time 2 is \(240lbs\).
- Relative standing has gone down so there is positive residualized change.
- Individual 2: Weight at time 1 was \(250lbs\) and weight at time 2 is \(230lbs\).
- Relative standing is the same so there is no residualized change.
Critique of Difference Score Model
There have also been some major historical critiques of differences scores (e.g. Cronbach and Furby (1970)).
Much of these critiques are based on reliability and the following rationale. Consider a typical model for a set of repeated measures,
\[ y_{1i} = y_{true,i} + e_{1i} \\ y_{2i} = y_{true,i} + e_{2i} \]
where
- \(y_{true,i}\) is the unobserved
truescore at both occasions - \(e_{i}\) is the unobserved random error that is independent over each occasion
Note, in this theoretical model the true score remains the same and all changes are based on random noise.
If this model holds then we could write a simple difference score as
\[ D_{i} = y_{2i} - y_{1i} \\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad= (y_{true,i} + e_{2i}) - (y_{true,i} + e_{1i})\\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad = (y_{true,i} - y_{true,i}) + (e_{2i} - e_{1i})\\ \quad\quad = (e_{2i} - e_{1i})\\ \]
where
- Variance of the difference score is entirely based on the variance of the differences in random error scores
- the reliability of the difference scores is zero
Alternative Interpretation
This has led many to many historical critiques of difference scores. However, other researchers have pointed out this conclusion is based on how one envisions change. If, for example, we have the following theoretical model for change,
\[ y_{1i} = y_{true,i} + e_{1i}\\ y_{2i} = (y_{1i} + \Delta y_{true,i}) + e_{2i} \] where
- \(y_{true}\) is the unobserved
truescore at both occasions - \(\Delta y_{true}\) is the unobserved
true changescore between occasions - \(e_{i}\) is the unobserved random error that is independent over each occasion
If this model holds, as opposed to the alternative model, then the difference scores
\[ D_{i} = y_{2i} - y_{1i} \\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad= (y_{true,i} + e_{2i}) - (y_{true,i} + e_{1i})\\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad = (y_{true,i} - y_{true,i}) + (e_{2i} - e_{1i})\\ \quad\quad \quad\quad\quad= \Delta y_{1} + (e_{2i} - e_{1i})\\ \]
where
- now the variance of the difference score is based on the variance of the differences in the random error scores and the gain in the true scores
- the relative size of the true score gain determines variance and reliability of the difference scores
- this implies difference scores may be an entirely appropriate means for measuring change