10.8 Critique of Difference Scores
There have also been some major historical critiques of differences scores (e.g. Cronbach and Furby (1970)).
Much of these critiques are based on reliability and the following rationale. Consider a typical model for a set of repeated measures,
\[ y_{1i} = y_{true,i} + e_{1i} \\ y_{2i} = y_{true,i} + e_{2i} \]
where
- \(y_{true,i}\) is the unobserved
truescore at both occasions - \(e_{i}\) is the unobserved random error that is independent over each occasion
Note, in this theoretical model the true score remains the same and all changes are based on random noise.
If this model holds then we could write a simple difference score as
\[ D_{i} = y_{2i} - y_{1i} \\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad= (y_{true,i} + e_{2i}) - (y_{true,i} + e_{1i})\\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad = (y_{true,i} - y_{true,i}) + (e_{2i} - e_{1i})\\ \quad\quad = (e_{2i} - e_{1i})\\ \]
where
- Variance of the difference score is entirely based on the variance of the differences in random error scores
- the reliability of the difference scores is zero
10.8.1 Alternative Interpretation
This has led many to many historical critiques of difference scores. However, other researchers have pointed out this conclusion is based on how one envisions change. If, for example, we have the following theoretical model for change,
\[ y_{1i} = y_{true,i} + e_{1i}\\ y_{2i} = (y_{1i} + \Delta y_{true,i}) + e_{2i} \] where
- \(y_{true}\) is the unobserved
truescore at both occasions - \(\Delta y_{true}\) is the unobserved
true changescore between occasions - \(e_{i}\) is the unobserved random error that is independent over each occasion
If this model holds, as opposed to the alternative model, then the difference scores
\[ D_{i} = y_{2i} - y_{1i} \\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad= (y_{true,i} + e_{2i}) - (y_{true,i} + e_{1i})\\ \quad\quad\: \quad\quad\:\quad\quad\:\quad\quad\quad = (y_{true,i} - y_{true,i}) + (e_{2i} - e_{1i})\\ \quad\quad \quad\quad\quad= \Delta y_{1} + (e_{2i} - e_{1i})\\ \]
where
- now the variance of the difference score is based on the variance of the differences in the random error scores and the gain in the true scores
- the relative size of the true score gain determines variance and reliability of the difference scores
- this implies difference scores may be an entirely appropriate means for measuring change