Observational Studies

5. Construct Validity


There is no argument that all variables important to an analysis should be well measured. There is also no argument that most variables will be measured in an imperfect manner. Therefore, two questions need to be addressed:

  1. Are the requisite variables included in the dataset? and
  2. How well are they measured?

As explained more fully later, it is as important to have well-measured response variables as it is to have good measures of the intervention(s) and comparison conditions, along with good measures of all confounders.

We focus, therefore, on measurement quality. Sometimes measurement quality is called construct validity.

To take a recent example, there is growing interest in measuring how effective colleges are in educating undergraduates. One possible measurement tool is the "College Learning Assessment" (CLA) instrument.  "The CLA focuses on the institution (rather than the student) as the unit of analysis.  Its goal is to provide a summative assessment of the value-added by the school's instructional and other programs (taken as a whole) with respect to certain important learning outcomes" (Klein et al., 2007:418). The natural question is how well the CLA achieves these aims.