Share

Observational Studies

5. Construct Validity

To take another example, research on the impact of early child care and educational interventions requires sensible measures of those activities. As Layzer and Goodson (2006):556 note "There is a widespread belief that high-quality early care and education can improve children's school readiness.  However, debate continues about the essential elements of high-quality experience, about whether quality means the same things across different types of care settings, about how to measure quality, and about the level of quality that might make a meaningful difference in the outcomes of children."

In their article they address four questions:

  1. How is the quality of child care environment commonly defined and measured?
  2. Do the most commonly used measures capture the child's experience?
  3. Do measures work well across all care settings?
  4. Are researchers drawing the correct conclusions from studies of child care environments and child outcomes?

Good measurement can be boiled down to two features: validity and reliability. Both in practice are matters of degree. For validity the issue is how well you are measuring what you think you are measuring.

Example 3

Substance abuse is often a chronic problem for which several interventions are needed so that each responds to where in the life course an individual falls. One implication is a shift from an acute care paradigm to a chronic care paradigm.  Rush and his colleagues report on the results of an intervention called "Recovery Management Checkups" (RMC) designed to help "people with substance abuse disorders by level of co-occurring mental disorders..." (Rush et al., 2008:7). "The RMC intervention targets individuals who have previously participated in treatment and are now living in the community using substances. The intervention ... aims to provide immediate linkage back to substance abuse treatment on the basis of need, thus expediting the recovery process.  Key components include, for example, assessing eligibility for the intervention and need of treatment, transferring participants in need of treatment from the interviewer to a linkage manager for a brief intervention, linking participants to the intake assessment, and ultimately linking participants to treatment" (Rush et al., 2008:8). A key measurement issue is to determine who is in need of treatment. For this study, such a person was defined as a study participant living in the community (vs. incarcerated or in treatment) who was not already in treatment and answered yes to any of the following questions:

  1. During the past 90 days, have you used alcohol, marijuana, cocaine, or other drugs on 13 or more days?
  2. During the past 90 days, have you gotten drunk or been high for most of 1 or more days?
  3. During the past 90 days, has your alcohol or drug use caused you not to meet your responsibilities at work/school/home on 1 or more days?
  4. During the past month, has your substance use caused you any problems?
  5. During the past week, have you had withdrawal symptoms when you tried to stop, cut down, or control your use?
  6. Do you feel that you need to return to treatment?"

The alpha (measuring internal consistency) reported for these items was .85.