Table of Contents  |  Search Technical Documentation  |  References

NAEP Assessment Weighting Procedures → Weighting Procedures and Variance Estimation for the 2004 Assessment → Computation of Full-Sample Weights for the 2004 Assessment → School and Student Nonresponse Adjustments for the 2004 Assessment

NAEP Technical DocumentationSchool and Student Nonresponse Adjustments for the 2004 Assessment

			School Nonresponse Adjustment Student Nonresponse Adjustment

Nonresponse is unavoidable in any voluntary survey of a human population. Nonresponse leads to the loss of sample data that must be compensated for in the weights of the responding sample members. The purpose of the nonresponse adjustments is to reduce the mean square error of survey estimates. While the nonresponse adjustment reduces the bias from the loss of sample, it also increases variability among the survey weights leading to increased variances. However, it is presumed that the reduction in bias more than compensates for the increase in the variance, thereby reducing the mean square error and thus improving the accuracy of survey estimates. Nonresponse adjustments are made in the NAEP surveys at both the school and the student levels: the responding schools receive a weighting adjustment to compensate for nonresponding schools, and responding students receive a weighting adjustment to compensate for nonresponding students.

The paradigm used for nonresponse adjustment in NAEP is the quasi-randomization approach (Oh and Scheuren 1983). In this approach, school response cells are based on characteristics of schools known to be related to both response and achievement levels, such as the socioeconomic status of the school. Likewise, student response cells are based on characteristics of the schools containing the students, and student characteristics, which are known to be related to both response and to achievement levels, such as student race, gender, and age.

Under this approach, sample members are assigned to mutually exclusive and exhaustive response cells based on pre-determined characteristics. The weighting adjustment for each responding unit will be equal to the summation of adjusted base weights for all units divided by the summation of adjusted base weights for all responding units. In this way, the weights of responding units in the cell are "weighted up" to represent the full set of responding and nonresponding units in the response cell. The weight used for the school nonresponse adjustment is the product of the school base weight, the school trimming factor, the school-level session assignment weight, and the estimated age enrollment of the school. The weight used for the student nonresponse adjustment is the product of the student base weight, school trimming factor, school nonresponse adjustment factor, and subject factor.

The quasi-randomization paradigm views nonresponse as another stage of sampling. Within each nonresponse cell, the paradigm assumes that the responding sample units are a simple random sample from the total set of all sample units. If this model is valid, then the use of the quasi-randomization weighting adjustment will eliminate any nonresponse bias. Even if this model is not valid, the weighting adjustments will eliminate bias if the achievement scores are homogeneous within the response cells (i.e., bias is eliminated if there is homogeneity either in response propensity or in achievement levels). See, for example, Chapter 4 of Little and Rubin (1987).

Printer-friendly Version