Table of Contents  |  Search Technical Documentation  |  References

NAEP Weighting Procedures → 2002 Weighting Procedures and Variance Estimation → State Assessment Weights → Quality Assurance for State NAEP 2002 Weighting → Weighted Enrollment Checks from the State NAEP 2002 Responding Student Samples

NAEP Technical DocumentationWeighted Enrollment Checks from the State NAEP 2002 Responding Student Samples

If the student nonresponse adjustments are working correctly to adjust for bias incurred from student nonresponse, the aggregations of the full student sample and the responding student sample should have the same expectation. Thus differences between them may indicate the presence of nonresponse bias not fully adjusted for in the weighting process.

Different student characteristics of enrollment (over which the weights were aggregated) were compared between the full and responding student samples. These characteristics were as follows:

• percent of males enrolled;
• older students (those with birthdates before the modal birthdate year¹);
• younger students (those with birthdates after the modal birthdate year);
• Title I students;
• race and ethnicity (Black, Hispanic, Asian/Pacific Islander, and Native American students);
• students with disabilities (SD) status;
• SD students who were excluded;
• English language learners (ELL) status; and
• ELL students who were excluded.

The school administration form was used to classify the students' race/ethnicity rather than the students' self-reports of race/ethnicity from the questionnaire.

Assuming that the null hypothesis is correct, namely that all of the expectations of differences are zero, the p-values for the two-sided t-test should have a uniform distribution in the interval [0,1]². The differences appear to have this pattern. One also can see that all of the differences for excluded students (SD and ELL) are zero. This finding is a direct effect of the weighting process, which assigns the same weight to excluded students before and after the student nonresponse weighting adjustment process (i.e., only eligible students are included in this weight adjustment process).

¹The modal birthdate year was the one October 1 through September 30 period which contained the birthdates for a majority of students (e.g., the majority of fourth-graders had birthdates in the October 1, 1991 through September 30, 1992 year).

²See for example Efron, B. (2007), Correlation and large-scale simultaneous significance testing, Journal of the American Statistical Association 102, p. 103.

Printer-friendly Version