NAEP Technical DocumentationWeighted Enrollment Checks from the State NAEP 2002 Responding School Samples

The responding school sample consisted of the eligible public schools that cooperated with the 2002 state assessment. This was a subset of the full school sample. The respective weights for responding schools received school nonresponse adjustments to account for the nonresponding schools, as described in Computation of School Nonresponse Adjustments. If the nonresponse adjustments do adjust successfully for nonresponse biases, the aggregated weighted estimated grade enrollment for the responding schools should correspond to the aggregated enrollment for the full school sample. Significant differences between these totals may point to nonresponse biases that are not accounted for in the nonresponse adjustment process.

In addition, the standard error of the difference between these weighted aggregations was measured (to test the null hypothesis of no difference). Both percentages (means) have sampling error. An approximate sampling error for the difference was developed by concatenating the original school sample and responding school sample files together (so that each responding school was represented twice on the file), and treating it for this purpose as a single "sample." The weights for the original school sample were the school base weights, and the weights for the responding school sample were the nonresponse-adjusted school weights. The sample error was computed for the difference using these special file weights. The replicate difference estimates in a sense are the differences computed with a portion of the school sample perturbed, with the same schools perturbed among the original schools and the responding schools. The replicate variance estimates for the difference thus are correct at the level of school sampling: the first stage of sample selection.

Some of the "take-all" jurisdictions showed differences between the responding and full school sample (due to nonresponse), but no standard errors. Many states such as Connecticut, Massachusetts, North Carolina and Pennsylvania, for example, had perfect school response and, thus, have standard errors that are also zero (with no p-values).

For the fourth-grade tabulations, there were only two cases of significant differences according to the replicate variance calculations that were also significant in a substantive sense. Kansas showed a 2.3 percent lower Black percentage in the responding sample compared to the full school sample. Washington showed a 3.9 percent lower Hispanic percentage in the responding sample compared to the full school sample. For the eighth grade tabulations, there was only one case: North Dakota. North Dakota showed a 2.9 percent smaller Native American percentage in the responding school sample as compared to the full school sample.

Last updated 26 June 2008 (PE)

Printer-friendly Version