NAEP Technical DocumentationComparison of State NAEP School Samples Against the Public School Sampling Frame

The full school sample is a probability sample from the frame for each jurisdiction. The summation of the probability-weighted estimated grade enrollment aggregations (estimated grade enrollment divided by the school probability of selection) should be unbiased estimates of the corresponding frame grade enrollment figures.

These aggregations were computed for percent Blacks, Hispanics, Asians, and American Indians, and for mean median income (the mean of the median incomes) and type of location, by jurisdiction. These aggregations were also computed for state achievement data, in those states for which we had data. Two-sided p-values were calculated to test the null hypothesis that the difference between sample and frame is zero, using the jackknife standard error of the sample aggregation (note that the frame aggregation is treated as having no sampling error, as there is no sampling process in developing the frame, except for the very limited area portion of the Private School Survey, and the district sample portion of the public new-school sample). It should be expected that many of the p-values would be small simply from sampling error, as so many p-values were calculated.

The results are summarized as follows:

Of the differences that were significant, all but four absolute differences involving percentages were less than a percentage point, with most being near zero.
Of 96 total differences that were calculated for median income, only 12 differences reached the nominal 5 percent level of significance.
Of 68 total differences that were calculated for achievement scores, only five differences reached the nominal 5 percent level of significance.

Last updated 09 June 2008 (LR)

Printer-friendly Version