NAEP Technical DocumentationNonresponse Bias Analysis for the 2008 LTT Assessment

School-level nonresponse bias analyses were conducted on private schools at all three ages (9, 13, and 17) since the school-level participation rate of each of these groups fell below the NCES statistical standard threshold of 85 percent participation. The analyses involved three components. The first analysis looked for potential bias introduced through the school nonresponse. The second analysis examined the remaining potential for nonresponse bias after accounting for the mitigating effects of school substitution. The third analysis examined the remaining potential for nonresponse bias after accounting for the mitigating effects of both school substitution and school-level nonresponse weight adjustments.

The analyses suggest that although school nonresponse was quite high for private schools at all three ages, overall there does not appear to be evidence of substantial bias. After accounting for school substitution and school-level nonresponse weight adjustments, there are no significant results remaining for age 13. For age 9, race/ethnicity remains significant, specifically for the percentage White, non-Hispanic. The relative bias is small, 2.3 percent on an estimate of about 77 percent. For age 17, race/ethnicity remains significant, specifically for the percentage Asian/Pacific Islander. The absolute bias of 2.3 percent on a 5.6 percent estimate produces a fairly large relative bias of 41.6 percent.

Additionally for age 17, school size did not become significant until after the nonresponse adjustment. In the Northeast non-Catholic schools, there was an unusually low response rate, specifically about 10 schools out of about 40 responded (16 were ineligible, about 20 were nonresponding). The largest eligible sampled school in this group had about 200 students and about 10 other schools had more than 50, while the largest responding school had less than 50 students. Non-Catholic private schools in the Northeast were grouped as a nonresponse class and about 10 responding schools in this group were given a large adjustment factor (6.3316) so that they could represent about 20 nonresponding schools. Since all of the responding schools were small, this means that small non-Catholic private schools in the Northeast are overrepresented in the final weights. This bias was accepted in order to preserve other factors that were considered more related to achievement, in particular, affiliation.

There were no nonresponse bias analyses conducted at the student level since the student-level participation rates for all groups were above the 85 percent participation threshold.

To view the entire report or detailed tables, see NAEP Nonresponse Bias Analysis Report (359KB PDF).

Last updated 16 August 2012 (GF)

Printer-friendly Version