NAEP Assessment Weighting Procedures → Weighting Procedures for the 2012 Assessment → Weighting Procedures for the 2012 Long-Term Trend (LTT) Assessment → Quality Control on Weighting Procedures for the 2012 LTT Assessment → Nonresponse Bias Analysis for the 2012 LTT Assessment

NAEP Technical DocumentationNonresponse Bias Analysis for the 2012 LTT Assessment

NCES Statistical standards call for a nonresponse bias analysis to be conducted for a sample with a response rate below 85 percent at any stage of sampling. Weighted school response rates for the 2012 assessment indicate a need for school nonresponse bias analyses for private school samples for ages 9, 13, and 17. No student nonresponse bias analyses were necessary since the student-level participation rates for all groups were above the 85 percent participation threshold. The school-level analyses were conducted separately at each age. Thus, three separate school-level analyses were conducted.

The procedures and results from these analyses are summarized briefly below. The analyses conducted consider only certain characteristics of schools and students. They do not directly consider the effects of the nonresponse on student achievement, the primary focus of NAEP. Thus, these analyses cannot be conclusive of either the existence or absence of nonresponse bias for student achievement. For more details, please see the NAEP 2012 LTT NRBA report  (337KB).

Each school-level analysis was conducted in three parts. The first part of the analysis looked for potential nonresponse bias that was introduced through school nonresponse. The second part of the analysis examined the remaining potential for nonresponse bias after accounting for the mitigating effects of substitution. The third part of the analysis examined the remaining potential for nonresponse bias after accounting for the mitigating effects of both school substitution and school-level nonresponse weight adjustments. The characteristics examined were census region, reporting subgroup (private school type), urban-centric locale, size of school (categorical), size of school (continuous), and race/ethnicity enrollment percentages.

Based on the school characteristics available, for the private school samples at ages 13 and 17, there does not appear to be evidence of substantial potential bias resulting from school substitution or school nonresponse. However, the analyses suggest that a potential for nonresponse bias remains for the age 9 private school samples for school percentage race/ethnicity characteristics.  Please see the full report for more details.

 

Printer-friendly Version