Table of Contents  |  Search Technical Documentation  |  References

NAEP Sample Design → NAEP 2002 Sample Design → 2002 State Assessment Sample Design → Substitute Schools

NAEP Technical DocumentationSubstitute Schools

			Selection of Substitute Schools

Anticipating school-level nonresponse, substitute schools were selected for many of the original sample schools within each jurisdiction. The theory behind using substitutes is that a nonresponding original sample school can be replaced with another school which is anticipated as yielding similar assessment results. If this situation is really the case, then using the substitute will reduce nonresponse bias. The decision as to whether to activate any or all of the substitutes for nonresponding schools in a given jurisdiction was made by the NAEP state coordinator, taking account of the response rate and the burden on schools. Otherwise, we relied on nonresponse adjustment alone.
           
A set of substitutes was selected by comparing each sampled school to each school not sampled in either the NAEP 2002 national main school sample or the field study school sample for the same grade,¹ which was in the same state and within the same urbanicity stratum. A distance measure was generated for each potential school pair, based on percentage in the most important minority and the next most important minority within the urbanicity stratum, estimated grade enrollment, and achievement score (in jurisdictions in which it was available) or median income (in jurisdictions in which achievement score was not available). For example, a school pair in which all minority percentage, grade enrollment, achievement score, and median income were all equal between the two schools would have a distance measure of zero. The larger the difference in the relevant values increase the distance measure. Only pairs of schools with small distance measures were selected to be original-substitute pairs. This selection was done to ensure that substitutes will be in the same state and urbanicity stratum, and will have minority percentages, grade enrollment, and achievement scores (or median income) close to that of the original school.

In many cases, no substitute candidates could be found for a sampled school. The first choice for a substitute was a school out-of-district, i.e., in a different district from the original school. If no such substitute was available, a substitute was allowed within the district. Out-of-district schools were preferred, as in many cases school nonresponse was determined by a district-level decision.

¹ Also excluded from the list of potential substitutes were all new schools (because of inadequate information to pair to an original) and all closed/ineligible schools (identified as such in the new school canvassing process).

Printer-friendly Version