Table of Contents  |  Search Technical Documentation  |  References

NAEP Sample Design → Sample Design for the 2001 Assessment → National Main Assessment Sample Design in 2001 → Sampling of Schools in the 2001 National Main Assessment → Assigning Measures of Size and Selecting School Samples for the 2001 National Main Assessment

NAEP Technical DocumentationAssigning Measures of Size and Selecting School Samples for the 2001 National Main Assessment

For each grade-level sample, schools were selected (without replacement) across all primary sampling units (PSUs), systematically from a sorted list with probabilities proportional to assigned measures of size. The sorting variables included the following variables:

• certainty/noncertainty PSU classification,

• NAEP region (Northeast, Southeast, Central, and West),

• public/nonpublic school classification,

• type of location,

• high/low minority classification,

• PSU stratum, and

• estimated grade enrollment.

Type of location is a Common Core of Data (CCD) field attached to each school defining the type of locality of the community: central city large city, central city mid-sized city, urban fringe large city, urban fringe mid-sized city, large town, small town, and rural. (Note that the two CCD rural subgroups are collapsed together here.) The order of the sort was different depending on public and private school classification and certainty/noncertainty PSU classification.

To increase cost-efficiency in sampling, samples were designed to include more nonpublic schools, high-minority public schools, and relatively large schools. Therefore, a measure of size was assigned to each school according to the following scheme. Let S_i denote the estimated number of grade-eligible students in school i. Let L = 60 for the assessment of grade 4, L = 100 for the assessment of grade 8, and L = 100 for the assessment of grade 12. The measure of size was defined as follows:

• 0.25K_i, if S_i was five or less;

• K_iS_i /20, if S_i was between six and nineteen;

• K_i, if S_i was between 20 and L; and

• K_iS_i /L, if S_i was greater than L;

where K_i equals

• 2.8, for nonpublic schools on the fourth- and eighth-grade frames (other than Bureau of Indian Affairs and Department of Defense domestic schools);

• 2.875, for nonpublic schools on the twelfth-grade frame (other than Bureau of Indian Affairs and Department of Defense domestic schools);

• 2, for high minority public schools; and

• 1, for low minority public schools.

This procedure was used so as to obtain approximately self-weighting samples of students (i.e., students selected with approximately equal overall probabilities) at each grade within oversampling domains. Three aspects of the school-level measure-of-size cause the resulting student weights to vary.

• Schools with fewer than 20 estimated grade-eligible students were assigned somewhat lower measures of size, and thus lower probabilities of selection. This was designed to increase cost efficiency.

• Each public school designated as high-minority was given double the measure of size of a public school not designated high-minority of similar size. Such high-minority schools were oversampled in order to enlarge the sample of Black and Hispanic students, thereby enhancing the reliability of estimates for these groups.

• Each public school designated as high-minority was given double the measure of size of a public school not designated high-minority of similar size. Such high-minority schools were oversampled in order to enlarge the sample of Black and Hispanic students, thereby enhancing the reliability of estimates for these groups. For a given overall size of sample, this procedure reduces somewhat the reliability of estimates for all students as a whole and for those not Black or Hispanic.

The actual school sample from the school frame for each grade was a systematic sample using these measures of size, with schools ordered to achieve an implicit stratification on school characteristics judged to be related to school achievement. The NAEP 2001 ordering was the same as in NAEP 2000. The frame schools for each grade were ordered first by dividing into three subsets: certainty PSU public schools, certainty PSU nonpublic schools, and noncertainty PSU schools. The certainty PSU public schools were ordered by NAEP region first, type of locality second, minority status third, PSU stratum fourth, and estimated grade enrollment fifth. The certainty PSU nonpublic schools were ordered by NAEP region first, school type second (Catholic, non-Catholic religious, non-religious private), PSU stratum third, and estimated grade enrollment fourth.

The noncertainty PSU schools were ordered by PSU stratum first, and then by school type (public and nonpublic). The public schools were then ordered by type of locality first, minority status second, and estimated grade enrollment third. The nonpublic schools were ordered by school type first (Catholic, non-Catholic religious, non-religious private), and estimated grade enrollment second.

The sort order was 'serpentine' at each level: an alternation of ascending to descending and descending to ascending within each higher level group. For example, for certainty PSU public schools, the sort order for estimated grade enrollment within PSU stratum was ascending to descending for the first PSU stratum, descending to ascending for the next PSU stratum, and so on.

Once the measures of size had been determined, the school samples were selected.

Printer-friendly Version