NAEP Technical DocumentationNew-School Sampling Frame for the 2009 State Assessment

The Common Core of Data (CCD) file used for the frame corresponds to the 2005-2006 school year, whereas the assessment year is the 2008-2009 school year. During this 3-year period, some schools closed, some changed structure (one school becoming two schools, for example), and others came into existence.

As was done in previous years, to achieve as close to full coverage as possible, the school frame was supplemented by a sample of new schools obtained from a sample of districts. Each sampled district was sent a list of the CCD schools and asked to add in any new schools or old schools that had become newly eligible for fourth, eighth or twelfth grades.

Since asking every school district to list new- and newly-eligible schools would have generated too much of a burden, a sample of districts was contacted to obtain a list of new schools. To represent the unsampled districts in the full sample of schools, weights for schools included in the new-school sample were adjusted to reflect the district selection probability.

The goal was to allow every new school a chance of selection, thereby fully covering the target population of schools in operation during the 2008-2009 school year. The first step in this process was the development of a new-school frame through the construction of a district-level file from the CCD school-level file. To develop the frame, the district-level file was divided into two files: one for small districts and a second for medium and large districts.

Small districts contained no more than three schools on the frame in total, with no more than one school at each targeted grade (fourth, eighth, and twelfth). New schools in small districts were identified during school recruitment and added to the sample if the old school was sampled. From a sampling perspective, the new school was viewed as an “annex” to the sampled school that had a well-defined probability of selection equal to that of the old school. The “frame” in this case was, in fact, the original frame; when the old school was sampled in a small district, the new school was automatically sampled as well.

The remaining districts were defined as medium and large districts. In these districts, a frame of new schools was developed based on information provided by the district. To limit the required effort, the new-school frame was created through developing information on a sample of medium and large public school districts in each jurisdiction. All districts were selected in the following classes of districts:

• jurisdictions where all schools were sampled with certainty at either grade 4, 8 or 12 (so that all new schools would be selected with certainty, as well),

• state-operated districts,

• districts in states with fewer than 10 districts,

• districts containing no schools other than charter schools, and

• TUDA districts.

The remaining districts in each jurisdiction (excepting the certainty jurisdictions) were separated into two strata of large- and medium-size districts. These strata were defined by computing an aggregate percentage of enrollment for each district within the state (removing districts in the certainty strata defined above) and sorting in descending order by percentage of jurisdiction enrollment represented by the district. All districts up to and including the first district at or above the 80th cumulative percentage were defined as large districts. The remaining districts were defined as medium districts.

An example is given below. A state's districts are ordered by percentage enrollment. The first six become large districts and the last six become medium districts.

The target sample sizes for each jurisdiction were 10 districts total: 8 large and 2 medium. More than 10 districts were sampled in the jurisdictions that were oversampled. In the example above, all six large districts and four of the medium districts were selected for the new-school inquiry.

If sampling was needed in the medium stratum (i.e., it was not a certainty jurisdiction), the medium districts were selected with equal probability. If sampling was needed in the large stratum, the large districts were sampled with probability proportional to enrollment. These probabilities were retained and used in all later stages of sampling and weighting, as the district probability then represented the number of other districts that were not sampled to be surveyed for new schools.

The selected districts in each jurisdiction were then sent a listing of all their schools that appeared on the 2005-2006 CCD file and were asked to provide information about the new schools not included in the file and grade span changes of existing schools. These listings provided by the selected districts were used as sampling frames for selection of new public schools and updates of existing schools. This process was conducted through the NAEP State Coordinator in each jurisdiction. The coordinators were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates.

The eligibility of a school was determined based on the grade span. A school also was classified as “new” if a change of grade span had occurred such that the school status changed from ineligible to eligible in a particular grade.

Printer-friendly Version