NAEP Technical DocumentationNew-School Sampling Frame for the 2010 Public School Assessment

The most current Common Core of Data (CCD) file available was used to construct the public school frame for NAEP 2010. However, the information on that file was 3 years out of date by the time of the NAEP assessment. During that 3-year period, some schools closed, others changed grade span, and still others came into existence.

One can improve coverage by asking districts to provide information on currently open schools that were not listed on the 2006-2007 CCD file used to create the NAEP public school frame, and also to report grade span changes that may have caused a CCD-listed school to become newly eligible for fourth, eighth, or twelfth grades. Asking all districts to do this would have imposed an undue burden, so instead, a random sample of districts was contacted to obtain lists of new and newly eligible schools. The goal was to allow every new or newly eligible school a chance of selection, thereby fully covering the target population of schools in operation during the 2009-2010 school year.

The first step in this process was the development of a new-school frame through the construction of a district-level file from the CCD school-level file. Since the operational assessment was to be conducted within 105 geographically defined areas selected as Primary Sampling Units (PSUs), only districts that fell within the boundaries of those PSUs were eligible for sampling. Once the district-level file was subset to just the targeted PSUs, it was divided into three files: the first containing state-operated and charter school districts, the second containing small districts, and the third containing large districts.

State-operated districts and districts containing no schools other than charter schools require special handling. In survey years when state-level assessments are conducted, State Coordinators are asked to provide the names of all new charter-only and state-run schools. However, these types of school districts tend not to be geographically compact, and it is not feasible to link such a district to a single PSU, except at the individual school level. The smaller the proportion of a state’s population falling within sampled PSUs, the less likely that a specific new school of this type will be added to the frame and the more likely that state personnel will have expended unnecessary effort in providing updated information that will not be used. For this reason, for the NAEP 2010 assessment, the charter-only and state-run district component of the new school procedure was implemented only in states where more than 60 percent of youth fell within sampled PSUs. This meant that this component of the new-school sampling frame procedure was implemented in 13 states plus the District of Columbia, which taken together contain about 46 percent of the nation's youth.

The remaining districts were classified as small or large. A small district contains no more than three schools on the frame in total, with no more than one school at each targeted grade (fourth, eighth, and twelfth). New schools in small districts were identified during school recruitment and added to the sample if the new-sampling frame school was sampled for the relevant grade. From a sampling perspective, the new school was viewed as an “annex” to the sampled school that had a well-defined probability of selection equal to that of the frame school. Thus when the frame school was sampled in a small district for a particular grade, any new school was automatically sampled as well, for that grade.

Large districts were divided into 77 strata based on the NAEP 2010 PSU sampling strata, with districts in certainty PSUs grouped together in a single stratum. The district sample was allocated to each of the 77 strata proportional to the percent of the U.S. population of fourth, eighth, and twelfth graders contained in that stratum, with the caveat that each stratum had to be allocated at least one district. Once the allocation to each stratum had been fixed, districts were sampled from a sorted list using systematic sampling with probability proportional to size and a random start, the measure of size being the count of fourth, eighth, and twelfth graders enrolled in the district. Within the certainty PSU stratum, districts were sorted in a serpentine manner by state and measure of size prior to sampling. In all other strata the districts were sorted by measure of size alone. District selection probabilities were retained and used in all subsequent stages of sampling and weighting.

The selected districts were then sent a listing of all their schools that appeared on the 2006-2007 CCD file and were asked to provide information about any schools missing from CCD, and grade span changes of existing schools. This information provided by the sampled districts was used to construct sampling frames for the selection of new or newly eligible public schools and also for updating the status of existing schools (e.g. school closings). This process was conducted through the NAEP State Coordinator in each jurisdiction. The coordinators were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates.

Last updated 07 June 2011 (JL)

Printer-friendly Version