NAEP Technical DocumentationNew-School Sampling Frame for the 2012 Public School Long-Term Trend (LTT) Assessment

The most current Common Core of Data (CCD) file available was used to construct the public school frame for NAEP 2012. However, the information on that file was 3 years out of date by the time of the NAEP assessment. During that 3-year period, some schools closed, others changed grade span, and still others came into existence.

One can improve coverage by asking districts to provide information on currently open schools that were not listed on the 2008-2009 CCD file used to create the NAEP public school frame, and also to report grade span changes that may have caused a CCD-listed school to become newly eligible for ages 9, 13, or 17. Asking all districts to do this would have imposed an undue burden, so instead, a random sample of districts was contacted to obtain lists of new and newly eligible schools. The goal was to allow every new or newly eligible school a chance of selection, thereby fully covering the target population of schools in operation during the 2011-2012 school year.

The first step in this process was the development of a new-school frame through the construction of a district-level file from the CCD school-level file. Since the operational assessment was to be conducted within 67 geographically defined areas selected as Primary Sampling Units (PSUs), only districts that fell within the boundaries of those PSUs were eligible for sampling. Once the district-level file was subset to just the targeted PSUs, it was divided into three files: the first containing state-operated and charter school districts, the second containing small districts, and the third containing large districts.

State-operated districts and districts containing no schools other than charter schools require special handling. In survey years when state-level assessments are conducted, State Coordinators are asked to provide the names of all new charter-only and state-run schools. However, these types of school districts tend not to be geographically compact, and it is not feasible to link such a district to a single PSU, except at the individual school level. The smaller the proportion of a state’s population falling within sampled PSUs, the less likely that a specific new school of this type will be added to the frame and the more likely that state personnel will have expended unnecessary effort in providing updated information that will not be used. For this reason, for the NAEP 2012 assessment, the charter-only and state-run district component of the new school procedure was implemented only in states where more than 60 percent of youth fell within sampled PSUs. This meant that this component of the new-school sampling frame procedure was implemented in 9 states plus the District of Columbia, which taken together contain about 42 percent of the nation's youth.

The remaining districts were classified as small or large. A small district contains no more than eleven schools on the frame in total, with no more than one school at each targeted grade (second through twelfth). All other districts were considered to be large. The large districts were divided into 66 strata based on the NAEP 2012 PSU sampling strata. The district sample was allocated to each of the 66 strata proportional to the percent of the U.S. population of students ages 9, 13 and 17 contained in that stratum, with the caveat that each stratum had to be allocated at least one district. Once the allocation to each stratum had been fixed, districts were sampled from a sorted list using systematic sampling with probability proportional to size and a random start, the measure of size being the estimate of youth ages 9, 13 and 17 enrolled in the district. The districts were sorted by the measure of size. District selection probabilities were retained and used in all subsequent stages of sampling and weighting.

The selected districts were then sent a listing of all their schools that appeared on the 2008-2009 CCD file and were asked to provide information about any schools missing from CCD, and grade span changes of existing schools. This information provided by the sampled districts was used to construct sampling frames for the selection of new or newly eligible public schools and also for updating the status of existing schools (e.g. school closings). This process was conducted through the NAEP State Coordinator in each jurisdiction. The coordinators were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates.

Because the long-term trend (LTT) assessments are age based, the new-schools-in-small-districts procedure was done in a different manner from a grade-based year because it is not always clear, based on grade alone, whether a school is age-eligible. Once the small districts were identified, we prepared listings of all schools in small districts that had an LTT sampled school. The listings in turn were sent to the state coordinators in the affected states for review and updating, in the same manner as for the large-district new-school procedure. The statistical staff reviewed the updated listings from the states and determined that no new schools from small districts needed to be added to the sample.

Last updated 13 August 2012 (DB)

Printer-friendly Version