NAEP Technical DocumentationDistrict Sampling for the New-School Sampling Frame of the 2002 State Assessment

The Common Core of Data (CCD) file for the school sampling frame used corresponds to the 1999–2000 school year, whereas the assessment year is the 2001–2002 school year. In this two-year period, there were schools that closed, schools that changed status (e.g., one school becoming two schools), and schools that came into existence. Schools that were identified as closed in the sample were classified as out-of-scope. Schools that changed were handled on a case-by-case basis.

The goal was to allow every new school a chance of selection, thereby fully covering the target population of schools in operation in the 2001–2002 school year. The first step in this process was the development of a new-school frame. The first step in developing the new-school frame was the construction of a district-level file from the CCD school-level file. The district-level file was divided into a "small" districts file, and a "medium and large" districts file that was used for this purpose.

Small districts consisted of those districts in which there were at most three schools on the CCD frame and no more than one fourth-, one eighth-, and one twelfth-grade school. New schools in small districts were identified during school recruitment, and added to the sample if the "old" school was sampled. From a sampling viewpoint, the new school was viewed as an "annex" to the sampled school, that had a well-defined probability of selection equal to that of the old school. The "frame" in this case was in fact the original frame: when the old school was sampled in a small district, the new school was automatically sampled as well.

The remainder of districts were denoted as "medium and large" districts. In these districts, a frame of new schools was developed based on information provided by the district. Given the burden of this process, a sample of medium and large public school districts was drawn in each jurisdiction that was identified as providing a new-school frame. All districts were selected in the following classes of districts:

districts in jurisdictions where all schools were sampled with certainty (so that all new schools were selected with certainty as well),
very large districts with more than 6000 fourth-grade students and 105 fourth-grade schools,
state-operated districts, and
districts in states with fewer than 10 districts.

The remaining districts in each jurisdiction (excepting the take-all jurisdictions) were separated into two strata of large and medium districts. These strata were defined by computing an aggregate percentage of enrollment for each district within the state (removing districts in the certainty strata defined above), and sorting in descending order by percentage enrollment. All districts up to and over the 80^th cumulative percentage were defined as large districts. The remaining districts were defined as medium districts.

An example is given below. A state’s districts are ordered by percentage enrollment. The first six became large districts and the last six became medium districts.

Example of a state's districts, national main assessment, by district: 2002
District	Percent enrollment	Cumulative percent enrollment	Stratum
District 1	20	20	L
District 2	20	40
District 3	15	55
District 4	10	65
District 5		75
District 6		85
District 7	5	90	M
District 8	2	92
District 9		94
District 10		96
District 11		98
District 12		100
NOTE: L = Large district; M = Medium district. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2002.

The target sample sizes for each jurisdiction were 10 districts altogether. Exceptions were jurisdictions with eight sampled districts (California, Florida, Maryland, and Texas), and jurisdictions with nine sampled districts (Hawaii, Illinois, Michigan, Nevada, New York, Pennsylvania, Virginia, and Wisconsin). The exceptions are due to the presence of certainty districts in these states. In the ten-sampled-district jurisdictions, up to eight districts in the large district stratum were sampled. If there were eight or more large districts, eight large districts and two medium districts were sampled. If there were less than eight large districts, all of the large districts were sampled, and then a total of 10 minus the number of large districts were sampled (so that the full sample across both strata is 10.) "Eight" and "ten" are replaced with "six" and "eight" in California, Florida, Maryland, and Texas; and with "seven" and "nine" in Hawaii, Illinois, Michigan, Nevada, New York, Pennsylvania, Virginia, and Wisconsin. In the table example, we would sample all six large districts and four of the medium districts.

If sampling was needed in the medium stratum (i.e., it was not a take-all circumstance), the medium districts were selected with equal probability. If sampling was needed in the large stratum, the large districts were sampled probability proportional to enrollment. These probabilities were retained and used in all later sampling and weights, as the district probability then represents the number of other districts which were not sampled to be canvassed for new schools. The selected districts in each jurisdiction were then sent a listing of all their schools that appeared on the file, and were asked to provide information about the new schools not included in the file. These listings, provided by the selected districts, were used as sampling frames for selection of new public schools. This process was conducted via the NAEP state coordinator. They were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates to the NAEP sampling staff.

The eligibility of a school was determined based on the grade span. A school was also classified as "new" if a change of grade span was such that the school status changed from ineligible to eligible. The average grade enrollment for these schools was set to the average grade enrollment before the grade-span change. The schools found eligible for sampling due to the grade-span change were added to the new-school selection frame.

Last updated 09 September 2008 (RF)

Printer-friendly Version