NAEP Assessment Sample Design → NAEP 2011 Sample Design → Sample Design for the 2011 State Assessment

NAEP Technical Documentation Sample Design for the 2011 State Assessment

			Target Population Sampling Frame Stratification of Schools School Sample Selection Ineligible Schools Student Sample Selection School and Student Participation

Each assessment cycle, a sample of students in designated grades within both public and private schools throughout the United States is selected for assessment. In state assessment years, of which 2011 is an example, the samples of public schools and their students in each state are large enough to support state-level estimates.

The NAEP 2011 state assessments covered fourth- and eighth-grade students in public schools for operational mathematics and reading. It also covered operational science but only for students at grade 8, not grade 4. A representative sample of students was drawn in each participating jurisdiction, including the 50 states, the District of Columbia, Bureau of Indian Education (BIE) schools, Department of Defense Education Activity (DoDEA) schools, and in school districts chosen for the Trial Urban District Assessment (TUDA) study.

All jurisdictions, including the TUDA districts, were included in the mathematics and reading assessments at grades 4 and 8. Only the 50 states, the District of Columbia, and DoDEA schools were included in the state science assessment. TUDA districts were not part of the state science assessment because they lacked the required number of students for this third subject.

The District of Columbia, which generally does not have enough students for an assessment in a third subject, participated in the grade 8 science assessment. This was accomplished by testing each student in two of the three assessment subjects.

Generally for the state assessments, each non-TUDA jurisdiction sample is designed to produce aggregate estimates with approximately equal precision for all the participating jurisdictions, as well as estimates for various subpopulations of interest. The target sample size for these jurisdictions is 3,150 for each operational subject.  In 2011, the samples for operational mathematics and reading at grades 4 and 8 were designed in this fashion. However, the grade 8 sample for operational science used a sample size that was 20 percent smaller (2,520) in order to ensure that there were enough students available for the various pilot test and special studies that were also being conducted in eighth grade.

In 2011, the overall target student sample size for the operational samples in each non-TUDA jurisdiction was 6,300 at grade 4 and 8,820 at grade 8 (except for BIE schools). Since BIE schools did not have enough students for a state-level assessment for science, its target sample size at  grade 8 was 6,600—3,150 each for mathematics and reading and 300 for science, enough samples so that it was adequately represented at the national level. Details can be found in the school sample selection.

The target population for the NAEP 2011 state assessment included students in public schools who were enrolled in the grades 4 and 8 at the time of assessment. The sampling frame included public schools having the relevant grade in each jurisdiction. The samples were selected based on a two-stage sample design:

1. selection of schools within participating strata, and
2. selection of students within schools.

From the stratified frame of public schools for each grade within each jurisdiction, a systematic random sample of grade-eligible schools was drawn with probability proportional to a measure of size based on the estimated grade-specific enrollment of the school.

For the TUDA study, schools were sampled from the 21 participating TUDA districts at the same time schools were selected for the jurisdiction samples. The TUDA districts are listed below. The ones in bold are those introduced in 2011.

• Albuquerque Public Schools, New Mexico;
• Atlanta Public Schools, Georgia;
• Austin Independent School District, Texas;
• Baltimore City Public Schools, Maryland;
• Boston Public Schools, Massachusetts;
• Charlotte-Mecklenburg Schools, North Carolina;
• Chicago Public Schools, Illinois;
• Cleveland Metropolitan School District, Ohio;
• Dallas Independent School District, Texas;
• Detroit Public Schools, Michigan;
• District of Columbia Public Schools, District of Columbia;
• Fresno Unified School District, California;
• Hillsborough County Public Schools, Florida;
• Houston Independent School District, Texas;
• Jefferson County Public Schools (Louisville) , Kentucky;
• Los Angeles Unified School District, California;
• Miami-Dade County Public Schools, Florida;
• Milwaukee Public Schools, Wisconsin;
• New York City Department of Education, New York;
• School District of Philadelphia, Pennsylvania; and
• San Diego Unified School District, California.

These subsamples affected the design of the state samples in those states where TUDA districts were oversampled. In each of these states, there were distinct sampling rates for each TUDA district and for the balance of the state (i.e., the rest of the state not in a TUDA district).

Each selected school provided a list of eligible enrolled students from which a systematic sample of students was drawn. In fourth-grade schools, 63 students, if possible, were selected from each school: 30 for mathematics, 30 for reading, and 3 for the pilot tests. In eighth-grade schools, the within-school target sample size ranged from 63 to 114. The target sample sizes depended upon the size of the state. Very small states did not have any pilot test/special study sample and larger states had somewhat more pilot test/special study sample than other states. This is to ensure that the samples for the pilot tests and special studies would be reasonably nationally representative. Details can be found in the student sample selection. 

Printer-friendly Version