NAEP Technical DocumentationSample Design for the 2017 State Assessment

			Target Population Sampling Frame Stratification of Schools School Sample Selection Ineligible Schools  Student Sample Selection  School and Student Participation

Each assessment cycle, a sample of students in designated grades within both public and private schools throughout the United States is selected for assessment. In state assessment years, of which 2017 is an example, the samples of public schools and their students in each state are large enough to support state-level estimates.

The NAEP 2017 state assessments covered fourth- and eighth-grade students in public schools for operational mathematics and reading. A representative sample of students was drawn in each participating jurisdiction, including the 50 states, the District of Columbia, Puerto Rico, Bureau of Indian Education (BIE) schools, Department of Defense Education Activity (DoDEA) schools, and in school districts chosen for the Trial Urban District Assessment (TUDA). 

All jurisdictions, including the TUDA districts, were included in the mathematics and reading assessments, with the exception of Puerto Rico, where a special mathematics assessment was conducted instead of the operational mathematics and reading assessments. Also, BIE was not designed as a reportable jurisdiction for the 2017 state assessments, but a nationally representative sample of students in BIE schools was selected.

Generally for the state assessments, each non-TUDA jurisdiction sample is designed to produce aggregate estimates with approximately equal precision for all the participating jurisdictions, as well as estimates for various subpopulations of interest. The target student sample size for these jurisdictions is 2,700 for each operational subject. In 2017, the samples for operational mathematics and reading at grades 4 and 8 were designed in this fashion.

At grades 4 and 8, the overall target assessed student sample size for the operational samples in each non-TUDA jurisdiction was 5,400: 2,700 each for mathematics and reading. Of these 2,700 per subject, 2,200 were digitally based assessments (DBA) and 500 were paper and pencil assessments (PBA). For the special mathematics assessment in Puerto Rico, the target assessed student sample size was 6,000 (3,000 DBA and 3,000 PBA) at grades 4 and 8.

The target population for the NAEP 2017 state assessment included students in public schools who were enrolled in grades 4 and 8 at the time of assessment. The sampling frame included public schools having the relevant grade in each jurisdiction. The samples were selected based on a two-stage sample design:

• selection of schools within participating jurisdictions; and
• selection of students within schools.

From the stratified frame of public schools for each grade within each jurisdiction, a systematic random sample of grade-eligible schools was drawn with probability proportional to a measure of size based on the estimated grade-specific enrollment of the school.

For the TUDA, schools were sampled from the 27 participating TUDA districts at the same time schools were selected for the non-TUDA jurisdiction samples. The TUDA districts are listed below:

• Albuquerque Public Schools, New Mexico;
• Atlanta Public Schools, Georgia;
• Austin Independent School District, Texas;
• Baltimore City Public Schools, Maryland;
• Boston Public Schools, Massachusetts;
• Charlotte-Mecklenburg Schools, North Carolina;
• Chicago Public Schools, Illinois;
• Clark County School District, Nevada;
• Cleveland Metropolitan School District, Ohio;
• Dallas Independent School District, Texas;
• Denver Public Schools, Colorado;
• Detroit Public Schools, Michigan;
• District of Columbia Public Schools, District of Columbia;
• Duval County Public Schools, Florida;
• Fort Worth Independent School District, Texas;
• Fresno Unified School District, California;
• Guilford County Schools, North Carolina;
• Hillsborough County Public Schools, Florida;
• Houston Independent School District, Texas;
• Jefferson County Public Schools, Kentucky;
• Los Angeles Unified School District, California;
• Miami-Dade County Public Schools, Florida;
• Milwaukee Public Schools, Wisconsin;
• New York City Department of Education, New York;
• San Diego Unified School District, California;
• School District of Philadelphia, Pennsylvania; and
• Shelby County Schools, Tennessee.

These subsamples affected the design of the state samples in those states where TUDA districts were oversampled. In each of these states, there were distinct sampling rates for each TUDA district and for the balance of the state (i.e., the rest of the state not in a TUDA district).

Each selected school provided a list of eligible enrolled students from which a systematic sample of students was drawn. Starting in 2017, student sampling was more complex because of the two different modes (DBA and PBA) assigned at each school. In addition, sample sizes differed by type of area in which the school was located: state, large TUDA, or small TUDA. Details can be found in the student sample selection page. 

Printer-friendly Version