NAEP Technical DocumentationStudent Sample Selection for the 2011 State Assessment

Within each sampled school, a sample of students was selected from a listing of the students in the grade such that every student had an equal chance of selection. The student lists were submitted in multiple ways. E-filing is an electronic submission system. Excel files are submitted for sampled schools by school coordinators or NAEP State Coordinators. Files can be submitted for one school at a time or for an entire jurisdiction at once. This method allows schools to easily submit student demographic data electronically with the student lists, easing the burden on NAEP field supervisors and school coordinators. Schools that are unable to submit their student lists using the e-filing system provide hard copy lists via the student listing form to NAEP field supervisors. In 2011, there were 18,023 schools that E-filed their student lists, while 781 lists were submitted using the student listing form.

In year-round, multi-track schools, students who were not scheduled to be in school on the assessment day were removed from the student lists prior to sampling. Student base weights were adjusted to account for these students.

The sampling process was the same, regardless of list submission type. The sampling process was systematic (e.g., if the sampling rate was one-half, a random starting point of one or two was chosen, and every other student on the list was selected). For E-filed schools only, where demographic data was submitted for every student on the frame, students were sorted by gender and race/ethnicity before the sample was selected to implicitly stratify the sample.

In the certainty jurisdictions, all students were sampled in all schools. Otherwise, the sample size for grade 4 was 63 students. The sample sizes for grade 8 varied due to the pilot and special study samples. Very small states did not have any pilot test/special study sample and larger states had somewhat more pilot test/special study sample than other states. This is so that the samples for the pilot tests and special studies would be reasonably nationally representative. The sample size for grade 8 ranged from 63 to 114 students with 89 students sampled in the majority of the jurisdictions. Larger schools may have been selected more than once in the sampling process and thus may have a larger sample size. In addition, most fourth-grade schools chose the option of taking all students when enrollment was less than 120. This increased the fourth-grade sample size in many states beyond the designated target.

Some students enrolled in the school after the sample was selected. In such cases, new enrollees were sampled at the same rate as the students on the original list.

In fourth-grade schools, sampled students were randomly assigned to mathematics, reading, and pilot as follows: 30 for mathematics, 30 for reading, and 3 for pilot. In eighth-grade Bureau of Indian Education (BIE) schools that did not participate in the state science assessment, the sampled students were randomly assigned to reading, mathematics, and science as follows: 30 for mathematics, 30 for reading, and 3 for national science. In the other eighth-grade schools that did participate in the state science assessment, students were randomly assigned to mathematics, reading, and science as follows: 30 for mathematics, 30 for reading, 24 for science, plus a varying number for pilot and special studies. This was implemented by spiraling: the booklets assigned to sampled students were provided from booklet packets that had, on average, equal numbers of each of the relevant assessments in a randomized order.

Some of the students who were English language learners (ELL) or students with disabilities (SD) were excluded from the assessment because they could not be assessed with the accommodations NAEP provides.

Last updated 26 February 2016 (GF)

Printer-friendly Version