NAEP Technical DocumentationStudent Selection for the 2012 Public School Long-Term Trend (LTT) Assessment

Within each sampled school, a sample of students was selected from a listing of the age-eligible students such that every student had an equal chance of selection. The student lists were submitted in multiple ways. E-Filing is an electronic submission system. Excel files are submitted for sampled schools by school coordinators or NAEP State Coordinators. Files can be submitted for one school at a time or for an entire jurisdiction at once. This method allows schools to easily submit student demographic data electronically with the student lists, easing burden on NAEP field supervisors and school coordinators. Schools that are unable to submit their student lists using the E-Filing system provide hardcopy lists via the student listing form to NAEP field supervisors.

In year-round, multi-track schools, students who were not scheduled to be in school on the assessment day were removed from the student lists prior to sampling. Student base weights were adjusted to account for these students.

The sampling process was the same regardless of list submission type. The sampling process was systematic (e.g., if the sampling rate was 1/2, a random starting point of one or two was chosen, and every other student on the list was selected). For E-Filed schools only, where demographic data are submitted for every student on the frame, students are sorted by sex and race before the sample is selected to implicitly stratify the sample.

Some students enrolled in the school after the sample was selected. In such cases, new enrollees were sampled at the same rate as the students on the original list.

Students were randomly assigned to a reading or mathematics operational assessment, or to a reading or mathematics operational assessment with a reading or mathematics integrated pilot assessment. The proportions assigned to these four types of assessments were approximately 3/8, 3/8, 1/8, and 1/8, respectively. The booklet assignment was implemented by spiraling: the booklets assigned to sampled students were provided from booklet packets which had, on average, the correct proportion of each of the relevant assessments in a randomized order.

Some of the students who were English language learners (ELL) or students with disabilities (SD) were excluded from the assessment because they could not be assessed with the accomodations NAEP provides. Excluded students were removed from the sample.

The student sample consisted of all age-eligible students in the school if the school had 64 or fewer students, or a sample of 64 students selected without replacement, if the school had 65 or more students.

In 2012 in addition to the regular assessments described above, there was an experimental SD/ELL decision tree that was piloted in a subsample of schools selected for the ages 9 and 13 LTT samples (see the school sampling page). To increase the number of students utilizing the decision tree, extra SD and ELL students in the selected schools were sampled where possible. In these schools only, and only if the school e-filed, a special sampling procedure was implemented which required sorting the student list into two groups (SD/ELL and non-SD/ELL) and sampling from each group based on a specific rate. The result of the oversampling was an increase of about 5 SD/ELL students in the student sample for each school in the decision tree subsample of schools.

Last updated 23 August 2012 (DB)

Printer-friendly Version