Data Collection

The 2018 NAEP Oral Reading Fluency assessment was administered using the new eNAEP platform on tablets to small groups of students at the same time in a single room, rather than with one-on-one administration. This enabled the assessment to be self-administered by the students in a group setting, with students wearing headsets with microphones. The group administration increased the efficiency of administration, yielded high-quality recordings, and provided a more relaxed testing atmosphere with fewer possible inconsistencies due to administrator interactions during the assessment.

The flowchart to the right shows the 2018 NAEP ORF study assessment administration procedure. The assessment started with a NAEP grade 4 reading assessment tutorial, followed by a reading assessment consisting of 8 out of 10 blocks of assessment items from the 2017 grade 4 NAEP reading assessment (two blocks were released). As is typical in the NAEP reading assessment, ORF students received two blocks of reading items. Then they responded to the ORF study task module with four ORF feedback questions. Lastly, they answered the NAEP fourth-grade student questionnaire for the reading assessment.

The ORF study task module consisted of four types of tasks in the following order: (1) repeating four sentences aloud to assess students’ speaking skills, (2) reading a word list aloud, (3) reading a pseudoword list aloud, and (4) reading four passages aloud with a single “yes/no” passage meaning question for each passage to encourage students to read for meaning. Before students were administered each task type, they were shown a tutorial that provided instructions with task-specific examples of a student demonstrating each upcoming task. After completing the four tasks, students answered the following four ORF feedback questions: (1) In this school year, how often have you read out loud in school or at home, or anywhere? (2) Who did you read to? (3) Where were you? and (4) How difficult was this reading out loud test?

Student recordings were transferred from the tablet to a NAEP database to be processed using a speech analysis system to produce a profile of scores to be analyzed for the study. For more information about the speech analysis system, see Scoring.