NAEP Technical DocumentationTraining the Scoring Trainers

The goal of train-the-trainer sessions is to provide trainers with orientation to NAEP scoring activities and to convey all training requirements and project specifications. This includes an overview of the program, specifics regarding the NAEP subject area frameworks, and details related to trend scoring requirements. Trend scoring is used to compare the consistency of scoring over time (i.e., cross-year interrater agreement).

Trainers receive instruction on how to use the trend CD, which contains responses from items from a prior assessment to inform the training of items. This is also an important opportunity to demonstrate to the trainers the features of the scoring system that will be necessary for them to monitor progress on their items. In addition, train-the-trainer sessions provide a forum for review of the items assigned to individual trainers and an opportunity to review best practices for training, with mock training activities incorporated into the sessions. Train-the-trainer is the trainer’s introduction to NAEP and highlights the trainer’s critical role in the delivery of high-quality scoring for NAEP. Shortly before scoring will commence and concurrent with supervisor training, trainers are presented with a demonstration of the electronic scoring system and the latest updates.

For trend subjects, the scoring staff conducts train-the-trainer sessions that include a thorough discussion of trend scoring, with explanation of the rationale for trend scoring, an explanation of the rules governing trend training, and the monitoring of cross-year consistency with trend responses.

Test development staff members have primary training responsibilities for pilot and field test items, as well as first-year operational items. Those who are highly experienced with NAEP scoring conduct subject specific scoring planning meetings in which they review guidelines for choosing appropriate training sets and training practices.

Last updated 11 February 2009 (RF)

Printer-friendly Version