NAEP Technical DocumentationWriting Scoring Specifications

The use of anchor and practice sets for the scoring of writing items differs somewhat from the scoring of all other subject areas.

Anchor Papers: Two anchor sets consisting of solid examples of each of the six levels of the scoring guide are used for training writing. To familiarize scorers with the range of responses that merit the same score (e.g., a high 6 and low 6), the first set features two responses for each score point. The second anchor set consists of six responses, one for each score point. The responses in this second set are not marked with scores, so that scorers can be asked to read, score, and arrange them in score point order. Once both sets have been presented, they are then combined by the scorers into a single set for reference while scoring.
Practice Sets: Two practice sets consisting of a mix of solid and borderline papers are used, each containing 5-6 responses. One set represents upper-level (score points 4-6) responses and the other set represents lower-level (score points 1-3) responses. This arrangement allows scorers to practice scoring without the pressure of using the whole range of score points. Scorers are told only that the responses are either upper- or lower-level. If in consultation with the trainer the coordinator deems it necessary, a third practice set may be prepared, containing 3-6 responses focusing on points of difficulty (e.g. 3s vs. 4s, 5s vs. 6s). After the scorers independently read and score each practice response, each scorer reads a response aloud and attempts to justify his or her score. During discussion, the accurate score is explained, and the trainer helps scorers justify scores using the language of the scoring guide and by comparison with the anchor papers.
Qualifying Sets: For writing, the qualifying sets are the same as for other subject areas.

Writing training also utilizes “consensus building” sets, which consist of a range of borderline and solid responses, mixed across all six levels of the scoring guide. Two or three sets (again, the number will be determined by consultation between the coordinator and trainer) of 10 responses each are identified as consensus building sets. After the scorers read each response and assign scores independently, each scorer reads a response aloud and attempts to justify his or her score. During discussion, the accurate score is explained, and the trainer encourages scorers who agree with the correct score to justify it using the language of the scoring guide and by comparison with the anchor papers.

Last updated 22 September 2008 (JL)

Printer-friendly Version