NAEP Technical DocumentationWriting Achievement Levels

The writing achievement levels that were set by the National Assessment Governing Board were used in reporting results for the 1998 assessment and subsequent assessments. They will be used until the Board determines that a new framework is needed to guide the development of the assessment.

The 1998 level setting process was different in some significant ways from earlier level setting processes that had been used in other NAEP subject areas. First, in the past panelists were given the preliminary descriptions of the levels developed initially by the framework consensus groups and asked to craft recommended descriptions during the process. The descriptions continued to be refined throughout the level-setting process, and usually were validated by a supplementary group of judges subsequent to the level-setting meetings.

In 1998, the finalized achievement level descriptions (ALDs) were predetermined, much like the framework and the item pool are predetermined. Prior to the level-setting meetings, focus groups in each NAEP region were conducted to evaluate the preliminary ALDs for writing. The focus group recommendations were reviewed by 88 expert content panelists, and appropriate modifications were made. The revised ALDs were then reviewed and evaluated by the framework consensus panels and all focus group participants. These proposed final ALDs were then reviewed and modified by the Board's Achievement Levels Committee and approved for use—without change—in the remainder of the process.

Two reports provide details on the writing achievement levels:

The Board's complete report on achievement levels for writing includes descriptions of achievement levels and cut scores; sample items illustrating achievement levels, and performance data.
Appendix J of the NAEP 1998 Technical Report (Allen and Donoghue 1996) provides detailed information related to the process for developing the achievement levels, selecting exemplar items, evaluating and validating the levels, mapping the levels onto the NAEP scale, as well as panel meeting participants and affiliations.

In 2011, a new framework was introduced by the Board to reflect changes in the field of writing. Along with the introduction of a new framework, new achievement level descriptions and cut scores were established. The assessment resulting from the 2011 framework started a new NAEP writing trend line so results from 2011 cannot be compared with results of previous writing assessments. Whenever changes are made to a framework, efforts are made to maintain the trend lines that permit the reporting of changes in student achievement over time. If, however, the nature of the changes made to an assessment is such that the results would not be comparable to earlier assessments, a new trend line is started. See description of the 2011 writing framework.

A complete description of the writing achievement levels for grades 8 and 12 is available at http://nces.ed.gov/nationsreportcard/writing/achieve.asp

Last updated 02 July 2014 (GF)

Printer-friendly Version