Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2007, Grade 12 Writing National Main Assessment, by Item and Block: 2007

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed–response items from previous assessment years, rescored in 2007, grade 12 writing national main assessment, by item and block: 2007

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Writing Assessment.
W4	W008402	1–6	800	54.50	—	0.73
W5	W011202	1–6	800	57.58	—	0.78
W6	W008602	1–6	800	52.38	—	0.74
W7	W008702	1–6	800	56.31	—	0.74
W9	W008902	1–6	800	52.05	—	0.69
W10	W011302	1–6	700	55.71	—	0.77
W11	W009102	1–6	700	58.69	—	0.74
W12	W009202	1–6	700	51.64	—	0.73
W13	W009302	1–6	700	51.23	—	0.75
W15	W009502	1–6	800	54.16	—	0.82
W17	W009702	1–6	800	56.06	—	0.81
W18	W009802	1–6	900	47.63	—	0.70
W19	W009902	1–6	800	49.38	—	0.73
W21	W011402	1–6	800	52.87	—	0.75
W22	W010202	1–6	800	55.43	—	0.78
W23	W010302	1–6	800	51.36	—	0.78
W24	W010402	1–6	800	54.80	—	0.81

Printer-friendly Version