NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 U.S. History National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 U.S. history national assessment, by item and block: 2010

Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 U.S. History Assessment.
H3	H065701	1-4	600	90	†	0.84
	H065801	1-4	600	98	†	0.99
	H066001	1-3	500	84	†	0.81
	H066501	1-4	500	97	†	0.98
	H066701	1-3	400	94	†	0.92
H4	H054301	1-3	600	85	†	0.84
	H054601	1-3	600	91	†	0.87
	H054801	1-3	600	97	†	0.97
	H054901	1-3	500	91	†	0.92
	H055301	1-3	500	93	†	0.93
H5	H055901	1-3	500	95	†	0.96
	H056101	1-3	500	93	†	0.93
	H056401	1-3	500	97	†	0.97
	H056601	1-3	600	97	†	0.96
	H056801	1-3	600	98	†	0.96
H6	H067401	1-3	500	95	†	0.94
	H067701	1-3	500	96	†	0.96
	H068001	1-4	600	91	†	0.89
	H068201	1-3	500	91	†	0.90
	H068301	1-3	500	88	†	0.87
	H068501	1-3	400	98	†	0.98
H7	H077402	1-3	500	92	†	0.92
	H078903	1-3	600	89	†	0.90
	H079301	1-4	400	93	†	0.94
H8	H078101	1-3	400	96	†	0.96
	H078201	1-3	400	91	†	0.89
	H078401	1-4	500	91	†	0.94
	H080401	1-4	500	97	†	0.99
	H080601	1-3	500	100	†	1.00
	H081001	1-3	500	92	†	0.92

Printer-friendly Version