NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 U.S. History National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 U.S. history national assessment, by item and block: 2010

Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 U.S. History Assessment.
H3	H035801	1-3	500	93	†	0.90
	H035901	1-3	600	95	†	0.92
	H035902	1-3	600	86	†	0.82
	H036101	1-3	500	88	†	0.86
	H036402	1-4	600	89	†	0.93
H4	H059001	1-3	600	90	†	0.93
	H059201	1-3	600	86	†	0.88
	H059701	1-3	500	89	†	0.91
	H059801	1-3	600	87	†	0.90
	H060201	1-3	500	89	†	0.89
H5	H038103	1-4	500	83	†	0.77
	H038301	1-3	500	93	†	0.93
	H038601	1-3	600	88	†	0.76
	H038702	1-3	500	87	†	0.80
	H039001	1-3	500	98	†	0.97
H6	H069001	1-3	500	85	†	0.85
	H069401	1-3	500	94	†	0.94
	H069601	1-3	600	94	†	0.92
	H069801	1-4	500	89	†	0.91
	H070001	1-2	500	96	0.92	†
H7	H082301	1-3	500	90	†	0.91
	H084301	1-4	500	88	†	0.90
	H084901	1-3	600	93	†	0.94
	H090201	1-4	500	89	†	0.93
H8	H082901	1-3	500	88	†	0.89
	H088401	1-4	600	87	†	0.92
	H091101	1-4	500	92	†	0.96
	H097801	1-4	600	83	†	0.90
H9	H082402	1-3	500	96	†	0.95
	H084801	1-3	500	92	†	0.88
	H087401	1-4	600	91	†	0.91
	H088301	1-3	500	93	†	0.88
	H097901	1-3	500	99	†	1.00
H10	H042201	1-3	500	95	†	0.86
	H042801	1-4	600	89	†	0.84
	H042902	1-3	500	96	†	0.97
	H043001	1-3	600	86	†	0.83
	H043101	1-3	500	94	†	0.93
H11	H070401	1-3	500	87	†	0.85
	H070701	1-4	500	100	†	0.99
	H070801	1-3	600	95	†	0.93
	H071001	1-4	600	90	†	0.87
	H071601	1-3	500	87	†	0.87
H12	H071901	1-3	500	88	†	0.90
	H072601	1-3	500	91	†	0.79
	H072701	1-4	500	90	†	0.93
	H073001	1-3	500	86	†	0.86
	H073101	1-3	500	84	†	0.86

Printer-friendly Version