NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2009, Grade 8 U.S. History National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2010, grade 8 U.S. history national assessment, by item and block: 2010

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 U.S. History Assessment.
H3	H035801	1-3	800	86	†	0.74
	H035901	1-3	700	89	†	0.88
	H035902	1-3	700	84	†	0.80
	H036101	1-3	700	85	†	0.85
	H036402	1-4	700	73	†	0.81
H4	H059001	1-3	700	85	†	0.88
	H059201	1-3	700	76	†	0.77
	H059701	1-3	700	86	†	0.87
	H059801	1-3	700	83	†	0.86
	H060201	1-3	700	81	†	0.81
H5	H038103	1-4	700	81	†	0.68
	H038301	1-3	700	87	†	0.89
	H038601	1-3	700	79	†	0.61
	H038702	1-3	700	81	†	0.71
	H039001	1-3	700	96	†	0.95
H6	H069001	1-3	700	69	†	0.72
	H069401	1-3	700	84	†	0.84
	H069601	1-3	700	90	†	0.86
	H069801	1-4	700	77	†	0.84
	H070001	1-2	700	93	0.84	†
H10	H042201	1-3	700	94	†	0.83
	H042801	1-4	700	89	†	0.88
	H042902	1-3	700	89	†	0.90
	H043001	1-3	800	78	†	0.69
	H043101	1-3	700	88	†	0.85
H11	H070401	1-3	700	84	†	0.85
	H070701	1-4	700	95	†	0.96
	H070801	1-3	700	87	†	0.82
	H071001	1-4	700	82	†	0.79
	H071601	1-3	700	79	†	0.77
H12	H071901	1-3	700	82	†	0.86
	H072601	1-3	700	82	†	0.62
	H072701	1-4	700	91	†	0.94
	H073001	1-3	700	83	†	0.84
	H073101	1-3	700	77	†	0.78

Printer-friendly Version