NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2010, Grade 8 Geography National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2010, grade 8 geography national assessment, by item and block: 2010

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 Geography Assessment.
G3	G013402	1-3	700	85	†	0.77
	G014001	1-3	700	95	†	0.95
	G014201	1-4	800	83	†	0.89
	G014301	1-3	700	79	†	0.77
	G014401	1-3	700	94	†	0.91
G4	G032901	1-3	800	90	†	0.84
	G033301	1-3	800	81	†	0.83
	G033501	1-4	800	69	†	0.76
	G033801	1-3	700	91	†	0.90
	G034101	1-3	800	75	†	0.63
G5	G034801	1-4	700	94	†	0.96
	G035001	1-4	700	77	†	0.78
	G035301	1-4	700	81	†	0.91
	G035501	1-4	700	80	†	0.77
G9	G019002	1-3	800	99	†	0.97
	G019003	1-3	800	89	†	0.86
	G019102	1-3	700	93	†	0.87
	G019202	1-3	700	87	†	0.83
	G019302	1-3	800	95	†	0.94
	G019402	1-3	700	90	†	0.91
	G019901	1-4	800	93	†	0.92
	G020001	1-3	700	96	†	0.84
	G020201	1-3	800	88	†	0.82
	G020302	1-3	700	90	†	0.89

Printer-friendly Version