NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Geography National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 geography national assessment, by item and block: 2010

Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 Geography Assessment.
G3	G013402	1-3	600	88	†	0.81
	G014001	1-3	600	96	†	0.94
	G014201	1-4	600	84	†	0.90
	G014301	1-3	600	86	†	0.86
	G014401	1-3	600	94	†	0.89
G4	G032901	1-3	600	89	†	0.85
	G033301	1-3	600	84	†	0.88
	G033501	1-4	600	70	†	0.78
	G033801	1-3	600	94	†	0.91
	G034101	1-3	500	80	†	0.67
G5	G046701	1-3	600	92	†	0.92
	G048702	1-2	600	95	0.90	†
	G050001	1-4	500	98	†	0.99
	G050301	1-4	500	87	†	0.87
	G050801	1-3	500	88	†	0.90
G6	G034801	1-4	600	98	†	0.99
	G035001	1-4	500	84	†	0.86
	G035301	1-4	500	86	†	0.93
	G035501	1-4	500	84	†	0.82
G7	G051601	1-4	600	100	†	1.00
	G052501	1-3	600	98	†	0.96
	G052901	1-3	600	98	†	0.98
	G053601	1-3	400	89	†	0.84
G8	G046301	1-4	600	96	†	0.97
	G048101	1-3	600	90	†	0.89
	G049001	1-3	600	98	†	0.98
	G049401	1-3	600	94	†	0.96
	G049701	1-3	500	87	†	0.86
G9	G019002	1-3	500	99	†	0.99
	G019003	1-3	600	95	†	0.95
	G019102	1-3	500	96	†	0.92
	G019202	1-3	500	94	†	0.93
	G019302	1-3	600	97	†	0.97
	G019402	1-3	600	93	†	0.93
	G019901	1-4	600	97	†	0.95
	G020001	1-3	500	98	†	0.93
	G020201	1-3	600	95	†	0.92
	G020302	1-3	500	95	†	0.96

Printer-friendly Version