NAEP Technical Documentation

Score range and percentage of exact agreement for the constructed-response items from the previous year 2010, rescored in 2014, grade 8 geography national assessment, by item and block: 2014

Block	Item	Range of response codes	Sample size	Percentage of exact agreement
GC	G013402	1-3	800	86
GC	G014001	1-3	800	96
GC	G014201	1-4	900	85
GC	G014301	1-3	800	82
GC	G014401	1-3	800	91
GD	G032901	1-3	800	88
GD	G033301	1-3	800	82
GD	G033501	1-4	800	68
GD	G033801	1-3	800	91
GD	G034101	1-3	800	82
GE	G050001	1-4	800	97
GE	G046701	1-3	800	81
GE	G048702	1-2	800	90
GE	G050801	1-3	800	81
GE	G050301	1-4	800	79
GF	G034801	1-4	800	95
GF	G035001	1-4	800	78
GF	G035301	1-4	800	83
GF	G035501	1-4	800	80
GG	G051601	1-4	800	99
GG	G052901	1-3	800	96
GG	G052501	1-3	800	97
GG	G053601	1-3	800	88
GH	G049001	1-3	800	93
GH	G049401	1-3	800	91
GH	G048101	1-3	800	86
GH	G046301	1-4	800	93
GH	G049701	1-3	800	81
GI	G019002	1-3	800	98
GI	G019003	1-3	800	89
GI	G019102	1-3	800	92
GI	G019202	1-3	800	88
GI	G019302	1-3	800	95
GI	G019402	1-3	800	88
GI	G019901	1-4	800	91
GI	G020001	1-3	800	97
GI	G020201	1-3	800	89
GI	G020302	1-3	800	92
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percentage of Exact Agreement measure. In 2014, grades 4 and 12 were not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2014 Geography Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from the previous year 2010, rescored in 2014, grade 8 geography national assessment, by item and block: 2014

Block	Item	Range of response codes	Sample size	Cohen’s Kappa	Intraclass correlation
GC	G013402	1-3	800	.70	.76
GC	G014001	1-3	800	.91	.95
GC	G014201	1-4	900	.76	.91
GC	G014301	1-3	800	.64	.77
GC	G014401	1-3	800	.81	.83
GD	G032901	1-3	800	.76	.82
GD	G033301	1-3	800	.68	.85
GD	G033501	1-4	800	.51	.74
GD	G033801	1-3	800	.83	.89
GD	G034101	1-3	800	.59	.66
GE	G050001	1-4	800	.95	.98
GE	G046701	1-3	800	.67	.78
GE	G048702	1-2	800	.78	†
GE	G050801	1-3	800	.68	.84
GE	G050301	1-4	800	.64	.79
GF	G034801	1-4	800	.93	.97
GF	G035001	1-4	800	.61	.79
GF	G035301	1-4	800	.74	.90
GF	G035501	1-4	800	.55	.77
GG	G051601	1-4	800	.98	1.00
GG	G052901	1-3	800	.94	.96
GG	G052501	1-3	800	.91	.95
GG	G053601	1-3	800	.69	.82
GH	G049001	1-3	800	.87	.91
GH	G049401	1-3	800	.85	.92
GH	G048101	1-3	800	.73	.83
GH	G046301	1-4	800	.89	.95
GH	G049701	1-3	800	.63	.77
GI	G019002	1-3	800	.97	.97
GI	G019003	1-3	800	.81	.86
GI	G019102	1-3	800	.82	.90
GI	G019202	1-3	800	.78	.85
GI	G019302	1-3	800	.92	.94
GI	G019402	1-3	800	.82	.88
GI	G019901	1-4	800	.85	.89
GI	G020001	1-3	800	.87	.89
GI	G020201	1-3	800	.80	.85
GI	G020302	1-3	800	.84	.91
† The Intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. In 2014, grades 4 and 12 were not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2014 Geography Assessment.

Printer-friendly Version