Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2013, Rescored in 2015, Grade 12 Mathematics National Assessment, by Item and Block: 2015

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2013, Rescored in 2015, Grade 12 Mathematics National Assessment, by Item and Block: 2015

Score range and percentage of exact agreement for the constructed-response items from previous year 2013, rescored in 2015, grade 12 mathematics national assessment, by item and block: 2015
Block	Item	Score range	Sample size	Percentage of exact agreement
M1	M175901	1-3	800	97
	M178001	1-3	900	96
	M183601	1-3	900	98
	M183901	1-2	1,100	98
	M186701	1-3	900	94
	M186702	1-3	900	88
M2	M302901	1-3	800	97
	M305901	1-2	800	99
	M305902	1-2	800	99
	M305903	1-2	800	100
	M306701	1-3	900	91
	M307501	1-3	800	90
M3	M179201	1-3	800	97
	M197303	1-2	1,000	99
	M197304	1-2	800	99
	M197501	1-3	800	90
	M197502	1-3	800	97
	M197503	1-3	800	95
M4	M174801	1-3	800	96
	M184201	1-2	800	99
	M184202	1-2	800	99
	M184203	1-2	800	99
	M184204	1-2	800	97
	M184205	1-2	800	99
	M184206	1-2	800	98
	M184207	1-3	800	87
	M197601	1-3	800	96
M5	M185201	1-2	800	98
	M185703	1-5	800	96
	M187701	1-3	800	98
M6	M175002	1-5	800	97
	M177202	1-2	900	95
	M183401	1-3	800	94
M7	M304001	1-3	800	96
	M310001	1-4	800	93
	M310401	1-2	800	99
M8	M188001	1-3	1,100	95
	M188201	1-5	800	97
	M189001	1-3	900	92
	M193101	1-2	1,600	100
	M194301	1-2	800	99
	M194302	1-2	800	100
M9	M191901	1-5	800	93
	M198701	1-2	800	100
	M198901	1-3	800	95
M10	M178501	1-3	900	98
	M189701	1-3	900	98
	M189702	1-2	900	97
	M189703	1-2	900	99
	M190801	1-3	800	97
	M191701	1-3	1,100	99
	M199201	1-3	800	98
M11	M176501	1-3	800	97
	M176502	1-2	800	95
	M176503	1-2	800	96
	M176504	1-2	800	96
	M181201	1-2	800	98
	M181202	1-3	800	98
	M181203	1-2	800	100
	M183101	1-2	800	99
M12	M178701	1-4	800	97
	M181801	1-2	800	99
	M186001	1-2	1,100	98
	M194001	1-3	1,000	98
M13	M302101	1-4	800	98
	M302701	1-3	800	99
	M305501	1-2	800	99
	M306501	1-2	800	99
M14	M308001	1-2	900	99
	M310501	1-3	1,000	99
	M311101	1-3	800	99
	M311102	1-3	800	95
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percentage of exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from previous year 2013, rescored in 2015, grade 12 mathematics national assessment, by item and block: 2015
Block	Item	Score range	Sample size	Cohen’s Kappa	Intraclass correlation
M1	M175901	1-3	700	.85	.88
	M178001	1-3	800	.93	.97
	M183601	1-3	900	.98	.98
	M183901	1-2	900	.98	†
	M186701	1-3	800	.90	.94
	M186702	1-3	800	.77	.86
M2	M302901	1-3	700	.96	.98
	M305901	1-2	800	.99	†
	M305902	1-2	800	.99	†
	M305903	1-2	800	1.00	†
	M306701	1-3	800	.85	.91
	M307501	1-3	700	.78	.86
M3	M179201	1-3	700	.89	.94
	M197303	1-2	900	.98	†
	M197304	1-2	700	.99	†
	M197501	1-3	800	.81	.84
	M197502	1-3	700	.90	.91
	M197503	1-3	700	.79	.90
M4	M174801	1-3	700	.86	.89
	M184201	1-2	800	.99	†
	M184202	1-2	800	.95	†
	M184203	1-2	800	.97	†
	M184204	1-2	800	.90	†
	M184205	1-2	800	.99	†
	M184206	1-2	800	.95	†
	M184207	1-3	700	.64	.80
	M197601	1-3	700	.93	.96
M5	M185201	1-2	700	.98	†
	M185703	1-5	700	.94	.98
	M187701	1-3	700	.98	.99
M6	M175002	1-5	800	.97	.99
	M177202	1-2	800	.88	†
	M183401	1-3	700	.85	.93
M7	M304001	1-3	700	.93	.95
	M310001	1-4	700	.91	.97
	M310401	1-2	800	.98	†
M8	M188001	1-3	1,000	.91	.97
	M188201	1-5	800	.95	.99
	M189001	1-3	800	.85	.86
	M193101	1-2	1,400	1.00	†
	M194301	1-2	800	.99	†
	M194302	1-2	700	1.00	†
M9	M191901	1-5	800	.89	.98
	M198701	1-2	700	1.00	†
	M198901	1-3	800	.91	.95
M10	M178501	1-3	800	.98	.99
	M189701	1-3	800	.96	.98
	M189702	1-2	700	.94	†
	M189703	1-2	700	.98	†
	M190801	1-3	800	.96	.97
	M191701	1-3	1,000	.99	1.00
	M199201	1-3	700	.95	.97
M11	M176501	1-3	800	.95	.98
	M176502	1-2	800	.88	†
	M176503	1-2	800	.90	†
	M176504	1-2	800	.91	†
	M181201	1-2	700	.98	†
	M181202	1-3	700	.97	.98
	M181203	1-2	700	.99	†
	M183101	1-2	700	.99	†
M12	M178701	1-4	700	.96	.97
	M181801	1-2	700	.99	†
	M186001	1-2	900	.99	†
	M194001	1-3	900	.96	.98
M13	M302101	1-4	800	.94	.97
	M302701	1-3	800	.99	1.00
	M305501	1-2	700	.99	†
	M306501	1-2	700	.99	†
M14	M308001	1-2	800	.99	†
	M310501	1-3	900	.99	.99
	M311101	1-3	700	.98	.99
	M311102	1-3	700	.77	.85
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Last updated 09 November 2021 (ML)

Printer-friendly Version