NAEP Technical Documentation Score Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Mathematics Combined National and State Assessment, by Block and Item: 2017

Score range and percentage of exact agreement for the constructed-response items used in scaling, grade 8 mathematics combined national and state assessment, by block and item: 2017

Block	Item	Score range	Sample size	Percentage of exact agreement
M1	M381603	1-2	1,300	98
	M381604	1-2	1,200	98
	M381605	1-2	1,200	99
	M381901	1-2	1,400	100
	M384402	1-3	1,400	100
	M385601	1-2	1,400	97
M2	M379901	1-3	1,200	99
	M382902	1-3	1,300	99
	M384001	1-2	1,400	100
	M384002	1-2	1,300	100
	M384003	1-2	1,400	99
	M386102	1-3	1,300	94
M3	M367501	1-2	1,400	100
	M381802	1-3	1,200	100
	M383401	1-2	1,500	100
	M384202	1-3	1,500	98
	M385901	1-3	1,400	100
	M385902	1-2	1,400	99
M4	M222301	1-3	1,400	99
	M232101	1-2	1,400	100
	M351901	1-2	1,500	100
	M351902	1-2	1,400	99
	M352301	1-3	1,400	99
M5	M223101	1-3	1,400	99
	M223301	1-3	1,400	91
	M223801	1-3	1,300	99
M6	M104901	1-3	1,300	97
	M163105	1-3	1,400	100
	M167901	1-3	1,300	96
	M233601	1-3	1,300	98
M7	M221701	1-2	1,400	99
	M231501	1-2	1,400	99
	M234001	1-2	1,400	99
M8	M386701	1-2	1,600	99
	M387303	1-2	1,400	98
	M387304	1-2	1,300	100
	M387305	1-2	1,200	99
	M388201	1-3	1,600	100
	M388300	1-3	1,600	100
M9	M355301	1-3	1,600	97
	M356401	1-2	1,600	100
	M356601	1-2	1,600	100
	M356602	1-2	1,600	99
	M356603	1-2	1,600	99
	M356901	1-2	1,500	100
M10	M234702	1-3	1,500	99
	M235501	1-2	1,500	100
	M237901	1-3	1,400	99
	M239001	1-3	1,400	99
NOTE: Special codes assigned to student responses including omitted, off-task, and illegible were included in the calculation of the percentage of exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2017 Mathematics Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items used in scaling, grade 8 mathematics combined national and state assessment, by block and item: 2017

Block	Item	Score range	Sample size	Cohen’s Kappa	Intraclass correlation
M1	M381603	1-2	1,300	.95	†
	M381604	1-2	1,200	.95	†
	M381605	1-2	1,100	.95	†
	M381901	1-2	1,400	.99	†
	M384402	1-3	1,400	1.00	1.00
	M385601	1-2	1,400	.93	†
M2	M379901	1-3	1,100	.99	.99
	M382902	1-3	1,300	.98	.99
	M384001	1-2	1,400	.99	†
	M384002	1-2	1,300	.99	†
	M384003	1-2	1,400	.99	†
	M386102	1-3	1,300	.83	.89
M3	M367501	1-2	1,400	.99	†
	M381802	1-3	1,200	1.00	1.00
	M383401	1-2	1,500	.99	†
	M384202	1-3	1,500	.93	.93
	M385901	1-3	1,400	1.00	1.00
	M385902	1-2	1,300	.96	†
M4	M222301	1-3	1,300	.99	.99
	M232101	1-2	1,400	.99	†
	M351901	1-2	1,500	.99	†
	M351902	1-2	1,400	.99	†
	M352301	1-3	1,400	.99	.99
M5	M223101	1-3	1,400	.98	.99
	M223301	1-3	1,400	.83	.86
	M223801	1-3	1,300	.99	1.00
M6	M104901	1-3	1,300	.95	.96
	M163105	1-3	1,400	1.00	1.00
	M167901	1-3	1,300	.84	.91
	M233601	1-3	1,300	.96	.98
M7	M221701	1-2	1,400	.99	†
	M231501	1-2	1,400	.98	†
	M234001	1-2	1,400	.97	†
M8	M386701	1-2	1,600	.99	†
	M387303	1-2	1,300	.95	†
	M387304	1-2	1,200	1.00	†
	M387305	1-2	1,200	.97	†
	M388201	1-3	1,500	.99	1.00
	M388300	1-3	1,600	.99	.99
M9	M355301	1-3	1,600	.95	.98
	M356401	1-2	1,600	.99	†
	M356601	1-2	1,600	.99	†
	M356602	1-2	1,600	.99	†
	M356603	1-2	1,600	.97	†
	M356901	1-2	1,500	.99	†
M10	M234702	1-3	1,500	.98	.99
	M235501	1-2	1,500	1.00	†
	M237901	1-3	1,400	.99	1.00
	M239001	1-3	1,400	.99	1.00
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including omitted, off-task, and illegible were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2017 Mathematics Assessment.

Printer-friendly Version