NAEP tdw - Score Range and Percent Agreement, for the Constructed-Response Items Used in Scaling, Grade 8 Mathematics Combined National and State Assessment, by Item and Block: 2013

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Mathematics Combined National and State Assessment, by Item and Block: 2013

NAEP Technical DocumentationScore range and percent agreement for the constructed-response items used in scaling, grade 8 mathematics combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size ¹	Percent exact agreement
¹ Blocks M4, M5, M9, M10, M11, and M12 have larger sample sizes due to their pairing with integrated pilot blocks. NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.
M3	M149901	1-3	31,500	.98
M3	M150801	1-3	31,500	.98
M3	M150901	1-2	31,500	.99
M3	M151201	1-2	31,500	.99
M4	M151501	1-3	32,800	.94
M4	M151601	1-3	32,800	.95
M4	M152602	1-5	32,800	.98
M5	M221203	1-3	32,700	.99
M5	M221204	1-3	32,700	.88
M5	M221701	1-2	32,700	.97
M5	M222301	1-3	32,700	.99
M6	M141351	1-3	31,600	.94
M6	M141601	1-2	31,600	.92
M6	M141901	1-5	31,600	.91
M7	M165301	1-3	35,300	.94
M7	M167001	1-2	35,300	.95
M7	M168501	1-2	35,300	.99
M7	M168502	1-2	35,300	.99
M7	M168503	1-3	35,300	.96
M7	M168701	1-3	35,300	.97
M7	M170101	1-3	35,300	.99
M8	M237901	1-3	32,800	.99
M8	M239001	1-3	32,800	.99
M8	M317301	1-3	32,800	.98
M8	M317302	1-3	32,800	.94
M9	M234702	1-3	36,900	.99
M9	M235501	1-2	36,900	.99
M9	M237201	1-2	36,900	.99
M9	M237202	1-3	36,900	.99
M9	M237203	1-3	36,900	.98
M9	M237204	1-3	36,900	.98
M10	M222701	1-3	32,900	.98
M10	M223101	1-3	32,900	.98
M10	M223301	1-3	32,900	.90
M10	M223801	1-3	32,900	.99
M11	M104901	1-3	32,800	.98
M11	M162901	1-3	32,800	.95
M11	M163101	1-3	32,800	.96
M11	M167901	1-3	32,800	.97
M11	M168301	1-3	32,800	.95
M12	M231501	1-2	32,800	.98
M12	M233501	1-2	32,800	.99
M12	M234001	1-2	32,800	1.00
M12	M234201	1-3	32,800	.90

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 mathematics combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size ¹	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items. ¹ Blocks M4, M5, M9, M10, M11, and M12 have larger sample sizes due to their pairing with integrated pilot blocks. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.
M3	M149901	1-3	30,600	.97	.98
M3	M150801	1-3	29,200	.95	.98
M3	M150901	1-2	30,500	.99	†
M3	M151201	1-2	28,200	1.00	†
M4	M151501	1-3	31,800	.91	.95
M4	M151601	1-3	29,500	.86	.93
M4	M152602	1-5	29,500	.97	.98
M5	M221203	1-3	31,300	.97	.97
M5	M221204	1-3	31,200	.77	.82
M5	M221701	1-2	30,300	.99	†
M5	M222301	1-3	26,100	.99	.99
M6	M141351	1-3	29,500	.91	.97
M6	M141601	1-2	28,200	.83	†
M6	M141901	1-5	26,700	.87	.95
M7	M165301	1-3	32,900	.90	.95
M7	M167001	1-2	31,900	.75	†
M7	M168501	1-2	29,800	.98	†
M7	M168502	1-2	29,000	.98	†
M7	M168503	1-3	25,200	.82	.89
M7	M168701	1-3	34,300	.95	.98
M7	M170101	1-3	33,800	.98	.99
M8	M237901	1-3	31,900	.99	.00
M8	M239001	1-3	32,600	.98	.99
M8	M317301	1-3	29,400	.96	.99
M8	M317302	1-3	25,300	.77	.87
M9	M234702	1-3	35,700	.99	1.00
M9	M235501	1-2	36,000	.98	†
M9	M237201	1-2	32,500	.99	†
M9	M237202	1-3	31,000	.98	.99
M9	M237203	1-3	31,400	.95	.96
M9	M237204	1-3	30,100	.97	.99
M10	M222701	1-3	31,000	.98	.99
M10	M223101	1-3	32,200	.99	1.00
M10	M223301	1-3	30,700	.86	.90
M10	M223801	1-3	28,200	.99	.99
M11	M104901	1-3	28,000	.96	.98
M11	M162901	1-3	31,400	.93	.95
M11	M163101	1-3	31,100	.94	.97
M11	M167901	1-3	29,800	.92	.96
M11	M168301	1-3	29,400	.94	.97
M12	M231501	1-2	31,900	.97	†
M12	M233501	1-2	32,400	.99	†
M12	M234001	1-2	32,200	.97	†
M12	M234201	1-3	26,500	.78	.88

Last updated 13 June 2017 (GF)

Printer-friendly Version