NAEP tdw - Score Range and Percent Agreement for the Constructed-Response Items From Previous Year 2011, Rescored in 2013, Grade 12 Mathematics Combined National and State Assessment, by Item and Block: 2013

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items From Previous Year 2011, Rescored in 2013, Grade 12 Mathematics Combined National and State Assessment, by Item and Block: 2013

Score range and percent agreement for the constructed-response items from the previous year 2011, rescored in 2013, grade 12 mathematics combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size	Percent agreement
M1	M175901	1-3	900	96
M1	M178001	1-3	1,700	95
M1	M183601	1-3	900	98
M1	M183901	1-2	900	98
M1	M186701	1-3	900	92
M1	M186702	1-3	900	86
M3	M179201	1-3	900	96
M3	M193401	1-3	900	91
M3	M197303	1-2	1,000	99
M3	M197304	1-2	900	99
M3	M197501	1-3	900	93
M3	M197502	1-3	900	95
M3	M197503	1-3	900	94
M4	M174801	1-3	900	96
M4	M184201	1-2	900	100
M4	M184202	1-2	900	99
M4	M184203	1-2	900	98
M4	M184204	1-2	900	97
M4	M184205	1-2	900	99
M4	M184206	1-2	900	98
M4	M184207	1-3	900	87
M4	M197601	1-3	900	93
M5	M177301	1-3	900	89
M5	M185201	1-2	900	98
M5	M185703	1-5	900	96
M5	M187701	1-3	900	97
M6	M175002	1-5	900	98
M6	M175601	1-3	900	96
M6	M177202	1-2	900	93
M6	M183401	1-3	900	96
M8	M188001	1-3	900	95
M8	M188201	1-5	900	97
M8	M189001	1-3	800	90
M8	M193101	1-2	800	99
M8	M194301	1-2	900	99
M8	M194302	1-2	900	99
M9	M191901	1-5	900	92
M9	M198701	1-2	1,000	100
M9	M198901	1-3	900	95
M10	M178501	1-3	900	98
M10	M189701	1-3	900	97
M10	M189702	1-2	900	98
M10	M189703	1-2	900	98
M10	M190801	1-3	900	98
M10	M191701	1-3	800	98
M10	M199201	1-3	800	98
M11	M176501	1-3	900	97
M11	M176502	1-2	900	94
M11	M176503	1-2	900	97
M11	M176504	1-2	900	96
M11	M181201	1-2	1,000	98
M11	M181202	1-3	900	98
M11	M181203	1-2	900	99
M11	M183101	1-2	900	99
M12	M178701	1-4	900	97
M12	M181801	1-2	900	99
M12	M186001	1-2	1,000	99
M12	M194001	1-3	900	96
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items from the previous year 2011, rescored in 2013, grade 12 mathematics combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size	Cohen’s Kappa	Intraclass correlation
M1	M175901	1-3	800	.84	.86
M1	M178001	1-3	1,600	.88	.94
M1	M183601	1-3	800	.97	.98
M1	M183901	1-2	800	.99	†
M1	M186701	1-3	800	.87	.92
M1	M186702	1-3	800	.75	.85
M3	M179201	1-3	800	.85	.92
M3	M193401	1-3	800	.75	.86
M3	M197303	1-2	900	.97	†
M3	M197304	1-2	800	1.00	†
M3	M197501	1-3	800	.87	.89
M3	M197502	1-3	800	.85	.90
M3	M197503	1-3	600	.76	.86
M4	M174801	1-3	800	.74	.76
M4	M184201	1-2	800	1.00	†
M4	M184202	1-2	800	.93	†
M4	M184203	1-2	800	.96	†
M4	M184204	1-2	800	.95	†
M4	M184205	1-2	800	.99	†
M4	M184206	1-2	800	.97	†
M4	M184207	1-3	800	.68	0.81
M4	M197601	1-3	800	.87	0.92
M5	M177301	1-3	800	.81	.89
M5	M185201	1-2	800	.98	†
M5	M185703	1-5	800	.94	.96
M5	M187701	1-3	800	.98	.99
M6	M175002	1-5	800	.97	.99
M6	M175601	1-3	800	.92	.94
M6	M177202	1-2	800	.87	†
M6	M183401	1-3	800	.87	.92
M8	M188001	1-3	800	.92	.97
M8	M188201	1-5	800	.96	.99
M8	M189001	1-3	800	.81	.84
M8	M193101	1-2	800	.99	†
M8	M194301	1-2	800	1.00	†
M8	M194302	1-2	700	.99	†
M9	M191901	1-5	800	.87	.97
M9	M198701	1-2	1,000	1.00	†
M9	M198901	1-3	800	.94	.96
M10	M178501	1-3	800	.96	.99
M10	M189701	1-3	800	.95	.98
M10	M189702	1-2	700	.96	†
M10	M189703	1-2	600	.96	†
M10	M190801	1-3	800	.98	.98
M10	M191701	1-3	800	.99	.99
M10	M199201	1-3	800	.97	.98
M11	M176501	1-3	800	.95	.98
M11	M176502	1-2	800	.88	†
M11	M176503	1-2	800	.92	†
M11	M176504	1-2	800	.91	†
M11	M181201	1-2	900	.97	†
M11	M181202	1-3	800	.97	.98
M11	M181203	1-2	800	.99	†
M11	M183101	1-2	800	.99	†
M12	M178701	1-4	800	.92	.95
M12	M181801	1-2	800	1.00	†
M12	M186001	1-2	900	1.00	†
M12	M194001	1-3	800	.96	.97
† The Intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen"s Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.

Last updated 13 June 2017 (GF)

Printer-friendly Version