NAEP tdw - Score Range and Percent Agreement, for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Assessment, by Item and Block: 2013

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Assessment, by Item and Block: 2013

NAEP Technical DocumentationScore range and percent agreement for the constructed-response items used in scaling, grade 4 mathematics combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size	Percent exact agreement
NOTE: Blocks M4, M5, M8, M9, M10, and M11 have larger sample sizes due to their pairing with integrated pilot blocks. Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.
M3	M145901	1-3	34,800	.97
M3	M146101	1-2	34,800	.97
M3	M146301	1-3	34,800	.97
M3	M146401	1-3	34,800	.96
M4	M146601	1-5	34,800	.87
M4	M146801	1-2	36,300	1.00
M4	M146901	1-3	36,300	.98
M4	M147001	1-3	36,300	.94
M4	M148101	1-3	36,300	.89
M5	M218502	1-2	36,500	.97
M5	M218504	1-2	36,500	.97
M5	M218505	1-3	36,500	.94
M5	M218801	1-2	36,500	.96
M5	M218901	1-2	36,500	.97
M5	M219501	1-3	36,500	.92
M6	M136501	1-2	35,200	.96
M6	M136601	1-3	35,200	.90
M6	M136951	1-3	35,200	.92
M7	M156701	1-2	37,200	.99
M7	M157001	1-3	37,200	.99
M7	M157301	1-2	37,200	.99
M7	M159401	1-4	37,200	.96
M7	M160701	1-3	37,200	.99
M7	M162401	1-3	37,200	.82
M8	M231101	1-2	38,900	.97
M8	M231102	1-2	38,900	.98
M8	M231103	1-2	38,900	.98
M8	M240401	1-2	38,900	1.00
M8	M240601	1-2	38,900	.99
M8	M240801	1-2	38,900	.99
M9	M224301	1-3	36,400	.99
M9	M225001	1-3	36,400	.98
M9	M226801	1-2	36,400	.97
M9	M338401	1-3	36,400	.98
M9	M338801	1-2	36,400	1.00
M10	M220001	1-3	36,900	.96
M10	M220401	1-3	36,900	.91
M10	M220701	1-3	36,900	.89
M10	M221101	1-3	36,900	.94
M11	M156101	1-3	36,700	.95
M11	M157501	1-3	36,700	.96
M11	M161001	1-2	36,700	.96
M11	M161002	1-3	36,700	.94
M11	M161601	1-3	36,700	.94
M11	M161901	1-3	36,700	.94
M11	M162201	1-3	36,700	.94
M12	M225201	1-3	35,200	.99
M12	M227201	1-3	35,200	.99
M12	M339401	1-3	35,200	.97

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 mathematics combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items. NOTE: Blocks M4, M5, M8, M9, M10, and M11 have larger sample sizes due to their pairing with integrated pilot blocks. The intraclass correlation is most appropriate for items with more than two categories. Cohen’s Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.
M3	M145901	1-3	34,500	.95	.97
M3	M146101	1-2	33,800	.94	†
M3	M146301	1-3	33,300	.96	.98
M4	M146401	1-3	33,400	.94	.96
M4	M146601	1-5	32,500	.83	.96
M4	M146801	1-2	35,400	1.00	†
M4	M146901	1-3	35,700	.98	.98
M4	M147001	1-3	34,600	.92	.95
M4	M148101	1-3	32,900	.74	.84
M5	M218502	1-2	35,700	.99	†
M5	M218504	1-2	36,000	.99	†
M5	M218505	1-3	35,400	.79	.89
M5	M218801	1-2	34,800	.97	†
M5	M218901	1-2	35,200	.97	†
M5	M219501	1-3	33,000	.89	.95
M6	M136501	1-2	34,000	.98	†
M6	M136601	1-3	33,500	.90	.95
M6	M136951	1-3	33,800	.94	.97
M7	M156701	1-2	36,300	.99	†
M7	M157001	1-3	36,100	.99	.99
M7	M157301	1-2	35,700	.98	†
M7	M159401	1-4	31,200	.92	.97
M7	M160701	1-3	35,000	.98	.98
M7	M162401	1-3	34,800	.67	.80
M8	M231101	1-2	24,200	.94	†
M8	M231102	1-2	35,100	.97	†
M8	M231103	1-2	34,700	.97	†
M8	M240401	1-2	34,500	.99	†
M8	M240601	1-2	38,400	.99	†
M8	M240801	1-2	38,200	.98	†
M9	M224301	1-3	36,000	.96	.97
M9	M225001	1-3	34,900	.97	.98
M9	M226801	1-2	33,300	.95	†
M9	M338401	1-3	36,000	.96	.99
M9	M338801	1-2	34,100	.99	†
M10	M220001	1-3	36,500	.97	.98
M10	M220401	1-3	36,300	.87	.88
M10	M220701	1-3	36,400	.82	.87
M10	M221101	1-3	35,500	.95	.98
M11	M156101	1-3	36,300	.97	.98
M11	M157501	1-3	35,100	.98	.99
M11	M161001	1-2	32,400	.98	†
M11	M161002	1-3	32,000	.92	.97
M11	M161601	1-3	35,900	.95	.97
M11	M161901	1-3	35,500	.94	.97
M11	M162201	1-3	33,900	.89	.90
M12	M225201	1-3	34,700	.98	.99
M12	M227201	1-3	34,900	.98	.99
M12	M339401	1-3	34,400	.95	.98

Last updated 21 May 2017 (DC)

Printer-friendly Version