Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 4 Mathematics Combined National and State Assessment, by Item and Block: 2015

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 4 Mathematics Combined National and State Assessment, by Item and Block: 2015

Score range and percentage of exact agreement for the constructed-response items from previous year 2013, rescored in 2015, grade 4 mathematics combined national and state assessment, by item and block: 2015
Block	Item	Score range	Sample size	Percentage of exact agreement
M4	M146801	1-2	2,300	99
	M146901	1-3	2,100	98
	M147001	1-3	2,100	96
	M148101	1-3	2,300	91
M5	M218502	1-2	2,200	99
	M218504	1-2	2,000	99
	M218505	1-3	2,000	93
	M218801	1-2	2,200	98
	M218901	1-2	2,200	99
	M219501	1-3	2,000	92
M8	M231101	1-2	2,200	96
	M231102	1-2	2,200	98
	M231103	1-2	2,200	98
	M240401	1-2	2,300	100
	M240601	1-2	2,500	99
	M240801	1-2	2,400	99
M9	M224301	1-3	2,300	98
	M225001	1-3	2,000	98
	M226801	1-2	2,100	97
	M338401	1-3	2,000	97
	M338801	1-2	2,100	99
M10	M220001	1-3	2,100	98
	M220401	1-3	1,900	93
	M220701	1-3	1,900	87
	M221101	1-3	1,900	97
M11	M156101	1-3	2,200	98
	M157501	1-3	2,100	99
	M161001	1-2	2,100	99
	M161002	1-3	2,100	95
	M161901	1-3	1,900	97
M12	M225201	1-3	2,100	95
	M227201	1-3	2,100	99
	M339401	1-3	2,000	98
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percentage of exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from previous year 2013, rescored in 2015, grade 4 mathematics combined national and state assessment, by item and block: 2015
Block	Item	Score range	Sample size	Cohen’s Kappa	Intraclass correlation
M4	M146801	1-2	2,000	.99	†
	M146901	1-3	1,800	.98	.99
	M147001	1-3	1,800	.93	.95
	M148101	1-3	1,900	.73	.81
M5	M218502	1-2	2,000	.99	†
	M218504	1-2	1,900	.98	†
	M218505	1-3	1,800	.77	.90
	M218801	1-2	2,000	.98	†
	M218901	1-2	2,000	.97	†
	M219501	1-3	1,800	.87	.94
M8	M231101	1-2	1,200	.94	†
	M231102	1-2	1,900	.97	†
	M231103	1-2	1,900	.98	†
	M240401	1-2	2,100	.99	†
	M240601	1-2	2,300	.99	†
	M240801	1-2	2,200	.99	†
M9	M224301	1-3	2,100	.96	.98
	M225001	1-3	1,800	.97	.98
	M226801	1-2	1,700	.95	†
	M338401	1-3	1,800	.94	.96
	M338801	1-2	1,900	.99	†
M10	M220001	1-3	1,900	.96	.98
	M220401	1-3	1,700	.85	.86
	M220701	1-3	1,700	.74	.81
	M221101	1-3	1,700	.96	.98
M11	M156101	1-3	2,000	.97	.98
	M157501	1-3	1,900	.98	.99
	M161001	1-2	1,900	.99	†
	M161002	1-3	1,900	.90	.96
	M161901	1-3	1,700	.96	.98
M12	M225201	1-3	1,900	.92	.95
	M227201	1-3	1,900	.99	.99
	M339401	1-3	1,800	.97	.99
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Last updated 09 November 2021 (ML)

Printer-friendly Version