NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items From Previous Years, Rescored in 2009, Grade 4 Mathematics Combined National and State Main Assessment, by Item and Block: 2009

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items from previous years, rescored in 2009, grade 4 mathematics combined national and state main assessment, by item and block: 2009

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 Mathematics Assessment.
M3	M145901	1–3	1,600	97	—	0.98
	M146101	1–2	1,600	96	0.93	—
	M146301	1–3	1,600	97	—	0.98
	M146401	1–3	1,600	96	—	0.95
	M146601	1–5	1,600	84	—	0.95
M4	M146801	1–2	1,600	100	0.99	—
	M146901	1–3	1,600	99	—	0.98
	M147001	1–3	1,600	95	—	0.95
	M148101	1–3	1,600	86	—	0.80
M5	M019701	1–2	1,600	100	0.99	—
	M020001	1–2	1,600	99	0.97	—
	M066301	1–3	1,600	97	—	0.96
	M066501	1–3	1,600	96	—	0.98
	M067901	1–3	1,600	97	—	0.98
	M085401	1–5	1,600	90	—	0.97
	M085701	1–3	1,600	98	—	0.98
	M085901	1–3	1,600	96	—	0.96
M6	M136501	1–2	1,600	99	0.99	—
	M136601	1–3	1,600	93	—	0.95
	M136901	1–3	1,600	96	—	0.97
M7	M156701	1–2	1,700	99	0.99	—
	M157001	1–3	1,700	99	—	0.99
	M157301	1–2	1,600	100	0.99	—
	M159401	1–4	1,600	94	—	0.96
	M160701	1–3	1,600	99	—	0.99
	M162401	1–3	1,600	79	—	0.81
M8	M010631	1–3	1,600	98	—	0.98
	M040001	1–3	1,600	97	—	0.98
	M072401	1–3	1,600	93	—	0.94
	M087001	1–3	1,600	96	—	0.97
	M087301	1–5	1,600	93	—	0.98
	M091201	1–3	1,600	92	—	0.94
M9	M155101	1–5	1,600	93	—	0.96
	M157601	1–3	1,700	80	—	0.80
	M160901	1–3	1,600	95	—	0.87
	M161201	1–3	1,700	94	—	0.93
	M162001	1–3	1,700	99	—	0.99
M10	M137501	1–2	1,600	99	0.97	—
	M138201	1–2	1,600	97	0.93	—
	M138401	1–3	1,600	85	—	0.85
	M138701	1–5	1,600	87	—	0.96
M11	M156101	1–3	1,700	98	—	0.98
	M157501	1–3	1,600	99	—	0.99
	M161001	1–2	1,600	99	0.98	—
	M161002	1–3	1,600	94	—	0.95
	M161601	1–3	1,700	96	—	0.97
	M161901	1–3	1,700	98	—	0.98
	M162201	1–3	1,600	95	—	0.88
M12	M148501	1–3	1,600	98	—	0.99
	M149201	1–3	1,600	96	—	0.94
	M149301	1–2	1,600	98	0.95	—
	M149701	1–5	1,600	92	—	0.97

Printer-friendly Version