NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 12 Mathematics National Main Assessment, by Item and Block: 2009

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 12 mathematics national main assessment, by item and block: 2009

Block1	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
1 Common blocks M13–M22 were previously named M3–M12 in 2005. — Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 Mathematics Assessment.
M1	M175901	1–3	1,300	99	—	0.95
	M178001	1–3	1,100	99	—	0.98
	M183601	1–3	1,500	99	—	0.99
	M183901	1–2	1,400	100	0.99	—
	M186701	1–3	1,500	94	—	0.93
	M186702	1–3	1,400	89	—	0.88
M2	M180901	1–3	1,100	96	—	0.93
	M180902	1–3	1,300	93	—	0.93
	M180903	1–3	1,200	96	—	0.97
	M190601	1–3	1,500	99	—	0.96
	M195001	1–3	1,200	98	—	0.96
	M196802	1–3	1,600	92	—	0.85
	M197001	1–3	1,400	99	—	0.99
M3	M179201	1–3	900	98	—	0.95
	M193401	1–3	1,600	93	—	0.89
	M197303	1–2	1,400	99	0.94	—
	M197304	1–2	1,400	100	0.99	—
	M197501	1–3	1,500	96	—	0.94
	M197502	1–3	1,300	98	—	0.94
	M197503	1–3	1,100	98	—	0.95
M4	M174801	1–3	1,100	99	—	0.89
	M184201	1–2	1,500	100	0.99	—
	M184202	1–2	1,600	100	0.99	—
	M184203	1–2	1,500	100	0.99	—
	M184204	1–2	1,500	99	0.98	—
	M184205	1–2	1,400	100	0.99	—
	M184206	1–2	1,400	99	0.98	—
	M184207	1–3	1,200	91	—	0.86
	M197601	1–3	1,300	97	—	0.97
M5	M177301	1–3	1,600	94	—	0.95
	M185201	1–2	1,500	99	0.99	—
	M185703	1–5	1,400	98	—	0.98
	M187701	1–3	1,400	99	—	1.00
M6	M175002	1–5	1,500	99	—	1.00
	M175601	1–3	1,600	98	—	0.97
	M177202	1–2	1,200	98	0.94	—
	M183401	1–3	800	98	—	0.96
M7	M180201	1–5	1,200	98	—	0.97
	M188301	1–2	1,300	100	0.99	—
	M189901	1–3	1,500	99	—	0.99
	M191401	1–3	1,400	98	—	0.98
	M195201	1–3	1,200	99	—	1.00
M8	M188001	1–3	1,400	98	—	0.99
	M188201	1–5	1,400	99	—	0.99
	M189001	1–3	1,300	91	—	0.83
	M193101	1–2	1,500	100	1.00	—
	M194301	1–2	1,400	100	1.00	—
	M194302	1–2	1,400	100	0.99	—
M9	M191901	1–5	1,400	96	—	0.98
	M198701	1–2	1,600	100	0.99	—
	M198901	1–3	1,300	97	—	0.97
M10	M178501	1–3	1,400	100	—	1.00
	M189701	1–3	1,200	99	—	0.97
	M189702	1–2	1,100	98	0.96	—
	M189703	1–2	1,000	99	0.97	—
	M190801	1–3	1,400	99	—	0.98
	M191701	1–3	1,500	100	—	1.00
	M199201	1–3	1,400	100	—	0.99
M11	M176501	1–3	1,400	97	—	0.98
	M176502	1–2	1,500	97	0.94	—
	M176503	1–2	1,500	98	0.95	—
	M176504	1–2	1,500	97	0.94	—
	M181201	1–2	1,500	99	0.99	—
	M181202	1–3	1,500	99	—	0.97
	M181203	1–2	1,500	100	0.99	—
	M183101	1–2	1,000	99	0.97	—
M12	M178701	1–4	900	99	—	0.96
	M181801	1–2	1,300	100	1.00	—
	M186001	1–2	1,500	100	1.00	—
	M194001	1–3	1,300	98	—	0.97
M13	M056801	1–2	500	100	0.99	—
	M076601	1–3	400	99	—	0.99
	M095001	1–3	500	97	—	0.98
	M095301	1–3	500	95	—	0.96
	M095401	1–4	400	98	—	0.99
M14	M057001	1–2	500	100	1.00	—
	M058901	1–3	500	99	—	0.99
	M071502	1–3	400	97	—	0.98
	M071602	1–3	500	93	—	0.93
	M071603	1–3	500	94	—	0.94
	M071604	1–3	500	94	—	0.95
	M071801	1–5	400	93	—	0.98
M15	M021401	1–2	500	99	0.99	—
	M021501	1–2	500	98	0.95	—
	M021502	1–2	500	100	0.99	—
	M071701	1–3	300	99	—	0.98
	M094201	1–4	500	93	—	0.97
	M094301	1–3	500	98	—	0.97
	M094701	1–5	500	89	—	0.95
M16	M021801	1–2	400	100	0.99	—
	M071001	1–3	400	96	—	0.95
	M132201	1–4	400	96	—	0.98
M17	M073801	1–3	500	99	—	0.99
	M073901	1–3	500	97	—	0.97
	M074101	1–5	400	95	—	0.98
	M076101	1–3	600	99	—	0.99
	M076901	1–3	400	99	—	0.98
M18	M020901	1–2	500	96	0.88	—
	M021001	1–2	500	100	0.99	—
	M021601	1–4	400	99	—	0.87
	M021602	1–2	300	98	0.89	—
	M071201	1–3	400	96	—	0.96
	M071301	1–3	300	99	—	0.97
	M071401	1–5	300	99	—	0.98
	M076701	1–3	300	98	—	0.98
	M092401	1–3	500	99	—	0.99
	M092601	1–3	500	98	—	0.98
M19	M011931	1–2	400	100	1.00	—
	M012031	1–3	400	100	—	1.00
	M013131	1–2	500	100	0.99	—
	M052401	1–2	500	95	0.88	—
	M057101	1–2	400	100	0.97	—
	M071101	1–3	500	99	—	0.99
	M074001	1–3	500	99	—	0.99
	M076801	1–3	300	99	—	0.99
	M092301	1–3	500	98	—	0.98
	M092901	1–5	300	97	—	0.99
M20	M021701	1–2	500	99	0.99	—
	M021702	1–2	500	98	0.96	—
	M053301	1–2	500	96	0.74	—
	M053401	1–5	500	84	—	0.94
	M059702	1–3	500	98	—	0.94
	M059801	1–2	400	98	0.86	—
	M070801	1–3	500	93	—	0.94
M21	M126001	1–2	500	100	1.00	—
	M129001	1–5	500	94	—	0.97
	M131201	1–2	500	100	0.98	—
M22	M133801	1–4	500	98	—	0.97
	M134601	1–2	500	98	0.94	—
	M135501	1–5	500	94	—	0.97

Printer-friendly Version