Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Mathematics Combined National and State Main Assessment, by Item and Block: 2007

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 mathematics combined national and state main assessment, by item and block: 2007

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007Mathematics Assessment.
M3	M149901	1–3	1,500	96.81	—	0.97
	M150801	1–3	1,400	97.44	—	0.98
	M150901	1–2	1,500	99.29	0.98	—
	M151201	1–2	1,400	99.72	0.99	—
M4	M151501	1–3	1,500	94.89	—	0.96
	M151601	1–3	1,400	94.54	—	0.94
	M152602	1–5	1,500	96.84	—	0.98
M5	M019701	1–2	1,600	99.88	0.99	—
	M020001	1–2	1,600	99.29	0.98	—
	M047901	1–3	1,400	99.15	—	0.98
	M053101	1–5	1,400	87.76	—	0.94
	M066301	1–3	1,500	96.95	—	0.94
	M066501	1–3	1,500	96.01	—	0.96
	M067901	1–3	1,400	97.68	—	0.97
	M085701	1–3	1,600	98.15	—	0.97
	M085901	1–3	1,500	94.60	—	0.93
M6	M141301	1–4	1,500	96.08	—	0.97
	M141601	1–2	1,400	85.58	0.69	—
	M141901	1–5	1,400	87.70	—	0.93
M7	M144201	1–2	1,600	99.42	0.99	—
	M144601	1–3	1,500	93.72	—	0.93
	M144901	1–4	1,400	90.41	—	0.92
	M145101	1–4	1,400	98.01	—	0.93
M8	M020901	1–2	1,400	90.94	0.82	—
	M021001	1–2	1,600	99.05	0.98	—
	M073601	1–5	1,400	84.91	—	0.94
	M092401	1–3	1,600	97.52	—	0.96
	M092601	1–3	1,500	97.69	—	0.97
M9	M013131	1–2	1,500	98.62	0.97	—
	M052401	1–2	1,600	91.37	0.82	—
	M072901	1–3	1,500	93.94	—	0.95
	M073501	1–3	1,500	95.88	—	0.94
	M075301	1–3	1,500	94.79	—	0.93
	M075801	1–3	1,400	85.58	—	0.87
	M093801	1–5	1,400	91.73	—	0.97
M10	M142801	1–2	1,400	99.65	0.99	—
	M143201	1–2	1,500	98.94	0.98	—
	M143501	1–5	1,400	84.09	—	0.92
M11	M106301	1–2	1,600	99.00	0.98	—
	M106901	1–2	1,500	87.34	0.70	—
	M107501	1–2	1,500	99.67	0.99	—
M12	M153201	1–3	1,600	88.95	—	0.88
	M153601	1–2	1,500	98.59	0.97	—
	M153901	1–3	1,400	86.52	—	0.89

Printer-friendly Version