Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 9 Mathematics Long-Term Trend Bridge Study: 2004

NAEP Technical DocumentationRange of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 9 mathematics long-term trend bridge study: 2004

Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen's Kappa
M1	N270001	1–2	600	99	0.98
	N270901	1–2	600	99	0.90
	N276101	1–2	600	99	0.98
	N276801	1–2	700	100	0.92
	N276802	1–2	600	99	0.96
	N276803	1–2	600	99	0.97
	N284001	1–2	600	99	0.96
	N284002	1–2	500	99	0.97
	N286101	1–2	600	99	0.97
M2	N271101	1–2	600	99	0.96
	N275401	1–2	600	99	0.97
	N276001	1–2	600	99	0.96
	N276002	1–2	500	100	0.99
	N277501	1–2	600	99	0.98
	N277601	1–2	600	100	0.99
	N277602	1–2	600	99	0.98
	N277603	1–2	600	99	0.99
	N286102	1–2	500	98	0.96
M21	N292101	1–2	1,200	100	0.99
	N292401	1–2	1,200	100	0.99
	N293001	1–2	1,200	100	0.99
	N293301	1–2	1,200	100	0.99
	N293501	1–2	1,200	100	0.99
M22	N294201	1–2	600	99	0.89
	N294301	1–2	600	100	1.00
	N295301	1–2	600	100	0.99
	N295401	1–2	600	99	0.99
	N295701	1–2	400	100	1.00
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Printer-friendly Version