Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Exact Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, by Block and Item, Age 17 Mathematics Long-Term Trend Bridge Study: 2004

NAEP Technical DocumentationRange of response codes, percent exact agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, by block and item, age 17 mathematics long-term trend bridge study: 2004

Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen's Kappa
M1	N251101	1 - 2	900	98	0.95
	N256101	1 - 2	1,000	100	0.96
	N260601	1 - 2	1,000	99	0.96
	N263001	1 - 2	900	99	0.97
	N264301	1 - 2	700	97	0.92
	N278501	1 - 2	800	98	0.96
	N278502	1 - 2	800	98	0.96
	N278503	1 - 2	800	98	0.95
	N287301	1 - 2	700	97	0.95
	N287302	1 - 2	800	96	0.93
M2	N255801	1 - 2	700	99	0.97
	N259001	1 - 2	800	97	0.93
	N260801	1 - 2	800	99	0.98
	N263101	1 - 2	900	99	0.98
	N280401	1 - 2	900	99	0.97
M21	N307301	1 - 2	900	99	0.98
	N307701	1 - 2	800	100	1.00
	N308101	1 - 2	900	99	0.99
	N308501	1 - 2	800	99	0.98
NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004 Mathematics Long-Term Trend Assessment.

Printer-friendly Version