NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 12 U.S. History National Assessment, by Item and Block: 2010

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 12 U.S. history national assessment, by item and block: 2010

Block	Item	Range of response codes	Sample size	Percent exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2010 U.S. History Assessment.
H3	H073501	1-3	600	86	†	0.88
	H074001	1-3	500	98	†	0.98
	H074401	1-3	500	81	†	0.82
	H074801	1-3	400	94	†	0.92
	H075101	1-4	500	79	†	0.80
H4	H045509	1-3	600	86	†	0.85
	H045901	1-4	600	83	†	0.85
	H046001	1-2	600	99	0.98	†
	H046101	1-3	600	99	†	0.99
	H046301	1-3	500	87	†	0.85
H5	H062001	1-3	600	86	†	0.78
	H062201	1-3	600	91	†	0.89
	H063101	1-3	600	92	†	0.92
	H063401	1-4	600	90	†	0.94
	H063601	1-3	500	87	†	0.86
H6	H048901	1-3	300	96	†	0.97
	H049401	1-4	500	93	†	0.76
	H049503	1-3	500	82	†	0.70
	H049601	1-4	500	88	†	0.86
	H049701	1-3	500	98	†	0.98
H7	H075301	1-3	600	90	†	0.85
	H075701	1-3	600	93	†	0.88
	H076401	1-3	600	90	†	0.90
	H076501	1-3	500	88	†	0.82
	H076801	1-3	600	93	†	0.93
H8	H091501	1-3	400	84	†	0.85
	H093501	1-4	600	85	†	0.91
	H094801	1-3	500	87	†	0.87
	H095901	1-3	500	86	†	0.85
	H097001	1-3	600	94	†	0.89
H9	H091801	1-4	500	88	†	0.94
	H093101	1-3	600	88	†	0.87
	H093803	1-3	500	81	†	0.77
	H094201	1-3	500	86	†	0.87
	H095401	1-4	500	89	†	0.92
H10	H042201	1-3	600	92	†	0.82
	H042801	1-4	600	85	†	0.87
	H042902	1-3	600	96	†	0.97
	H043001	1-3	600	81	†	0.81
	H043101	1-3	600	93	†	0.93
H11	H063801	1-3	600	86	†	0.85
	H064001	1-3	600	86	†	0.85
	H064101	1-3	600	83	†	0.85
	H064401	1-4	500	80	†	0.83
	H064901	1-3	600	83	†	0.83
	H065101	1-3	600	85	†	0.85
	H065201	1-3	600	82	†	0.84
	H065301	1-3	600	81	†	0.81
	H065401	1-4	500	78	†	0.87

Printer-friendly Version