NAEP Analysis and Scaling - Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Civics Assessment, by Item and Block: 2006

Home TDW NAEP Analysis and Scaling Initial Activities Constructed-Response Interrater ReliabilityRange of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Civics Assessment, by Item and Block: 2006

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 civics assessment, by item and block: 2006
Block	Item	Range of response codes	Sample size	Percent of exact agreement	Cohen’s Kappa	Intraclass correlation
† The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2006 Civics Assessment.
P3	P030004	1–3	600	77	†	0.75
	P030005	1–3	600	83	†	0.83
	P030007	1–3	500	87	†	0.86
	P030010	1–4	500	84	†	0.87
P4	P040102	1–3	600	86	†	0.87
	P040105	1–3	600	79	†	0.75
	P040109	1–3	500	94	†	0.92
	P040111	1–3	500	79	†	0.81
P5	P040203	1–4	500	91	†	0.90
	P040206	1–3	600	93	†	0.77
	P040209	1–3	500	85	†	0.80
P6	P042122	1–3	600	81	†	0.80
	P042125	1–3	600	87	†	0.70
	P042129	1–3	500	83	†	0.81
	P042132	1–3	500	87	†	0.83
P7	P042136	1–3	500	84	†	0.85
	P042140	1–4	600	82	†	0.92
	P042143	1–4	500	86	†	0.90
	P042147	1–4	500	82	†	0.80
P8	P040502	1–4	600	90	†	0.95
	P040507	1–3	600	86	†	0.70
	P040510	1–3	500	81	†	0.80

Last updated 19 November 2009 (GF)