NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 8 Arts (Music) Assessment, by Item and Block: 2008

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 8 arts (music) assessment, by item and block: 2008

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. Item statistics in this table are described on the page Blocks of Items. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2008 Arts (Music) Assessment.
U3	UC00004	1–2	500	100	1.00	—
	UC00006	1–3	400	91	—	0.89
	UC00010	1–4	400	87	—	0.87
	UC00012	1–2	400	97	0.94	—
	UC000A2	1–2	500	93	0.84	—
	UC000A3	1–3	500	91	—	0.88
	UC000B1	1–3	500	96	—	0.81
	UC000B2	1–2	200	98	0.92	—
	UC000B3	1–3	400	87	—	0.85
	UC00B14	1–3	500	93	—	0.82
	UC00B15	1–3	500	88	—	0.76
	UC00C14	1–3	500	90	—	0.82
	UC00C15	1–3	500	84	—	0.68
	UC00D15	1–3	500	86	—	0.72
U4	UD00007	1–3	400	87	—	0.82
	UD000A5	1–2	500	99	0.95	—
	UD000A6	1–3	500	95	—	0.96
	UD000B5	1–2	500	99	0.98	—
	UD000B6	1–3	500	93	—	0.95
	UD000B9	1–3	500	96	—	0.95
	UD000C5	1–2	500	93	0.85	—
	UD000C6	1–2	500	95	0.86	—
	UD000D5	1–2	500	96	0.91	—
	UD000D6	1–3	500	97	—	0.97
	UD000E5	1–3	400	92	—	0.82
	UD000E6	1–3	500	99	—	0.99
U5	UE00004	1–4	400	97	—	0.99
	UE00010	1–3	500	81	—	0.79
	UE00015	1–3	500	92	—	0.85
	UE00016	1–3	400	82	—	0.85
	UE000A6	1–2	300	99	0.97	—
	UE000A8	1–3	500	97	—	0.98
	UE000A9	1–3	500	94	—	0.94
	UE000B6	1–2	200	100	0.99	—
	UE000B8	1–3	400	95	—	0.94
	UE000B9	1–3	400	89	—	0.89
	UE000C8	1–2	400	99	0.97	—
	UE000C9	1–3	400	91	—	0.91
	UE00B11	1–3	500	97	—	0.97
	UE00B12	1–3	500	97	—	0.97
U6	UF00003	1–4	500	81	—	0.87
	UF00005	1–5	500	91	—	0.81
	UF000B1	1–3	300	94	—	0.94
	UF000B2	1–3	500	98	—	0.80
	UF000H1	1–3	300	95	—	0.93
	UF000L1	1–3	300	96	—	0.96
	UF000N1	1–3	300	97	—	0.97
	UF000P1	1–3	200	97	—	0.96
	UF000T1	1–3	200	98	—	0.98
	UF001A6	1–3	500	96	—	0.92
	UF001B6	1–4	500	85	—	0.85
	UF001C6	1–3	400	79	—	0.72
	UF002A6	1–3	400	95	—	0.92
	UF002B6	1–4	400	86	—	0.82
	UF002C6	1–3	400	82	—	0.73
	UF003A6	1–3	400	93	—	0.87
	UF003B6	1–4	400	87	—	0.83
	UF003C6	1–3	400	84	—	0.75
	UF004A6	1–3	500	97	—	0.89
	UF004B6	1–4	500	85	—	0.85
	UF004C6	1–3	400	85	—	0.86

Printer-friendly Version