Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Main Assessment, by Item and Block: 2007

NAEP Technical DocumentationRange of response codes, percent agreement, and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 reading combined national and state main assessment, by item and block: 2007

Block	Item	Range of response codes	Sample size	Percent agreement	Cohen’s Kappa	Intraclass correlation
— Not available. The intraclass correlation is not reported for dichotomously scored items; Cohen's Kappa is not reported for polytomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The intraclass correlation coefficient is most appropriate for items with more than two categories. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2007 Reading Assessment.
R3	R053901	1–3	1,900	86.60	—	0.83
	R054001	1–3	1,800	84.36	—	0.76
	R054201	1–4	1,800	76.79	—	0.85
	R054301	1–3	1,600	87.35	—	0.81
R4	R052904	1–3	1,800	89.35	—	0.90
	R052906	1–2	1,800	90.92	0.81	—
	R052907	1–4	1,700	78.74	—	0.82
	R052909	1–2	1,600	94.95	0.87	—
R5	R012601	1–2	1,800	92.60	0.83	—
	R012604	1–2	1,800	95.78	0.87	—
	R012607	1–4	1,600	86.29	—	0.84
	R012612	1–2	1,400	95.49	0.91	—
R6	R017301	1–2	1,900	95.24	0.86	—
	R017303	1–3	1,800	92.48	—	0.93
	R017307	1–4	1,600	86.87	—	0.87
	R017309	1–3	1,400	85.76	—	0.86
	R017310	1–3	1,700	96.52	—	0.97
R7	R034601	1–3	1,800	91.27	—	0.91
	R034901	1–3	1,700	91.85	—	0.91
	R035001	1–3	1,700	89.16	—	0.91
	R035201	1–3	1,600	95.03	—	0.95
	R035301	1–4	1,600	84.68	—	0.90
R8	R054801	1–3	1,800	97.60	—	0.98
	R055101	1–4	1,700	91.30	—	0.96
	R055301	1–3	1,600	86.88	—	0.85
	R055401	1–3	1,500	94.19	—	0.93
R9	R053003	1–2	1,900	97.62	0.94	—
	R053004	1–3	1,900	96.16	—	0.96
	R053006	1–4	1,800	86.30	—	0.94
	R053009	1–2	1,700	92.00	0.77	—
	R053010	1–3	1,600	89.96	—	0.88
R10	R053101	1–2	1,900	92.19	0.75	—
	R053105	1–3	1,800	86.99	—	0.87
	R053106	1–4	1,700	85.69	—	0.93
	R053108	1–3	1,500	91.25	—	0.89
R11	R020401	1–3	1,900	95.52	—	0.94
	R020701	1–2	1,800	94.62	0.85	—
	R021201	1–3	1,700	89.63	—	0.88
	R021701	1–3	1,800	86.51	—	0.85
R12	R023201	1–2	1,600	96.12	0.92	—
	R023501	1–3	1,800	90.21	—	0.92
	R023601	1–3	1,800	91.80	—	0.91
	R024001	1–3	1,700	88.95	—	0.89

Printer-friendly Version