NAEP tdw - Score Range and Percent Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2013

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2013

NAEP Technical DocumentationScore range and percent agreement for the constructed-response items used in scaling, grade 4 reading combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size ¹	Percent exact agreement
¹ Blocks R1, R2, R4, R6, R7, and R9 have larger sample sizes due to their pairing with integrated pilot blocks NOTE: Special codes assigned to responses including blank, off-task, and not scorable were included in the calculation of the Percent Exact Agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment.
R1	R068904	1-2	36,200	.95
R1	R068907	1-4	36,200	.78
R1	R068910	1-3	36,200	.81
R2	R057806	1-4	36,300	.82
R2	R057807	1-2	36,300	.93
R2	R057809	1-3	36,300	.89
R3	R068704	1-3	33,800	.83
R3	R068707	1-4	33,800	.86
R3	R068708	1-3	33,800	.83
R3	R068711	1-3	33,800	.91
R4	R070204	1-3	36,300	.91
R4	R070206	1-4	36,300	.84
R4	R070210	1-3	36,300	.85
R5	R058504	1-3	34,000	.82
R5	R058508	1-4	34,000	.78
R5	R058509	1-2	34,000	.95
R6	R058602	1-3	35,500	.78
R6	R058606	1-3	35,500	.91
R6	R058608	1-4	35,500	.85
R7	R058805	1-3	36,500	.84
R7	R058807	1-4	36,500	.77
R7	R058810	1-3	36,500	.84
R8	R059106	1-4	33,900	.78
R8	R059108	1-3	33,900	.90
R8	R059109	1-3	33,900	.84
R8	R059110	1-2	33,900	.93
R9	R058302	1-2	35,500	.96
R9	R058304	1-3	35,500	.92
R9	R058307	1-4	35,500	.85
R10	R069305	1-2	33,800	.93
R10	R069308	1-4	33,800	.76

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items used in scaling, grade 4 reading combined national and state assessment, by item and block: 2013
Block	Item	Range of response codes	Sample size ¹	Cohen’s Kappa	Intraclass correlation
¹ Blocks R1, R2, R4, R6, R7, and R9 have larger sample sizes due to their pairing with integrated pilot blocks † The Intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa and intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment.
R1	R068904	1-2	34,800	.89	†
R1	R068907	1-4	33,700	.65	.79
R1	R068910	1-3	31,700	.67	.80
R2	R057806	1-4	34,100	.71	.83
R2	R057807	1-2	32,500	.79	†
R2	R057809	1-3	29,300	.75	.81
R3	R068704	1-3	32,300	.70	.81
R3	R068707	1-4	30,900	.78	.84
R3	R068708	1-3	29,400	.71	.81
R3	R068711	1-3	26,400	.82	.88
R4	R070204	1-3	35,200	.81	.84
R4	R070206	1-4	33,800	.74	.89
R4	R070210	1-3	31,000	.73	.79
R5	R058504	1-3	31,300	.67	.74
R5	R058508	1-4	28,200	.64	.81
R5	R058509	1-2	24,900	.86	†
R6	R058602	1-3	33,900	.63	.72
R6	R058606	1-3	31,600	.83	.84
R6	R058608	1-4	29,100	.71	.84
R7	R058805	1-3	34,400	.73	.82
R7	R058807	1-4	31,900	.64	.82
R7	R058810	1-3	28,400	.68	.77
R8	R059106	1-4	28,300	.56	.70
R8	R059108	1-3	26,800	.78	.88
R8	R059109	1-3	32,500	.72	.83
R8	R059110	1-2	25,200	.81	†
R9	R058302	1-2	34,800	.92	†
R9	R058304	1-3	34,200	.88	.94
R9	R058307	1-4	31,500	.78	.93
R10	R069305	1-2	32,300	.85	†
R10	R069308	1-4	31,100	.63	.81

Last updated 21 May 2017 (DC)

Printer-friendly Version