Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items from Previous Year 2011, Rescored in 2013, Grade 8 Reading Combined National and State Assessment, by Item and Block: 2013
Score range and percent agreement for the constructed-response Items from previous year 2011, rescored in 2013, grade 8 reading combined national and state assessment, by item and block: 2013
BlockItemRange of response codesSample sizePercent agreement
RAR0596041-21,60092
RAR0596051-31,60078
RAR0596061-31,60083
RAR0596071-31,60087
RAR0596101-41,60082
RCR0601031-21,60085
RCR0601051-21,60093
RCR0601081-41,60070
RDR0603011-21,60091
RDR0603051-31,60084
RDR0603061-41,60070
RDR0603091-31,60079
RER0585041-31,60078
RER0585081-41,60073
RER0585091-21,60091
RFR0586021-31,60082
RFR0586061-31,70086
RFR0586081-41,60076
RGR0588051-31,60082
RGR0588071-41,60079
RGR0588101-31,60082
RHR0591061-41,60066
RHR0591081-31,60084
RHR0591091-31,60076
RHR0591101-21,60092
RIR0605031-31,60091
RIR0605051-41,60077
RIR0605071-31,60069
RIR0605081-31,60076
RIR0605091-31,60083
RJR0606031-31,60080
RJR0606051-31,60087
RJR0606061-41,60076
RJR0606071-31,60081
RJR0606081-31,60089
RLR0610031-31,60084
RLR0610041-21,60096
RLR0610061-21,60091
RLR0610071-41,60068
RLR0610101-31,60076
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment.

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items from the previous year 2011, rescored in 2013, grade 8 reading combined national and state assessment, by item and block: 2013
Block Item Range of response codes Sample size Cohen’s Kappa Intraclass correlation
R1 R059604 1-2 1,500 .72 †
R1 R059605 1-3 1,500 .59 .71
R1 R059606 1-3 1,400 .69 .80
R1 R059607 1-3 1,500 .76 .87
R1 R059610 1-4 1,500 .72 .86
R3 R060103 1-2 1,500 .64 †
R3 R060105 1-2 1,500 .84 †
R3 R060108 1-4 1,800 .55 .77
R4 R060301 1-2 1,500 .73 †
R4 R060305 1-3 1,500 .68 .75
R4 R060306 1-4 1,800 .54 .62
R4 R060309 1-3 1,400 .61 .69
R5 R058504 1-3 1,400 .58 .66
R5 R058508 1-4 1,500 .56 .77
R5 R058509 1-2 1,500 .80 †
R6 R058602 1-3 1,400 .68 .76
R6 R058606 1-3 1,500 .77 .81
R6 R058608 1-4 1,400 .64 .84
R7 R058805 1-3 1,900 .65 .74
R7 R058807 1-4 1,400 .65 .83
R7 R058810 1-3 1,500 .65 .72
R8 R059106 1-4 2,000 .46 .69
R8 R059108 1-3 1,500 .73 .84
R8 R059109 1-3 1,500 .60 .74
R8 R059110 1-2 1,600 .79 †
R9 R060503 1-3 1,400 .85 .91
R9 R060505 1-4 1,400 .71 .88
R9 R060507 1-3 1,600 .44 .49
R9 R060508 1-3 2,900 .55 .63
R9 R060509 1-3 1,400 .70 .80
R10 R060603 1-3 1,400 .65 .77
R10 R060605 1-3 1,500 .73 .78
R10 R060606 1-4 1,400 .58 .77
R10 R060607 1-3 1,500 .59 .67
R10 R060608 1-3 1,500 .82 .88
R12 R061003 1-3 1,500 .73 .82
R12 R061004 1-2 1,500 .81 †
R12 R061006 1-2 1,400 .80 †
R12 R061007 1-4 1,400 .44 .59
R12 R061010 1-3 1,400 .55 .67
† The Intraclass correlation is not reported for dichotomously scored items.
NOTE:The intraclass correlation is most appropriate for items with more than two categories. Cohen"s Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
Last updated 13 June 2017 (GF)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences