Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items from Previous Year 2011, Rescored in 2013, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2013
Score range and percent agreement for the constructed-response Items from previous year 2011, rescored in 2013, grade 4 reading combined national and state assessment, by item and block: 2013
BlockItemRange of response codesSample sizePercent exact agreement
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment.
R2R0578061-41,600.81
R2R0578071-21,700.93
R2R0578091-31,600.87
R3R0687041-31,700.81
R3R0687071-41,600.82
R3R0687081-31,700.79
R3R0687111-31,800.90
R5R0585041-31,600.81
R5R0585081-41,600.72
R5R0585091-21,700.94
R6R0586021-31,600.76
R6R0586061-31,700.89
R6R0586081-41,600.79
R7R0588051-32,100.83
R7R0588071-41,600.72
R7R0588101-31,600.79
R8R0591061-42,100.70
R8R0591081-31,600.87
R8R0591091-31,600.81
R8R0591101-21,700.92
R9R0583021-21,800.96
R9R0583041-31,600.93
R9R0583071-41,600.85

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items from the previous year 2011, rescored in 2013, grade 4 reading combined national and state assessment, by item and block: 2013
BlockItemRange of response codesSample sizeCohen’s KappaIntraclass correlation
RBR0578061-41,600.69.83
RBR0578071-21,700.81†
RBR0578091-31,600.73.80
RCR0687041-31,700.64.76
RCR0687071-41,600.74.82
RCR0687081-31,700.64.75
RCR0687111-31,800.82.88
RER0585041-31,600.63.72
RER0585081-41,600.57.78
RER0585091-21,700.86†
RFR0586021-31,600.59.71
RFR0586061-31,700.80.84
RFR0586081-41,600.65.84
RGR0588051-31,600.71.81
RGR0588071-41,600.57.79
RGR0588101-31,600.64.74
RHR0591061-41,500.48.70
RHR0591081-31,600.75.84
RHR0591091-31,600.66.74
RHR0591101-21,700.84†
RIR0583021-21,800.93†
RIR0583041-31,600.87.93
RIR0583071-41,600.77.91
† The Intraclass correlation is not reported for dichotomously scored items.
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment.
Last updated 13 June 2017 (GF)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences