Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 8 Reading Combined National and State Assessment, by Item and Block: 2015

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 8 Reading Combined National and State Assessment, by Item and Block: 2015

Score range and percentage of exact agreement for the constructed-response items from previous year 2013, rescored in 2015, grade 8 reading combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Percentage of exact agreement
R1R0596041-21,90091
R0596051-31,70076
R0596061-31,90085
R0596071-31,60089
R0596101-41,60082
R2R0698011-31,60093
R0698041-31,90082
R0698061-41,60072
R0698101-31,60082
R3R0601031-21,70084
R0601051-21,90092
R0601081-41,60068
R6R0586021-31,90082
R0586061-31,90088
R0586081-41,60076
R7R0588051-31,60085
R0588071-41,60078
R0588101-31,60079
R9R0605031-31,60092
R0605051-41,60078
R0605071-31,60073
R0605081-31,60078
R0605091-31,90083
R10R0606031-31,90081
R0606051-31,60086
R0606061-41,60077
R0606071-31,90079
R0606081-31,60085
R11R0696031-31,60092
R0696061-31,60082
R0696071-41,60077
R0696091-31,90083
R12R0610031-31,70085
R0610041-21,90095
R0610061-21,60092
R0610071-41,60070
R0610101-31,60078
R13R0697051-21,90092
R0697061-31,60087
R0697071-41,60079
R0697081-31,60072
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percentage of exact agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from previous year 2013, rescored in 2015, grade 8 reading combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Cohen’s KappaIntraclass correlation
R1R0596041-21,700.66†
R0596051-31,500.57.72
R0596061-31,700.72.83
R0596071-31,400.80.89
R0596101-41,400.73.88
R2R0698011-31,400.85.87
R0698041-31,700.68.79
R0698061-41,400.54.74
R0698101-31,400.70.81
R3R0601031-21,500.62†
R0601051-21,700.82†
R0601081-41,400.52.72
R6R0586021-31,700.69.76
R0586061-31,700.80.82
R0586081-41,400.63.83
R7R0588051-31,400.72.79
R0588071-41,400.64.84
R0588101-31,500.61.70
R9R0605031-31,500.88.92
R0605051-41,400.68.86
R0605071-31,400.47.54
R0605081-31,500.58.67
R0605091-31,700.70.79
R10R0606031-31,700.68.80
R0606051-31,400.72.77
R0606061-41,400.58.76
R0606071-31,700.51.59
R0606081-31,400.76.83
R11R0696031-31,400.86.92
R0696061-31,400.60.67
R0696071-41,400.64.82
R0696091-31,700.73.84
R12R0610031-31,500.74.84
R0610041-21,700.78†
R0610061-21,400.82†
R0610071-41,400.51.70
R0610101-31,400.59.70
R13R0697051-21,700.78†
R0697061-31,400.75.83
R0697071-41,500.64.84
R0697081-31,400.52.67
† Not applicable. The intraclass correlation is not reported for dichotomously scored items.
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment.


Last updated 15 November 2021 (ML)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences