Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2015

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2015

Score range and percentage of exact agreement for the constructed-response items used in scaling, grade 4 reading combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Percentage of exact agreement
R1R0689041-21,40091
R0689071-41,40076
R0689101-31,40086
R2R0578061-41,40083
R0578071-21,40093
R0578091-31,40088
R3R0793061-31,40084
R0793071-41,40078
R0793091-31,40083
R4R0702041-31,40089
R0702061-41,40084
R0702101-31,40084
R5R0795041-31,40084
R0795071-41,40083
R0795081-21,40093
R0795101-31,40081
R6R0586021-31,40078
R0586061-31,40090
R0586081-41,40081
R7R0588051-31,40091
R0588071-41,40075
R0588101-31,40085
R8R0797031-31,40080
R0797061-31,40095
R0797071-41,40091
R0797091-31,40088
R9R0583021-21,40097
R0583041-31,40093
R0583071-41,40082
R10R0693051-21,40091
R0693081-41,40073
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percentage of exact agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items used in scaling, grade 4 reading combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Cohen’s KappaIntraclass correlation
R1R0689041-21,300.81†
R0689071-41,300.62.78
R0689101-31,200.74.83
R2R0578061-41,300.72.84
R0578071-21,300.79†
R0578091-31,100.73.78
R3R0793061-31,300.73.83
R0793071-41,200.67.78
R0793091-31,100.67.78
R4R0702041-31,400.77.82
R0702061-41,300.74.88
R0702101-31,200.72.82
R5R0795041-31,400.74.83
R0795071-41,300.75.91
R0795081-21,200.85†
R0795101-31,200.66.78
R6R0586021-31,300.64.74
R0586061-31,200.81.84
R0586081-41,200.64.79
R7R0588051-31,300.86.91
R0588071-41,200.61.80
R0588101-31,100.69.78
R8R0797031-31,300.68.74
R0797061-31,300.91.96
R0797071-41,200.85.94
R0797091-31,100.71.73
R9R0583021-21,400.94†
R0583041-31,400.89.94
R0583071-41,300.73.89
R10R0693051-21,300.80†
R0693081-41,300.57.75
† Not applicable. The intraclass correlation is not reported for dichotomously scored items.
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment.


Last updated 09 November 2021 (ML)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences