Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2009, Rescored in 2015, Grade 4 Science Combined National and State Assessment, by Item and Block: 2015

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items from Previous Year 2009, Rescored in 2015, Grade 4 Science Combined National and State Assessment, by Item and Block: 2015

Score range and percentage of exact agreement for the constructed-response items from previous year 2009, rescored in 2015, grade 4 science combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Percentage of exact agreement
S3K0776021-41,70090
K0804041-31,90090
K1530011-31,80082
K1531021-41,60092
K1067011-31,90084
S4K0789021-31,90093
K0843011-21,90090
K0843021-21,70092
K1533011-31,70088
K0795011-41,60090
S5K0816011-31,60093
K0873011-31,70090
K0818011-31,70089
K0825011-31,60091
K0842021-31,60091
S6K1079021-31,60086
K0869011-21,60099
K0869021-21,60099
K0869031-21,600100
K0869041-21,60099
K0765011-31,90090
K0874021-41,60081
K1091021-31,60079
S8K0775011-31,90087
K0777011-31,90094
K0781021-41,60087
K1097011-31,70088
K1097021-31,70091
K1097031-31,70087
K0783011-31,70094
S9K0883021-31,60096
K0884011-51,60077
K1094021-31,60096
K1537011-31,60088
K1537021-31,60093
S10K0829011-31,90093
K0829021-31,90093
K0829031-31,90093
K1540011-31,90088
K0835021-31,70085
K0853021-31,60088
K0834021-41,60087
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percentage of exact agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Science Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items from previous year 2009, rescored in 2015, grade 4 science combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Cohen’s KappaIntraclass correlation
S3K0776021-41,500.85.94
K0804041-31,700.82.87
K1530011-31,700.68.77
K1531021-41,400.86.93
K1067011-31,700.72.80
S4K0789021-31,700.85.88
K0843011-21,700.81†
K0843021-21,500.85†
K1533011-31,500.79.87
K0795011-41,400.83.90
S5K0816011-31,400.80.90
K0873011-31,500.83.89
K0818011-31,500.80.87
K0825011-31,400.85.90
K0842021-31,400.74.83
S6K1079021-31,400.75.86
K0869011-21,500.97†
K0869021-21,500.99†
K0869031-21,500.99†
K0869041-21,400.99†
K0765011-31,700.82.89
K0874021-41,400.69.81
K1091021-31,400.57.70
S8K0775011-31,700.78.87
K0777011-31,700.91.96
K0781021-41,400.77.91
K1097011-31,500.73.78
K1097021-31,400.70.74
K1097031-31,300.69.73
K0783011-31,500.90.94
S9K0883021-31,400.94.95
K0884011-51,400.66.89
K1094021-31,400.89.95
K1537011-31,400.78.85
K1537021-31,400.86.89
S10K0829011-31,700.85.88
K0829021-31,700.86.88
K0829031-31,800.86.90
K1540011-31,700.76.86
K0835021-31,500.71.78
K0853021-31,400.77.82
K0834021-41,400.80.92
† Not applicable. The intraclass correlation is not reported for dichotomously scored items.
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Science Assessment.

Last updated 15 November 2021 (ML)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences