Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Assessment, by Item and Block: 2015

NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Assessment, by Item and Block: 2015

Score range and percentage of exact agreement for the constructed-response items used in scaling, grade 4 mathematics combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Percentage of exact agreement
M3
M3418011-21,40099
M3408011-31,40099
M3424011-31,40098
M3424021-21,40099
M3424031-21,40098
M3415011-21,400100
M4M1468011-21,40099
M1469011-31,40099
M1470011-31,40095
M1481011-31,40088
M5M2185021-21,40098
M2185041-21,40098
M2185051-31,40094
M2188011-21,40097
M2189011-21,40097
M2195011-31,40093
M6M3402011-31,40099
M3421011-31,40097
M3432011-31,40095
M3430011-21,40099
M3444011-31,400100
M7M3461011-31,50097
M3468011-21,50098
M3468021-21,50098
M3479011-31,50097
M2303011-21,50098
M3483011-21,50097
M8M2406011-21,50098
M2408011-21,50098
M2311011-21,50096
M2311021-21,50097
M2311031-21,50097
M2404011-21,50098
M9M2243011-31,40099
M3384011-31,40097
M2250011-31,40099
M2268011-21,40098
M3388011-21,40099
M10M2200011-31,40097
M2204011-31,40095
M2207011-31,40090
M2211011-31,40095
M11M1561011-31,40097
M1616011-31,40095
M1619011-31,40097
M1575011-31,40098
M1622011-31,40098
M1610011-21,40099
M1610021-31,40094
M12M2272011-31,50099
M2252011-31,50098
M3394011-31,50097
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the percentage of exact agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.

Score range, Cohen's Kappa, and intraclass correlation for the constructed-response items used in scaling, grade 4 mathematics combined national and state assessment, by item and block: 2015
BlockItemScore range
Sample size
Cohen’s KappaIntraclass correlation
M3M3418011-21,400.98†
M3408011-31,400.99.99
M3424011-31,400.94.97
M3424021-21,400.96†
M3424031-21,300.96†
M3415011-21,300.99†
M4M1468011-21,400.98†
M1469011-31,400.99.99
M1470011-31,300.93.95
M1481011-31,200.71.80
M5M2185021-21,400.99†
M2185041-21,400.98†
M2185051-31,400.75.88
M2188011-21,300.98†
M2189011-21,300.94†
M2195011-31,200.90.96
M6M3402011-31,400.97.98
M3421011-31,300.96.98
M3432011-31,300.92.95
M3430011-21,300.98†
M3444011-31,2001.001.00
M7M3461011-31,500.99.99
M3468011-21,400.99†
M3468021-21,4001.00†
M3479011-31,300.98.99
M2303011-21,200.99†
M3483011-21,100.99†
M8M2406011-21,500.99†
M2408011-21,500.98†
M2311011-2900.94†
M2311021-21,300.98†
M2311031-21,300.98†
M2404011-21,300.99†
M9M2243011-31,400.97.98
M3384011-31,400.94.97
M2250011-31,300.98.99
M2268011-21,300.97†
M3388011-21,300.99†
M10M2200011-31,400.94.96
M2204011-31,400.90.91
M2207011-31,400.82.87
M2211011-31,300.94.97
M11M1561011-31,400.96.97
M1616011-31,400.93.96
M1619011-31,300.94.97
M1575011-31,300.97.98
M1622011-31,300.90.90
M1610011-21,200.98†
M1610021-31,200.88.95
M12M2272011-31,400.98.99
M2252011-31,400.97.98
M3394011-31,400.96.98
† Not applicable. The intraclass correlation is not reported for dichotomously scored items.
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.




Last updated 09 November 2021 (ML)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences