Skip to main content
IES    
NCES
National Center for
Education Statistics
Menu
Contact
Tools
ED.gov
Newsflash
Search IES

Table of Contents | Search Technical Documentation | References

NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items From Previous Year 2011, Rescored in 2013, Grade 4 Mathematics Combined National and State Assessment, by Item and Block 2013
Score range and percent agreement for the constructed-response Items from previous year 2011, rescored in 2013, grade 4 mathematics combined national and state assessment, by item and block: 2013
BlockItemRange of response codesSample sizePercent agreement
M3M1459011-31,60096
M3M1461011-21,60097
M3M1463011-31,60098
M3M1464011-31,60097
M3M1466011-51,60087
M4M1468011-21,60098
M4M1469011-31,60098
M4M1470011-31,60094
M4M1481011-31,60090
M5M2185021-21,60099
M5M2185041-21,60099
M5M2185051-31,60094
M5M2188011-21,70099
M5M2189011-21,60099
M5M2195011-31,60092
M6M1365011-21,60098
M6M1366011-31,60093
M6M1369011-41,60096
M7M1567011-21,90099
M7M1570011-31,80099
M7M1573011-21,60098
M7M1594011-41,60095
M7M1607011-31,60099
M7M1624011-31,60084
M10M2200011-31,70099
M10M2204011-31,60092
M10M2207011-31,60089
M10M2211011-31,60097
M11M1561011-31,80098
M11M1575011-31,60098
M11M1610011-21,60099
M11M1610021-31,60095
M11M1616011-31,60096
M11M1619011-31,60097
M11M1622011-31,60097
NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.

Score range and Cohen's Kappa or intraclass correlation for the constructed-response items from the previous year 2011, rescored in 2013, grade 4 mathematics combined national and state assessment, by item and block: 2013
Block Item Range of response codes Sample size Cohen’s Kappa Intraclass correlation
M3 M145901 1-3 1,600 .94 .97
M3 M146101 1-2 1,600 .95 †
M3 M146301 1-3 1,600 .96 .98
M3 M146401 1-3 1,600 .95 .97
M3 M146601 1-5 1,600 .82 .95
M4 M146801 1-2 1,600 .99 †
M4 M146901 1-3 1,600 .98 .99
M4 M147001 1-3 1,600 .92 .95
M4 M148101 1-3 1,600 .76 .86
M5 M218502 1-2 1,600 .99 †
M5 M218504 1-2 1,600 .99 †
M5 M218505 1-3 1,600 .81 .92
M5 M218801 1-2 1,700 .98 †
M5 M218901 1-2 1,600 .95 †
M5 M219501 1-3 1,600 .87 .94
M6 M136501 1-2 1,600 .98 †
M6 M136601 1-3 1,600 .90 .94
M6 M136901 1-4 1,600 .94 .97
M7 M156701 1-2 1,900 .98 †
M7 M157001 1-3 1,800 .99 1.00
M7 M157301 1-2 1,600 .99 †
M7 M159401 1-4 1,600 .91 .97
M7 M160701 1-3 1,600 .99 .99
M7 M162401 1-3 1,600 .70 .82
M10 M220001 1-3 1,700 .97 .98
M10 M220401 1-3 1,600 .82 .81
M10 M220701 1-3 1,600 .77 .83
M10 M221101 1-3 1,600 .96 .98
M11 M156101 1-3 1,800 .97 .98
M11 M157501 1-3 1,600 .98 .98
M11 M161001 1-2 1,600 .98 †
M11 M161002 1-3 1,600 .90 .95
M11 M161601 1-3 1,600 .95 .97
M11 M161901 1-3 1,600 .94 .97
M11 M162201 1-3 1,600 .91 .92
† The Intraclass correlation is not reported for dichotomously scored items.
NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen"s Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation.
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.
Last updated 13 June 2017 (GF)
IES    
NCES
National Center for
Education Statistics
Explore the Institute of Education Sciences