- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Twitter
Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 8 Mathematics Combined National and State Assessment, by Item and Block: 2015
NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 8 Mathematics Combined National and State Assessment, by Item and Block: 2015
| Block | Item | Score range | Sample size | Percentage of exact agreement |
|---|---|---|---|---|
| M4 | M151501 | 1-3 | 2,100 | 94 |
| M151601 | 1-3 | 2,100 | 96 | |
| M152602 | 1-2 | 2,400 | 96 | |
| M5 | M221203 | 1-2 | 2,100 | 99 |
| M221204 | 1-3 | 2,100 | 88 | |
| M221701 | 1-3 | 2,100 | 100 | |
| M222301 | 1-5 | 2,100 | 99 | |
| M8 | M237901 | 1-3 | 2,800 | 99 |
| M239001 | 1-3 | 2,800 | 98 | |
| M317301 | 1-2 | 2,400 | 97 | |
| M317302 | 1-3 | 2,400 | 95 | |
| M9 | M234702 | 1-3 | 2,000 | 99 |
| M235501 | 1-2 | 2,000 | 99 | |
| M237201 | 1-5 | 2,000 | 99 | |
| M237202 | 1-3 | 2,000 | 99 | |
| M237203 | 1-2 | 2,000 | 98 | |
| M237204 | 1-2 | 2,000 | 97 | |
| M10 | M222701 | 1-2 | 1,900 | 96 |
| M223101 | 1-3 | 2,300 | 98 | |
| M223301 | 1-3 | 2,000 | 93 | |
| M223801 | 1-3 | 2,100 | 98 | |
| M11 | M104901 | 1-3 | 1,900 | 97 |
| M162901 | 1-3 | 2,200 | 97 | |
| M163101 | 1-3 | 2,100 | 94 | |
| M167901 | 1-3 | 2,000 | 98 | |
| M168301 | 1-3 | 2,100 | 96 | |
| M12 | M231501 | 1-3 | 2,300 | 99 |
| M233501 | 1-3 | 2,300 | 98 | |
| M234001 | 1-3 | 2,100 | 99 | |
| M234201 | 1-3 | 1,900 | 92 | |
| NOTE: Special codes assigned to student responses including blank,
off-task, and not-scorable were included in the calculation of the
percentage of exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment. | ||||
| Block | Item | Score range | Sample size | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|
| M4 | M151501 | 1-3 | 1,900 | .91 | .95 |
| M151601 | 1-3 | 1,700 | .85 | .93 | |
| M152602 | 1-5 | 2,100 | .95 | .96 | |
| M5 | M221203 | 1-3 | 1,800 | .97 | .98 |
| M221204 | 1-3 | 1,800 | .76 | .81 | |
| M221701 | 1-2 | 1,800 | .99 | † | |
| M222301 | 1-3 | 1,800 | .98 | .99 | |
| M8 | M237901 | 1-3 | 2,500 | .99 | 1.00 |
| M239001 | 1-3 | 2,600 | .97 | .98 | |
| M317301 | 1-3 | 2,000 | .97 | .99 | |
| M317302 | 1-3 | 1,700 | .77 | .89 | |
| M9 | M234702 | 1-3 | 1,800 | .99 | 1.00 |
| M235501 | 1-2 | 1,800 | .99 | † | |
| M237201 | 1-2 | 1,700 | .99 | † | |
| M237202 | 1-3 | 1,600 | .99 | .99 | |
| M237203 | 1-3 | 1,800 | .96 | .97 | |
| M237204 | 1-3 | 1,700 | .96 | .98 | |
| M10 | M222701 | 1-3 | 1,600 | .93 | .97 |
| M223101 | 1-3 | 2,100 | .97 | .99 | |
| M223301 | 1-3 | 1,700 | .88 | .92 | |
| M223801 | 1-3 | 1,800 | .98 | .99 | |
| M11 | M104901 | 1-3 | 1,700 | .93 | .96 |
| M162901 | 1-3 | 1,900 | .95 | .97 | |
| M163101 | 1-3 | 1,800 | .91 | .95 | |
| M167901 | 1-3 | 1,800 | .94 | .97 | |
| M168301 | 1-3 | 1,800 | .94 | .98 | |
| M12 | M231501 | 1-2 | 2,100 | .99 | † |
| M233501 | 1-2 | 2,100 | .98 | † | |
| M234001 | 1-2 | 1,900 | .98 | † | |
| M234201 | 1-3 | 1,600 | .80 | .88 | |
|
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment. | |||||
Last updated 09 November 2021 (ML)