- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Twitter
Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2015
NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items From Previous Year 2013, Rescored in 2015, Grade 4 Reading Combined National and State Assessment, by Item and Block: 2015
| Block | Item | Score range | Sample size | Percentage of exact agreement |
|---|---|---|---|---|
| R1 | R068904 | 1-2 | 1,900 | 92 |
| R068907 | 1-4 | 1,600 | 76 | |
| R068910 | 1-3 | 1,600 | 76 | |
| R2 | R057806 | 1-4 | 1,600 | 81 |
| R057807 | 1-3 | 1,900 | 93 | |
| R057809 | 1-3 | 1,600 | 86 | |
| R4 | R070204 | 1-3 | 1,900 | 87 |
| R070206 | 1-4 | 1,600 | 82 | |
| R070210 | 1-3 | 1,600 | 81 | |
| R6 | R058602 | 1-3 | 1,800 | 76 |
| R058606 | 1-3 | 1,700 | 89 | |
| R058608 | 1-4 | 1,600 | 80 | |
| R7 | R058805 | 1-3 | 1,600 | 85 |
| R058807 | 1-4 | 1,600 | 74 | |
| R058810 | 1-3 | 1,600 | 78 | |
| R9 | R058302 | 1-2 | 1,900 | 95 |
| R058304 | 1-3 | 1,700 | 92 | |
| R058307 | 1-4 | 1,600 | 81 | |
| R10 | R069305 | 1-2 | 1,600 | 91 |
| R069308 | 1-4 | 1,600 | 74 | |
| NOTE: Special codes assigned to student responses including blank,
off-task, and not-scorable were included in the calculation of the
percentage of exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment. | ||||
| Block | Item | Score range | Sample size | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|
| R1 | R068904 | 1-2 | 1,700 | .82 | † |
| R068907 | 1-4 | 1,400 | .62 | .78 | |
| R068910 | 1-3 | 1,400 | .59 | .76 | |
| R2 | R057806 | 1-4 | 1,400 | .68 | .81 |
| R057807 | 1-3 | 1,700 | .76 | .76 | |
| R057809 | 1-3 | 1,400 | .71 | .76 | |
| R4 | R070204 | 1-3 | 1,700 | .74 | .80 |
| R070206 | 1-4 | 1,400 | .69 | .87 | |
| R070210 | 1-3 | 1,400 | .66 | .76 | |
| R6 | R058602 | 1-3 | 1,600 | .60 | .72 |
| R058606 | 1-3 | 1,500 | .80 | .81 | |
| R058608 | 1-4 | 1,400 | .67 | .84 | |
| R7 | R058805 | 1-3 | 1,400 | .74 | .83 |
| R058807 | 1-4 | 1,400 | .61 | .81 | |
| R058810 | 1-3 | 1,400 | .61 | .74 | |
| R9 | R058302 | 1-2 | 1,700 | .92 | † |
| R058304 | 1-3 | 1,500 | .88 | .93 | |
| R058307 | 1-4 | 1,400 | .71 | .90 | |
| R10 | R069305 | 1-2 | 1,400 | .79 | † |
| R069308 | 1-4 | 1,400 | .59 | .80 | |
|
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment. | |||||
Last updated 15 November 2021 (ML)