- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Twitter
Table of Contents | Search Technical Documentation | References
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Score Range and Percent Agreement for the Constructed-Response Items Used in Scaling, Grade 12 Reading Combined National and State Assessment, by Item and Block: 2013
| Block | Item | Range of response codes | Sample size | Percent exact agreement |
|---|---|---|---|---|
| NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment. | ||||
| R1 | R059604 | 1-2 | 6,600 | .93 |
| R1 | R059605 | 1-3 | 6,600 | .78 |
| R1 | R059606 | 1-3 | 6,600 | .88 |
| R1 | R059607 | 1-3 | 6,600 | .91 |
| R1 | R059610 | 1-4 | 6,600 | .81 |
| R2 | R069801 | 1-3 | 6,500 | .93 |
| R2 | R069804 | 1-3 | 6,500 | .84 |
| R2 | R069806 | 1-4 | 6,500 | .74 |
| R2 | R069810 | 1-3 | 6,500 | .83 |
| R3 | R060103 | 1-2 | 6,600 | .86 |
| R3 | R060105 | 1-2 | 6,600 | .93 |
| R3 | R060108 | 1-4 | 6,600 | .76 |
| R4 | R060301 | 1-2 | 6,500 | .87 |
| R4 | R060305 | 1-3 | 6,500 | .86 |
| R4 | R060306 | 1-4 | 6,500 | .74 |
| R4 | R060309 | 1-3 | 6,500 | .81 |
| R5 | R061602 | 1-3 | 6,600 | .89 |
| R5 | R061604 | 1-3 | 6,600 | .84 |
| R5 | R061606 | 1-3 | 6,600 | .82 |
| R5 | R061607 | 1-4 | 6,600 | .73 |
| R5 | R061608 | 1-2 | 6,600 | .90 |
| R6 | R078903 | 1-3 | 6,600 | .86 |
| R6 | R078905 | 1-4 | 6,600 | .78 |
| R6 | R078907 | 1-2 | 6,600 | .91 |
| R7 | R062203 | 1-3 | 6,600 | .87 |
| R7 | R062205 | 1-2 | 6,600 | .94 |
| R7 | R062207 | 1-4 | 6,600 | .77 |
| R7 | R062210 | 1-3 | 6,600 | .85 |
| R8 | R062303 | 1-3 | 6,500 | .84 |
| R8 | R062304 | 1-3 | 6,500 | .86 |
| R8 | R062308 | 1-4 | 6,500 | .78 |
| R8 | R062311 | 1-3 | 6,500 | .80 |
| R9 | R062501 | 1-3 | 6,600 | .83 |
| R9 | R062503 | 1-3 | 6,600 | .77 |
| R9 | R062505 | 1-3 | 6,600 | .79 |
| R9 | R062508 | 1-4 | 6,600 | .67 |
| R10 | R062705 | 1-3 | 6,500 | .78 |
| R10 | R062706 | 1-4 | 6,500 | .75 |
| R10 | R062708 | 1-2 | 6,500 | .93 |
| R10 | R062710 | 1-3 | 6,500 | .84 |
| R11 | R062902 | 1-3 | 6,500 | .81 |
| R11 | R062903 | 1-3 | 6,500 | .75 |
| R11 | R062908 | 1-4 | 6,500 | .70 |
| R11 | R062909 | 1-3 | 6,500 | .74 |
| R12 | R063101 | 1-3 | 6,600 | .84 |
| R12 | R063105 | 1-4 | 6,600 | .85 |
| R12 | R063109 | 1-3 | 6,600 | .97 |
| R12 | R063801 | 1-3 | 6,600 | .83 |
| R13 | R079101 | 1-3 | 6,600 | .81 |
| R13 | R079106 | 1-4 | 6,600 | .76 |
| R13 | R079107 | 1-2 | 6,600 | .91 |
| R13 | R079110 | 1-3 | 6,600 | .92 |
| R13 | R079111 | 1-3 | 6,600 | .82 |
| Block | Item | Range of response codes | Sample size | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|
| † The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen’s Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment. | |||||
| R1 | R059604 | 1-2 | 6,600 | .62 | † |
| R1 | R059605 | 1-3 | 6,100 | .61 | .73 |
| R1 | R059606 | 1-3 | 6,300 | .74 | .85 |
| R1 | R059607 | 1-3 | 6,100 | .78 | .88 |
| R1 | R059610 | 1-4 | 6,100 | .70 | .85 |
| R2 | R069801 | 1-3 | 6,400 | .85 | .87 |
| R2 | R069804 | 1-3 | 6,300 | .72 | .80 |
| R2 | R069806 | 1-4 | 6,100 | .57 | .73 |
| R2 | R069810 | 1-3 | 6,200 | .73 | .83 |
| R3 | R060103 | 1-2 | 6,500 | .59 | † |
| R3 | R060105 | 1-2 | 6,400 | .82 | † |
| R3 | R060108 | 1-4 | 6,200 | .64 | .83 |
| R4 | R060301 | 1-2 | 6,500 | .59 | † |
| R4 | R060305 | 1-3 | 6,400 | .65 | .70 |
| R4 | R060306 | 1-4 | 6,300 | .63 | .74 |
| R4 | R060309 | 1-3 | 6,200 | .68 | .75 |
| R5 | R061602 | 1-3 | 6,400 | .80 | .89 |
| R5 | R061604 | 1-3 | 6,400 | .71 | .76 |
| R5 | R061606 | 1-3 | 5,900 | .69 | .84 |
| R5 | R061607 | 1-4 | 6,200 | .57 | .79 |
| R5 | R061608 | 1-2 | 6,000 | .58 | † |
| R6 | R078903 | 1-3 | 6,300 | .78 | .88 |
| R6 | R078905 | 1-4 | 6,200 | .66 | .81 |
| R6 | R078907 | 1-2 | 6,100 | .81 | † |
| R7 | R062203 | 1-3 | 6,300 | .72 | .72 |
| R7 | R062205 | 1-2 | 6,300 | .89 | † |
| R7 | R062207 | 1-4 | 5,900 | .60 | .81 |
| R7 | R062210 | 1-3 | 5,900 | .66 | .72 |
| R8 | R062303 | 1-3 | 6,200 | .69 | .74 |
| R8 | R062304 | 1-3 | 6,100 | .72 | .80 |
| R8 | R062308 | 1-4 | 5,900 | .65 | .85 |
| R8 | R062311 | 1-3 | 5,700 | .62 | .76 |
| R9 | R062501 | 1-3 | 6,400 | .64 | .77 |
| R9 | R062503 | 1-3 | 5,900 | .61 | .77 |
| R9 | R062505 | 1-3 | 6,200 | .61 | .70 |
| R9 | R062508 | 1-4 | 6,000 | .50 | .72 |
| R10 | R062705 | 1-3 | 6,200 | .63 | .75 |
| R10 | R062706 | 1-4 | 6,200 | .64 | .85 |
| R10 | R062708 | 1-2 | 6,200 | .81 | † |
| R10 | R062710 | 1-3 | 6,000 | .73 | .79 |
| R11 | R062902 | 1-3 | 6,300 | .68 | .76 |
| R11 | R062903 | 1-3 | 6,300 | .59 | .72 |
| R11 | R062908 | 1-4 | 5,800 | .53 | .75 |
| R11 | R062909 | 1-3 | 5,400 | .52 | .67 |
| R12 | R063101 | 1-3 | 6,300 | .74 | .85 |
| R12 | R063105 | 1-4 | 5,600 | .73 | .87 |
| R12 | R063109 | 1-3 | 5,900 | .95 | .98 |
| R12 | R063801 | 1-3 | 6,200 | .72 | .82 |
| R13 | R079101 | 1-3 | 6,300 | .68 | .82 |
| R13 | R079106 | 1-4 | 6,000 | .64 | .84 |
| R13 | R079107 | 1-2 | 6,300 | .81 | † |
| R13 | R079110 | 1-3 | 6,000 | .85 | .91 |
| R13 | R079111 | 1-3 | 5,900 | .69 | .81 |
Last updated 21 May 2017 (DC)