- Surveys & Programs +
- Data & Tools
+
- Downloads Microdata/Raw Data
- Downloads Microdata/Raw Data
- EDAT Delta Cost Project IPEDS Data Center How to apply for Restricted Use License
- Online Analysis
- School and College Search
- School and College Search
-
Search for Schools and Colleges
College Navigator
Public Schools Public School Districts Private Schools
- Comparison Tools
- Questionnaire Tools
- Questionnaire Tools
- NAEP Question Tool NAAL Questions Tool
- Mapping
- Mapping
- MapEd
- Other Tools
- Other Tools
- Bibliography
- Downloads Microdata/Raw Data
- Fast Facts +
- News & Events +
- Publications & Products +
- About Us
+
- About Us
- About NCES Commissioner Contact NCES Staff Help
Twitter
Table of Contents | Search Technical Documentation | References
HomeTDWNAEP Analysis and ScalingInitial ActivitiesConstructed-Response Interrater ReliabilityScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Reading Combined National and State Assessment, by Item and Block: 2015
NAEP Technical DocumentationScore Range and Percentage of Exact Agreement for the Constructed-Response Items Used in Scaling, Grade 8 Reading Combined National and State Assessment, by Item and Block: 2015
| Block | Item | Score range | Sample size | Percentage of exact agreement |
|---|---|---|---|---|
| R1 | R059604 | 1-2 | 1,100 | 92 |
| R059605 | 1-3 | 1,100 | 78 | |
| R059606 | 1-3 | 1,100 | 85 | |
| R059607 | 1-3 | 1,100 | 90 | |
| R059610 | 1-4 | 1,100 | 83 | |
| R2 | R069801 | 1-3 | 1,100 | 91 |
| R069804 | 1-3 | 1,100 | 80 | |
| R069806 | 1-4 | 1,000 | 76 | |
| R069810 | 1-3 | 1,100 | 84 | |
| R3 | R060103 | 1-2 | 1,100 | 85 |
| R060105 | 1-2 | 1,100 | 94 | |
| R060108 | 1-4 | 1,100 | 75 | |
| R4 | R080005 | 1-4 | 1,100 | 86 |
| R080007 | 1-2 | 1,100 | 90 | |
| R080009 | 1-3 | 1,100 | 86 | |
| R080010 | 1-3 | 1,100 | 84 | |
| R5 | R079504 | 1-3 | 1,100 | 85 |
| R079507 | 1-4 | 1,100 | 84 | |
| R079508 | 1-2 | 1,100 | 95 | |
| R079510 | 1-3 | 1,100 | 81 | |
| R6 | R058602 | 1-3 | 1,100 | 81 |
| R058606 | 1-3 | 1,100 | 84 | |
| R058608 | 1-4 | 1,100 | 75 | |
| R7 | R058805 | 1-3 | 1,100 | 88 |
| R058807 | 1-4 | 1,100 | 78 | |
| R058810 | 1-3 | 1,100 | 80 | |
| R8 | R079703 | 1-3 | 1,100 | 79 |
| R079706 | 1-3 | 1,100 | 97 | |
| R079707 | 1-4 | 1,100 | 86 | |
| R079709 | 1-3 | 1,100 | 83 | |
| R9 | R060503 | 1-3 | 1,100 | 92 |
| R060505 | 1-4 | 1,000 | 79 | |
| R060507 | 1-3 | 1,100 | 73 | |
| R060508 | 1-3 | 1,100 | 78 | |
| R060509 | 1-3 | 1,100 | 85 | |
| R10 | R060603 | 1-3 | 1,100 | 83 |
| R060605 | 1-3 | 1,100 | 86 | |
| R060606 | 1-4 | 1,100 | 82 | |
| R060607 | 1-3 | 1,100 | 79 | |
| R060608 | 1-3 | 1,100 | 82 | |
| R11 | R069603 | 1-3 | 1,000 | 92 |
| R069606 | 1-3 | 1,100 | 84 | |
| R069607 | 1-4 | 1,100 | 80 | |
| R069609 | 1-3 | 1,100 | 85 | |
| R12 | R061003 | 1-3 | 1,100 | 85 |
| R061004 | 1-2 | 1,100 | 94 | |
| R061006 | 1-2 | 1,100 | 90 | |
| R061007 | 1-4 | 1,100 | 73 | |
| R061010 | 1-3 | 1,100 | 83 | |
| R13 | R069705 | 1-2 | 1,100 | 90 |
| R069706 | 1-3 | 1,100 | 86 | |
| R069707 | 1-4 | 1,100 | 77 | |
| R069708 | 1-3 | 1,100 | 80 | |
| NOTE: Special codes assigned to student responses including blank,
off-task, and not-scorable were included in the calculation of the
percentage of exact agreement measure. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment. | ||||
| Block | Item | Score range | Sample size | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|
| R1 | R059604 | 1-2 | 1,000 | .73 | † |
| R059605 | 1-3 | 900 | .60 | .72 | |
| R059606 | 1-3 | 1,000 | .73 | .84 | |
| R059607 | 1-3 | 900 | .80 | .89 | |
| R059610 | 1-4 | 1,000 | .75 | .89 | |
| R2 | R069801 | 1-3 | 1,000 | .80 | .82 |
| R069804 | 1-3 | 1,000 | .65 | .77 | |
| R069806 | 1-4 | 1,000 | .60 | .75 | |
| R069810 | 1-3 | 900 | .73 | .85 | |
| R3 | R060103 | 1-2 | 1,000 | .66 | † |
| R060105 | 1-2 | 1,000 | .87 | † | |
| R060108 | 1-4 | 1,000 | .63 | .80 | |
| R4 | R080005 | 1-4 | 1,000 | .79 | .92 |
| R080007 | 1-2 | 1,000 | .75 | † | |
| R080009 | 1-3 | 800 | .71 | .78 | |
| R080010 | 1-3 | 800 | .64 | .75 | |
| R5 | R079504 | 1-3 | 1,100 | .74 | .84 |
| R079507 | 1-4 | 1,000 | .73 | .89 | |
| R079508 | 1-2 | 1,000 | .83 | † | |
| R079510 | 1-3 | 1,000 | .67 | .79 | |
| R6 | R058602 | 1-3 | 1,000 | .69 | .76 |
| R058606 | 1-3 | 1,000 | .75 | .79 | |
| R058608 | 1-4 | 1,000 | .62 | .83 | |
| R7 | R058805 | 1-3 | 1,000 | .78 | .83 |
| R058807 | 1-4 | 1,000 | .64 | .80 | |
| R058810 | 1-3 | 1,000 | .64 | .73 | |
| R8 | R079703 | 1-3 | 1,000 | .67 | .74 |
| R079706 | 1-3 | 1,000 | .93 | .97 | |
| R079707 | 1-4 | 1,000 | .79 | .91 | |
| R079709 | 1-3 | 1,000 | .60 | .69 | |
| R9 | R060503 | 1-3 | 1,000 | .88 | .92 |
| R060505 | 1-4 | 1,000 | .69 | .87 | |
| R060507 | 1-3 | 1,000 | .56 | .57 | |
| R060508 | 1-3 | 1,000 | .58 | .64 | |
| R060509 | 1-3 | 900 | .73 | .83 | |
| R10 | R060603 | 1-3 | 1,000 | .71 | .82 |
| R060605 | 1-3 | 1,000 | .74 | .80 | |
| R060606 | 1-4 | 1,000 | .63 | .72 | |
| R060607 | 1-3 | 1,100 | .54 | .62 | |
| R060608 | 1-3 | 1,000 | .72 | .81 | |
| R11 | R069603 | 1-3 | 1,000 | .86 | .91 |
| R069606 | 1-3 | 1,000 | .69 | .75 | |
| R069607 | 1-4 | 1,000 | .68 | .83 | |
| R069609 | 1-3 | 900 | .74 | .85 | |
| R12 | R061003 | 1-3 | 1,000 | .77 | .85 |
| R061004 | 1-2 | 1,000 | .78 | † | |
| R061006 | 1-2 | 1,000 | .79 | † | |
| R061007 | 1-4 | 900 | .52 | .70 | |
| R061010 | 1-3 | 800 | .60 | .70 | |
| R13 | R069705 | 1-2 | 1,000 | .74 | † |
| R069706 | 1-3 | 1,000 | .76 | .84 | |
| R069707 | 1-4 | 900 | .59 | .82 | |
| R069708 | 1-3 | 900 | .65 | .75 | |
|
† Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: The intraclass correlation is most appropriate for items with more than two categories. Cohen's Kappa is a measure of reliability that is appropriate for both dichotomously and polytomously scored items. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Reading Assessment. | |||||
Last updated 09 November 2021 (ML)