- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Twitter
NAEP Analysis and Scaling → Initial Activities → Constructed-Response Interrater Reliability → Range of Response Codes, Percent Agreement, and Cohen's Kappa or Intraclass Correlation for the Constructed-Response Items Used in Scaling, Grade 4 Mathematics Combined National and State Assessments, by Item and Block: 2011
| Block | Item | Range of response codes |
Sample size | Percent exact agreement |
|---|---|---|---|---|
| NOTE: Special codes assigned to student responses including blank, off-task, and not-scorable were included in the calculation of the Percent Exact Agreement. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2011 Mathematics Assessment. |
||||
| M3 | M145901 | 1-3 | 2,100 | 97 |
| M146101 | 1-2 | 2,100 | 97 | |
| M146301 | 1-3 | 2,100 | 98 | |
| M146401 | 1-3 | 2,100 | 96 | |
| M146601 | 1-5 | 2,100 | 87 | |
| M4 | M146801 | 1-2 | 2,100 | 100 |
| M146901 | 1-3 | 2,100 | 98 | |
| M147001 | 1-3 | 2,100 | 93 | |
| M148101 | 1-3 | 2,100 | 90 | |
| M5 | M218502 | 1-2 | 2,100 | 99 |
| M218504 | 1-2 | 2,100 | 100 | |
| M218505 | 1-3 | 2,100 | 96 | |
| M218801 | 1-2 | 2,100 | 99 | |
| M218901 | 1-2 | 2,100 | 99 | |
| M219501 | 1-3 | 2,100 | 95 | |
| M6 | M136501 | 1-2 | 2,300 | 96 |
| M136601 | 1-3 | 2,300 | 92 | |
| M136951 | 1-3 | 2,300 | 93 | |
| M7 | M156701 | 1-2 | 2,100 | 99 |
| M157001 | 1-3 | 2,100 | 99 | |
| M157301 | 1-2 | 2,100 | 98 | |
| M159401 | 1-4 | 2,100 | 96 | |
| M160701 | 1-3 | 2,100 | 99 | |
| M162401 | 1-3 | 2,100 | 85 | |
| M8 | M010631 | 1-3 | 2,300 | 95 |
| M040001 | 1-3 | 2,300 | 96 | |
| M072401 | 1-3 | 2,300 | 93 | |
| M087001 | 1-3 | 2,300 | 95 | |
| M087301 | 1-5 | 2,300 | 94 | |
| M091201 | 1-3 | 2,300 | 93 | |
| M9 | M155101 | 1-5 | 2,100 | 95 |
| M157601 | 1-3 | 2,100 | 89 | |
| M160901 | 1-3 | 2,100 | 94 | |
| M161201 | 1-3 | 2,100 | 96 | |
| M162001 | 1-3 | 2,100 | 99 | |
| M10 | M220001 | 1-3 | 2,100 | 99 |
| M220401 | 1-3 | 2,100 | 96 | |
| M220701 | 1-3 | 2,100 | 93 | |
| M221101 | 1-3 | 2,100 | 98 | |
| M11 | M156101 | 1-3 | 2,300 | 94 |
| M157501 | 1-3 | 2,300 | 95 | |
| M161001 | 1-2 | 2,300 | 97 | |
| M161002 | 1-3 | 2,300 | 93 | |
| M161601 | 1-3 | 2,300 | 92 | |
| M161901 | 1-3 | 2,300 | 94 | |
| M162201 | 1-3 | 2,300 | 96 | |
| M12 | M148501 | 1-3 | 2,300 | 94 |
| M149201 | 1-3 | 2,300 | 94 | |
| M149301 | 1-2 | 2,300 | 93 | |
| M149701 | 1-5 | 2,300 | 91 | |
| Block | Item | Range of response codes |
Sample size | Cohen’s Kappa | Intraclass correlation |
|---|---|---|---|---|---|
| † Not applicable. The intraclass correlation is not reported for dichotomously scored items. NOTE: Cohen's Kappa is a measure of reliability that is appropriate for items that are dichotomously scored. The Intraclass Correlation Coefficient is most appropriate for items with more than two categories. Special codes assigned to student responses including blank, off-task, and not-scorable were not included in the calculation of Cohen's Kappa or intraclass correlation. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2011 Mathematics Assessment. |
|||||
| M3 | M145901 | 1-3 | 2,100 | 0.95 | 0.98 |
| M146101 | 1-2 | 2,000 | 0.94 | † | |
| M146301 | 1-3 | 2,000 | 0.96 | 0.99 | |
| M146401 | 1-3 | 2,000 | 0.93 | 0.95 | |
| M146601 | 1-5 | 2,000 | 0.83 | 0.96 | |
| M4 | M146801 | 1-2 | 2,000 | 1.00 | † |
| M146901 | 1-3 | 2,100 | 0.98 | 0.99 | |
| M147001 | 1-3 | 2,000 | 0.90 | 0.93 | |
| M148101 | 1-3 | 1,900 | 0.77 | 0.85 | |
| M5 | M218502 | 1-2 | 2,000 | 0.99 | † |
| M218504 | 1-2 | 2,100 | 0.99 | † | |
| M218505 | 1-3 | 2,000 | 0.78 | 0.89 | |
| M218801 | 1-2 | 2,000 | 0.99 | † | |
| M218901 | 1-2 | 2,000 | 0.96 | † | |
| M219501 | 1-3 | 1,900 | 0.92 | 0.96 | |
| M6 | M136501 | 1-2 | 2,200 | 0.99 | † |
| M136601 | 1-3 | 2,100 | 0.92 | 0.96 | |
| M136951 | 1-3 | 2,100 | 0.94 | 0.97 | |
| M7 | M156701 | 1-2 | 2,100 | 0.99 | † |
| M157001 | 1-3 | 2,100 | 0.99 | 1.00 | |
| M157301 | 1-2 | 2,000 | 0.98 | † | |
| M159401 | 1-4 | 1,800 | 0.92 | 0.97 | |
| M160701 | 1-3 | 2,000 | 0.99 | 0.99 | |
| M162401 | 1-3 | 2,000 | 0.73 | 0.85 | |
| M8 | M010631 | 1-3 | 2,200 | 0.97 | 0.98 |
| M040001 | 1-3 | 2,100 | 0.97 | 0.99 | |
| M072401 | 1-3 | 1,600 | 0.90 | 0.95 | |
| M087001 | 1-3 | 2,200 | 0.97 | 0.99 | |
| M087301 | 1-5 | 1,300 | 0.89 | 0.96 | |
| M091201 | 1-3 | 1,800 | 0.91 | 0.96 | |
| M9 | M155101 | 1-5 | 2,000 | 0.92 | 0.96 |
| M157601 | 1-3 | 2,000 | 0.75 | 0.84 | |
| M160901 | 1-3 | 2,000 | 0.75 | 0.85 | |
| M161201 | 1-3 | 2,100 | 0.92 | 0.95 | |
| M162001 | 1-3 | 2,100 | 0.99 | 1.00 | |
| M10 | M220001 | 1-3 | 2,100 | 0.97 | 0.98 |
| M220401 | 1-3 | 2,100 | 0.89 | 0.90 | |
| M220701 | 1-3 | 2,100 | 0.85 | 0.89 | |
| M221101 | 1-3 | 2,000 | 0.97 | 0.99 | |
| M11 | M156101 | 1-3 | 2,200 | 0.95 | 0.97 |
| M157501 | 1-3 | 2,100 | 0.96 | 0.98 | |
| M161001 | 1-2 | 1,900 | 0.99 | † | |
| M161002 | 1-3 | 1,900 | 0.90 | 0.96 | |
| M161601 | 1-3 | 2,200 | 0.94 | 0.96 | |
| M161901 | 1-3 | 2,200 | 0.96 | 0.98 | |
| M162201 | 1-3 | 2,100 | 0.90 | 0.91 | |
| M12 | M148501 | 1-3 | 2,200 | 0.97 | 0.99 |
| M149201 | 1-3 | 2,000 | 0.93 | 0.95 | |
| M149301 | 1-2 | 2,100 | 0.93 | † | |
| M149701 | 1-5 | 1,900 | 0.90 | 0.96 | |
Last updated 06 January 2016 (GF)