Table of Contents | Search Technical Documentation | References

The NAEP Database → Database Quality Control: 2013 to 2017 → 2017 Summary Comparison Tables

NAEP Technical Documentation2017 Summary Comparison Tables

The purpose of the tables below is to summarize the differences in percentages across item-response categories for the 2017 NAEP assessments. For each subject's instrument, the presented statistics include the differences in item response-category percentages across all the items in a group. In some circumstances, an item may have more than one response option or prompt. The percentages presented include the minimum, maximum, average, and median difference, all of which are expressed based upon the absolute value of the difference at the item-response level.

Summary of quality control error analysis for data processing of cognitive blocks in student test forms, by grade and assessment: 2017
Assessment	Grade	Number of response options/prompts	Average number of respondents in Contractor 2 database per option/prompt	Average difference in number of respondents between Contractor 2 database and Contractor 1 scored data	Difference in percentage differences at the response-category level
Assessment	Grade	Number of response options/prompts			Minimum	Maximum¹	Average	Median
Reading	4	97	29,684	672	#	1.70	0.27	0.18
Reading	8	130	21,769	776	#	1.20	0.12	0.06
Mathematics	4	430	29,653	1,231	#	3.29	0.12	0.05
Mathematics	8	419	28,718	1,281	#	2.94	0.06	0.03
Mathematics in Puerto Rico	4	522	351	1	#	4.00	0.29	0.01
Mathematics in Puerto Rico	8	624	481	1	#	3.39	0.13	#
# Rounds to zero. ¹ A component of digital assessment quality control is a cross-validation process which compares captured data to actual keystroke data. This activity revealed some misalignment of captured data, in part due to students' use of the elimination tool to select responses for multiple-choice items. Therefore some moderately higher rates of differences in percentages for some students exist. NOTE: Average difference in number of respondents is the average, across all response options, of the absolute value of the difference in the number of respondents for each response option in Contractor 1's file and the number in Contractor 2's file. In 2017, grade 12 was not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2017 Assessment.

Summary of quality control error analysis for data processing of student survey questionnaires, by grade and assessment: 2017
Assessment	Grade	Number of response options/prompts	Average number of respondents in Contractor 2 database per option/prompt	Average difference in number of respondents between Contractor 2 database and Contractor 1 scored data	Difference in percentage differences at the response-category level
Assessment	Grade	Number of response options/prompts			Minimum	Maximum	Average	Median
Core (reading and mathematics)	4	49	252,907	12,239	#	2.12	0.26	0.12
Core (reading and mathematics)	8	53	236,971	8,998	#	2.84	0.39	0.11
Reading	4	30	136,347	458	#	0.26	0.05	0.02
Reading	8	49	122,778	147	#	0.11	0.01	0.01
Mathematics	4	22	136,530	1,506	0.03	0.53	0.26	0.22
Mathematics	8	40	130,256	2,731	#	1.98	0.02	#
Mathematics in Puerto Rico	4	85	2,993	5	#	0.25	0.02	#
Mathematics in Puerto Rico	8	112	3,139	0	#	0.35	0.06	0.03
# Rounds to zero. NOTE: Average difference in number of respondents is the average, across all response options, of the absolute value of the difference in the number of respondents for each response option in Contractor 1's file and the number in Contractor 2's file. In 2017, grade 12 was not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2017 Assessment.

Summary of quality control error analysis for data processing of teacher questionnaires, by grade and assessment: 2017
Assessment	Grade	Number of response options/prompts	Average number of respondents in Contractor 2 database per option/prompt	Average difference in number of respondents between Contractor 2 database and Contractor 1 scored data	Difference in percentage differences at the response-category level
Assessment	Grade	Number of response options/prompts			Minimum	Maximum	Average	Median
Reading and mathematics	4	173	28,225	9,552	#	6.68	0.70	0.39
Core (reading and mathematics)	8	66	30,880	8,346	#	3.80	0.64	0.46
Reading	8	49	15,457	9,138	#	11.25	0.96	0.62
Mathematics	8	53	14,763	11,101	#	13.62	1.21	0.90
Mathematics in Puerto Rico	4	117	229	129	#	19.51	1.97	1.27
Mathematics in Puerto Rico	8	119	197	100	#	24.59	3.26	2.08
# Rounds to zero. NOTE: Average difference in number of respondents is the average, across all response options, of the absolute value of the difference in the number of respondents for each response option in Contractor 1's file and the number in Contractor 2's file. In 2017, grade 12 was not assessed. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2017 Assessment.

Although not shown separately, the results for the Mathematics assessment in Puerto Rico are included in the following table. The school questions in Puerto Rico are the same set of questions that is administered in the operational Mathematics assessment.

Summary of quality control error analysis for data processing of school questionnaires, by grade and assessment: 2017
Assessment	Grade	Number of response options/prompts	Average number of respondents in Contractor 2 database per option/prompt	Average difference in number of respondents between Contractor 2 database and Contractor 1 scored data	Difference in percentage differences at the response-category level
Assessment	Grade	Number of response options/prompts			Minimum	Maximum	Average	Median
Reading and mathematics	4	170	9,017	150	#	1.94	0.19	0.08
Reading and mathematics	8	157	8,011	82	#	4.71	0.06	#
# Rounds to zero. NOTE: Average difference in number of respondents is the average, across all response options, of the absolute value of the difference in the number of respondents for each response option in Contractor 1's file and the number in Contractor 2's file. In 2017, grade 12 was not assessed. Results for the Mathematics assessment in Puerto Rico are included in the counts in this table. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2017 Assessment.

Last updated 08 August 2022 (ML)

Printer-friendly Version