NAEP Technical DocumentationIndices Based on Questionnaire Data

			2018 Civics Indices Geography Indices Technology and Engineering Literacy (TEL) Indices U.S. History Indices 2017 Grade 4 Mathematics Indices Grade 8 Mathematics Indices Grade 4 Reading Indices Grade 8 Reading Indices 2015 Mathematics Indices Reading Indices Science Indices 2014 Social Studies Indices Technology and Engineering Literacy (TEL) Indices

The NAEP survey questionnaires are given to students, teachers, and school administrators who participate in a NAEP assessment. These questionnaires collect contextual information that helps put student achievement results into context and allow for meaningful student group comparisons. 

Historically, NAEP has designed its survey questionnaires around single questions, and questionnaire results were therefore reported for single questions. In 2014, the program moved towards an enhanced survey questionnaire design and reporting approach to examine information of key interest to NAEP audiences. Specifically, while some survey questions are still analyzed and reported as single items (e.g., gender), several questions on the same topic are combined into indices measuring a single underlying construct or concept. This approach aims to provide both breadth and depth of coverage of the factors being examined.  

The current approach to survey questionnaire design and reporting directly addresses the National Assessment Governing Board's policy principles laid out in their 2012 policy statement¹, particularly the principles that "NAEP reporting should be enriched by greater use of contextual data derived from background or non-cognitive questions asked of students, teachers, and schools" (National Assessment Governing Board 2012, p. 2). It also aligns with the design and reporting approaches followed by large-scale international assessments (e.g., PISA, TIMSS, and PIRLS) and student surveys (e.g., Gallup Student Poll).  

The table below summarizes the difference between the historic and current approach in terms of both questionnaire design and reporting. 

 

The multi-item indices create more robust reporting of survey questionnaire results. Aggregating questions into indices helps minimize wording effects of individual contextual questions and increase measurement precision. The indices also enrich the trend data on important contextual student factors. 

In 2014 and 2015, indices were created from existing survey questions accompanying the social studies, mathematics, reading, and science assessments. Beginning with the 2014 technology and engineering literacy (TEL) assessment and 2017 mathematics and reading assessments, new survey questions were developed with the intention of developing indices to measure specific constructs of interest (e.g., students' enjoyment of complex problems or TEL confidence). More information about the development and creation of specific indices for the questionnaire data can be found by clicking on the page links in the sidebar on the right-hand side of the page. 

¹¹ National Assessment Governing Board. (2012). Policy statement on NAEP background questions and the use of contextual data in NAEP reporting. Washington, DC: U.S. Department of Education, National Assessment Governing Board. 

Printer-friendly Version