NAEP Technical DocumentationProportion of Scale Score Variance Accounted for by the Population-Structure Models

For each assessment scale, the proportions of scale score variance accounted for by the population-structure model are given for each grade level. The numbers of principal components and group-defining variable contrasts used in the population-structure model are also provided.

The proportions in the tables indicate how strongly the scale score distributions are related to the groups to which students belong, as represented by the principal components. If the proportion of scale score variance accounted for by the population-structure model is one, all of the variability in the scores can be related to group membership as defined by the full set of variables in the population-structure model. The proportion of scale score variance accounted for by the population-structure model is

where the total variance includes sampling error, measurement error, and other sources of error.

Values of the proportion of scale score variance accounted for by the population-structure model range from approximately 0.50 to 0.98 in the national sample, depending on the subject area scale and sample for which it is calculated. The values are provided as documentation for each sample. However, for the purposes of NAEP, it is not necessary for the population-structure model to explain a high proportion of scale score variance. Student group membership cannot explain all the variance in the scale scores. The variation in the scale scores may also be due to individual ability variations which are not related to student group. There could also be purely random noise in the scale scores.

Links to proportion of scale score variance accounted for by the population-structure model, national and state assessments, by subject area and year: Various years, 2000–2018
Subject area	Year	National assessment	Grade 4 state assessment	Grade 8 state assessment	Grade 12 state assessment
Arts	2016	R3	†	†	†
Arts	2008	R3	†	†	†
Civics	2018	R3	†	†	†
	2014	R3	†	†	†
	2010	R3	†	†	†
	2006	R3	†	†	†
Economics	2012	R3	†	†	†
Economics	2006	R3	†	†	†
Geography	2018	R3	†	†	†
	2014	R3	†	†	†
	2010	R3	†	†	†
	2001	R2/R3	†	†	†
Mathematics	2017	R3	R3	R3	†
	2015	R3	R3	R3	†
	2013	R3	R3	R3	R3
	2011	R3	R3	R3	†
	2009	R3	R3	R3	R3
	2007	R3	R3	R3	†
	2005	R3	R3	R3	†
	2003	R3	R3	R3	†
	2000	R2/R3	R2/R3	R2/R3	†
Reading	2017	R3	R3	R3	†
	2015	R3	R3	R3	†
	2013	R3	R3	R3	R3
	2011	R3	R3	R3	†
	2009	R3	R3	R3	R3
	2007	R3	R3	R3	†
	2005	R3	R3	R3	†
	2003	R3	R3	R3	†
	2002	R3	R3	R3	†
	2000	R2/R3	†	†	†
Science	2015	R3	R3	R3	†
	2011	R3	†	R3	†
	2009	R3	R3	R3	†
	2005	R3	R3	R3	†
	2000	R2/R3	R2/R3	R2/R3	†
Technology and engineering literacy (TEL)	2018	R3	†	†	†
Technology and engineering literacy (TEL)	2014	R3	†	†	†
U.S. history	2018	R3	†	†	†
	2014	R3	†	†	†
	2010	R3	†	†	†
	2006	R3	†	†	†
	2001	R2/R3	†	†	†
Vocabulary	2015	R3	†	†	†
	2011	R3	†	†	†
	2009	R3	†	†	†
Writing	2011	R3	†	†	†
	2007	R3	†	—	—
	2002	R3	R3	R3	†
— Not available. † Not applicable; subject was not assessed at this grade in this year. NOTE: In NAEP, vocabulary, reading vocabulary, and meaning vocabulary refer to the same reporting scale. R2 is the non-accommodated reporting sample; R3 is the accommodated reporting sample. If sampled students are classified as students with disabilities (SD) or English learners (EL), and school officials, using NAEP guidelines, determine that they can meaningfully participate in the NAEP assessment with accommodation, those students are included in the NAEP assessment with accommodation along with other sampled students including SD/EL students who do not need accommodations. The R3 sample is more inclusive than the R2 sample type and excludes a smaller proportion of sampled students. The R3 sample is the only reporting sample used in NAEP after 2001. Because preliminary analyses of students' writing performance in the 2017 NAEP writing assessments at grades 4 and 8 revealed potentially confounding factors in measuring performance, results will not be publicly reported. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 2000–2018 Assessments.

Links to proportion of scale score variance accounted for by the population-structure model, long-term trend assessments, by subject area and year: 2004, 2008, and 2012
Subject area	Year	Assessment	National assessment
Mathematics	2012	Long-term trend	R3
Reading	2012	Long-term trend	R3
Mathematics	2008	Long-term trend	R3
Reading	2008	Long-term trend	R3
Mathematics	2004	Long-term trend	R3
Reading	2004	Long-term trend	R3
NOTE: R3 is the accommodated reporting sample. If sampled students are classified as students with disabilities (SD) or English learners (EL), and school officials, using NAEP guidelines, determine that they can meaningfully participate in the NAEP assessment with accommodation, those students are included in the NAEP assessment with accommodation along with other sampled students including SD/EL students who do not need accommodations. The R3 sample is more inclusive than the R2 sample type and excludes a smaller proportion of sampled students. The R3 sample is the only reporting sample used in NAEP after 2001. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004, 2008, and 2012 Long-Term Trend Assessments.

Last updated 02 November 2022 (SK)

Printer-friendly Version