NAEP Technical DocumentationA Numerical Example

To illustrate how plausible values are used in subsequent analyses, this subsection gives some of the steps in the calculation of the 1992 grade 4 reading composite mean and its estimation-error variance. This illustration is an example of the calculation of NAEP means and variances and can be used to understand their calculation for any NAEP assessment.

The weighted mean of the first plausible values of the reading composite for the grade 4 students in the sample is 217.79, and the jackknife variance of these values is 0.833. Were these values true θ values, then 217.79 would be the estimate of the mean and 0.833 would be the estimation-error variance. The weighted mean of the second plausible values of the same students, however, is 217.62; the third, fourth, and fifth plausible values give weighted means of 217.74, 218.24, and 218.05. Since all of these figures are based on precisely the same sample of students, the variation among them is due to uncertainty about the students' θs, having observed their item responses and background variables. Consequently, our best estimate of the mean for grade 4 students is the average of the five plausible values: 217.89. Taking the jackknife variance estimate from the first plausible value, 0.833, as our estimate U* of sampling variance and the variance among the five weighted means, .063, as our estimate B of uncertainty due to not observing θ, we obtain as the final estimate V of total error variance 0.833 + (1 + 5^-1) 0.063 = 0.909.

It is also possible to partition the estimation error variance of a statistic using these same variance components. The proportion of error variance due to sampling students from the population is U*/V, and the proportion due to the latent nature of θ is (1 + M ^-1)B/V. The results are shown in the table. The value of U*/V roughly corresponds to reliability in classical test theory and indicates the amount of information about an average individual's θ present in the observed responses of the individual. It should be recalled again that the objective of NAEP is not to estimate and compare values of individual examinees, the accuracy of which is gauged by reliability coefficients. The objective of NAEP, rather, is to estimate population and subpopulation characteristics, and the marginal estimation methods in NAEP analyses have been designed to do so consistently regardless of the values of reliability coefficients.

Estimation error variance and related coefficients for the composite (based on five plausible values), grade 4 reading national main assessment: 1992
U*	Proportion of variance due to...
	Student sampling: U*/V	Latency of θ: (1 + 5^-1)B/V
	0.833	0.076	0.908	0.921	0.080
SOURCE: NAEP Reading Assessment: 2000

Last updated 25 February 2008 (TS)

Printer-friendly Version