Table of Contents  |  Search Technical Documentation  |  References

NAEP Analysis and Scaling → Drawing Inferences from NAEP Results → Comparison of Two Groups → Estimation of the Degrees of Freedom

NAEP Technical DocumentationEstimation of the Degrees of Freedom

Because of clustering and differential weighting in the sample, the degrees of freedom are less than for a simple random sample of the same size. The degrees of freedom of this t test are defined by a Satterthwaite (Johnson and Rust 1992) approximation as follows:

where N is the number of student groups involved, and the estimate df_Ak is as follows:

where m is the number of jackknife replicates (usually 62 in NAEP), t_j is the j^th replicated estimate for the mean of a student group, and t_k is the estimate of the group mean using the overall student group weights and the first plausible value.

The number of degrees of freedom for the variance equals the number of independent pieces of information used to generate the variance. In the case of data from NAEP, the 62 pieces of information are the squared differences (t_jk-t_k)², each supplying at most one degree of freedom (regardless of how many individuals were sampled within primary sampling units (PSUs). If some of the squared differences (t_jk-t_k)² are much larger than others, the variance estimate of m_k is predominantly estimating the sum of these larger components, which dominate the remaining terms. The effective degrees of freedom of S_Ak in this case will be nearer to the number of dominant terms. The estimate df_Ak reflects these relationships.

The two formulas above illustrate that when the estimate df_Ak is small, the degrees of freedom for the t test, df, will also be small. This will tend to be the case when only a few PSU pairs have information about student group differences relevant to a t test. It will also be the case when a few PSU pairs have group differences much larger than other PSU pairs.

Printer-friendly Version