NAEP Technical DocumentationComputation of Replicate Weights for the 2004 Assessment

Defining Replicate Strata and Forming Replicates

Computation of School-Level Replicate Base Weights

Computation of Student-Level Replicate Base Weights

Replicate Variance Estimation

In addition to the full-sample weight, a set of 62 replicate weights was provided for each student. These replicate weights are used in calculating the sampling variance of estimates obtained from the data, using the jackknife repeated replication method. (The theory that underlies the jackknife variance estimators used in NAEP studies is discussed on the web page Replicate Variance Estimation.) The method of deriving these weights was aimed at reflecting the features of the sample design appropriately for each sample, so that when the jackknife variance estimation procedure is implemented, approximately unbiased estimates of sampling variance are obtained. This page gives the specifics for generating the replicate weights for the 2004 assessment samples.

For each sample, replicates were formed in two steps. First, each school was assigned to one or more of 62 replicate strata. In the next step, a random subset of schools in each replicate stratum was dropped. The remaining subset and all schools in the other replicate strata then constituted one of the 62 replicates.

Of the 62 replicate weights formed for each student, 30 replicate weights act to reflect the amount of sampling variance contributed by the noncertainty strata of PSUs, while the remaining 32 replicate weights reflect the variance contribution of the certainty PSU samples.

Each replicate weight was calculated using weighting procedures similar to those used for the full-sample weight. The replicate base weights contain an additional component, known as a replicate factor, to account for the subsetting of the sample to form the replicate. By repeating the various weighting procedures on each set of replicate base weights, the impact of these procedures on the sampling variance of an estimate is appropriately reflected in the variance estimate.

Each of the 62 replicate weights can be expressed as follows:

where

STU_BWT_jsk(r) is the replicate student base weight for replicate r,
SCH_NRAF_js(r) is the school-level nonresponse adjustment factor for replicate r,
STU_NRAF_jk(r) is the student-level nonresponse adjustment factor for replicate r,
SCH_TRIM_js is the school-level weight trimming adjustment factor, and
STU_TRIM_jk is the student-level weight trimming adjustment factor.

The student adjustment factor does not include a subscript s to reflect the fact that not every student in school s falls into the same student nonresponse cell or the same training group.

Specific school and student nonresponse adjustment factors were calculated separately for each replicate, thus the use of the index (r), and applied to the replicate student base weights. Computing separate school and student nonresponse adjustment factors for each replicate is generally known as replicating the school and student nonresponse adjustment factors. This allows resulting variances from the use of the final student replicate weights to reflect components of variance due to the school and student nonresponse adjustments.

School and student weight trimming adjustments were not replicated, that is, not calculated separately for each replicate. Instead, each replicate used the school and student trimming adjustment factors derived for the full sample. Statistical theory for replicating trimming adjustments under the jackknife approach has not been developed in the literature. Due to the absence of a statistical framework, and since relatively few school and student weights in NAEP require trimming, the weight trimming adjustments were not replicated.

Last updated 26 October 2009 (JL)

Printer-friendly Version