Table of Contents  |  Search Technical Documentation  |  References

NAEP Weighting Procedures → 2003 Weighting Procedures and Variance Estimation → Trimming of Student Weights

NAEP Technical DocumentationTrimming of Student Weights

			Distribution of Student Weight Trimming Factors for Student Groups

Student weight trimming is a weighting adjustment procedure that involves detecting and reducing extremely large student weights. Unusually large weights are likely to produce large sampling variances of statistics of interest, especially when the large weights are associated with sample cases reflective of rare or atypical characteristics. To reduce the impact of these large weights on variances, weight reduction methods are typically employed. The motivation behind weight reduction methods is to reduce the mean square error of survey estimates. While the trimming of large weights reduces variances, it also introduces a small bias. However, it is presumed that the reduction in the variances more than compensates for the increase in the bias, thereby reducing the mean square error and thus improving the accuracy of survey estimates.

Large student weights generally result from compounding nonresponse adjustments at the school and student levels with artificially low first-stage selection probabilities, which result at times from inaccurate enrollment data on the school frame used to define the school size measure. Even though measures are in place to limit the number and size of excessively large weights—such as the implementation of adjustment factor size constraints in both the school and student nonresponse procedures and the use of the school trimming procedure—large student weights can occur due to compounding effects of the various weighting components.

The student weight trimming procedure used the multiple median rule to detect excessively large student weights. Any student weight within a given trimming group greater than a specified multiple of the median weight value of the given trimming group had its weight scaled back to that threshold. Student weight trimming was implemented separately by grade, subject, and school type (public/private). The multiples used were 3.5 for public school trimming groups and 4.5 for private school trimming groups. Trimming groups were defined by jurisdiction/Trial Urban District Assessment (TUDA) districts for public school samples and by private school student group (Catholic, Lutheran, conservative Christian, and other private schools) for the private school sample.

The procedure computed the median of the nonresponse-adjusted student weights in the trimming group g for a given grade and subject sample. Any student k with a weight more than M times the median (where M = 3.5 for public school and 4.5 for private school trimming groups) received a trimming factor calculated as follows:

where,

• M is the trimming multiple;

• MEDIAN_j(g) is the median nonresponse-adjusted student weight in trimming group g; and

• STUWGT_jk is the student weight for student k in jurisdiction j in trimming group g.

The adjustment factor and the components of its formula do not include a subscript s to reflect the fact that not every student in a school s necessarily falls into the same trimming group.

It should be noted that excluded students were not factored into the calculation of the median, but were still eligible for student weight trimming. Using excluded students in the median calculation would lower trimming thresholds since excluded students do not experience student nonresponse and, thus, generally have smaller weights than assessed students. However, it was plausible for excluded students to have excessively large weights due to the compounding of the other weighting components.

In the 2003 assessment, relatively few students had weights considered excessively large. Out of the total of 741,000 students included in the combined 2003 assessment samples, only 600 students required student weight trimming.

Printer-friendly Version