NAEP Technical DocumentationStudent Weight Raking Adjustment for the 2015 Assessment

			Development of Final Raking Dimensions Raking Adjustment Control Totals Raking Adjustment Factor Calculation

Weighted estimates of population totals for student-level subgroups for a given grade will vary across subjects even though the student samples for each subject generally come from the same schools. These differences are the result of sampling error associated with the random assignment of subjects to students through a process known as spiraling. For state assessments in particular, any difference in demographic estimates between subjects, no matter how small, may raise concerns about data quality. To remove these random differences and potential data quality concerns, a new step was added to the NAEP weighting procedure starting in 2009. This step adjusts the student weights in such a way that the weighted sums of population totals for specific student groups are the same across all subjects. It was implemented using a raking procedure and applied only to state-level assessments.

Raking is a weighting procedure based on the iterative proportional fitting process developed by Deming and Stephan (1940) and involves simultaneous ratio adjustments to two or more marginal distributions of population totals. Each set of marginal population totals is known as a dimension, and each population total in a dimension is referred to as a control total. Raking is carried out in a sequence of adjustments. Sampling weights are adjusted to one marginal distribution and then to the second marginal distribution, and so on. One cycle of sequential adjustments to the marginal distributions is called an iteration. The procedure is repeated until convergence is achieved. The criterion for convergence can be specified either as the maximum number of iterations or an absolute difference (or relative absolute difference) from the marginal population totals. More discussion on raking can be found in Oh and Scheuren (1987).

For NAEP 2015, the student raking adjustment was carried out separately in each state for the mathematics, reading, and science public school samples at grades 4 and 8. The dimensions used in the raking process were race/ethnicity, SD/EL status, and gender. (National School Lunch Program (NSLP) eligibility, which has been used as a raking dimension in previous NAEP years, was not used as a raking dimension in 2015 because of the instability of this data in many states.) The control totals for these dimensions were obtained from the NAEP student sample weights of the mathematics, reading, and science public samples combined.

Printer-friendly Version