NAEP Technical DocumentationStepwise Regression Analysis Results for Primary Sampling Unit (PSU) Stratification for the 2008 Assessment

The object was to find the optimum set of Primary Sampling Unit (PSU)-level sociodemographic characteristics in terms of strength of relationship to achievement. The PSU-level values of these characteristics were derived from the 2000 Census Summary Files and the 2003 county population estimates, computed by combining the county-level data (using county youth estimates as the relative weighting factor for each county within the PSU). The characteristics used, and their abbreviations as used in the tables were as follows:

• race/ethnicity percentages in schools (percent Black, Hispanic, or American Indian/Alaska Native – "Pct BHI;" percent Black; percent Hispanic – "Hsp;" percent Asian; percent American Indian/Alaska Native; percent Two or more races);
• income levels (median household income – "Med Inc;" percent children below the poverty line – "Cld pov");
• education levels in population (i.e., percent of persons age 25 and over who completed high school but have no college degree – "HS grd;" percent of persons age 25 and over with college degrees – "CG grd");
• percent of renters (i.e., percent of householders who rent rather than own their place of residence); and
• percent of female householders living alone.

These PSU-level Census characteristics were examined within each of the four NAEP 2000 assessment values: grade 4 mathematics achievement, grade 4 science achievement, grade 8 mathematics achievement, and grade 8 science achievement. These PSU-level values for achievement were computed using the 2000 state NAEP database. The criterion was that good strata should be heterogeneous for each of the four characteristics (i.e., within-stratum variance for each assessment value should be low and between-stratum variance high), so that strata are defined that do a good job for both mathematics and science, in both grades, not just the best possible job for one subject and one grade. This will prevent overfitting to some extent.

The analysis was done separately within each of the eight primary strata (Census region by metro status), using a forward stepwise regression approach, with a p-value cutoff of 20 percent. The results are given in the tables below. The order of the regressors is the order of entry in the stepwise procedure. The p-value is for an F-test for entry of the regressor into the forward stepwise model. The minus or plus sign indicates the direction of effect (negative indicates that increase in the regressor is related to reduced achievement; positive indicates that increase in the regressor is related to increased achievement). The regressor is in italics if the direction of the effect is unexpected (i.e., negative when we generally expect a positive effect, or vice versa). The stratifiers chosen by the statisticians to generate the final PSU strata are indicated in a note below the regression analysis result tables.

The intent is that the results of this stepwise regression analysis and stratification were to be used for multiple design years and subject matter. They were used previously in 2006. Periodically, this analysis and stratification will be conducted according to the availability of Census data and key assessment scores.

Printer-friendly Version