NAEP Technical DocumentationStepwise Regression Analysis Results for Primary Sampling Unit (PSU) Stratification for the 2012 Assessments

The objective was to find the optimum set of primary sampling unit (PSU)-level sociodemographic characteristics in terms of strength of relationship to achievement. The PSU-level values of these characteristics were derived from the 2000 Census Summary Files and the 2003 county population estimates, computed by combining the county-level data (using county youth estimates as the relative weighting factor for each county within the PSU). The characteristics used, and their abbreviations as used in the tables, were as follows:

• race/ethnicity percentages in schools (percent Black, Hispanic, or American Indian/Alaska Native – "Pct BHI" below; percent Black; percent Hispanic – "Hsp" below; percent Asian; percent American Indian/Alaska Native; percent two or more races);

• income levels (median household income – "Med Inc" below, percent children below the poverty line – "Cld pov" below);

• education levels in population (i.e., percent of persons age 25 and over who completed high school but have no college degree – "HS grd" below, percent of persons age 25 and over with college degrees – "CG grd" below);

• percent of renters (i.e., percent of householders who rent rather than own their place of residence); and

• percent of female householders living alone.

These PSU-level census characteristics were examined within each of the four NAEP 2000 assessment values: fourth-grade mathematics achievement, fourth-grade science achievement, eighth-grade mathematics achievement, and eighth-grade science achievement. These PSU-level values for achievement were computed using the 2000 state NAEP database. The criterion was that good strata should be heterogeneous for each of the five characteristics (i.e., within-stratum variance for each assessment value should be low and between-stratum variance high), so that strata are defined that do a good job for both mathematics and science, in both grades, not just the best possible job for one subject and one grade. This prevents overfitting to some extent.

The analysis was done separately within each of the eight primary strata (census region by metro status), using a forward stepwise regression approach, with a p-value cutoff of 20 percent. The results are given in the tables below. The order of the regressors is the order of entry into the stepwise procedure. The p-value is for an F-test for entry of the regressor into the forward stepwise model. The minus or plus sign indicates the direction of effect (negative indicates that increase in the regressor is related to lower achievement; positive indicates that increase in the regressor is related to higher achievement). The regressor is in italics if the direction of the effect is unexpected (i.e., negative when we generally expect a positive effect, or vice versa). The stratifiers chosen to generate the final PSU strata are indicated in a note below the regression analysis result tables.  

Printer-friendly Version