NAEP Technical DocumentationComparisons of the 2003 and 2004 Sampling Frames

One approach towards checking on the accuracy and consistency of a frame is to compare it to the frame for the previous NAEP year. In the current situation, the 2004 frame is age-based, whereas that for 2003 was grade-based. Nonetheless, comparisons of the frame characteristics should be useful in assessing the current year's frame.

To recap, the NAEP 2004 frame consists of schools containing students of ages 9, 13, and 17 years in a sample of PSUs; the NAEP 2003 frame, on the other hand, was a grade-based school frame, consisting of schools containing a fourth grade, eighth grade, and twelfth grade, respectively. The comparisons are based on estimated enrollment. On the NAEP 2004 frame, enrollment is estimated by taking estimated age-eligible enrollment in each school, dividing these by the PSU probabilities of selection, and aggregating them to the national level. For NAEP 2003, enrollment is estimated by taking estimated weighted grade enrollment for the schools in the grade-based frame and aggregating these to the national level.

The characteristics used in the comparisons are as follows¹:

race/ethnicity (Black, Hispanic, Asian, American Indian) and
degree of urbanization.

The race/ethnicity percentages come from the CCD and PSS records for each school. Type of location is the urbanicity variable treated as an interval variable.² The tabulations are weighted tabulations over the schools in the frame for that NAEP region, with the estimated grade or age-eligible enrollment the weight.

Comparisons of Black and Hispanic student percentages from the 2003 grade-based frame and the 2004 age-based frame: By age, 2004
Age/ Grade	Black pct 2004	Black pct 2003	Black pct diff 2004-2003	Hsp pct 2004	Hsp pct 2003	Hsp pct diff 2004-2003
Age 9, Grade 4	16.5	17.0	-0.5	19.5	18.7	0.8
Age 13, Grade 8	16.0	16.5	-0.6	16.6	16.2	0.5
Age 17, Grade 12	13.5	13.7	-0.2	14.5	14.0	0.5
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004.

Comparisons of Asian and American Indian student percentages from the 2003 grade-based frame and the 2004 age-based frame: By age, 2004
Age/ Grade	Asian pct 2004	Asian pct 2003	Asian pct diff 2004-2003	Am Ind pct 2004	Am Ind pct 2003	Am Ind pct diff 2004-2003
Age 9, Grade 4	4.1	4.0	0.1	1.1	1.2	-0.2
Age 13, Grade 8	4.0	4.0	0.0	1.1	1.2	-0.2
Age 17, Grade 12	4.5	4.4	0.1	0.9	1.1	-0.2
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004.

Comparisons of type of location means from the 2003 grade-based frame and the 2004 age-based frame: By age, 2004
Age/ Grade	Type of location mean 2004	Type of location mean 2003	Type of location mean diff 2004-2003
Age 9, Grade 4	3.9	4.0	-0.1
Age 13, Grade 8	4.0	4.0	-0.1
Age 17, Grade 12	4.0	4.1	-0.1
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2004.

¹ Median income is also generally presented in these comparisons, but in this case the 2000-01 CCD uses 1990 median income information and the 2001-02 CCD uses 2000 median income information, making any comparisons meaningless.
² Type of location is an interval variable with the following levels: 1=center of large city, 2=center of mid-size city, 3= urban fringe of large city, 4=urban fringe of mid-size city, 5=large town, 6=small town, 7=rural area, 8=rural area in MSA. A mean value of 6.0 for example can mean all in small town, or half in urban fringe of mid-size city and half in rural area of MSA stratum.

Last updated 16 March 2009 (RF)

Printer-friendly Version