NAEP Assessment Sample Design → NAEP 2011 Sample Design → Sample Design for the 2011 National Assessment → 2011 Writing Computer-Based Assessment (WCBA) → Private School 2011 Writing Computer-Based Assessment (WCBA) → Sampling of Private Schools for the 2011 Writing Computer-Based Assessment (WCBA) → Computation of Measures of Size for the 2011 Private School Writing Computer-Based Assessment (WCBA)

NAEP Technical DocumentationComputation of Measures of Size for the 2011 Private School Writing Computer-Based Assessment (WCBA)

In the design of each school sample, five objectives underlie the process of determining the probability of selection for each school and how many students are to be sampled from each selected school containing grade-eligible students:

• to meet the target student sample size;

• to select an equal-probability sample of students;

• to limit the number of students who are selected from a school;

• to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included; and

• to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools.

The goal in determining the school's measure of size is to optimize across the last four objectives in terms of maintaining the accuracy of estimates and the cost-effectiveness of the sample design. The following algorithm was used to assign a measure of size to each school based on its estimated grade enrollment as indicated on the sampling frame.

The measures of size vary by enrollment size. The initial measures of size (MOS) were set as follows, for both eighth and twelfth grades:

where X_js is the estimated grade enrollment for grade j (j = 8, 12) in school s, PSCHWT_s= the Private School Universe Survey (PSS) area frame weight for school s, computed by the U.S. Census Bureau, and PSU_WT_s = the primary sampling unit (PSU) weight for school s.

An adjustment to the initial measure of size was made for some schools. Schools in the PSU containing Honolulu County had their measure of size increased by a factor of two in order to double their probability of selection.

The school measure of size was then rescaled to create an expected number of hits by applying a multiplicative constant b_j, which varies by grade and school type. For the national WCBA sample, by design, a school could not be selected or "hit" in the sampling process more than once.

 The rescaled measure of size, E_js, was defined as:

For grade 8 only, a final adjustment was made to the measures of size (E_js) in the national sample to attempt to reduce school burden by minimizing the number of schools that were selected for simultaneous administration of the WCBA, the operational private school assessments (mathematics, reading, and science), and the Trends in International Mathematics and Science Study (TIMSS). The NAEP 2011 studies for grade 8 used an adaptation of the Keyfitz process to compute conditional measures of size that, by their design, minimized the overlap of schools selected for the three types of assessment. Grade 12 did not have any operational assessments or a TIMSS sample in 2011.

The school's probability of selection π_js was given by:

One can choose a value of b_j such that the expected overall student sample yield matches the desired targets specified by the design, where the expected yield is calculated by summing the product of an individual school’s probability and its student sample yield across all schools in the frame.

In addition, new and newly eligible schools were sampled from the new-school frame. The assigned measures of size for these schools,

used the b_j value from the main school sample for the grade and school type (i.e., the same sampling rates as for the main school sample). The variable π_djs is the probability of selection of the diocese into the new-school diocese (d) sample.

 

Printer-friendly Version