- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Twitter
NAEP Technical DocumentationNew-School Sampling Frame for the 2009 Private School Assessments
Whereas the Private School Universe Survey (PSS) file used for the frame corresponds to the 2005-2006 school year, the NAEP assessment year was the 2008-2009 school year. During this 3-year period, some schools closed, some changed their grade span, and still others came into existence.
To achieve as close to full coverage as possible, the private school frame was supplemented by a sample of new Catholic schools. The goal was to allow every such school a chance of selection, thereby fully covering the target population of Catholic schools in operation during the 2008-2009 school year. The first step in this process was the development of a new-school frame through the construction of a diocese-level file from the PSS school-level file. To develop the frame, the diocese-level file was divided into two files: one for small dioceses and a second for medium and large dioceses.
Small dioceses contained no more than three schools on the frame in total, with no more than one school at each targeted grade (fourth, eighth, and twelfth). New schools in small dioceses were identified during school recruitment and added to the sample if the old school in the same diocese was sampled at the relevant grade. From a sampling perspective, the new school was viewed as an “annex” to the sampled school that had a well-defined probability of selection equal to that of the old school. The “frame” in this case was, in fact, the original frame; when the old school was sampled in a small diocese, the new school was automatically sampled as well.
To limit respondent burden and keep the level of effort within reasonable bounds, the new-school frame was created using information obtained from a sample of the remaining dioceses. The remaining dioceses were separated into two strata of large- and medium-size dioceses. These strata were defined by computing the percentage of the nation’s total Catholic school enrollment each diocese represents, sorting the dioceses in descending order by that percentage, and cumulating the percentages across the sorted file. All dioceses up to and including the first diocese at or above the 80th cumulative percentage were defined as large dioceses. The remaining dioceses were defined as medium dioceses.
A simplified example is given below. Dioceses are ordered by percentage enrollment. The first six become large dioceses and the last six become medium dioceses.
| Diocese | Percent enrollment | Cumulative percentage enrollment | Stratum |
|---|---|---|---|
| Diocese 1 | 20 | 20 | L |
| Diocese 2 | 20 | 40 | L |
| Diocese 3 | 15 | 55 | L |
| Diocese 4 | 10 | 65 | L |
| Diocese 5 | 10 | 75 | L |
| Diocese 6 | 10 | 85 | L |
| Diocese 7 | 5 | 90 | M |
| Diocese 8 | 2 | 92 | M |
| Diocese 9 | 2 | 94 | M |
| Diocese 10 | 2 | 96 | M |
| Diocese 11 | 2 | 98 | M |
| Diocese 12 | 2 | 100 | M |
| SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2009 Assessment. | |||
In actuality there were 72 large and 101 medium dioceses in the sampling frame.
The target sample size was 10 dioceses total: 8 large and 2 medium. In the medium stratum, the dioceses were selected with equal probability. In the large stratum, dioceses were sampled with probability proportional to enrollment. These probabilities were retained and used in all later stages of sampling and weighting in order to represent all dioceses, whether or not they had been sampled to be surveyed for new schools.
Each selected diocese was sent a listing of its schools extracted from the 2005-2006 PSS file and was asked to provide information about new schools and any changes to grade span in existing schools. This information provided by the selected dioceses was used to create sampling frames for the selection of new Catholic schools. The process of obtaining the information was conducted with the help of the National Catholic Educational Association (NCEA). NCEA was sent the school lists for the 10 sampled dioceses and was responsible for returning the completed updates.
The eligibility of a new school at a particular grade was determined by its grade span. A school already on PSS also was classified as “new” if a change of grade span had occurred such that the school status changed from ineligible to eligible at a particular grade.
Last updated 16 August 2010 (JL)