Table of Contents | Search Technical Documentation | References

NAEP Technical Documentation Student Test Form and Booklet Block Design

Since 2017, most NAEP assessments are conducted on a digitally based platform, including mathematics, reading, science, civics, U.S. history, and technology and engineering literacy (TEL). NAEP uses a focused balanced incomplete block (BIB) or partially balanced incomplete block (pBIB) design to pair blocks of cognitive items which define student forms. Because of the BIB and pBIB form designs and the way NAEP assigns forms to students, NAEP can sample enough students to obtain precise results for each test question while generally consuming an average of about an hour and a half of each student's time.

The "focused" aspect of NAEP's form design requires that each student answer questions from only one subject area. The "BIB" or "pBIB" design ensures that students receive different interlocking sections of the assessment forms, enabling NAEP to check for any unusual interactions that may occur between different samples of students and different sets of assessment questions.

In a BIB design, the cognitive blocks are balanced; each cognitive block appears an equal number of times in every possible position (within form, e.g., first or second). Each cognitive block is also paired with every other cognitive block in a test form exactly the same number of times. This "connectedness" (pairs of blocks being taken by given students) that such designs afford enables the calculation of correlations between blocks which improves the quality of the scaling analyses. In a pBIB design, cognitive blocks may not appear an equal number of times in each position, or may not be paired with every other cognitive block an equal number of times. NAEP form design varies according to subject area (e.g., mathematics, reading, science, civics, U.S. history, and TEL) and by type of assessment (e.g., operational or pilot). On occasion, there might be distinct form types (e.g., discrete item-only, scenario-based task-only) which require a unique bookmap designed for that specific situation.

Once the instrument developer has identified the set of blocks to be administered, bookmaps can be developed by the design team. Digitally based assessments in NAEP require much more extensive information than was the case with paper-based assessments. In digitally based assessments it is necessary for the bookmaps to provide the delivery system with timing details, accommodation form substitutions, what to present to students in welcome and thank you screens, tutorial information, and more. However, the essence of the bookmap from a content perspective remains the same: identifying which cognitive blocks and survey questionnaire blocks are placed in each specific form. The figure below shows, in a simplified bookmap format, how the content of each form can be documented. Such information is needed to develop the complete digital bookmap and also contributes to the development of the bundle maps (spiraling) of the student forms.

Prior to 2017, most NAEP assessments utilized a paper-based rather than the digitally based format. The block design for printed test booklets was similar to the block design described above for the computer delivery of the assessments. The figure below provides an example of a bookmap format that was used for paper-based assessments. Once the instrument developer had laid out the configuration of all blocks for each booklet in a booklet map with the following column headings, the number of rows (booklet numbers) provided the booklet spiral design information needed for the bundling of the student booklets.