Table of Contents | Search Technical Documentation | References

NAEP Instruments → Cognitive Items and Instruments → Development of the Technology and Engineering Literacy Cognitive Items, Scenario-Based Tasks, and Instruments

NAEP Technical DocumentationDevelopment of the Technology and Engineering Literacy Cognitive Items, Scenario-Based Tasks (SBTs), and Instruments

Digital test forms: 2014

Number of items: 2014

Assembly units: 2014

In 2014, the Technology and Engineering Literacy (TEL) assessment was administered for the first time to eighth-grade students on laptop computers. The TEL instrument was composed of newly developed blocks of discrete questions as well as interactive scenario-based tasks (SBTs) that measured TEL concepts and problem-solving skills. For the first administration of the TEL assessment, 4 SBTs and 20 discrete cognitive items were released to the public. The SBTs and related student performance data from the assessment can be accessed via the 2014 TEL Report Card. The discrete items and related student performance data can be accessed via the NAEP Questions Tool.

The TEL framework and the specifications documents guided TEL cognitive Item development. Cognitive items were administered via 20 SBTs (comprised of 130 SBT items) and 97 discrete items; individual students responded to only a portion of the entire assessment. SBTs and discrete items were developed by NAEP item development staff and members of the 2014 Technology and Engineering Literacy Assessment Standing Committee. All assessment materials were reviewed by specialists in technology and engineering education, measurement, assessment development, accessibility, and bias. The cognitive items were assembled into SBT and discrete item blocks covering three main content areas:

technology and society,
design and systems, and
information and communication technology.

Following approval from the National Center for Education Statistics (NCES), the assessment blocks were assembled into digital test forms and distributed to schools for administration. The SBT and discrete item blocks conformed to a 30-minute per section timing standard established for all NAEP computer-based assessments. SBT and discrete blocks were grouped into 30-minute cognitive sections referred to as assembly units (AU). Each AU was in one of the following configurations:

a single 30-minute SBT block,
a 30-minute discrete block, or
a pair of blocks (one SBT and one discrete block) whose time summed to 30 minutes.

Students had 60 minutes of total testing time for the cognitive SBTs and discrete items. Each digital test form contained two 30-minute AUs and two supplemental sections presented to students who completed the two AUs in less than 60 minutes. With the exception of the special test form for low mobility accommodation, none of the test forms were comprised entirely of discrete items. Since some students with physical impairments or a temporary disability could not use a mouse to access computer-based assessments, one special discrete-only test form was created to include items that are completely keyboard navigable.

Last updated 14 July 2021 (SK)

Printer-friendly Version