Mathematics Online Study

The Mathematics Online (MOL) study is one of three field investigations in NAEP's Technology-Based Assessment Project, which explores the use of new technology in administering NAEP. The study focused on issues associated with delivering existing multiple-choice and constructed-response mathematics items on computer.

Results of the study are available in the report Online Assessment in Mathematics and Writing: Reports from the NAEP Technology-Based Assessment Project, Research and Development Series.

The MOL study was guided by the mathematics assessment framework and focused on the issues associated with translating existing multiple-choice and constructed-response mathematics questions from paper and pencil to computer delivery. Explore sample mathematics items in the NAEP Questions Tool. The primary goals of MOL were to understand how computer delivery affects the measurement of NAEP math skills, to gain insights into the operational and logistical mechanics of computer-delivered assessments, and to evaluate the ability of fourth- and eighth-graders to deal with mathematics assessments delivered on computer. At grade 8, an additional goal was to investigate the technical feasibility of generating alternate versions of multiple-choice and constructed-response items using an "on-the-fly" (OTF) technology. Test items generated using OTF technology were based on models derived from original, or "parent" versions of items.

Before taking the assessment, students were asked to complete a tutorial.
 

Data were collected in spring 2001 from more than 100 schools at grades 4 and 100 schools at grade 8. Over 1,000 fourth-grade students and 1,000 eighth-grade students took a test on a computer via the Web or on laptop computers taken into schools.

The study collected background data concerning students' access to, uses of, and attitudes toward computers. In addition, students were administered hands-on exercises designed to measure their ability to input information into a computer.

Over 2,700 students at grade 8 took comparable paper-and-pencil tests. The students taking paper-and pencil tests were assigned randomly to one of three forms. One paper-and-pencil form, which presented identical items to the grade 8 computer-based test, provides the main comparisons for the effect of computer delivery versus paper delivery. The other two paper-and-pencil forms were used to study psychometric questions related to the automatic generation of test questions.

Explore the Writing Online study.

Learn about the Problem Solving in Technology-Rich Environments study.