- Surveys & Programs
- Data & Tools
- Fast Facts
- News & Events
- Publications & Products
- About Us
Twitter
Search Scenario
Below is a detailed description of the TRE Search scenario. The discussion of the design and components of the Search scenario is accompanied by selected screen shots. Due to technical limitations, it is not possible to provide the actual Search scenario interface for demonstration purposes.
Search Scenario Directions
Before beginning the Search scenario, students answered a set of questions regarding prior knowledge of science and computers. The prior-knowledge questions were intended to collect additional information about students’ skills and to give a rough measure of their familiarity with the material being assessed. Students then worked through a brief (5 minute) tutorial to introduce them to the Search interface.
Although the Search interface was designed to be as close to a standard web search browser as possible, some features—such as buttons for reading directions and for accessing the window to enter answers—are particular to the TRE software; therefore, a tutorial was included to ensure that students had an opportunity to become familiar with the interface before beginning the scenario.
The scenario directions, shown below, were designed to introduce students to the tasks they would be performing and to let them know the basis on which their responses would be evaluated: their searching skills, the quality of the information they located, and the quality of their answers to the question posed to motivate their searching.
Search Scenario Interface
After the directions screen, students moved to the Search interface (see figure below) to which they had been introduced in the tutorial. The question intended to motivate students’ searching was on the left-hand side of the screen and was always visible. Also always visible was a summary of scoring criteria for students’ work. On the right side of the screen was a web browser created for the purposes of this TRE scenario. At the top of the browser was a toolbar that included buttons for moving back and forth among pages, returning to the search page, bookmarking, viewing bookmarks, getting more extensive directions, receiving science help, and going to a page to take notes or answer the motivating problem. The center of the browser page provided a space for entering queries and a link to tips for searching.
Motivating Problem
The motivating problem in the left-hand column on the screen shown in the figure below was developed over many iterations and pilot tests of the scenario with groups of students. The problem was designed to be open enough to encourage searching, and yet specific enough so that reasonably skilled searching would supply substantive information to answer it within the 40 minutes allotted for the Search scenario.
The motivating problem and scoring criteria read as follows:
Some scientists study space with large helium gas balloons. These balloons are usually launched from the ground into space but can also be launched from spacecraft near other planets.
Why do scientists use these gas balloons to explore outer space and the atmosphere instead of using satellites, rockets, or other tools? Be sure to explain at least three advantages of using gas balloons.
Base your answer on more than one web page or site. Be sure to write your answer in your own words!
You will be scored on:
Skilled searching using relevant terms from the motivating problem and methods for focusing searches (e.g., quotation marks, use of “near” and “or”) yielded a list of pages, including some suitable for answering the question. Unskilled searching that employed only generic terms from the motivating problem (e.g., “balloon”), on the other hand, yielded less-relevant or irrelevant results.
To ensure that the TRE universe was as authentic as possible and would yield results ranging from the irrelevant to the highly relevant, with many gradations in between, during the development of the Search scenario skilled and unskilled searches were conducted to identify the kinds of pages students would find by searching the real World Wide Web (WWW). About five thousand pages from the WWW constituted the TRE web universe.
To maximize test-taker initiative, students could use the tool bar to search and bookmark at any time, even while they were responding to the motivating problem. The figure above shows the window for entering the response with the toolbar visible. Students were permitted to take notes but were told in the initial directions that their notes would not be scored. To ensure that some information was collected about the performance of each student, students could not leave the Search task without entering some text into the answer space. Once students had made some attempt to answer the question, they were given (assuming time had not run out) the option of reviewing their work.
Final Synthesizing Questions
Once students had responded to the motivating problem, they moved to a set of four multiple-choice questions designed to test how well they had synthesized the information that they had found about the use of helium gas balloons in space exploration. As with the constructed-response question, students could search while answering these questions. The figure below displays one of the synthesizing questions.
The text of the multiple-choice question shown above is as follows:
Why are scientific gas balloons only partially filled with helium before launch?
A. To prevent them from going too high.
B. To allow room for the gas to expand.
C. To keep the balloons from rising too slowly.
D. To keep the balloons from becoming too heavy.
E. To allow the balloons to be filled (launched) more quickly.
F. To save money on an expensive gas.
The correct answer is B.
Students had to answer all four synthesizing questions before they could leave the Search scenario. After completing the scenario, students responded to background questions intended to gather information about their demographic characteristics, school science classes and activities, and computer familiarity. A detailed discussion of the percentages of students in various background question-response categories appears in chapter 3 of the report Problem Solving in Technology-Rich Environments. See the full text of the background questions in Appendix D of the report.
Read about the Simulation scenario.
For more information, see the Technology-Rich Environments overview page.
Last updated 15 June 2007 (RF)