Science Teachers’ Search Behaviors, Resource Preferences, and Classroom Technology Lecia Barker School of Information University of Texas United States
[email protected]
Abstract: This paper presents results of a triangulated study of teachers’ available classroom and lab technologies, information seeking behaviors, and preferences for online resources. The first two phases of the study were qualitative, one based in an economically advantaged, majority school district and the other in an economically disadvantaged and more diverse school district. Qualitative findings were used to inform an online survey administered to 622 science teachers across the U.S. Teachers in this study highly valued online resources for their teaching. However, their ability to effectively search for resources was limited. Teaching preservice and inservice teachers to search effectively is a priority, especially in financial times that limit textbook and other resource purchases. Teachers valued web sites that were specifically designed for teachers. The teacher preferences expressed in this study can support improved development of resources for teachers and information retrieval services.
Introduction Science teachers seek and use online resources in their classrooms and in computer labs at an increasing rate (Hanson & Carlson, 2005; Perrault, 2007). Teachers seek ways to engage students with images, simulations, and current and local events; take advantage of tutorials, learning units, and vocabulary and content acquisition activities; and assign students “web quests,” through which students learn while seeking information from web sites. The abundance of high-quality, free materials is important to teachers. Indeed, for teachers in poor schools, online resources may be the only way to teach up-to-date science. Increased access does not necessarily mean effective information seeking, however: teachers are about as good at discovering information as the average Internet user (Ford, Miller, & Moss, 2005; Pattuelli, 2008; Perrault, 2007; Recker, 2006). Teachers also have more constraints than the average Internet user, since they seek resources not just to satisfy their own needs (which are perhaps only implicitly apparent to them), but that can be used in their local classroom environment and for specific students’ needs. Thus, they must explicitly understand the needs of the classroom context and the cognitive abilities of the students on behalf of whom they search. For example, in many poorer school districts, teachers cannot assign online homework or activities because they cannot assume that their students’ families have access to computers at home. Understanding how teachers search for resources and under what circumstances they are likely to use what they find can support the work of resource providers as they develop web sites and resources for teacher use. It can also support the work of teacher educators as they prepare their students for the specific socio-technical environments in which they will teach. The proposed paper will present the results of a three-part, mixed-method study of teachers’ information seeking behaviors, their preferences for online resources, and available classroom and lab technologies. The findings presented here are part of a study originally intended to identify science teachers’ perceptions of and uses of the Digital Library for Earth System Education (DLESE, dlese.org), the geosciences collection of the National Science Digital Library. While the study originated as an evaluation of DLESE, it became clear that many teachers were completely unaware of digital libraries and depended on their own search habits to find resources. Therefore, the second two phases of the study were focused on what teachers do naturally, with or without knowledge of DLESE. Findings specific to teachers’ perceptions and use of DLESE can be found in Barker, 2009.
Methodology: Data Collection in Three Phases Phase One was an exploratory study of how 12 teachers in the Williamsburg-James City County School District in Williamsburg, Virginia might use the Digital Library for Earth System Education (DLESE). The school district is
- 1340 -
71% white, 21% African American, and 4% Hispanic. It has a 93% graduation rate and 84% reading proficiency. Only 21% of students are considered economically disadvantaged (as measured by free and/or reduced lunch data). Teachers involved in the study had exceptional access to computer technology and electronic white boards in their classrooms. The study began with DLESE outreach staff training teachers on the benefits of DLESE and how to use DLESE. Clearly, the training may have introduced bias to the study. The teachers were then asked to use DLESE during the academic year in exchange for a $300 stipend. Data collection included email reports, classroom observations, and telephone interviews. Phase Two focused on teachers’ behaviors in searching for and evaluating resources; they were not “primed” by training or introduction to DLESE. Seventeen teachers from Denver Public Schools in Denver, Colorado participated. In contrast to the Williamsburg schools, Denver Public Schools is 56% Hispanic, 21% white, and 18% African American. It has a 63% graduation rate and 71% reading proficiency. Sixty-five percent of students are considered economically disadvantaged. Phase Two teachers were offered $300 to participate in one of three fourhour sessions of six teachers and at least six researchers each. The research staff was led by the author and included three colleagues with relevant Ph.D.s, several graduate students trained for the project, and two professional research assistants. After ice breaking/trust building sessions (including food), each teacher was paired with a researcher and computer. The teachers completed a recorded, structured interview about their relationship with technology, then were asked to think of a topic they would teach the next week and to find a resource to support their teaching, with no instruction. Each computer started out with the desktop showing, so that the teachers had to find and open a browser before searching. The search session lasted 90 minutes, was audio-recorded, and used screen capture to record teachers’ search choices. As they searched, teachers were asked to “think out loud” to describe their choices and observations. After they had found (or sometimes not found) something useful too them, they were introduced to DLESE. Teachers were asked what they thought of DLESE and if they would use it. The teachers then were given pizza and salad while participating in video-recorded focus groups to discuss what resources they typically search for, how and why they use technology with their students, what they liked and disliked about DLESE, and in support of Phase Three, what incentives would convince other teachers to take a survey. Phase Three began with the development and administration of an online survey. The survey contents were derived from the findings of Phases One and Two as well as several individual difference variables: years of teaching experience, highest degree earned, perceived reading level of students (used as a proxy of socio-economic status), type and number of hardware/electronic devices in classroom, working/Internet-connected computers in the classroom, and availability of computer labs that teachers could freely access. The sample was acquired using a snowball technique. A survey link was sent out to science coordinators and teachers involved in both studies, to teachers who had participated as “DLESE Ambassadors,” and to other projects across the U.S. to which members of the research team had access. The email invitation offered each teacher a $10 gift card to either Amazon or iTunes and asked the respondents to forward the link to other science teachers they knew. It also asked teachers not to take the survey more than once; there is no evidence that teachers gamed the system, especially since the gift certificate was sent to individual email addresses. This snowball method resulted in 622 usable responses, including at least one respondent from all but three of the 50 United States (not North Dakota, West Virginia, or Utah); and one each from Guam, the District of Columbia, and Ontario.
Findings Qualitative Study Highlights – Phases One and Two Phase One of the study revealed that teachers used online resources to enhance student engagement. Several teachers stated that students’ expectations for presentation of information are changing as a result of their being exposed to so much multimedia in their extracurricular lives. Some teachers thought they would like to require their students to use resources at home for homework or to provide parents with information about what their children were learning. These teachers, used to parents who were involved with their children’s education and who almost all had broadband access at home, believed this was a real possibility and suggested having a “My DLESE” sort of feature where they could keep track of resources and that parents could log into. Despite their positive perceptions of online resources, these teachers largely privileged tangible/touchable resources in the classroom. Still, several expressed a desire for faster computers, data projectors, electronic whiteboards, better access to computer labs, and more consistent access to the Internet. Finally, some Williamsburg teachers expressed frustration in finding resources, even in DLESE (and even when the researcher knew a resource being sought was in DLESE and could easily find it). These teachers said they were overwhelmed by search results and gave up easily. Hypothesizing that many teachers had poor search skills and that they may have developed specific, yet tacit expectations for the appearance
- 1341 -
of results pages and the functionality of search sites, Phase Two was designed as a naturalistic study to find out what teachers would do if asked to use the computer the way they do in everyday searching. Phase Two was partly designed to understand what search strategies secondary science teachers use to locate educational resources online, where they begin their searches (e.g., a search engine or educational web site), and how they evaluate and use results pages. Teachers demonstrated search and evaluation competencies at opposite ends of a spectrum and in between. For example, two teachers could be categorized as “super searchers,” four as “wanderers,” and the rest using techniques from both ends of this skill continuum. Behaviors of teachers falling into these extremes are described below. The super searchers started with search engines and used very specific search terms, stating that they wanted to limit the number of results they got. For example, the two supersearchers used a high degree of specificity in their choice of search terms and expressed a desire to limit the number of results returned. Both used Boolean operators, quotation marks in the search box (putting quotation marks around a search term commands the search engine to search for the exact string of characters), numerous search terms (e.g., complete phrases or sentences rather than one word, such as “chemistry”), and even terms that they were not looking for, but which would mark a site as educational, such as “lesson plan,” as shown in Figure 2. Both of these teachers believed that using online resources saved them time, as compared to having to develop their own worksheets and other tasks. One treated her searches like a game in which she sought “lovely” search terms which would provide “invitational” results. Both used results pages strategically, looking for the number of search terms that were bolded in the description of a site and scanning down the results to find additional words for revising a search. If search terms were not bolded in the description, neither teacher would follow that link. When they followed links, they made instant judgments about suitability for their students and teaching goals. Sometimes this was based on the perceived credibility of the source. For example, asked why she followed a particular link about fossil fuel lessons, a teacher said, “it’s from the Geological society and I figure they have a whole department working on it.” In short, “super searchers” applied a critical process to choice of search terms, were protective of their time evaluating results, were not afraid to skip sites or change search terms to find the most promising resources, and were choosy about the sites they actually accessed.
Figure 2: Super Searcher Search Results
In contrast, “wanderers” had poor search skills, which resulted in time wasted and much more limited access to resources. For example, one teacher began her search session by entering the words ‘fossil fuels’ (without double quotation marks) into the search box. There was no clear pattern to the speed with which she evaluated the resulting websites, at times very quickly or very slowly, nor did she articulate a rationale for how she evaluated the websites, other than noting the domain and/or source of the site. Not finding something useful for her class, she changed her search terms to ‘fossil fuel statistics’. Despite saying that she sought user-friendly hand-outs for her students, her search terms did not specify these terms. After little success with these two search attempts, the teacher said she was bored and changed her search term to just ‘earthquakes’. She was surprised this search offered a sports team called the Earthquakes – not surprising given the breadth of the single key word. After following a couple of the 31,000,000 results links, she changed her search term to ‘volcanoes’, saying she wanted images. This wanderer did little to advance the likelihood of success, but seemed confident that she would stumble upon something good, in
- 1342 -
time. In summary, wanderers who started with search engines used terms that were very general and used few specialized search techniques. They often spent several minutes with each page of results, sometimes perusing several results pages without finding something suitable, and frustrated when there were tens of millions of results. They likely held misconceptions about how a search engine works also, since as one pointed out, the last result might be “the one.” Therefore, checking all links was the strategy of choice, ending in frustration, since time was clearly a scarce resource for teachers. In focus groups, Phase Two teachers discussed the kinds of resources they looked for, their constraints in using resources, and their preferences for educational web sites. In contrast to what might be expected, they were not often searching for individual learning modules for students to use either in a lab or for homework. This was because they were constrained in their ability to use the labs in their schools: several teachers described “lab hogs,” or teachers who signed up for lab time for the entire semester or school year, keeping any other teacher who wanted to use the lab from using it. Few teachers had access to laptop carts; one teacher in an extraordinarily poor school told of laptop theft. Teachers also sought lesson plans, quizzes, crossword puzzles for vocabulary learning and other similar resources they could, but preferred not to, produce themselves. Some teachers stated that they wanted to be able to download such resources and use them intact, while others wanted to be able to customize them (and some of these were confused about “this pdf format that can’t be modified”). Teachers often sought images, if they could display them. However, one teacher said flatly that she had no use for images or any other kind of resources: she had no computer in her classroom and no way to use them. Another teacher described an inability to transfer an image to the computer in her classroom, since the image was too large. Finally, teachers talked about features of web sites that might be perceived important. The observed search techniques, resource types, technical restraints, and several of the web site features (e.g., “reviewed by teachers,” the ability to store links to resources in their own account) were included in the survey in order to understand the generalizabity of these findings as well as the conditions under which variables were more or less likely to appear. Survey Results The results of Phases One and Two informed a survey designed to test the generalizability of findings. Both qualitative studies could have introduced bias, since the teachers were in situations where they would develop relationships with researchers or were biased in favor of online resources. In addition to simply describing the technology conditions under which teachers teach, teachers’ search choices, and their preferences for online resources, differences in teacher profiles in relationship to search and preferences were analyzed. Analyses of variance were conducted to see test these for these. The findings are summarized below. Where there were differences across teacher profiles categories, they are presented in each section below only when they were statistically significant. There were fewer differences than expected, however, perhaps because the sample was collected through email (in other words, this was already a technology using group). Teacher Profile, Conditions. 37% of teachers had only one working computer in the classroom; most of these computers were Internet capable. About one quarter of teachers had five or more Internet connected computers in their classrooms. While only 3% of teachers said they had no access to a computer lab, only 12% stated that they had access to a lab with plenty of computers anytime they wanted. Others ranged from difficulty with scheduling (20%) to access only if scheduled well in advance (60%). About 57% of teachers used their home computer for high-speed access. Teachers indicated other resources, such as television with VCR (most common – 87%), computer projector (69%), or hand-held computer (7%). Unlike the teachers in Williamsburg, only 16% of teachers in the sample had an electronic whiteboard. Teachers in this study were highly educated, with a majority having master’s degrees and even a few with doctorates. Years of teaching ranged from one to 30, with a median of seven. Teacher Search Choices, Skills. A majority of teachers were likely to begin a resource search by using a search engine (in contrast to going directly to a preferred educational web site, for example). They were also likely to go to a site designed specifically for teachers (although teachers with Ph.D.s were significantly less likely to do so, p < .01). Most teachers used specialized search techniques less than might be considered useful. While Google assumes an “and” when two or more words are typed in, most teachers never or rarely used a “not” (or dash) to avoid material a teacher would not want (e.g., in searching for Venus, a teacher might include “-sex” to avoid pornography). In addition, teachers were relatively unlikely to specify a web suffix (e.g., .org, .gov, or .edu) to avoid commercial sites. They were much more likely to include reading level (43% sometimes, 21% always), type of resource (e.g., “lesson plan,” 51% sometimes, 28% always), and very likely to include the subject area (70% always). With respect to refining searches, teachers reported using fewer strategies. The most frequent strategy used by teachers was scanning the results page to identify new or better search terms (most said “sometimes”). Less
- 1343 -
frequently used strategies were looking at the type of document, using a spelling suggestion (e.g., “did you mean…”), using the same search terms at different search sites, or changing the order of search terms. 59% of teachers said they scan two of more pages of results, while 11% said they only look at the first page. Technology/Device Television (e.g., with VCR) Overhead projector (for transparencies)
% Use 87% 83%
LCD or data projector (for computer projection)
69%
Analog audio (e.g., tape, CD player)
39%
Television that can display the teacher's computer screen 37% Laptop cart Graphing calculators Smart board/electronic white board Digital audio equipment (e.g., MP3 player) Portable microphone for classroom GPS (Global Positioning System)
32% 21% 16% 13% 9% 8%
Hand-held computer, personal digital assistant
7%
Table 2: Teachers’ Use of Various Digital Devices for Teaching
Preferred Qualities of Web Sites, Resources. The most important qualities rated by teachers were “free, no cost website,” followed by “resources marked by grade level and reading level.” Descriptions of time and resources needed, and sites containing graphics and visual images were also rated highly. Less highly rated were resources marked with standards, affiliations with science organizations, and if the resource was local. Least valued was having a saved history of past searches. Teachers expressed a stronger preference for being able to customize resources than for downloading them intact. They also valued being able to print a resource for classroom use. Resources that had ratings by other users were seen as less important. Teachers who used more electronic devices in their teaching and teachers with more than one working computer in their classroom valued the Internet more for their teaching. Direction of this relationship is not clear, however: teachers with more devices may seek out more resources or teachers who find more resources may find acquire and use more hardware. Not surprisingly, teachers whose students were more proficient readers had more hardware resources in their classrooms. Perceived Value of the Internet for Teaching. Overall, the teachers in this sampled very strongly valued the Internet for finding resources and using them in their teaching, as shown in Table 3. However, preferences varied based on teachers’ profiles. The following null hypothesis was tested to understand whether there were differences across teaching conditions: H0: Teachers will not value access to digital resources differently depending on the reading level of their students, the technology used in their classrooms, or other teacher characteristics. A one-way analysis of variance was calculated to determine whether teachers’ ratings of the importance of the Internet to teaching and the usefulness of the Internet for finding teaching resources differed by years of teaching experience, highest degree, reading proficiency of students, amount of hardware/electronic media in classroom, number of working computers in the classroom, and lab availability. The null hypothesis was rejected for the two variables related to hardware in the classroom, lending support to a positive relationship between computer availability and total device use with perceived value of the Internet for teaching. Specifically, teachers who had more than one working computer in their classroom rated the importance of the Internet to their teaching more highly than did teachers who had no or only one working computer in their classrooms (F(4,614)=3.28, p=.013); and teachers who heavy users of non-computer hardware/electronic media in their classrooms rated the Internet as more important to their teaching than did teachers who were light users (F(3,612)=4.522, p=.004).
Table 3: Perceived Value of Internet for Teaching and Finding Resources
- 1344 -
Discussion/Conclusion Teachers in this study highly valued online resources for their teaching. However, their ability to effectively search for and therefore retrieve resources was limited. There were no significant differences for years of teaching experience: teachers with 1-3 years and teachers with more than 20 years teaching experience searched in about the same ways. Clearly, the digital divide is alive and well. Teachers who ranked their students as highly proficient readers had more devices to use in their classrooms and more highly valued the Internet. However, these teachers did not demonstrate better search skills than their less-equipped counterparts. Not surprisingly, teachers valued web sites that were specifically designed by or for teachers. Marking resources for reading level or grade level was seen as very important, for example. The Internet is rich with up-to-date science resources in the form of tutorials, lesson plans, worksheets, images, and all manner of pedagogical tools and techniques. Teacher access to these is limited by their ability to search and retrieve them as well as the technology conditions under which they teach. Teaching preservice and inservice teachers to search effectively is a priority, especially in financial times that limit textbook and other resource purchases and when teachers are assigned to teach topics for which they have no training. As Ingersoll (2003) pointed out, “…highly qualified teachers may actually become highly unqualified if they are assigned to teach subjects for which they have little background.” This could be especially problematic for those teachers who rarely limited use of specialized search techniques, effectively limiting their access to improved understanding for themselves and improved pedagogies in the classroom. The results presented here are expected to be useful for developers of teacher resources, who can get a better understanding of the technology conditions under which teachers teach as well as teachers’ ability to retrieve their resources.
References Barker, L. (2009). Science Teachers’ Use of Online Resources and the Digital Library for Earth System Education. Proceedings of the 9th ACM/IEEE-CS Joint Conference on Digital Libraries, June 15-19, 2009, Austin, Texas, 1-10. Ford, N., Miller, D., & Moss, N. (2005). Web Search Strategies and Human Individual Differences: A Combined Analysis. Journal of American Society for Information Science and Technology, 56(7), 757-764. Hanson, K., & Carlson, B. (2005). Effective Access: Teachers’ Use of Digital Resources in STEM Teaching. Newton, MA: Education Development Center. Ingersoll, R.M. (2003). Out-of-field teaching and the limits of teacher policy. Research Report by the Center for the Study of Teaching and Policy and the Consortium for Policy Research in Education. Pattuelli, M. C. (2008). Teachers'Perspectives and Contextual Dimensions to Guide The Design of N.C. History Learning Objects and Ontology. Inf. Process. Manage., 44(2), 635-646. Perrault, A. M. (2007). An Exploratory Study of Biology Teachers’ Online Information Seeking Practices. School Library Media Research, 10. http://www.ala.org/ala/mgrps/divs/aasl/aaslpubsandjournals /slmrb/slmrcontents/volume10/perrault_biologyteachers.cfm Recker, M. (2006). Perspectives on Teachers as Digital Library Users: Consumers, Contributors, And Designers. DLib Magazine, 12.
Acknowledgments This study would not have been possible without Julie Naughton, Susan Buhr, Tim Weston, Christopher Hovey, Ed Geary, Bryan Aivazian, Susan Lynds, Rachel Matz, Terra Morris, M.G. Saldivar, Matthew Vorell, Stephen Williams, Jenifer Martin, Harrison Archer, Amy Hofmockel, Evan Hofmockel, Russ Mehring, and Quinn Rennerfeldt. Funded by the National Science Foundation under Awards #0306694 and #0439669.
- 1345 -
