ZEKI SIMSEK JOHN F. VEIGA

11 downloads 203 Views 165KB Size Report
JOHN F. VEIGA. University of Connecticut. Even though ... back thousands of years (Erdos, 1983; Rossi, Wright, & Anderson, 1983). However, until recently, mail ...
ORGANIZA Simsek, Veiga TIONALRESEARCH / ELECTRONIC SUR METHODS VEYTECHNIQUE

ZEKI SIMSEK JOHN F. VEIGA University of Connecticut

Even though e-mail is the most widely used computer-mediated communication medium, its considerable potential as a survey technique has received little attention from management scholars. Using a three-dimensional framework focused on sampling issues, nonsampling errors, and comparative performance, the authors review and integrate previous research on the electronic survey technique and provide an assessment of the comparative trade-offs vis-à-vis other techniques. Moreover, they provide recommendations for future researchers interested in using this approach. Finally, they conclude that although this approach poses unique challenges and drawbacks, when an unbiased sampling frame exists or can be constructed, it allows researchers to inexpensively gather data with less effort than other available approaches.

Surveying techniques—usually classified by the communication medium used, such as face-to-face, telephone, mail, or electronic—rely on questioning individuals to elicit particular information to look for patterns among facts, values, behaviors, and so on to make generalizations about a population from which only some individuals are surveyed. Over the years, the use of such techniques has been, by far, the most common method of data collection in several fields, and this is anticipated to remain such, at least for the foreseeable future (Aaker, Kumar, & Day, 1995; Chadwick, Bahr, & Albrecht, 1984; Malhotra, 1993; Synodinos & Brennan, 1988). Despite its wellknown inherent weaknesses relative to experimental methods, gathering data via surveys has been more prevalent in management research arguably because of costs and obstacles associated with carrying out experiments and because the basic locus of many research questions has involved phenomena in the field. Simply put, if you want to find out what managers are thinking, you need to ask them (Zikmund, 1994). Albeit sketchy, the history of surveys and thus surveying techniques can be traced back thousands of years (Erdos, 1983; Rossi, Wright, & Anderson, 1983). However, until recently, mail questionnaires, field interviews, and telephone surveys were the

Authors’ Note: We wish to thank Monica Maciel Lopes and Melissa Foreman for their help during the preparation of this manuscript. We also thank two anonymous reviewers for their suggestions. Organizational Research Methods, Vol. 3 No. 1, January 2000 93-115 © 2000 Sage Publications, Inc.

93

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

94

only convenient techniques to collect survey information. Gates and Jarboe (1987) argue that developments in electronic technology, computer software, and environmental forces that oppose traditional data collection techniques have contributed to the change in data collection techniques today. Of these, computer technology has been a fundamental force behind the growth of alternative survey techniques and has led to improved data collection (Malhotra, 1993). Indeed, the emergence of this technology has affected not only data collection techniques but also has had a dramatic impact on almost every phase of survey research, including instrument design, sampling, field monitoring, coding and data editing, data capture, data cleaning, scale index construction, database organization, database retrieval, data analysis, and documentation (Anderson & Gansender, 1995; Karweit & Meyers, 1983; Neal, 1989; Saltzman, 1995). It has been aptly acknowledged that currently there are probably as many surveying techniques as there are different forms of communication technology (Aaker et al., 1995). For example, computer-assisted personal interviewing (Baker, 1992; Couper & Burt, 1994), computer-assisted telephone interviewing (Havice, 1990), fully automated telephone interviewing (Dacko, 1995), computer disk by mail (Higgins, Dimnik, & Greenwood, 1987; Saltzman, 1995), fax surveys (Vazzana & Bachmann, 1994), online World Wide Web (WWW) surveys, and focus groups (Gaiser, 1997) are only a few of the new techniques that have evolved. This evolution is the result of rapid developments in computer and communication technologies. The ever-increasing preference for computer-mediated communication, the opening of the Internet to the public, the introduction of WWW in 1989 at the European Particle Physics Laboratory in Europe, and the low-cost dispersion of software and hardware are fundamental forces that have shaped the viability of the e-mail survey technique (EST). EST holds considerable promise to obliterate the time and geographical constraints usually associated with surveys, facilitate interaction between surveyors and respondents, and reduce cost, time, and data entry errors per response (Bachmann, Elfrink, & Vazzana, 1996; Kiesler & Sproull, 1986; Mehta & Sivadas, 1995; L. Parker, 1992). Nonetheless, despite this potential, our review of the literature revealed that research on EST (a) is dispersed across the literature of almost two decades and several fields; (b) has not been systematically evaluated, let alone integrated; (c) consists mostly of empirical studies dealing with either response rates or quality of collected data or commentaries that juxtapose the pros and cons; and (d) has not embraced replications and lacks theoretical arguments and a conceptual framework. Indeed, to date, no one has attempted to integrate and assess both the theoretical and practical concerns (cf. Kiesler & Sproull, 1986; Kittleson, 1995; Oppermann, 1995; Schuldt & Totten, 1994). Thus, it seems that EST is, like the weather, something about which everybody is talking but nobody is doing much about it. We believe that this state of research on EST is unfortunate for several important reasons. First, literally hundreds of organizations conduct e-mail or web-based surveys for private and organizational consumers who in turn base their decisions on these data. Second, on any given day, numerous researchers conduct surveys using some conventional techniques, some of which could be done more efficiently and effectively using EST. Third, although e-mail is the most widely used computermediated communication medium, its full utility in sample surveying is not thoroughly assessed, let alone realized. Indeed, it took 2,500 years until basic postal serv-

95

ORGANIZATIONAL RESEARCH METHODS

ices became available to most individuals after being established by King Cyrus of the Persian Empire in the 6th century B.C. (Kochmer & Northwest, 1993, p. 47), but it took only 50 years until the numbers of computers rose to 1 computer per 44 persons from no more than dozens in the world (Perl & Whitley, 1995, p. 6). According to a recent estimate, electronic messages sent in the United States went from 776 billion to 2.6 trillion from 1994 to 1997 and are projected to reach 6.6 trillion by 2000 (Gwyne & Dikerson, 1997). Fourth, with the exception of earlier work by Kiesler and Sproull (1986), there is a paucity of focused discussion on EST in the management literature. Accordingly, using a three-dimensional framework focused on sampling issues, nonsampling errors, and comparative performance, we review and integrate previous research on EST and provide an assessment of the comparative trade-offs vis-à-vis other techniques. Moreover, we provide recommendations for future researchers interested in using this approach.

Although the basic notion for an electronic mail system has been around since the 1840s, e-mail has only emerged in the past three decades as a result of the convergence of computer and communication technologies (Helliwell, 1986; Mortensen, 1985). The use of e-mail was first started on the ARPAnet during the 1960s. At this early stage, the limited access and primitive nature of the systems hampered widespread usage, and it was not until local-area networks (LANs) were developed that e-mail acquired its popularity. E-mail is simply a combination of software, hardware, and communication technologies that allows a user to send and receive messages or documents to and from a user or set of users. The Electronic Mail Association (EMA), a Washington-based trade association, defines e-mail as “the generic term for the non-interactive communication of data, images or voice messages between a sender and designated recipient(s) by systems utilizing telecommunication links.” Although this definition encompasses technologies such as facsimile, telex, and communicating word processors, in this article, e-mail refers to the transmission of text message and (in some advanced systems) graphics, video, and sound over telephone lines or wireless technology from computer to computer. Within this context, EST can simply be defined as a computerized selfadministered questionnaire in which the researcher sends a questionnaire and the respondents receive, complete, and return the questionnaire through e-mail systems that bring together capabilities of both computers and telecommunication networks (Rice, 1990). The researcher can send a separate e-mail with the survey embedded to each respondent or multiple respondents, or the researcher can ask each respondent to access a web site where the survey is housed. In this latter case, on completion, the survey is submitted either as an e-mail to the researcher, or it can be downloaded to a data file. Anyone with a computer, modem, and telephone line can use EST if he or she has access to an online service, a commercial carrier, a LAN, or an Internet service provider. EST differs from WWW surveys that depend on the transmission of a questionnaire over the Internet to a database located at the site of the study (Subramanian, McAfee, & Getzinger, 1997) but rely on chance that somebody might come across the question-

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

96

naire, become interested in it, copy the document, complete it, and then return it (Swoboda, Muhlberger, Weitkunat, & Scheneeweib, 1997). Assuming no specialized controls such as passwords are used, anyone who has access to the Internet could potentially respond to such a survey. As such, although a WWW survey does not have the sampling controllability as EST (Stanton, 1998), it has the potential, with protection, to provide greater anonymity. Furthermore, unlike EST, WWW surveys can incorporate images, graphics, sound, and so on into the survey. Another advantage of a WWW survey is that it is always present and available, whereas EST is inherently episodic (Stanton, 1998). EST, however, does not require the researcher to develop appropriate data-screening methods and develop links that respondents should access to participate in the survey. EST also differs from surveys that are done through newsgroups or list servers, although sending e-mail questionnaires through newsgroups and list servers is clearly another form of EST. Like WWW surveys, these surveys suffer from low sampling control as well as self-selection bias because individuals on newsgroups or list servers are usually interested in particular issues. For example, a list server survey typically relies on surveying individuals with a special interest in a certain topic (Stanton, 1998). The potential receivers are often unknown to the sender and are characterized only by their interest in a particular subject (Batinic, 1997). There is virtually no control over individuals who are to complete the survey by the researcher (Swoboda et al., 1997).

Once the target population is identified, determination of the sampling frame, selection of a sampling procedure (probability vs. nonprobability), and computation of the sample size are important sampling-related issues that must be addressed while preparing for collecting data through EST. Although the latter two issues are rather straightforward statistical topics, we will attend primarily to the availability or construction of the sampling frame. The sampling frame can significantly reduce difficulties involved in the sampling process because it determines the sampling control and guides the direction of the inquiry in EST. The sampling frame is a master listing of population members usually used to draw a random sample from which data will be collected. Depending on the objective of the research, a sampling frame can be a listing of all the managers who work at a company, all the executives from Fortune 500 companies, and so forth. The quality of such a list primarily determines sampling biases. The ideal sampling frame is one in which every element of the population is not only represented but also only represented once. Clearly, when one is interested in sampling an appreciable segment of the human population, there will be problems with this ideal (Sudman, 1996; Tull & Hawkins, 1993). With respect to telephone and mail survey techniques, problems such as mailing lists being out of date or incomplete and phone directories not including unlisted numbers have been addressed. However, mostly because of lack of reliable documentation on addresses and profiles of e-mail users among different segments of the society, problems associated with sampling frames of EST have not been documented yet.

97

ORGANIZATIONAL RESEARCH METHODS

To our best knowledge, there is no good frame that lists individuals or households using or even having access to e-mail, although it has been reported that the Internet’s Network Information Center is working to produce a master directory of all users called InterNIC (T. Parker, 1995). We also do not know of any private firm that can currently offer unbiased frames for many populations. Indeed, there are many list brokers, list servers, and sources of opt-in lists on the Internet, the lists that are developed from people who have agreed to receive unsolicited e-mail. Yet, it has been suggested that commercial lists tend to be seriously flawed (Comley, 1996). Although it might be possible to create a good EST sampling frame by using e-mail addresses from online service providers or e-mail carriers, these companies observe privacy laws and policies, so this is not a viable option. Finally, even though there are several utilities such as Finger, Whois, and Netfind, to collect the Internet e-mail addresses, it has been claimed that only 1% or 2% of all the Internet users can be located with one of these methods (T. Parker, 1995). Accordingly, obtaining or constructing an unbiased or at least a useable sampling frame that allows probability sampling is currently the most serious challenge that EST imposes on researchers. Indeed, whether EST can successfully use probability sampling for general populations has not yet been established. On the other hand, even when available, many if not all e-mail frames should be used cautiously because they usually lack universal coverage of the population. As a rule, such claims as those echoed in the popular media on availability or accessibility of e-mail should not be considered as a surrogate measure of feasibility. In making a case, such aggregate numbers are of limited use because the availability of e-mail does not guarantee acceptance, usage, or compatibility (Kerr & Hiltz, 1982; Komsky, 1991). Although researchers have generally been able to assume that people receive their mail at their postal address, they cannot simply assume that all e-mail addressees are active. Moreover, although some of the problems in constructing e-mail frames pertain to self-administration in general (e.g., literacy and blindness), others may only relate to the ability to have and use a computer. The systematic exclusion associated with e-mail frames is severe, particularly because of e-mail’s relation to purchasing power. Research on computer usage reveals that computer users still share similar demographic characteristics of being young, well educated, and above average in income (Oppermann, 1995). Likewise, Couper and Rowe (1996) found that less educated, older respondents and those with less computer experience were less likely to complete a self-administered component of a computer-assisted personal interview survey on self-images, suggesting that the use of the computer may add additional constraints on the willingness or ability of respondents to complete an e-mail questionnaire. Clearly then, whether researchers can currently reach a representative sample through EST primarily hinges on the population under investigation. For example, if the investigation involves low-income households, minorities, or elderly populations, even simple random sampling attempts will be flawed because of high noncoverage error (Anderson & Gansender, 1995; Dillman, 1991). On the other hand, EST can prove quite beneficial for obtaining opinions related to new software. Likewise, because most large firms and their managerial/professional employees have access to e-mail, sample surveys of these populations are possible. In any case, using a stratified sampling approach rather than a random sample should lessen the degree of potential noncoverage error (Oppermann, 1995). However, the researcher should keep in mind

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

98

that almost all hypothesis testing and estimation procedures assume simple random samples, not stratified samples.

Nonsampling errors, which are often viewed more important than sampling errors, are all the other errors in a survey except those due to sampling method and sample size (Sudman, 1996; Tull & Hawkins, 1993). More specifically, they include coverage error—which has been discussed—nonresponse, and measurement error (Lavrakas, 1996). Nonresponse errors. A high number of nonresponses raise the question of whether those who responded to the survey are different from those who did not. Even in the absence of sampling biases, if nonresponses are not randomly distributed, then the data generated by EST will be biased because careful attempts at sample randomization have been eroded. In turn, such biased data severely influence the validity of the research and often result in invalid inferences (Gilbert, Longmate, & Branch, 1992; Dillman, 1991; Dillon, Madden, & Firtle, 1987). Nonresponses, even when random, may reduce what was an adequate sample to an inadequate one, forcing the researcher to either survey additional respondents or to find a remedy for these missing responses through postsurvey estimates (Hair, Anderson, Tatham, & Black, 1995). Nonetheless, increasing the response rate—as opposed to postsurvey adjustments, such as weighting cases by estimated probabilities of cooperation and known population quantities, imputation, and selection bias models that can work under certain assumptions to a limited extent (e.g., Hair et al., 1995; Kalton, 1983)—is clearly the safest strategy to reduce nonsampling errors. Overall, nonresponse errors in EST can generally be attributed to noncontacts (i.e., unreachables and refusals). It is promising to note that EST has been used as an effective means of gathering data in several academic and institutional settings (Anderson & Gansender, 1995), especially when one takes into account that most responses were attained without the use of any response inducement technique. Researchers using EST have reported response rates ranging from 19.3% to 76%. For example, Kiesler and Sproull (1986), on examining the response rate associated with EST and a postal survey, found a higher response rate for the paper survey (75% vs. 67%). When comparing EST with face-to-face interviews in a Fortune 500 manufacturing company, Sproull (1986) found a participation rate of 73% for EST versus 87% for interviews. In a survey of a major corporation’s overseas employees, L. Parker (1992) reported that the response rate associated with EST (68%) was significantly higher than when mail pouches (38%) were used. Although Schuldt and Totten (1994) reported a response rate of 56.5% for a mailed survey and 19.3% for EST, Kittleson (1995) obtained a response rate of 28.1% for EST and 76.5% for a postcard survey. Anderson and Gansender (1995), employing a survey to assess how and why people used a network system, obtained a response rate of 76% from 488 Free-Net users of a metropolitan area. Walsh, Kiesler, Sproull, and Hesse (1992) attained a response rate of 76% from a 93-item online survey of 300 science-net subscribers. Finally, in another study involving business school deans and division chairpersons on the use of total quality management, Bachmann, Elfrink, and Vazzana (1996) had a response rate of 65.6% for the mail questionnaire and 52.5% for EST.

99

ORGANIZATIONAL RESEARCH METHODS

Although further research is need on factors causing response rate differences across studies, these findings overall indicate that EST has been used as an effective means of gathering data in terms of response rates. They also indicate that EST tends to have a lower response rate compared to mail surveys, thereby pointing to the need for more research on response inducement techniques in the EST context. However, augmenting responses to EST through some incentives and procedures virtually has not been explored. Even fewer studies have investigated the influence of such tactics and incentives on response speed and response content. Although the researcher has no control over the unreachables, refusals stemming from such factors as survey design can be influenced as the research on mail survey suggests (Linsky, 1975; Yammarino, Skinner, & Childers, 1991). In the Recommendations section of this article, we further touch on this issue. Measurement errors. Measurement error is simply the deviation between the “true” and the observed responses (Dillman, 1991). Many types of errors stemming primarily from the data collection method used, such as those committed during the transformation of the data, are always at work in a survey. Broadly speaking, there are three sources of measurement error due to the survey instrument, the respondent, and/or the data collection technique. With respect to EST, measurement errors due to the survey instrument generally occur at the presurvey scale development stage. In this respect, EST does not differ from other surveying techniques in that it also requires the researcher to do everything that is needed in terms of developing a reliable and valid scale. Given the newness of EST, we were able to find only one study that detailed how the scale for a study using EST was developed (Clayton, Applebee, & Pascoe, 1996). With respect to respondent-based errors, some researchers have examined response content of computerized data collection techniques in general and EST in particular. Tourangeau and Smith (1996) noted that computer-assisted, selfadministered surveys produced similar outcomes to the more conventional selfadministered techniques. They further claimed that computerization by itself had little influence on the response quality and that better data quality often associated with the computer-administered questionnaires may be the result of the self-administration. However, in a comprehensive review on computerized data collection techniques, Leeuw, Hox, and Snijkers (1995) contended that in general, computerized techniques such as computer-assisted personal interviewing (CAPI) and computer-assisted selfinterviewing (CASI) have a positive effect on data quality. In the case of EST, these findings are largely consistent with the findings reported on response quantity, but they are inconclusive with respect to response quality because how responses are influenced by e-mail communication itself has not been conclusively demonstrated. Some researchers have asserted that EST generally conveys little social information, so respondents experience less evaluation anxiety than when they respond using other survey modalities (Kiesler & Sproull, 1986; Kiesler, Zubrow, & Moses, 1985; Sproull, 1986). Kiesler (1989) and Sproull (1986) suggest that because e-mail tends to reduce social concerns and constraints on individuals, EST respondents are less concerned about reporting negative and socially inappropriate things about themselves. Couper and Rowe (1996) found that those who completed a self-administered computer interview reported a more positive self-image than those who had the interviewer help, after controlling for respondent characteristics related to self-image. Ayidiya and McClendon (1990) noted that EST might influence the acquiescence of

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

100

respondents, thereby lessening some respondents’ propensity to agree with survey statements more often than they would do with a pencil-and-paper survey. In another study, Sproull (1986) observed that questionnaire data were slightly more complete when using conventional mail than when e-mail was used. However, when comparing these methods, it was found that although there were no differences in the nature of answers provided by the participants, the e-mail survey elicited more extreme responses. Perhaps this is because, as Kiesler (1989) has suggested, e-mail communication loosens social concerns and constraints on people, so that they are less concerned about saying negative things and/or revealing socially inappropriate beliefs. Indeed, in a meta-analysis of self-disclosure on computer forms in general, Weisband and Kiesler (1996) found that across 39 studies using 100 measures, computer administration was associated with increased self-disclosure compared to face-to-face interviews. The researchers speculated that this finding might be because working on a computer creates a sense of privacy. Corman (1990) directly attempted to validate data generated by a computerized survey by comparing them with postal survey data. To do this, data from two separate groups were compared in three different ways, including test-retest reliability, criterion validity, and internal consistency. The results indicated that the computerized survey method produced considerably higher criterion validity and slightly higher test-retest reliability than did the postal survey. Corman attributed these findings to the novelty of the computerized data collection approach in that computer respondents may have taken greater care in filling out the survey. With respect to measurement errors introduced by the data collection technique itself, several researchers have looked at how EST compares to the postal survey technique in terms of item completeness and responses to open-ended questions. Liefeld (1988) compared the response effects of a computer-administered questionnaire to self-completion and personal interview techniques. Liefeld found that with the exception of multiresponse/knowledge-type questions, there was little difference in response patterns among techniques. The computer-assisted technique, however, produced higher means for most of the items. Bachmann et al. (1996) also found that there were no significant differences in the responses and respondents’ tendency to leave an item blank or to comment on questions between individuals receiving a mail or an e-mail questionnaire. Yet, the e-mail respondents showed a greater willingness to respond to open-ended questions (21.9% vs. 4.8%). Kiesler and Sproull (1986) found similar responses between a paper and an electronic survey but also reported greater item completeness resulting for the electronic survey. Finally, Schaefer and Dillman (1998) found that EST generated greater item completion and lengthier responses to open-ended questions. In turn, all this suggests that EST might be particularly useful to conduct surveys involving open-ended questions. On the other hand, some have expressed concern that self-administered questionnaires can suffer from sequence biasing because respondents are able to see the whole questionnaire and may consider questions together rather than each question individually (Churchill, 1995, p. 371). Hence, responses to an earlier question can prime particular beliefs and make them more accessible, serve as a standard of comparison for subsequent items, or be a source for consistency pressure (Lockhart & Russo, 1996). Although sequencing bias is a concern for conventional paper surveys, e-mail questionnaires may be sent in such a way that the computer displays each question

101

ORGANIZATIONAL RESEARCH METHODS

exactly as wished and does not display all the questions until previous ones have been completed. Table 1 displays some other specific possible sources of nonsampling errors and some techniques for reducing or controlling them when using EST. Table 1 is based on the extant research on self-administered questionnaires in general. Thus, most of our suggestions are not unique to EST because they represent general guidelines for identifying or controlling some nonsampling errors.

The recent proliferation of data collection techniques has made it important that researchers have a comparative knowledge of these techniques to make an appropriate decision. For example, although an electronic survey may generate a lower response rate compared to a postal survey, the choice of which one to use might also depend on such issues as survey costs, speed, convenience, and the like. Several researchers have indicated that the assessment of any data collection technique should take into account its comparative performance as well (Lockhart & Russo, 1996, p. 125). Salant and Dillman (1994) noted that “no single method can be judged superior to the others in the abstract. Instead each should be evaluated in terms of a specific study topic and population, as well as budget, staff, and time constraints” (p. 35). More specifically, Kiesler and Sproull (1986) aptly suggest comparing EST with other available alternatives as a good first step. Accordingly, Table 2 compares EST with personal interviewing, telephone interviewing, and the mail questionnaire technique along several dimensions considered to be important determinants of the choice of a data collection technique (cf. Dillman, 1978, 1991; Dillon et al., 1987; Erdos, 1983; Markus, 1994; Watson, 1998). These determinants include sampling issues, cost/efficiency and convenience, information richness, respondent issues, response outcomes, and future prospects in terms of usage. Given that we have already addressed sampling issues, we will first turn to issues of cost, efficiency, and convenience. EST has the potential of radically changing the economics of conducting surveys. With the file transfer capability of computers, EST does not require usage of paper at any stage, thereby avoiding the costs associated with the manual entry of raw data or electronic scanning. With EST, according to one estimate, the marginal cost of storage, communication, and dissemination of a 30-page document can be less than a penny (Kambil, 1995). In many cases, a hard copy of questionnaires may not be necessary, which in turn eliminates the need to print labels, type addresses, purchase envelopes, and so on. Furthermore, although the costs of the other techniques tend to be proportional to the size of the sample, the cost associated with adding additional respondents in EST is practically zero. The primary costs of EST include assembling and checking the e-mail list(s), creating or buying software and supporting databases, and accessing e-mail. When these are available, the cost of EST is trivial to the researcher. In any case, the marginal costs of collecting and communicating data through EST are much lower than costs of interviewing, telephoning, and sending questionnaires through postal services (Mehta & Sivadas, 1995). With respect to speed, sending questionnaires out and receiving them via EST are definitely very fast. An e-mail questionnaire can be sent to one thousand people as easily as to one person automatically, and all potential respondents immediately receive (text continued on p. 104)

102

Respondents exhibit a persistent tendency to respond favorably or unfavorably, have no opinion, or provide inconsistent responses.

NOTE: EST = e-mail survey technique.

Exceptions

Unintentional Respondents fail to provide their real thinking because respondent errors they misunderstand the questions, or they guess.

Although respondents participate in the study, their responses are not complete or incorrect. Intentional Respondents fail to tell the truth or do not provide respondent errors responses to some items.

Refusals

Respondent errors

Respondents do not respond to the questionnaire.

Unreachables

Definition Failure to obtain information from some elements of the sample. Designated respondent cannot be reached via EST.

Nonresponse

Type

1. 2. 3. 1. 2. 3. 4. 5. 1. 2. 3.

8. 9. 10.

3. 4. 5. 6. 7.

1. 2. 1. 2.

Emphasize anonymity. Use validation checks. Use a third-person technique when possible. Screen survey questions before using. Use validated scale. Provide good questionnaire instructions and, when possible, use examples. Use reversals of scale end points or question statements. Use visual aids when possible. Recheck the questions’ wording. Exclude inconsistent responses. Treat some respondents as outliers.

Check the e-mail address for accuracy. Check for temporary local and nonlocal systemwide e-mail problems. Use prior e-mail notifications. Attempt to convince the respondent of the value of the research and his or her participation. Ensure anonymity and confidentiality. Increase credibility through sponsorship manipulation. Offer some incentives such as gifts or money to motivate. Shorten the questionnaire when possible. Send individual e-mail questionnaires rather than forwarding or using chain sending. Make the questionnaire appealing through visual aids. Use an e-mail follow-up. Oversampling.

Some Possible Techniques for Handling

Table 1 Some Nonsampling Errors in EST and Techniques for Handling Them

103 Low transmission of nonverbal cues, conveying low sense of personalization, timely feedback, and transmission of medium varied language. High respondent convenience— time and discretion to respond. Some anonymity is possible. Requires literacy and computer skills; compatibility of e-mail packages could be a problem.

Information richness

Respondent issues

Maximum respondent convenience—time and discretion to respond. Full anonymity possible. Requires literacy.

Based on the same four criteria of media richness, its richness is the lowest.

Least expensive and most efficient. Low cost/moderately efficient.

Cost efficiency

Sampling frame for many populations exist and are usually easy to obtain and construct. Limited possibility of sampling control.

Sampling frames for many populations do not exist and are usually difficult to obtain and construct. Limited possibility of sampling control.

Mail Questionnaire

Sampling Issues

EST

Telephone Interview

Based on the same four criteria of medium richness, it is the second richest of the four techniques.

Highest cost and least efficient.

Low respondent convenience— Low respondent convenience— no time and discretion to respond. no time and discretion to respond.

Based on the same four criteria of medium richness, it is the richest of the four techniques.

High cost and least efficient.

Sampling frames are usually Sampling frames for some easily obtained and constructed. populations already exist and High potential for sampling are usually easy to obtain control. and construct. High sampling control.

Personal Interview

Table 2 EST, Mail, Personal, and Telephone Data Collection Techniques Compared

104

usage.

Privacy concerns may limit its

High likelihood of wide popularity.

Self-selection bias possibility. Less prone to field errors.

Very easy to process and analyze.

No control over response speed.

for eliciting sensitive responses.

biasing possibility. Appropriate

involuntary error, and sequence

distortion, less socially desirable, thoughtful responses, low

Medium to high. No interviewer

NOTE: EST = e-mail survey technique.

tions,

Among

muni-

near

Future prospects age

ten

bility

over

sequence

from

likeli-

Response outcomes of

Highest. Possibility of interviewer

Medium to high. Possibility

not be influenced by increasing

High likelihood that its usage will

Low item nonresponse.

data. Often difficult to process.

appropriate for eliciting sensitive

Control over its speed. Not

sequence biasing possibility.

from others avoided. Low-

interviewing, and given videoconferencing capability of computers, its usage among affluent populations might decrease.

assisted interviewing, telephone

increasing use of computer-

EST and telephone interviewing. use of e-mail surveys. However,

proportionate to increase in

Decreasing popularity,

process and analyze. Selfselection bias possibility.

over its speed. Easy to

sensitive responses. No control

Appropriate for eliciting

sequence biasing possibility.

low involuntary error, and

its usage might decrease.

high e-mail user popula-

cation gets cheaper.

future as telephone com-

will remain high in the

High likelihood that its us-

easy to process. Low item Nonresponse.

to elicit sensitive data. Of-

its speed. Limited applica-

biasing possibility. Control

others avoided. Low-

hood that contamination

distortion, less socially distortion and social desirability. interviewer distortion and desirable, thoughtful responses, High likelihood that contamination social desirability. High

Medium to low. No interviewer

105

ORGANIZATIONAL RESEARCH METHODS

the questionnaire regardless of their location (Kiesler & Sproull, 1986). Likewise, responses can flow back just as rapidly when respondents check their e-mail daily. None of the existing survey techniques, including facsimiles, can provide researchers with such speed in reaching specified individuals. Thus, EST promises to provide data in a timely manner when a quick answer is being sought. Indeed, as Mehta and Sivadas (1995) have suggested, EST generates fast data not only because e-mail is a fast communication medium but also because individuals are likely to respond more quickly to an e-mail, whereas in comparison, a mail questionnaire may remain on an individual’s desk for a long time. Likewise, EST is one of the most convenient data collection techniques. Once the survey instrument is developed, it can be e-mailed easily. EST saves all the time that regular postal survey requires for photocopying questionnaires, stuffing envelops, and addressing outgoing mail. Because a copy of all outgoing e-mail can be saved in an electronic mailbox, EST also makes repeated communications with respondents, such as sending follow-up questionnaires, extremely easy. Given that EST allows the researcher to know in a moment if the message has been received—and even when it is opened—identification, elimination, and replacement of unreachable respondents are easily accomplished. EST can also reduce many field and administration errors, such as deciphering respondents’ handwriting and allowing researchers to encode data without transcribing from paper. According to information richness theory (Daft & Lengel, 1984; Daft, Lengel, & Trevino, 1987), computer-mediated communications, such as electronic mail, are less rich in information-carrying capacity than face-to-face communication. Within this perspective, face-to-face interaction is seen as the richest medium, followed by telephone, electronic mail, letters, and memos. In effect, e-mail offers limited interactivity and informational cues compared to face-to-face interactions. Indeed, compared to other surveying techniques such as personal and telephone interviewing, EST involves low transmission of nonverbal cues, varied language, timely feedback, and low sense of personalization. With EST, lack of complete anonymity is also a concern. Truly anonymous responses are not possible with EST. When a respondent returns a questionnaire using the reply function in an e-mail package, his or her e-mail address, including the name and affiliation, is automatically conveyed to the surveyor. This lack of anonymity might in turn affect response rates as well as response content in EST. Moreover, if respondents complete the survey at their place of employment, it is possible that an electronic trail will remain and that their responses could be uncovered, which, depending on the nature of the survey questions, could raise confidentiality issues as well. Previous research has indicated that respondents’ beliefs about anonymity affect responses to computer-based surveys (Kantor, 1991). There is also a wide recognition among researchers that whether anonymity is provided affects responses in mail surveys (Albaum, 1987). However, if anonymity or confidentiality is a major concern, we suggest one of two approaches: First, use a web-based survey because when respondents submit their answers back to the researcher, their identifying information is not automatically conveyed, and second, respondents could be directed to go through a free Internet e-mail account such as “Hotmail” and, if necessary, use a fictitious name. With respect to our second suggestion, note that it is extremely difficult, if not impossi-

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

106

ble, for an outsider to obtain the identity of a respondent, given the privacy assurance policies that such providers offer. Indeed, most providers indicate that the only exception to such a policy is in the case of an alleged crime. Hence, we believe that this approach would provide a high degree of confidentiality. However, having said that, irrespective of the actual degree of confidentiality achievable by such approaches, EST researchers should be aware that as long as respondents perceive that complete confidentiality is necessary and could be compromised, response rates are likely to suffer. Noncompatibility among e-mail packages is another important disadvantage of EST compared to the other three techniques. E-mail is still not a standardized medium, despite the growing demand of users for standardization. For instance, LAN e-mail packages are different not only from one another but also from the Internet e-mail. Although most online service providers and LAN-based e-mail systems permit transfer of binary files, the Internet still uses an ASCII format. Nonstandardization among various e-mail systems causes discrepancies between the form of questionnaire sent and that received by respondents (Oppermann, 1995). Relatedly, researchers should also be concerned with such issues as free hard disk space (either in their or their service provider’s hard drive), e-mail bandwidth, and server capacity while using EST. For instance, a large number of responses could create problems because returned e-mail takes up a lot of space on a system. It is essential to download responses in a timely manner while using EST for full-scale surveys. Finally, with respect to the future prospects of EST, Bloom, Milne, and Adler (1995) have noted that although new information technologies increase efficiency and effectiveness of data collection, their haphazard use can lead to some legal difficulties. For example, although legislation concerning privacy of e-mail communication is still in its infancy, such legislation would effectively destroy the use of EST in many contexts. As e-mail addresses are considered to be more personal than mail addresses, sending unsolicited e-mail questionnaires might be considered an intrusion on a person’s seclusion or solitude or into his or her private affairs (Dyson, 1994). Although such a case has yet to be made in the courts to our knowledge, the use of e-mail for mass mailing, known as “spamming,” has been very much debated. A popular online service was sued in three states for deceptive advertising because some of its customers were using the system for mass mailing instead of personal messaging (Shannon & Rosenthal, 1993). The e-mail overload that many individuals increasingly face is likely to be another important disadvantage of EST (Clayton et al., 1996). Some researchers have indicated that the increasing e-mail overload can cause individuals to feel overwhelmed (Garton & Wellman, 1995). It is likely that the more individuals receive e-mail, the less likely they are to spend time responding to EST. Although individuals do not have many clues for judging the importance of e-mail they receive, it is very easy for them to sort out and delete e-mail that they are not interested in. In turn, this makes it particularly important that researchers successfully employ some incentives and response inducement strategies to make respondents complete the survey. In sum, researchers have to make a number of trade-offs when they decide to use EST. Using EST requires harmony among a variety of resources, including human, hardware, and software. It is thus essential that the decision to use EST takes into account all relevant issues, including sampling issues, nonsampling errors, and com-

107

ORGANIZATIONAL RESEARCH METHODS

parative performance that are discussed throughout this article. We next provide some practical recommendations as to how EST can be used productively.

We suggest that researchers first go through relevant directories and then check the following sources: The Usenet Addressees Database, Knowbot Information Server (KIS), GOPHER, WAIS, The Usenet Newsgroup, and Netfind. Many of these sources can be used to locate individuals’ e-mail addresses or names. For example, Mehta and Sivadas (1995) wrote a program that collected e-mail addresses and signatures of the people who posted articles on newsgroups; then, based on confirmation, they determined mailing addresses of individuals. Put simply, because some Internet users already have organized themselves into mailing lists and discussion groups according to their interests, it is possible to use these lists or combine several such lists to construct specific sample frames. The directories seem to be especially beneficial if the population of interest is academic staff because a growing number of institutions are putting their staff and student directories online in publicly accessible formats and are being incorporated into Gopher and WAIS (Kochmer & Northwest, 1993, p. 53). It is also possible that the researcher can develop sampling frames using more conventional sources such as mailing lists and phone directories and then use these sources to determine e-mail addresses of individuals through several Internet search engines or the aforementioned sources. In fact, some of the sources, such as Finger and KIS, provide additional information such as telephone number, postal address, and so on of individuals, thereby allowing the information obtained from the conventional sources to be double-checked.

Based on our experience with e-mail surveys and the extant literature, we recommend the following broad approaches. First, because many e-mail users have strong concerns about the use of their e-mail boxes and consider them to be more private than their mail addresses, it may prove beneficial to notify sample members about the incoming e-mail questionnaire through an e-mail or postal prior notification (Emery, 1995, p. 344). The prior notification should not only ask for permission but also let the respondent know the purpose of the survey, why their involvement is important, how responses will be used, the sponsor of the survey, person(s) to contact for questions, expected date of the survey, and a statement indicating the strict confidentiality of the respondent’s e-mail address and response. Yu and Cooper (1983) suggest that a prior notification should simultaneously include the following: (a) a social utility appeal that emphasizes the worthiness of the survey, (b) an egoistic appeal that stresses the respondent’s place and importance in completing the survey, and (c) an appeal to help the researcher in completing an important project. Given concerns over anonymity in

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

108

EST, it is also important that the cover letter assure respondents that their responses will be held confidentially and mention some possible steps that will be taken toward this goal. For example, the researcher may state that screen headers will be deleted once the responses are received (Goree & Marszalek, 1995). The researcher could also offer some options for responding anonymously such as placing the questionnaire on the WWW or mentioning the possibility that the respondent could send the completed questionnaire through regular mail. Second, questionnaire layout and design issues should also be taken into account. EST should be accompanied by very clear and simple instructions, such as how to reply, that will not consume much of the respondents’ time. In particular, “extra” features that would minimize questionnaire completion time and maximize respondent convenience should be pursued. For example, scrolling, jump screen, quitting, no automatic next, no keyboard responses, help screens, and a progress thermometer indicating completed percentage of the questionnaire were incorporated and successfully used by Beebe, Mika, Harrison, Anderson, and Fulkerson (1997). Like many other researchers (e.g., Johnston & Walton, 1995), we also believe that whenever possible, researchers should use simple graphics-animated questionnaires because many people’s perceptions of computers are similar to that of TV rather than postal mail. But we caution that such devices consume enormous amounts of memory and make opening such messages time-consuming and frustrating if the individual is on an older modem. Graphics, sounds, and special formatting of the questionnaire may not translate across various e-mail software packages. To solve such problems, the researcher could check the major e-mail systems used by respondents and make sure that the formatting and appearance of the questionnaire remain the same after transmission (Tse, 1998). However, unless the researcher knows the capabilities of the e-mail systems of the people included in the survey, we suggest that it is best to keep the survey as simple and short as possible. As an aside, it should be remembered that respondents who use commercial online services effectively incur some cost in sending and receiving messages. Only after incorporating such approaches should the researcher attempt to manipulate some incentives and factors to increase responses to EST without eschewing that different populations may react differently to the factors, which the large body of research on mail survey strongly points out (Childers, Pride, & Ferrell, 1980; Jobber & Sanderson; 1985; Kaldenberg, 1994). A plethora of studies have been undertaken to identify factors that might potentially influence responses to a mail survey, including monetary offerings, lottery tickets, contributions to a charity, an offer of survey results, cover letter, personalization, anonymity, topical interest, sponsorship, questionnaire design, prior notification, follow-up, humor, type of mailing, and deadline. Excellent reviews of this body of research have been written by Church (1993); Fox, Crask, and Kim (1988); Heberlin and Baumgartner (1978); Jobber (1986); Linsky (1975); Veiga (1984); Yammarino et al. (1991); and Yu and Cooper (1983). Although EST may resemble a postal survey and share some characteristics of it, not all of these response inducement techniques are transferable to the EST context because EST has its own unique features. For example, it is impossible to attach a monetary incentive, such as a dollar bill, to an electronic survey or to attach a nonmonetary incentive such as a pen.

109

ORGANIZATIONAL RESEARCH METHODS

In a recent review of the literature on factors inducing responses to mail surveys, Roth and BeVier (1998) concluded that prior notification, follow-up, monetary incentives, personalization, and salience of the issues investigated have consistently been found to positively influence response rates. Although not conclusive, a few recent studies indicate that responses to EST can also be increased through these strategies (e.g., Kittleson, 1997; Schaefer & Dillman, 1998). In addition to these factors, we also expect that sponsorship might positively influence responses to EST. For example, alumni may respond to a university or business school–sponsored surveys more readily because of psychological indebtedness (Paxson, 1995). However, there is still lack of focused research on determining how strategies such as sponsorship, the opportunity to complete the survey through regular mail or the WWW, summary of the research, donations to a charity, purchase credits, and discounts affect response rates and response content in EST.

Although we have primarily focused on the use of EST as an alternative, independent surveying technique, EST can also be used in combination with some other techniques or at different stages of a research project. In fact, it is sometimes desirable to combine several techniques, thereby offsetting the strengths and limitations of any single technique (Aaker et al., 1995; Lockhart & Russo, 1996). In addition, using EST with some other techniques such as postal surveys can allow experimentation with much more diverse populations, not only with populations having nearly universal coverage. This strategy can reduce coverage error that is usually associated with EST as well (Schaefer & Dillman, 1998). EST can be used in combination with almost any data collection technique, including telephone interviews, personal interviews, postal surveys, or the other Internetbased surveys, as well as to send prior notifications and follow-ups. Schaefer and Dillman (1998) suggested that because of its cost and speed advantages, EST is ideal for a first mode of contact in surveys, such that the researcher could begin with EST and use progressively more expensive methods until enough responses are obtained. Or the researcher could simply use EST among respondents having e-mail addresses and use the postal technique to survey those without access. Likewise, one of the greatest benefits of EST may be realized when it is used to send prior notifications and followups to increase responses to postal surveys and to EST itself. EST can also be helpful in pretesting a survey instrument to increase the quality and quantity of responses in a full-scale survey (Swoboda et al., 1997). The cost and speed advantages of EST make it possible to conduct surveys aimed at establishing reliability and validity of survey instruments. This initial process might also result in early respondents commenting on the process of filling out the survey as well. Indeed, several researchers have successfully used the pretest to get feedback for identifying the optimal approach for conducting a mail survey (e.g., Hunt, Sparkman, & Wilcox, 1982). A study by Clayton et al. (1996) demonstrates how EST can be used in combination with other techniques, as well as for pilot surveying purposes. After developing the survey instrument through two nominal group discussions, the researcher used the EST to send the instrument for pilot survey purposes. They then sent a paper follow-up survey to those who did not respond to the EST version. Once the survey instrument

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

110

was fully developed, the researcher used EST and then used a final paper-based survey among those who did not respond to the electronic version. Through this mixed usage of EST, the researchers were able to increase reliability of the survey instrument as well as response rates while reducing the cost of the survey.

When a representative sample can be formed, it is apparent that EST can be used in several types of organizational studies employing a self-administered data collection technique. It can provide a good opportunity for those researchers who have a limited research budget or who are interested in fast data gathering. Because e-mail obliterates time and zone constraints, surveying with e-mail can prove very beneficial when the sample is scattered or mobile or consists of members from such populations as executives who will not participate in personal or phone interviewing but may respond to an e-mail questionnaire at their convenience. Indeed, EST has provided researchers with the ability to reach rare, hidden, and geographically disperse populations (O’Lear, 1996; Sell, 1997). Moreover, because e-mail addresses are personal, sending the questionnaire to the right person can be more effective via EST than a mailed questionnaire sent to a position wherein it is not always clear who is responding or usually results in questionnaires being thrown away before reaching the person who has the required information. That is, an e-mail survey intended for an individual is more likely to be read and answered by that individual (Mehta & Sivadas, 1995). Likewise, compared to other noncomputerized surveying techniques, EST is inexpensive, fast, and less prone to many known sources of nonsampling errors such as data collection and data processing. In addition, compared to surveys over the Internet such as newsgroup surveys, EST is the easiest to use and has better sampling control. On the other hand, EST’s brimming potential is at present inhibited by its lack of universal coverage, biased sampling frames, incompatibility of current e-mail systems, restricted binary file transfer, and technicalities involved in sending and receiving questionnaires. Of them, noncoverage error arguably presents the most significant impediment to the increased use of EST. It makes EST unsuitable for conducting surveys of many populations. There are no e-mail lists for most populations that can serve as sampling frames, and constructing them can be very difficult, costly, and timeconsuming. Even when they exist, such frames are usually biased, primarily because e-mail users by gender, age, race, income, education, and other major demographic characteristics are very different from their populations, in a sense creating a unique population. In particular, when the research project involves sample surveying of heterogeneous populations such as households, the researcher should be extremely cautious in the decision to employ EST in isolation. We believe that until local sites on the Internet develop and maintain local e-mail lists of general populations, e-mail lists will usually suffer from being incomplete and outdated, much like traditional mailing lists and telephone directories. With respect to future research, we feel that the best research will likely come from taking an interdisciplinary focus because EST is a multifaceted phenomenon. Within this context, we feel that each component of the assessment framework should and could be scrutinized for development of theoretical arguments. Meanwhile, empirical

111

ORGANIZATIONAL RESEARCH METHODS

research should investigate the comparative performance of EST, particularly vis-àvis postal surveying. So far, EST has mostly been used within organizations and educational institutions, so it will be fruitful and interesting to see findings from studies conducted in different organizational and institutional settings. Some research must be undertaken to explore different incentives to augment EST response rates because with time, as the novelty of e-mail fades, reactions of computer users toward unsolicited e-mail messages may become more negative. Because there is no way of knowing whether approaches used to increase response rates are only initial stimulators, the influences of these approaches on the quality of the data and the randomness of the sample must be simultaneously investigated. Furthermore, we believe such issues should be investigated through some experimentally designed studies. We urge researchers to use a factorial experimental design to provide greater precision for estimating overall variable effects, determine the interactions between the factors, and allow the range of validity of the conclusions to be extended by the insertion of additional variables (Cox, 1992; Montgomery, 1991). In particular, fractionated-factorial experiments allow a wide range of factors to be tested with small sample sizes (Box & Hunter, 1961; Devor, Chang, & Sutherland, 1992). We finally urge researchers to undertake research that focuses on EST as its primary goal rather than treating it as a topic of secondary importance and making post hoc investigations and predictions from project data that had another major agenda in mind. Unfortunately, many studies that we reviewed on EST are of this type. In sum, the rapid growth of global telecommunication networks, particularly the Internet, has placed emphasis on EST as a surveying technique. EST is attractive because it facilitates easy data management, location flexibility, and rapid transmission of the survey to all respondents across time and space. Yet, our review suggests that it is too early to declare that EST has become a rival or a better technique than major noncomputerized data collection techniques. Given current trends of rapidly increasing e-mail availability, computer expertise, e-mail packages’compatibility, and decreasing computer hardware and software cost, it is, however, conceivable that in the near future, electronic surveying of many diverse populations will be possible. Predicting this trend, several companies have recently introduced survey software packages that work with e-mail systems to create, collect, and tabulate survey results. In the long term, it seems that the most serious threats to EST will be legal ones and potentially negative attitudes toward responding to a survey even in electronic format.

Aaker, D. A., Kumar, V., & Day, G. S. (1995). Marketing research. New York: John Wiley. Albaum, G. (1987). Do source and anonymity affect mail survey results. Academy of Marketing Science, 15, 74-81. Anderson, S. E., & Gansender, B. M. (1995). Using electronic mail surveys and computermonitored data for studying computer-mediated communication systems. Social Science Computer Review, 13, 33-46. Ayidiya, S. A., & McClendon, J. M. (1990). Response effects in mail surveys. Public Opinion Quarterly, 54, 229-247. Bachmann, D., Elfrink, J., & Vazzana, G. (1996). Tracking the progress of e-mail vs. snail-mail. Marketing Research, 8, 31-35.

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

112

Baker, R. P. (1992). New technology in survey research: Computer-assisted personal interviewing (CAPI). Social Science Computer Review, 10, 145-156. Batinic, B. (1997). How to make an Internet based survey. Advances in Statistical Software, 6, 125-132. Beebe, T. J., Mika, T., Harrison, P. A., Anderson, R. E., & Fulkerson, J. A. (1997). Computerized school surveys. Social Science Computer Review, 15, 159-169. Bloom, P. N., Milne, G. R., & Adler, R. (1995). Avoiding misuse of new information technologies: Legal and societal considerations. Journal of Marketing, 58, 98-110. Box, G. E., & Hunter, J. S. (1961). The 2k-p factorial design part I. Technometrics, 3, 311-351. Chadwick, B. A., Bahr, H. M., & Albrecht, S. L. (1984). Social science research methods. Englewood Cliffs, NJ: Prentice Hall. Childers, T. L., Pride, W. M., & Ferrell, O. C. (1980). A reassessment of the effects of appeals on response to mail surveys. Journal of Marketing Research, 17, 365-370. Church, A. H. (1993). Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opinion Quarterly, 57, 62-79. Churchill, G. A. (1995). Marketing research methodological foundations. New York: Dryden. Clayton, P., Applebee, A., & Pascoe, C. (1996). E-mail surveys: Old problems with a new delivery medium. LASIE, 27, 30-39. Comley, P. (1996). The use of the Internet as a data collection method [Online]. Available: http://www.sga.co.uk/esomar.html. Corman, S. R. (1990). Computerized vs. pencil and paper collection of network data. Social Networks, 12, 375-384. Couper, M. P., & Burt, G. (1994). Interviewer attitudes toward computer-assisted personal interviewing (CAPI). Social Science Computer Review, 12, 38-54. Couper, M. P., & Rowe, B. (1996). Evaluation of a computer-assisted self-interview component in a computer-assisted personal interview survey. Public Opinion Quarterly, 60, 89-105. Cox, D. R. (1992). Planning of experiments. New York: John Wiley. Dacko, S. G. (1995). Data collection should not be manual labor. Marketing News, 29, 31-35. Daft, R. L., & Lengel, R. H. (1984). Information richness: Anew approach to managerial behavior and organizational design. Research in Organizational Behavior, 6, 191-233. Daft, R. L., Lengel, R. H., & Trevino, L. K. (1987). Message equivocality, media selection, and managerial performance: Implications for information systems. MIS Quarterly, 11, 355-366. Devor, R. E., Chang, T., & Sutherland, J. W. (1992). Statistical quality design and control: Contemporary concepts and methods. New York: Macmillan. Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: John Wiley. Dillman, D. A. (1991). The design and administration of mail surveys. Annual Review Sociology, 17, 225-249. Dillon, W. R., Madden, T. J., & Firtle, N. H. (1987). Marketing research in a marketing environment. Homewood, IL: Irwin. Dyson, E. (1994). If nothing on e-mail is private, just say so. Computerworld, 27, 22, 33. Emery, V. (1995). How to grow your business on the internet. New York: Coriolis. Erdos, P. L. (1983). Professional mail surveys. Malabar, FL: Krieger. Fox, R. J., Crask, M. R., & Kim, J. (1988). Mail survey response rate: A meta analysis of selected techniques for inducing response. Public Opinion Quarterly, 52, 229-247. Gaiser, T. J. (1997). Conducting on-line focus groups. Social Science Computer Review, 15, 135-144. Garton, L., & Wellman, B. (1995). Social impacts of electronic e-mail in organizations: A review of the research literature. In B. R. Burleson (Ed.), Communication yearbook (Vol. 18, pp. 434-452). Thousand Oaks, CA: Sage.

113

ORGANIZATIONAL RESEARCH METHODS

Gates, R. H., & Jarboe, G. R. (1987). Changing trends in data acquisition for marketing research. Journal of Data Collection, 27, 25-29. Gilbert, G. H., Longmate, J., & Branch, L. G. (1992). Factors influencing mailed health surveys. Public Health Reports, 107, 576-584. Goree, C. T., & Marszalek, J. F., III. (1995). Electronic surveys: Ethical issues for researchers. College Student Affairs Journal, 15, 75-79. Gwyne, S. C., & Dickerson, J. F. (1997, April 21). Lost in the e-mail. Time, pp. 88-90. Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (1995). Multivariate data analysis with readings. Englewood Cliffs, NJ: Prentice Hall. Havice, M. J. (1990). Measuring non-responses and refusals to an electronic survey. Journalism Quarterly, 67, 521-530. Heberlin, T. A., & Baumgartner, R. (1978). Factors affecting response rate to mailed questionnaires: A quantitative analysis of the published literature. American Sociological Review, 3, 447-462. Helliwell, J. (1986). Global standards promise fluency in electronic mail. PC Week, 3, 73-77. Higgins, C. A., Dimnik, T. P., & Greenwood, H. P. (1987). The DISKQ survey method. Journal of the Market Research Society, 29, 437-445. Hunt, S. D., Sparkman, R. D., & Wilcox, J. (1982). The pretest in survey research: Issues and preliminary findings. Journal of Marketing Research, 19, 269-273. Jobber, D. (1986). Improving response rates in industrial mail surveys. Industrial Marketing Management, 15, 183-195. Jobber, D., & Sanderson, S. M. (1985). Evaluating the effectiveness of two variables on industrial mail survey response rates. Industrial Marketing Management, 14, 119-121. Johnston, J., & Walton, C. (1995). Reducing response effects for sensitive questions: A computer-assisted self interview with audio. Social Science Computer Review, 13, 304-319. Kaldenberg, D. O. (1994). Mail survey response patterns in a population of elderly: Does response deteriorate with age? Public Opinion Quarterly, 58, 68-76. Kalton, G. (1983). Compensating for missing survey data. Ann Arbor: University of Michigan Press. Kambil, A. (1995). Electronic commerce: Implications of the Internet for business practice and strategy. Business Economics, 30, 27-32. Kantor, J. (1991). The effects of computer administration and identification on the Job Descriptive Index (JDI). Journal of Business & Psychology, 5, 309-323. Karweit, N., & Meyers, E. D. (1983). Computers in survey research. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of survey research (pp. 379-413). New York: Academic Press. Kerr, E. B., & Hiltz, S. R. (1982). Computer-mediated communication systems. New York: Academy Press. Kiesler, S. (1989). E-mail brings out the best and worst in group interaction. Infoworld, 4-9. Kiesler, S., & Sproull, L. S. (1986). Response effects in the electronic survey. Public Opinion Quarterly, 50, 402-413. Kiesler, S., Zubrow, D., & Moses, A. M. (1985). Affect in computer-mediated communication: An experiment in synchronous terminal-to-terminal discussion. Human-Computer Interaction, 1, 77-104. Kittleson, M. J. (1995). An assessment of the response rate via the postal service and e-mail. Health Values, 18, 27-29. Kittleson, M. J. (1997). Determining effective follow-up of e-mail surveys. American Journal of Health Behavior, 21, 193-196. Kochmer, J., & Northwest. (1993). Northwestnet’s guide to our world Internet (4th ed.). Washington, DC: Northwest Academic Computing Consortium.

Simsek, Veiga / ELECTRONIC SURVEY TECHNIQUE

114

Komsky, S. H. (1991). A profile of users of electronic mail in a university. Management Communication Quarterly, 4, 310-340. Lavrakas, P. C. (1996). To err is human. Marketing Research, 8, 30-36. Leeuw, E. D., Hox, J. J., & Snijkers, G. (1995). The effect of computer-assisted interviewing on data quality: A review. Journal of the Market Research Society, 37, 325-335. Liefeld, J. P. (1988). Response effects in computer-administered questioning. Journal of Marketing Research, 25, 405-409. Linsky, A. S. (1975). Stimulating responses to mailed questionnaires: A review. Public Opinion Quarterly, 3, 82-101. Lockhart, D. C., & Russo, J. R. (1996). Mail and telephone surveys in marketing research: A perspective from the field. In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 117161). Cambridge, UK: Blackwell. Malhotra, N. K. (1993). Marketing research: An applied orientation. Englewood Cliffs, NJ: Prentice Hall. Markus, M. L. (1994). Electronic mail as the medium of managerial choice. Organization Science, 5, 502-527. Mehta, R., & Sivadas, E. (1995). Comparing response rates and response content in mail versus electronic mail surveys. Journal of the Market Research Society, 37, 429-439. Montgomery, D. C. (1991). Design and analysis of experiments. New York: John Wiley. Mortensen, E. (1985). Electronic mail: An ever changing technology. The Office, 102, 101-103. Neal, W. D. (1989). Computering and software in marketing research: The US perspective. Journal of the Market Research Society, 31, 567-576. O’Lear, S.R.M. (1996). Using electronic mail (e-mail) surveys for geographic research: Lessons from a survey of Russian environmentalists. Professional Geographer, 48, 209-217. Oppermann, M. (1995). E-mail surveys: Potential and pitfalls. Marketing Research: A Magazine of Management & Applications, 7, 29-33. Parker, L. (1992). Collecting data the e-mail way. Training & Development, 46, 52-54. Parker, T. (1995). How internet e-mail works. In M. A. Pike (Ed.), Using the Internet (pp. 283-322). Indianapolis, IN: Que. Paxson, M. C. (1995). Increasing survey response rates: Practical instructions from the totaldesign method. Cornell Hotel & Restaurant Administration Quarterly, 36, 66-74. Perl, Y., & Whitley, E. (1995, December). The environmental contribution of personal computers: A state of the art report. Computers and Society, pp. 6-8. Rice, R. E. (1990). Computer-mediated communication system network data: Theoretical concerns and empirical examples. International Journal of Man-Machine Studies, 32, 627-647. Rossi, P. H., Wright, J. D., & Anderson, A. B. (1983). Sample surveys: History, current practice, and future prospects. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of survey research (pp. 1-20). New York: Academic Press. Roth, P. L., & BeVier, C. A. (1998). Response rates in HRM/OB survey research: Norms and correlates, 1990-1994. Journal of Management, 24, 97-118. Salant, P., & Dillman, D. A. (1994). How to conduct your own survey. New York: John Wiley. Saltzman, A. (1995). Improving response rates in disk-by-mail surveys. Marketing Research: A Magazine of Management & Applications, 5, 32-39. Schaefer, D. R., & Dillman, D. A. (1998). Development of standard e-mail methodology: Results of an experiment. Public Opinion Quarterly, 62, 378-386. Schuldt, B. H., & Totten, J. W. (1994). Electronic mail vs. mail survey response rate. Marketing Research: A Magazine of Management & Applications, 6, 36-44. Sell, R. L. (1997). Research and the Internet: An e-mail survey of sexual orientation. American Journal of Public Health, 2, 297. Shannon, J. H., & Rosenthal, D. A. (1993). Electronic mail and privacy: Can the conflicts be resolved? Business Forum, 18, 31-42.

115

ORGANIZATIONAL RESEARCH METHODS

Sproull, L. S. (1986). Using electronic mail for data collection in organizational research. Academy of Management Journal, 29, 159-169. Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51, 709-725. Subramanian, A. K., McAfee, A. T., & Getzinger, J. P. (1997). Use of the World Wide Web for multisite data collection. Academic Emergency Medicine, 4, 811-817. Sudman, S. (1996). Sampling. In R. P. Bagozzi (Ed.), Principles of marketing research (pp. 73115). Cambridge, UK: Blackwell. Swoboda, W. J., Muhlberger, N., Weitkunat, R., & Scheneeweib, S. (1997). Internet surveys by direct mailing: An innovative way of collecting data. Social Science Computer Review, 15, 242-255. Synodinos, N. E., & Brennan, J. M. (1988). Computer active interviewing in survey research. Psychology & Marketing, 5, 117-137. Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60, 275-304. Tse, A.C.B. (1998). Comparing the response rate, response speed and response quality of tow methods of sending questionnaires: E-mail vs. mail. Journal of the Market Research Society, 40, 353-361. Tull, D. S., & Hawkins, D. I. (1993). Marketing research: Measurement & method. New York: Macmillan. Vazzana, G. S., & Bachmann, D. (1994). Fax attracts. Marketing Research: A Magazine of Management & Applications, 6, 19-25. Veiga, J. F. (1984). Getting the mail questionnaire returned: Some practical research considerations. Journal of Applied Psychology, 59, 217-238. Walsh, J. P., Kiesler, S., Sproull, L. S., & Hesse, B. W. (1992). Self-selected and randomly selected respondents in a computer network survey. Public Opinion Quarterly, 56, 241-244. Watson, S. C. (1998). A primer in survey research. Journal of Continuing Higher Education, 46, 31-40. Weisband, S., & Kiesler, S. (1996). Self disclosure on computer forms: Meta analysis and implications. Human Factors in Computing Systems Conference Proceedings, 3-10. Yammarino, F. J., Skinner, S. J., & Childers, T. L. (1991). Understanding mail survey response behavior: A meta-analysis. Public Opinion Quarterly, 55, 613-639. Yu, J., & Cooper, H. (1983). A quantitative review of research design effects on response rates to questionnaires. Journal of Marketing Research, 20, 36-44. Zikmund, W. G. (1994). Business research methods. New York: Harcourt Brace. Zeki Simsek is a graduate student in the School of Business Administration, University of Connecticut. His research interests include new data collection techniques, intraorganizational networks, and interorganizational relations. John F. “Jack” Veiga (DBA Kent State) is the Airbus Industrie International Scholar and head of the Management Department, School of Business Administration, University of Connecticut. His current research interests include technology acceptance, cross-cultural behavior, and top management teams.