Determining Success for Different Website Goals - CiteSeerX

Proceedings of the 39th Hawaii International Conference on System Sciences - 2006

Determining Success for Different Website Goals L. Christian Schaupp*, Weiguo Fan**, France Belanger** *University of North Carolina at Wilmington **Virginia Polytechnic Institute and State University [email protected], [email protected], [email protected] Abstract In building a successful website it is imperative that the design matches the organization’s objectives, which should be well defined. Clearly, different types of websites will have different goals. Thus, determining success across websites is both goal and context specific. One measure of website success is satisfaction, and the resulting intent to return to a website. Built on theories from the IS success and information technology adoption literatures, this study investigated four variables believed to impact website satisfaction: information quality, system quality, perceived effectiveness, and social influence. Data was collected by surveying regular users of two different websites, each from a different category within a taxonomy of websites. Structural equation modeling techniques were used to test the proposed model of website success for each studied site. The results indicate that the determinants of satisfaction and overall successfulness of websites are both context dependent and goal specific.

1

Introduction For decades, information systems researchers have investigated factors that lead to successful use of information systems. Most often, researchers tie determinants of success to user perceptions about an information system [10, 43]. Nowhere is this more evident than in the case of Internet websites, where ecommerce success has been studied through the lenses of various adoption theories [e.g., 11, e.g., 12, 42]. In less than a decade, the Internet has extended into nearly every facet of society, from commerce to education to entertainment, and is employed in a variety of uses, from scholarly research to casual browsing (surfing). Although the Internet remains an unfinished product, it has accumulated enough of a history to permit meaningful analysis of the trends characterizing its evolution. Since the early 1990’s when the Internet exploded into the mainstream, it has made an enormous impact on the way people access information and on the way business operations are carried out. It has changed more than just the way individuals gather information about a particular topic

or product, or the way businesses try to streamline operations. The Internet has redefined peoples’ expectations about the accessibility of information. No longer is the computer savvy person willing to wait on hold or stand in line to get a specific form or piece of information. They want to just point and click, accessing the required information within a matter of seconds. This cultural change is not isolated to the United States or to one particular ethnic group; it breaks traditional borders and puts everyone on the same level playing field with accessibility of information. The business world has also experienced a dramatic culture change. The Internet has forced businesses to rethink and adapt existing business models to further emphasize productivity, efficiency, and the streamlining of all business processes. Most large traditional brick-mortar organizations now have an online presence --- their own websites. These online websites have served as virtual facades, allowing companies to engage customers, provide clear explanations of their business processes and models, and allow online transactions to streamline the business process. There are even companies that are completely virtual (e.g. Amazon.com), offering no traditional brick-mortar outlet. The dotcom bust of the late 1990s led to the reevaluation of these virtual stores, but as time has passed, e-retailer giants such as Amazon.com and eBay.com have further strengthened their grip on the marketplace, and sustained long-term profits are becoming a reality [24]. Does the long-term profitability of a company’s electronic outlet hold the answer to the company’s success? What if a company’s purpose for being on the web is more than just to have an Internet presence or to sell products? How is success defined? What qualifies as a successful website and what does not? In order to answer these questions, we turn to the predominant IS success models [12, 31, 34]; and IT adoption models [10, 40, 41, 43]. Specifically, this research proposes an integrated model of website success, incorporating constructs from both the IS success and IT adoption literature. Researchers have called for research to empirically validate and extend these existing IS success models into differing contexts

0-7695-2507-5/06/$20.00 (C) 2006 IEEE

1


[11, 12], specifically Internet websites [31]. Motivated by these calls for research, we seek to answers the following questions in this study: Q1: From the perspective of the end user, what is the appropriate model of website success? Q2: From the perspective of the end user, does the proposed model of website success “fit” when extended to a broader typology of websites than exists in the current IS literature? The remainder of this paper proceeds as follows. Section 2 builds the theoretical arguments for the proposed research model. In Section 3, we present the background for a preliminary study that tested this model in two different contexts of websites. The results are presented in Section 4, based on a sample of 2036 real users of two different types of websites. We provide a discussion of the findings, an agenda for future research, and limitations in Section 5. Finally, concluding comments are presented in Section 6. 2

Theoretical Development Although organizations have spent millions of dollars on the creation and maintenance of their websites, they still struggle with how to effectively evaluate user satisfaction and how to measure the success of their website. For several years, researchers from various disciplines have studied different perspectives of website success and have generated quite a number of interesting, yet isolated findings. These findings have provided different, although sometimes overlapping, perspectives on how to evaluate and determine website success from a variety of contexts [20, 22, 44]. In the information technology domain (information systems, information technology, computer science), prior research has typically focused on the evaluation of generic information systems to determine effectiveness. More specifically, most of the studies have examined generic organizational information systems embedded within organizational settings. As a result, the systems examined are often mandated systems, where the users and their characteristics are known. Prior literature examining website success, where there is a larger user base that breaks the traditional organizational boundaries in which users are relatively unknown, has been limited primarily to e-commerce contexts. These studies usually take the consumer or end-user point of view [20, 33]. Studies taking the organizational perspective usually address the implementation of an information system and primarily focus on user resistance, implementation, and intention to use the system, but do not directly address the issue of determining website “success” [9, 10, 21].

2.1 Website Success Measuring success is a difficult task because the definition of success changes depending on the perspective that the stakeholder adopts. There are two opposing perspectives that can be taken in the determination of success: 1) the website user and 2) the organization or party who hosts the website. From the perspective of the end-users, their expectations need to be met and their interaction with the website has to be a positive experience, in order for the website to be considered successful. The predominant way of determining success is to evaluate the user’s satisfaction and their likelihood of return through the use of surveys, as done in the present research. From the perspective of the firm, success varies depending upon the objectives and goals of the site. For example, an e-commerce site’s objective would be to sell their products or services and to maximize profit. However, the objective of a search engine’s website, such as Google.com, would be to quickly gather relevant information in a timely manner with the goal of creating repeat visits. The perspective taken is critical in the determination of success. From the organization’s perspective, the definition of success is the website’s ability to create an on-going relationship with a consumer (user), which will either immediately or eventually lead to a transaction of some sort. Gathering clickstream data is the predominant way of determining success from the firm’s perspective. A firm gathers clickstream data from the traffic on its site and makes inferences regarding the site’s success or effectiveness. However, organizational metrics to measure site effectiveness need to be tied not only to the navigation patterns of its users, but also to their specific goals, which are not directly highlighted in this kind of research. 2.2 Website Satisfaction Customer satisfaction is critical for establishing long-term client relationships and plays a significant role in sustaining profitability and determining the overall success of a website [30]. Web customer satisfaction (called e-satisfaction by Szymanski and Hise [39] is of great importance [22]. The need for further research in e-satisfaction has been accentuated by the increasing demand for long-term profitability of both dotcom companies and traditional companies that are “net-enhanced” [37]. General levels of e-satisfaction have been reported [6, 15] and research has been done in an e-commerce context attempting to identify e-satisfaction determinants [22, 39]. Although the antecedents to customer satisfaction are well documented in classical contexts [28, 29, 38, 46], customer satisfaction in an Internet context, specifically website satisfaction, has

2


not been subjected to conceptual or empirical scrutiny beyond the e-commerce setting. 2.3 IS Success In contrast to the technology acceptance literature, system and information characteristics have been core elements in the literature predicting satisfaction [12]. The predominant models in the IS success literature to date have only been tested in mandated usage contexts or in an e-commerce setting. Rai et al. [31] call for future research to examine how IS success models perform in different contexts. They state that models of IS success need to be critically evaluated, refined, and tested in emergent IS use settings, such as e-commerce. However, e-commerce is only a single type of website present on the Internet. Rai et al. [31] also point out that traditional IS systems were targeted at internal organization users at operational, tactical, and strategic levels. The focus of a majority of the past IS success research has been on explaining IS use in these settings, where IS use has been typically assessed by how much time was spent using the system [31]. However, a majority of organizations now have some sort of web presence, and these systems are targeted at external users. The DeLone and McLean [12] and Seddon [34] success models are the two most notable in the literature to date. Both models have been empirically validated in prior literature. Rai et al. [31] compared the two models in a mandatory usage context and called for future research to apply the existing success models [12, 34] to specific contexts. The existing models have yet to be applied to varying website contexts. 2.4 IT Adoption Traditionally, IT adoption models have attempted to predict usage. The technology acceptance literature highlights the predictors of usage by linking behaviors to attitudes and beliefs (ease of use and usefulness). The most predominant in predicting usage being the TAM model [10]. However, TAM provides limited design and implementation guidance [40, 43]. Up to this point in the literature, the technology acceptance research has included empirical tests, model comparisons, model variants, and model extensions [45]. Venkatesh et al. [43] provide a comprehensive examination of eight predominant technology acceptance models and produce a unified theory of acceptance and use of technology. 2.5 An Integrated Model of IS Success and IT Adoption To accurately predict intention, beliefs and attitudes must be specified in a manner that is

consistent with the behavior of interest [16]. This is essential when beliefs and attitudes about a specific behavior (e.g., the use of website), at a particular point in time, in a specific context are found to be predictive of intention to use [45]. Four factors and two success measures form the foundation of the research model proposed in this study. These six constructs are collected from the most well researched IS success models in the literature. Based on the nature of website development the proposed models by Davis et al. [10], DeLone and McLean [11], Rai et al. [31], Seddon [34], McKinney et al. [22], and Venkatesh et al. [43], we posit that website satisfaction and overall success have distinctive sources unique to the context of the website. The sections below provide a more detailed explanation of how the individual constructs were chosen for inclusion in the model (see Appendix 1). Success Factors Information quality is defined as the degree to which information produced by the website is accurate, relevant, complete, and in the format required by the user. Information quality has been shown to be a prominent success factor when investigating overall IS success [11, 34]. It has also been shown to be significant predictor of satisfaction, specifically in the context of websites [22]. As a result, the information quality construct was chosen for inclusion into the research model. Information quality was measured in terms of a seven-point Likert-type scale including seven items that capture the degree with which the target website generated information possessing the attributes: content, accuracy, and format. These items were patterned after items used by Doll and Torkzadeh [14] and subsequently Rai et al. [31]. These attributes comprising information quality are among the most widely studied in the IS literature[3, 4, 26, 32, 35]. System quality is defined as the degree to which the system is easy to use for the purposes of accomplishing some task. System quality has been represented in prior research by ease of use, which is defined as the degree to which a system is “user friendly” [14, p. 259]. Adams et al. [1], Chin and Todd [8], Davis [9], DeLone and McLean [11-13], Hendrickson et al. [17], Seddon [34] and Segars and Grover [36] investigated ease of use as a measure of IS success. McKinney et al. [22] also studied system quality in the context of online satisfaction. This study patterns the system quality construct after effort expectancy [43] and the system quality construct used in DeLone and McLean [13] and Seddon [34]. System quality was measured using a seven point Likert type scale consisting of two items selected from

3


Doll and Torkzadeh’s [14] user satisfaction instrument as well as an item from McKinney et al [22] which were adapted specifically to the website context. Perceived Effectiveness is defined as the degree to which an individual believes that using the website will help them accomplish some task. Perceived effectiveness is patterned after several constructs from the existing success models including: perceived usefulness [10, 34], performance expectancy [43], and individual impacts [12]. Perceived effectiveness was measured using a seven point Likert type scale consisting of four items taken directly from Venkatesh et al.’s [43] user satisfaction instrument and adapted specifically to the website context being examined. Social influence is defined as the degree to which an individual perceives that others believe they should use the website. Thompson et al. [41] use the term “social norms” to describe their construct and acknowledge its similarity with “subjective norm” within the Theory of Reasoned Action. Both theories contain the explicit or implicit notion that the individuals’ behavior is influenced by how they think using the specific technology will affect how others view them [43]. Image has been found to be a significant predictor of IT adoption [23], and is comprised in Venkatesh et al.’s [43] measure of social influence. However, Venkatesh et al. found their social influence construct to be significant in predicting usage only in a mandatory use setting. In addition to usage studies, there have also been studies of ecommerce settings that have found image to be a significant predictor of intention to purchase [7]. There has also been extensive marketing research investigating the link between a brand’s image and its website, known as e-branding [25]. Prior studies have been done in voluntary settings, but almost exclusively in e-commerce contexts. This study will investigate the significance of the social influence construct in various voluntary website contexts. Social Influence was measured using a seven point Likert type scale consisting of four items taken directly from Venkatesh et al.’s [43] user satisfaction instrument and adapted specifically to the website context being examined. Success Measures Satisfaction and intention to re-use are commonly used measures of success and adoption [10-12, 22, 34, 43, 45]. Website satisfaction is defined as the degree to which the user is satisfied with the website in question. Although the antecedents to satisfaction are well documented in classical contexts [28, 29, 38, 46], satisfaction in the context of a website, has not been subjected to conceptual or empirical scrutiny beyond

the e-commerce setting. General levels of esatisfaction have been reported [6, 15] and research has been done in an e-commerce context attempting to identify e-satisfaction determinants [22, 39]. However, there has been no empirical research done to determine the antecedents of website satisfaction and re-use in a context other than that of an e-commerce website. In this study, where subjects are regular users of the websites, satisfaction is imperative Website satisfaction was measured using a seven point Likert-type scale consisting of three items taken directly from McKinney et al. [22] and adapted to fit the context of this study. Behavioral intention to re-use was also measured using a seven point Likert-type scale consisting of three items patterned after Venkatesh et al. [43] and adapted specifically to the website context being examined. Figure 1: Detailed Research Model Critical Success Factors Information Quality System Quality

Success Measures Website Satisfaction

Intention to Reuse Site

Perceived Effectiveness Social Influence 2.6 Taxonomy of Website Goals To study website success in different contexts, we need to consider the various types of website goals that may exist. To date, websites have typically been classified based on functionality: online storefronts, web presence sites, content sites, malls, incentive sites, and search agents [18]. Belanger et al. [5] produced a taxonomy of websites based on a set of user goals (see Table 1). According to this taxonomy each website can have many different audiences with many different goals. In practice, two websites may have similar functions or features but different goals, and therefore, different definitions of user satisfaction. In many cases a website will combine multiple classifications from the taxonomy. For example, an "Informed Decision, Biased" website may extend to "Individual Ecommerce". This simply means that the goals and success measures from these two classifications will be combined.

4


Table 1. Taxonomy of Website Goals [5] Website goal Definition Informed decision biased Informed decision – unbiased Life enrichment Online learning Entertainment Knowledge enhancement E-commerce

Online community

Information specific search Interactive service management Online application

Gives product information with the goal of influencing the decision process of users Helps the user make an informed decision but without bias towards a particular decision Increases the general awareness of a topic, but not necessarily of a product Offers a forum for educational purposes Offers entertainment (games, music, etc.) Quickly informs visitors of current events or a specific topic Allows transactions online with another party (supplier, customer, partner, government, etc.) Gathers and shares information on a certain topic or area of interest and acts as a forum for people with similar interests Provides the ability to search and find relevant information on a particular topic Allows individuals or organizations to service their accounts online Allows individuals or organizations to access applications on a webbased platform

In this study, we focus on two of the categories of website goals to perform an initial comparison: online community and information-specific search. For both types of sites, we surveyed actual users of the sites. 3

Methods To investigate user perceptions of success for the website goals selected, we conducted surveys of actual users of such websites. Survey research is the principle method of study in technology adoption research [43, 45]. After using the particular website to accomplish a given task, respondents were asked a series of questions aimed at assessing their overall satisfaction with the site and their intention to return to the site. 3.1 Research Instrument The survey instrument was created by combining existing measures of IS success and IT adoption [22, 31, 43], and wording was modified to fit the website contexts to be studied. The questionnaire contained 28

randomly ordered items, with some question reverse worded. Each question was measured on a 7-point, Likert-type scale, ranging from 1 (Strongly disagree) to 7 (Strongly agree). The instrument was pre-tested with academics and practitioners, and then pilot tested with over 200 undergraduate students at a large midAtlantic university. Constructs in the pilot test showed internal consistency levels exceeding 0.70, as measured by Cronbach’s alpha [27]. 3.2 Sample To obtain study participants, an e-mail announcement was sent to regular users of our two target sites. Each site user was provided a link to a web based survey that they could access. All surveys were confidential; no identifying personal information was required. Demographics and response rates are provided in Table 2. 3.2.1 Information Specific Search (ISS) Website This website’s primary purpose is to allow searches for specific information. This website goal is most often associated with traditional search engines (i.e. Yahoo.com, Google.com, etc.). Users typically visit this particular website to search for information and then use hyperlinks provided in the search results to find the information on the topic of interest. The website chosen for analysis is one of the most popular search engine websites that exists presently. Of those surveyed, 94.2% reported that their primary purpose for using the website was to search for information on a particular topic. This indicates that the placement of this website into the information specific search (ISS) was accurate. The overwhelming majority of the subjects stated that they used the website at least several times per week. There were a total of 199 usable survey responses in the information specific search group. The subjects comprised employees of several commercial organizations (102), as well as some upper level undergraduate students (97). The subjects were all experienced Internet users, with 96.2% of respondents having accessed the Internet for over 5 years. Of those surveyed, 86.7 % reported having shopped online several times in the past year with 43.6% reporting that they shopped online at least once a month. There were slightly more males (52 %) than females in the sample, and 59.4% of the respondents were over age of 35. 3.2.2 Online Community (OC) Website The online community (OC) website’s purpose is to allow users to gather and share information on a certain topic or area of interest and act as a forum for people with similar interests [5]. This type of website is typically used by users with a specific common interest

5


to collect and disseminate information on that one particular topic area or a closely related area. There were a total of 1837 usable surveys in the online community category. The subject pool comprised active members of the online community from around the world. This website provides them with a forum to post and collect information regarding their job, upcoming conferences, etc. 54.5% of the subjects access the target website at least once a month. 46% of respondents stated that their primary purpose for visiting the website was to collect professional information, and 40.9% stated that they primarily visited the site to search for information relating to a particular topic area. Respondents were also very experienced Internet users with 93.2% of respondents having accessed the Internet for over 5 years. Of those surveyed, 66.6% were over the age of 40, and 97.5% had at least a 4-year college degree. There were considerably more females (71.4%) than males in the sample.

analysis, factor analysis, and linear regression into a theoretical causal model for analysis of latent constructs and measurable variables, allowing simultaneous estimation of both measurement and structural sub-models [2].

Table 2. Demographics and Response Rates

The reported goodness-of-fit index (GFI) of 0.804 is within the suggested range of 0.80-0.90 to be considered a moderately good fit of the model [19]. The Comparative Fit Index is often times the preferred fit index to report when structural equation modeling (SEM) techniques are employed. This data collection reports a CFI of 0.913 which is considered to be a good fit for the model [19]. The Tucker-Lewis Index (TLI) for this data had a value of 0.901, which would represent a good fit for the model too. The TuckerLewis Index is most appropriate to use when working with small sample sizes [19], sample sizes below the 200-250 range. With an appropriate model fit, we look at individual path coefficients, as presented in Table 4. Information quality, system quality, and perceived effectiveness were significant predictors of website satisfaction and website satisfaction was found to be a significant predictor of behavioral intention to return in the information specific search category.

OC Surveyed

Age

Gender

Surveyed Total Resp. % 18-20 21-24 25-30 31-35 36-40 40+ Male Female

Education

Site usage

Internet experience

4

4-year college degree Every day Every month > 5 years

3500 1837 53 % 5 85 188 174 161 1221 533 (29 %) 1304 (71 %) 97.5 %

ISS worker

ISS student

150 102 68 % 3 10 19 21 16 32

200 97 49 % 68 29 0 0 0 0 103 (52 %) 96 (48 %) 42 %

n/a

68.9 %

54.5 %

98.2 %

93 %

96.2 %

Results The research model was tested using structural equation modeling (SEM) techniques, which is a comprehensive approach to testing hypotheses about relations among observed and latent variables [19], and can be used for either predictive applications or theory testing. The statistical approach incorporates path

4.1 Information Specific Search Website This section presents the structural equation modeling techniques results for the information specific search website category, as described above. A summary of the fit indices used in this analysis is presented in Table 3. Table 3. Fit Indices - Information Specific Search Fit Measure Default Model Goodness-of-Fit (GFI)

0.804

Tucker-Lewis Index (TLI)

0.901

Comparative Fit Index (CFI)

0.913

Table 4. Information Specific Search Results Dependent Independent PSupport? Variable Variable Value Website Information Yes 0.001 Satisfaction Quality System Yes 0.006 Quality Perceived Yes 0.000 Effectiveness Social 0.174 No Influence Behavioral Website Yes 0.000 Intention Satisfaction

6


Using these findings, we propose a new model of website success for Information specific search websites, as presented in Figure 2. Figure 2. Revised Research Model - Information Specific Search Success Measures Information Quality Outcome Measures System Quality

Website Satisfaction


Perceived Effectiveness

4.2 Fit Indices for Online Community Website This section presents the structural equation modeling techniques results for the online community website category. A summary of the fit indices for the proposed model of website success in the online community category is presented in Table 5.

effectiveness, and social influence were significant predictors of website satisfaction, as shown in Table 6. Website satisfaction was also found to a significant predictor of behavioral intention. System quality was not found to significantly predict website satisfaction in this category. Table 6. Online Community Results Dependent Independent P-Value Support? Variable Variable Website Information Yes 0.009 Satisfaction Quality System 0.185 No Quality Perceived Yes 0.000 Effectiveness Social Yes 0.000 Influence Behavioral Website Yes 0.000 Intention Satisfaction The revised model of website success for the online community category is presented in Figure 3. Figure 3. Revised Research Model – Online Community Success Measures

Table 5. Fit Indices for Online Community Website Fit Measure

Default Model

Goodness-of-Fit (GFI)

0.815

Tucker-Lewis Index (TLI)

0.902

Comparative Fit Index (CFI)

0.915

The reported goodness-of-fit index (GFI) of 0.815 is within the suggested range of 0.80-0.90 for a moderately good fit of the model [19]. The sample size of the data collected is a large sample size, well above the ideal 200-250 sample size preferred for SEM techniques to be employed [19]. The Tucker-Lewis Index (TLI) for this data had a value of 0.902 which would be considered a good fit of the model; however, the Tucker-Lewis Index is most appropriate when working with a small sample size [19]. The comparative fit index (CFI) is often the fit index of choice when structural equation modeling (SEM) techniques are employed, specifically when dealing with large sample sizes, as is the case with this category. This data collection reports a CFI of 0.915, which shows a good fit of the model. With appropriate fit, we turn to path coefficient and find that information quality, perceived

Information Quality Outcome Measures Perceived Effectiveness

Website Satisfaction


Social Influence

5.

Discussion The findings of this study integrate concepts from the user satisfaction, IS success, and IT adoption literature into a single research model. These findings provide a preliminary test of the viability of the research model within two different website contexts: 1) an information specific search website (search engine), and 2) an online community website. There is an absence of empirical studies to date which have attempted to empirically measure these variables’ concurrent impacts on outcomes in varying website contexts. This study was oriented towards a model

7


building approach in which the potential linkages between the variables and outcome criteria were investigated. Once the research model’s fit was investigated for each website context, direct effects were then examined and the results of each were evaluated to identify any differences. The findings of this study highlight the discrepancies between the two categories of websites. Information quality was found to significantly predict satisfaction in both categories of websites, indicating that the quality of information provided on the site plays a vital role in satisfying the end user. System quality was only found to be significant in the information specific search category. This highlights the need for an easy-to-use website when searching for information. These results are not surprising since a user who can not use a website at all can not use it to find information. Intuitively, it makes sense that an easy-to-use searching function would lead to satisfaction in a scenario where the users’ goal is to search for a specific piece of information. However, in the online community website, system quality was not shown to be a significant predictor of website satisfaction. These results indicate that users, in this context, put more value on the quality of the information being provided, the usefulness of the website, and their peers also using the site, as opposed to the overall usability of the website. Perceived effectiveness showed similar results in the two website contexts. In both contexts evaluated (information specific search and online community), perceived effectiveness was found to be a significant predictor of website satisfaction. The results indicate that perceived effectiveness plays a vital role in satisfying the end user of the website. This finding is not surprising due to the fact that the perceived usefulness construct has been found to be a significant predictor of satisfaction in prior IS success [12, 31, 34] and IT adoption [43] literature. The results found in this study underscore the point that goal accomplishment is imperative to users’ overall satisfaction with a website. Social influence exhibited different results in the two varying website contexts. The results indicate that the significance of social influence in satisfying the end user of the website is context dependent. The significance of social influence is very evident in the context of the online community where the perception of a user’s peers plays an important role in satisfaction and reuse. This is due to the fact that online communities are settings where users gather and share information on a certain topic or area of interest and which act as forums for people with similar interests. It is intuitive to think that the more people with common interests who use the website, the more people would

be satisfied with and reuse the website. However, based on the results obtained in this study, there is no evidence to suggest that social influence is a significant predictor of website satisfaction in the information specific search category. Website satisfaction was not only a dependent variable, but also an independent variable. As a dependent variable, website satisfaction was found to be dependent upon different factors for each context of website that was being evaluated. In the information specific search context, website satisfaction was found to be dependent upon three factors in the model: information quality, system quality, and perceived effectiveness. In the online community context, website satisfaction was also found to be dependent upon three factors in the research model: information quality, perceived effectiveness, and social influence. This validates the fact that website satisfaction is goal and context dependent, demonstrating that there will be different determinants of satisfaction depending upon what type of website is being evaluated. Website satisfaction is goal as well as context specific. These findings highlight the fact that there is not just one model that is able to fully capture the determinants of success in multiple contexts of websites presently on the Internet from the user perspective. The website satisfaction model presented earlier is a starting point for evaluation of the many contexts of websites. As indicated by the results, it is necessary to fully evaluate each category of website goal in order to create multiple models of success which are context dependent. As suggested by DeLone and McLean [12], IS success models are case specific, and appropriate measures need to be included on a case-by-case basis in order to paint an accurate picture. Similar to previous satisfaction literature [22, 39], the findings of this study indicate that satisfaction is a prerequisite to website reuse. This is not to say that other factors do not play a role, but that the underlying determinant is that the user must be satisfied in order for them to return. The findings of this study highlight the fact that, regardless of the context being studied, satisfaction is the ultimate determinant of reuse intentions from the perspective of the user. In each category of website goal evaluated in this research, website satisfaction was found to be a significant predictor of intention to reuse the individual website. This path was the most significant path identified in the research model in both contexts. 5.1 Contributions and Limitations This research provides many contributions to the field of information systems. Findings from this research can help bridge the conceptual gap between accomplishing the objectives of a site and satisfying

8


users; leading to a successful site. One thing that becomes inherently clear from the results obtained in this study is that researchers have to start looking at websites in a different light. Specifically, they need to classify the websites that are being studied. For practitioners, it is imperative to understand the goals of those who use the website most frequently. The findings of this study show that depending upon the goal of the user, their determinants of satisfaction will vary. Thus, by better understanding the goals of the user it allows for the website to be designed accordingly. There are several limitations to this study that should be noted. The most notable is the number of websites that were evaluated. While valid results were produced from testing these two categories, there was only one website evaluated within each category. Future research should attempt to validate the findings of this study by testing multiple websites within each category. Another limitation is that the data for this study was collected by having respondents fill out a survey. There is therefore a limitation of self-report biases of respondents. In particular, obtaining perceived measures for the outcomes is subject to the respondents’ perceptions of their satisfaction with the website being evaluated. Another limitation was the lack of direct control over the subjects while taking the survey. The survey was distributed via e-mail to potential subjects. The subjects were able click on the link provided to them in the e-mail which directed them to the survey posted on the Internet. Once the users accessed the survey, it was up to them to complete the survey at their own pace in a place of their choosing. While the time that each respondent spent on completing the survey was monitored, the respondents could not be monitored directly. It would undoubtedly be beneficial to administer the survey to respondents in a controlled environment where they could be directly monitored. In a controlled environment, any doubt as to whether the subjects were completing the survey as instructed could be eliminated. 6.

Conclusion Acknowledging the limitations of this study, this research has still made several significant contributions to the field of information systems research. This study is a first step towards understanding experienced website users’ determinants of satisfaction and overall success. In particular, it has shown that website users’ evaluation of satisfaction and overall successfulness is dependent upon the context of the website being evaluated and that determinants of satisfaction are goal specific. It also serves as a foundation for further

exploration into a broader, more diverse array of the website contexts presently on the Internet. This study serves as a building block towards a greater understanding of overall website success from the end user perspective by assessing their users’ goals and objectives for the website. It is imperative for an organization to understand their users’ goals in order to design their website accordingly. 7.

References

[1] Adams, D. A., Nelson, R. R., and Todd, P. A., "Perceived usefulness, ease of use, and usage of information technology: A replication," MIS Quarterly, vol. 16, pp. 227-247, 1992. [2] Anderson, J. C. and Gerbing, D. W., "Structural equation modeling in practice: A review and Recommended two-step approach," Psychological Bulletin, vol. 103, pp. 411-423, 1988. [3] Bailey, J. E. and Pearson, S. W., "Development of a tool for measuring and analyzing computer user satisfaction," Management Science, vol. 29, pp. 530545, 1983. [4] Baroudi, J. J. and Orlikowski, W. J., "A short form measure if user information satisfaction: A psychometric evaluation and notes on use," Journal of Management Information Systems, vol. 4, pp. 44-59, 1988. [5] Belanger, F., Fan, W., Schaupp, L. C., Krishan, A., Everhart, J., Poteet, D., and Nakamoto, K., "Taxonomy and Success Metrics of Online Websites," Communications of the ACM, forthcoming. [6] Buskin, J., "Tales from the front: A firsthand look at buying online," in Wall Street Journal, 1998, pp. R6. [7] Chen, Q. and Wells, W., "Attitude toward the site," Journal of Advertising Research, vol. 39, pp. 27-37, 1999. [8] Chin, W. W. and Todd, P. A., "On the use, usefulness, and ease of use of structural equation modeling in MIS research: A note of caution," MIS Quarterly, vol. 19, pp. 237-246, 1995. [9] Davis, F., "Perceived usefulness, perceived ease of use, and user acceptance of information technology," MIS Quarterly, vol. 13, pp. 319-339, 1989. [10] Davis, F., Bagozzi, R., and Warshaw, P., "User acceptance of computer technology: a comparison of two theoretical models," Management Science, vol. 35, pp. 982-1002, 1989. [11] DeLone, W. H. and McLean, E. R., "The DeLone and McLean model of information success: A ten year update," Journal of Management Information Systems, vol. 19, pp. 9-30, 2003. [12] DeLone, W. H. and McLean, E. R., "Information systems success: The quest for the dependent variable," Information Systems Research, vol. 3, pp. 6095, 1992. [13] DeLone, W. H. and McLean, E. R., "Measuring eCommerce success: Applying the DeLone & McLean information systems success model," International

9


[14]

[15] [16] [17]

[18]

[19]

[20]

[21] [22]

[23]

[24]

[25] [26]

[27] [28]

[29] [30]

Journal of Electronic Commerce, vol. 9, pp. 31-47, 2004. Doll, W. and Torkzadeh, G., "The measurement of end user computing satisfaction," MIS Quarterly vol. 12, pp. 259-274, 1988. Ernst_and_Young, "Internet Shopping: An Ernst and Young Special Report," Ernst and Young LLP, 1999. Fishbein, M. and Ajzen, I., Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley, 1975. Hendrickson, A. R., Glorfeld, K., and Cronan, T. P., "On the repeated test-retest reliability of the end user computing instrument: A comment," Decision Sciences, vol. 25, pp. 655-667, 1994. Hoffman, D., Novak, T., and Chatterjee, P., "Commercial scenarios for the Web: Opportunities and challenges," Journal of Computer-Mediated Communication, vol. 1, pp. 1-20, 1996. Hoyle, R. H., Structural Equation Modeling: Concepts, Issues, and Applications, 1st ed: Sage Publications, 1995. Lin, J. C.-C. and Lu, H., "Towards an understanding of the behavioural intention to use a web site," International Journal of Information Management, vol. 20, pp. 197-208, 2000. Mahmood, M., "Systems development methods - A comparative investigation," MIS Quarterly, vol. 11, pp. 293-311, 1987. McKinney, V., Yoon, K., and Zahedi, F., "The measurement of Web-Customer satisfaction: An expectation and disconfirmation approach," Information Systems Research, vol. 13, pp. 296-315, 2002. Moore, G. and Benbasat, I., "Development of an instrument to measure the perceptions of adopting an information technology innovation," Information Systems Research, vol. 2, pp. 192-222, 1991. Mullaney, T. J., "The web is finally catching profits soon, more than 100 Net companies could be in the black," BusinessWeek, pp. 68, 2003. Muller, B. and Chandon, J.-L., "The impact of visiting a brand website on brand personality," Electronic Markets, vol. 13, pp. 210-221, 2003. Myers, B. L., Kappelman, L. A., and Prybutok, V. R., "A comprehensive model for assessing the quality and productivity of the information systems function: Toward a theory for information systems assessment," Information Resources Management, vol. 10, pp. 6-25, 1997. Nunnally, J. C., Psychometric Theory. New York, NY: McGraw-Hill, 1978. Oliver, R., "A cognitive model of the antecedents and consequences of satisfaction decisions," Journal of Marketing Research, vol. 17, pp. 460-469, 1980. Oliver, R., Satisfaction: A behavioral perspective on the consumer. New York: McGraw-Hill, 1997. Patterson, P., Johnson, L., and Spreng, R., "Modeling the determinants of customer satisfaction for business to business professional services," Acad. Marketing Science Journal, vol. 25, pp. 4-17, 1997.

[31] Rai, A., Lang, S., and Welker, R., "Assessing the validity of IS success models: An empirical test and theoretical analysis," Information Systems Research, vol. 13, pp. 50-69, 2002. [32] Rainer, R. K. and Watson, H. J., "The keys to executive information systems success," Journal of Management Information Systems, vol. 12, pp. 83-98, 1995. [33] Schubert, P., "Extended Web assessment method (EWAM): Evaluation of electronic commerce applications from the customer’s viewpoint," International Journal of Electronic Commerce, vol. 7, pp. 51–80, 2002-3. [34] Seddon, P. B., "A respecification and extension of the DeLone and McLean model of IS success," Information Systems Research, vol. 8, pp. 240-253, 1997. [35] Seddon, P. B. and Yip, S. K., "An empirical evaluation of user information satisfaction (UIS) measures for use with general ledger accounting software," Journal of Information Systems, vol. 6, pp. 75-92, 1992. [36] Segars, A. H. and Grover, V., "Re-examining perceived ease of use and usefulness: A confirmatory factor analysis " MIS Quarterly, vol. 17, pp. 517-522, 1993. [37] Straub, D. W., Hoffman, D. L., Weber, B. W., and Steinfield, C., "Toward new metrics for net-enhanced organizations," Information Systems Research, vol. 13, pp. 227-238, 2002. [38] Szymanski, D. and Henard, D., "Customer satisfaction: A meta-analysis of the empirical evidence," Journal of the Academy of Marketing Science, vol. 29, pp. 16-35, 2001. [39] Szymanzki, D. and Hise, R., "e-Satisfaction: An intial examination," Journal of Retailing, vol. 76, pp. 309322, 2000. [40] Taylor, S. and Todd, P., "Assessing IT usage: the role of prior experience," MIS Quarterly, vol. 19, pp. 561570, 1995. [41] Thompson, R., Higgins, C., and Howell, J., "Personal computing: toward a conceptual model of utilization," MIS Quarterly, vol. 15, pp. 124-143, 1991. [42] Van Slyke, C., Bélanger, F., and Comunale, C., "Adopting business-to-consumer electronic commerce: The effects of trust and perceived innovation characteristics," The Data Base for Advances in Information Systems, vol. 35, pp. 32-49, 2004. [43] Venkatesh, V., Morris, M., Davis, G., and Davis, F., "User acceptance of information technology: Toward a unified view," MIS Quarterly, vol. 27, pp. 425-478, 2003. [44] Waite, K. and Harrison, T., "Consumer expectations of online information provided by bank websites," Journal of Financial Services Marketing, vol. 6, pp. 309-322, 2002. [45] Wixom, B. H. and Todd, P. A., "A theoretical integration of user satisfaction and technology acceptance," Information Systems Research, vol. 16, pp. 85-102, 2005. [46] Yi, Y., "A critical review of customer satisfaction," in Review of Marketing, V. A. Zeithaml, Ed. Chicago: American Marketing Association, 1990.

10