Assessing Decision Making in Face-to-Face Teams versus Virtual

0 downloads 0 Views 627KB Size Report
face teams versus virtual teams in a virtual world, Journal of Decision ... Group report, virtual work in the form of virtual teams will account for 60% of global work.
Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Assessing Decision Making in Face-to-Face Teams versus Virtual Teams in a Virtual World Jeannie Pridmore – Gloria Phillips-Wren Sellinger School of Business and Management Loyola University Maryland 4501 N. Charles Street Baltimore, MD 21210 {jpridmore, gwren}@loyola.edu

ABSTRACT. Virtual teams use communication technologies to interact and accomplish work without physical presence. Prior research has shown that virtual teams can encounter difficulties in achieving efficient decision making processes and good decision outcomes. However, newer internet communication technologies (ICT) have enabled computer-based simulated environments that look like real locations and in which avatars (i.e. computer representation of the person) can carry on conversations and interact with each other. Although there is significant research on decision making in teams meeting face-to-face, literature on virtual teams in virtual worlds is just emerging. The objective of this research is to investigate how virtual world technology impacts team decision making, and to compare with teams meeting face-to-face. We collected data on teams operating in both environments and used an analytical hierarchy process model to compare their decision quality. We found that virtual teams in the virtual world took a somewhat longer time to reach a decision, but their decision accuracy was better than face-toface teams. The results were surprising and suggest that virtual world technology can be an effective team operating environment. KEYWORDS. decision making, virtual worlds, virtual teams, analytic hierarchy process

1|P age

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

1.0 Introduction Over the past several years, it has become the norm for innovative organizations to create geographically dispersed teams often referred to as virtual teams. According to a 2008 Gartner Group report, virtual work in the form of virtual teams will account for 60% of global work assignments in the near future (Bell, 2005). This trend suggests that business professionals will most likely be part of a virtual team at some point in their career. In addition, the effective use of virtual teams has been recognized as one of the “five imperatives” emerging from information technology that will force organizations to change structures and human behavior processes (Herman, 2001). To date, research on virtual teams has focused on understanding the challenges and the difficulties encountered when using Internet technologies such as web conferencing, email, chat, and discussion boards. Previous studies have shown that communication in virtual teams can be difficult. “The problem… [with virtual teams is]…usually some breakdown of communication. This is true in any group of people coming together; it’s just exacerbated in a virtual environment” (Chase, 1999). The lack of regular visual and nonverbal cues also inhibits the normal development of a team, and a breakdown in communication can be disruptive to a team’s decision making process and the quality of the decisions. Uses of virtual world technology have shown that immersive worlds can support a variety of human activities and interactions that impart a wealth of new opportunities and challenges for enriching how we learn, work, and play (Boulos, Hetherington and Wheeler, 2007; PrasolovaForland, Sourin and Sourina, 2006). Organizations can easily collaborate and make decisions in an immersive, three-dimensional environment regardless of physical distance. Given the 2|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

increasing use of virtual teams in business as well as the introduction of virtual world technology, it is necessary for practitioners and researchers to understand the process of working together in a virtual world, to determine how virtual world technology impacts the decision making process, and to assess the quality of the decisions made by the team. There are many virtual worlds in existence today: Open Sim, Active Worlds, and Second Life, to name a few. Currently, Second Life has the largest population with approximately 18 million residents. To better understand the complex social interactions that can occur in a virtual team operating in a virtual world and to increase our knowledge of how team decision quality is impacted through the use of virtual world technology, we investigated the use of Second Life in a structured virtual team exercise. This paper addresses the following research questions: 1) Does virtual world technology impact a team’s decision making process? 2) Can virtual teams using virtual world technology achieve equivalent decision quality compared to teams who meet face-to-face? Studies of technology supporting virtual teams provide important information for researchers and practitioners on phenomena related to communication and information sharing (Majchrzak and Malhotra, 2003; Pinsonneault and Caya, 2005; Powell, Piccoli, and Ives, 2004) as well as the challenge of overcoming limits to location and dispersion (Cousins and Robey, 2005; Jin and Robey, 2008; Sotto, 1997). This stream of research has implications for virtual world technology and the use of virtual teams in practice. While there is an established body of research on virtual teams, few researchers have addressed the impact of virtual world technology on decision making and decision value. This study adds insights to this stream of research. 3|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

The paper is organized as follows. First, we discuss virtual world technology and virtual teams. Next, we discuss a multi-criteria model of decision value that considers both the process of, and outcomes from, decision making by comparing decision making in teams operating faceto-face versus virtual teams operating in a virtual world. In the next section the methods and experiment are explained, and the results are presented and analyzed. Finally, the research findings are discussed and summarized, and our contribution to literature is addressed. 1.1 Virtual World Technology Virtual worlds, also known as metaverses, are three-dimensional communities that mimic the real world without its physical limitations. They reinvent the ideas of "being together" and awareness for distributed teams. Within virtual worlds, globally dispersed individuals can create avatars and work together in virtual teams as if they were operating in a face-to-face situation. Virtual world avatars (i.e. virtual persons) provide team members with a level of realism that includes social cues and non-verbal interaction such as gestures, facial expression, and body language. Avatars represent the embodiment of the user in a virtual world which allows for a greater sense of control within the immersive environment, allowing users to more readily engage in events as they were occurring in real time (Gazzard, 2009). The vivid and interactive features of virtual worlds allow users to experience circumstances that may influence their awareness and subsequent decision making. Figure 1 and Figure 2 display pictures of Loyola University Maryland’s Second Life Campus. Second Life is the virtual world used for this study.

4|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Figure 1: Inside the Sellinger Business Building

Figure 2: Outside on the University’s Grounds

Previous research suggests that the appearance of avatars in the virtual world coupled with their interactions, can affect a person’s sense of presence and decision making (Biocca, Harris, and Burgoon, 2003; Blascovich, 2002; Lombard and Ditton, 1997). A key theme of presence is the idea that realistic interactions with responsive representation in contextually 5|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

accurate settings could enhance the quality of interaction (Blascovich et al., 2002). This increase in presence of the user is due to their avatar living in the virtual world. The use of virtual world technology offers a feeling of presence that naturally allows for more complex interactions as well as increased interactivity and achieving a good level of decision quality. Research on the impact of virtual world technology and team decision making has yet to be examined in-depth (Davis et al., 2009).

1.2 Face-to-Face and Virtual Teams Teams provide a method for combining various skills, talents and perspectives to achieve business goals (Siebdrat, Hoegl, and Ernst, 2009). The importance and use of teams in organizations has been well established (Paul et al., 2004) to take on a variety of issues such as product customization, new product development, and strategic direction. Organizations believe that teams operating effectively can lead to a competitive advantage (Cooke, Kiekel, and Helm, 2001). As the workplace has globalized and technology has delivered the ability to communicate across geographic boundaries, virtual teams have become a fundamental business proposition. Virtual teams can be more complex than traditional face-to-face teams, because they cut across time, distance, and even organizations. Recent organizational studies have shown that distributed work teams are assigned some of the most important tasks in innovative, knowledge-intensive organizations including making decisions and solving problems (Mazneviskia and Chuboda, 2000; Leinonen, Jarvela, and Hakkinen, 2005).

6|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Research continues to support the findings of Mehrabian (1971) that human communication is predominately nonverbal. However, the majority of technology presently used to support virtual teams such as email, chat, web conferencing, and discussion boards has not been able to offer non-verbal cues (Wilson, 2003). Kezsbom (2001) found that: “Even in the best videoconferencing facial expressions can be difficult to pick up if the transmission is poor, if someone is off camera, or when the mute button is pressed” (p.35). Consequently, prior research on virtual teams has focused on communication issues such as the loss of social cues due to lean communication technology in comparison to that in face-to-face teams. Several challenges and difficulties associated with using traditional Internet technologies for virtual team work are: 1) the need to compensate for the lack of face-to-face interaction. When meeting in person, team members can learn from verbal and non-verbal interactions, such as voice levels, smiles, and facial expressions (Kezsbom, 2001). 2) problems with building relationships in virtual teams. Successful teams are founded on good relationships. It is easier to build face-to-face teams than in virtual teams (Rockett, Valor, Miller, and Naude, 1998). 3) difficulties accessing and leveraging the unique knowledge of each member to successfully achieve the team’s goal. According to Anderson and Shane (2002), “While a team is working on a project, usually the project is like a puzzle, and each team member is responsible to work on an assigned piece of the project” (p.6). The team must develop a system for sharing information so that nothing relevant to solving the problem is lost. What some team members may consider the most insignificant piece of information may prove to be critical to the final solution. 7|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Virtual teams rely on technology to carry out tasks (Davis et al., 2009, Dube and Pare, 2004, Zigurs, 2003). Face-to-face communication is limited if it occurs at all. Virtual world technologies offer virtual teams an entirely new way of working together by providing an environment that has the potential for richer and more engaging collaboration than has been possible with traditional technologies, leading to more efficient processes and better outcomes. 2.0 Assessing Decision Value The process of decision making involves the procedures and steps that a decision maker utilizes in making a decision. Table 1 displays four well-known decision making models. Groups and teams use specific processes such as collaboration within each of the phases in order to arrive at a decision (Phillips-Wren et al., 2009). Table 1: Comparison of Decision Making Process Models. Simon (1977)

Boyd (1950)

Turban and Aronson (1998)

Decision Making Steps (Forgionne 1999)

Intelligence

Observe -Observe unfolding circumstances-Gather outside information

Intelligence -Organizational objectivesData collection-Problem identification, ownership, classification, statement

Identifying objectives Recognizing the problem

Design

Orient -Perceive opportunities and threats-Focus thinking on particular direction

Design -Formulate model-Set criteria for choice-Search for alternatives-Predict and measure outcomes

Gathering data Generating alternatives Establishing criteria Evaluating alternatives

Choice

Decide -Make a decision Choice -EvaluationSensitivity analysis

Choice -Solution to problemSensitivity analysisSelection of best (good) alternatives-Plan for implementation

Making a choice

8|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Implementation

Act -Act on the decision

Implementation -Implement the decision

Implementing the choice Learning Synthesizing

All of the models describe similar phases in the decision making process. The decision making phases generally proceed sequentially, with iteration and feedback loops as more is known about the problem. Sometimes the phases are so tightly linked that they are difficult to separate. Simon’s (1977) model was based on managerial decision making, and it is best known along with its added fourth phase. Boyd’s (1950) model was developed for real-time events that change quickly, and Turban and Aronson’s (1998) description is close to that of Simon. Forgionne (1999) separated out Learning as a distinct phase, although some researchers argue that learning is embedded in Simon’s model. Forgionne described the process as a series of steps, with feedback loops, and related them to Simon’s model. All of the models in Table 1 describe a process culminating in a decision. Thus, we have included both the process of, and outcome from, decision making in our evaluation of the decision. A review of the literature shows that researchers include both process and outcome metrics to evaluate decision making and decision support systems (Phillips-Wren et al., 2009). Process metrics could be, for example, faster decision making or better data during the intelligence phase. Outcomes from decision making depend on the specific decision problem, and examples of outcome metrics are accuracy or user satisfaction. A generalized multiplecriteria model to determine decision value is shown in Figure 3 and incorporates Simon’s phases

9|P ag e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

(intelligence, design, choice, implementation and learning) as well as outcomes from decision making.

Figure 3. Generalized multiple-criteria model for decision value.

The model shown in Figure 3 can be evaluated with the Analytic Hierarchy Process (AHP). AHP is a multiple-criteria method that can incorporate both qualitative and quantitative criteria into a single metric (Saaty, 1977, 1994). The AHP is a plausible method that provides a logical and scientific basis for multiple-criteria decision making (Harker, 1988) and has been widely applied to both individual and group decision making scenarios since the early 1980s (Wind and Saaty, 1980; Saaty and Vargas, 1994). In order to evaluate the model using the AHP, pairwise comparisons are made between alternatives on each element at the lowest level of the hierarchy. The AHP model then computes the decision quality by combining the criteria in the model such that the sum of the relative rankings of the alternatives at each step in the AHP is one. In the research in this paper, we will compare virtual teams in a virtual world (VT) to face-to-face (FF) teams. The relative 10 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

comparison of VT to FF will be an input for each element at the lowest level of the hierarchy. The AHP will then compute the ratio VT/FF for Process and for Outcome. At the top of the hierarchy, the AHP will provide the ratio VT/FF. We posit that there is no difference in Decision Quality between VT operating in a virtual world and FF teams. We will present quantitative metrics from the empirical experiment that will provide the input for the initial comparison. Our hypotheses are then: H1: Virtual teams operating in a virtual world and face-to-face teams will not have a significant difference in the process of decision making. H2: Virtual teams operating in a virtual world and face-to-face teams will not have a significant difference in the outcome of decision making. H3: Virtual teams operating in a virtual world and face-to-face teams will not have a significant difference in decision quality.

3.0 Research Methodology The focus of our study is the decision making process and team decision outcomes for teams working face-to-face and in Second Life. This design led to the identification of subjects from an undergraduate information technology business course. Before teams were formed and before the experiment was begun, subjects were introduced to Second Life via two sessions with a total of three hours in a computer lab. During this time, subjects were given help to create Second Life accounts, create Second Life avatars, and interact within Second Life while working together in the classroom. In addition, they were given two Second Life practice exercises to complete outside of the computer lab, and they were provided with headsets to ensure that they 11 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

utilized all of the functionality such as verbal conversation provided by Second Life. The focus of these assignments was to give subjects time to acclimate themselves to their avatars and to using Second Life. The decision problems chosen for this study included two different survival decisions (a survival at sea scenario and a survival in space scenario) with a known optimal solution completed as a team. The objective was to decide on the relative ranking of 15 items in each scenario important to survival. Teams were given a survival scenario and were instructed to record the start and stop time used in the phases of decision making, including ranking the fifteen items in order of importance. Each person completed two decisions in teams, once with the at sea scenario and once with the space scenario. A total of twenty-one groups with three team members and four groups of four team members were formed. The research design is consistent with research reported by Fjermstad and Hiltz (1999) regarding number of groups, subjects per group, and total number of subjects. To avoid issues with team familiarity, teams were randomly formed for the face-to-face decision and for the virtual world decision. To help ensure validity and to reduce the possibility of order effects, half of the teams operated first in Second Life and then face-to-face, while the other half of the teams began meeting face-to-face. 4.0 Results Teams recorded the time it took for various decision-making tasks and submitted their final decisions. In addition, they responded to qualitative questions designed to capture their opinions of decision making in the two environments. The mean amount of time for the decisionmaking tasks for virtual teams and face-to-face teams is shown in Table 2. 12 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Table 2. Mean times for virtual teams and face-to-face teams. Mean time to:

Virtual Team

Understand the problem Generate Alternatives and Make Choice Decision Making Process Total

1.76 12.97

Face-to-Face Team 1.24 10.92

14.87

12.57

Since the rankings of the various items in each scenario have a known solution, the difference between the correct answer and the team decision can be calculated. The mean differences are shown in Table 3. Table 3. Mean difference from correct decision for virtual teams and face-to-face teams. Virtual Team 47.5

Face-to-Face Team 57.9

The hypotheses H1, H2 and H3 can be tested using the model in Figure 2 and the results in Tables 2 and 3. In our decision problem, the Intelligence phase component of Process is measured by the time to complete this stage. Subjects discussed alternatives and determined final ranking of items with an iterative approach as they progressed from Item 1 to Item 2, etc. Thus, we combined the Design and Choice phases in Figure 3. We did not measure Implementation or Learning. The Outcome component of Decision Quality in Figure 3 is measured by the accuracy of the final decision as determined by the mean difference between the team decision and the correct decision as shown in Table 3. 5.0 Analysis of Results

13 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Our hypotheses H1, H2 and H3 are that virtual teams operating in a virtual world and face-to-face teams will not have a significant difference in the process of decision making, the outcome from decision making, and decision quality. We tested the hypotheses two ways: (1) H1 and H2 were tested with a two sample t-test with a level of significance of 0.05, and (2) H3 was tested with a multi-criteria model implemented with the analytic hierarchy process. We used group-level aggregate scores for the variables in our statistical analyses. The use of groups as the unit of analysis is consistent with prior research (Gallupe and DeSanctis, 1988; Valacich et al., 1992). H1 proposes that the process of decision making would not be significantly different in the two environments. The metric used for comparison was the mean time the teams spent making their decisions, that is, does it take longer to make a decision in one environment compared to the other. The t-test results are shown in Table 4 and reveal that there is no significant difference in the amount of time to make a decision for virtual teams in a virtual world and face-to-face teams at the 0.05 level of significance. The results support H1. Table 4. t-test results for mean difference in time to make a decision (process of decision making) for virtual teams in a virtual world and face-to-face teams.

Face -to-face teams Virtual World teams

Mean St. Dev. 12.57 5.9 14.87 6.72

95% CI (-5.21, 0.61) p-value = 0.119 H2 proposes that the outcome of decision making would not be significantly different in the two environments. The metric used for comparison was the mean difference in the ranking 14 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

of items in the decision, that is, is the accuracy better one environment compared to the other. The t-test results are shown in Table 5 and reveal that there is no significant difference in decision accuracy for virtual teams in a virtual world and face-to-face teams at the 0.05 level of significance. The results support H2. Table 5. t-test results for mean difference from correct decision (outcome of decision making) for virtual teams in a virtual world and face-to-face teams.

Face -to-face teams Virtual World teams

Mean St. Dev. 57.9 15.9 47.46 9.71

95% CI (-5.21, 5.86) p-value = 0.907

H3 proposes that the overall decision value would not be significantly different in the two environments. We used the multi-criteria model shown in Figure 3 and implemented it with the analytic hierarchy process as shown in Figure 4. At the bottom of the hierarchy the two alternatives (face-to-face and virtual teams) are compared on each criterion such that the sum is equal to one. The times to recognize the problem and the time to generate alternatives/make a choice provided the relative comparisons under Process, where we have associated shorter time as better process. The mean difference from the correct decision was used to compare the alternatives under Outcome Value. The AHP computes higher level comparisons, with the Decision Value computed as 0.5009/0.4991 for face-to-face versus virtual teams where we have equally weighted each criterion. Although the AHP does not provide a statistical comparison, the values are close, with a slight advantage to face-to-face teams. Interestingly, we can see that 15 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

virtual teams have better decision accuracy or Outcome, while face-to-face teams show better process. The results suggest support for H3, i.e. the decision value achieved by face-to-face teams was the same as teams operating in a virtual world.

6.0 Discussion In this research, we compared the decision value of teams in face-to-face dialogue and communicating through a virtual world, Second Life. We found that there was no significant difference in the process of, and outcome from, decision making. The decision value was close for the two environments. These results show promise for virtual team decision making. The results indicate that the additional presence achieved through virtual world technology may help overcome some of the limitations that have been previously reported in the literature with the use of virtual teams.

16 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Qualitative information was obtained from the experiment by soliciting comments about the experience of decision making in a virtual team in Second Life. A summary of the strengths listed are located in Table 6, and a summary of the weaknesses are located in Table 7.

Table 6. Strengths of decision making for virtual teams in a virtual world as reported by participants. Strengths We were able to agree on the rankings in a timely matter. In typical team projects, we have a leader and 2 minions. However in the virtual world, we were all equal. Cooperation between our team members was a big strength, because we all listened to each other and agreed together. Being able to work from home. It was efficient and saved time. Being able to share our ideas easily and efficiently. It was easy to roll out of bed and go to work. We were able to share ideas and collaborate. It felt like we were all standing together. It made the decision process easier. Using the voice part really increase my feeling of being together. We all felt a significant level of presence that contributed us being able to reach consensus easily. We had great verbal communication It was easy to express our ideas and come to a consensus.

Table 7. Weaknesses of decision making for virtual teams in a virtual world as reported by participants. Weaknesses

17 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

As much as I was expecting weaknesses of working as a team to complete this exercise in the virtual world, there weren't any. Everyone was comfortable in Second Life. I am comfortable enough with the online world, that I didn't see a huge difference between the two assignments. We experienced some problems with the sound. A little confusing communicating at first. It took longer than when worked face-to-face. I don't think there were any weaknesses. Using the sound was difficult at times due to feedback. It was annoying when we started working together in Second Life. We experienced some technical difficulties at first. Getting used to using Second Life as a team was a bit confusing at first. There were no clear weaknesses to our team. It was a little awkward because we could not actually see each other. It was a little hard to read people. It was a little difficult to organize the discussion.

Our results provided some interesting findings. First even though there are more strengths listed than weaknesses, several of the participants reported difficulties getting started as a team in Second Life. Participants reported technical problems such as sound and discomfort of working as a team in Second Life even with training in the virtual world. Post-experiment comments were consistent with the t-test results of how hard it was to discuss the problem with their team (shown in Table 8), and how hard was it to discuss the most important item with their team (shown in Table 9). Virtual teams had significantly more difficulty than face-to-face teams on both items. Table 8. t-test of difficulty in discussing the decision problem. How hard was it to discuss the problem with your team? Mean St. Dev. Face-to-face 1.84 1.19 Virtual World 2.76 0.971 18 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

95% CI (-1.43, -0.42) p-value = 0.000

Table 9. t-test of difficulty in discussing the most important item in the decision problem. How hard was it to discuss the most important item? Mean St. Dev. Face-to-face 1.86 1.29 Virtual World 2.74 0.98 95% CI (-1.40, -0.34) p-value = 0.002 The results obtained from the t-tests comparing the participant’s perceptions between face-to-face team work and virtual team work showed that that there was a significant difference perceived in the difficulty level during the beginning stages of the team decision making. However, the t-test comparisons between the face-to-face teams and virtual teams show no significant difference in the level of difficulty when making their team decisions. Table 10 and Table 11 display our results. Table 10. t-test of difficulty in discussing the rankings of the items in the decision problem. How hard was it to discuss the rankings with your team? Mean St. Dev. Face-to-face 2.49 1.17 Virtual World 2.79 0.905

19 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

95% CI (-0.79, 0.18) p-value = 0.215

Table 11. t-test of difficulty in agreeing on the rankings of the items in the decision problem. How hard was it to agree on the rankings with your team? Mean St. Dev. Face-to-face 2.11 0.94 Virtual World 2.4 0.89 95% CI (-0.71, 0.13) p-value = 0.178

Coupling the qualitative comments and the t-test results of difficulty provides insight into how the participants perceived team decision making in Second Life. Although individuals perceived difficulty in initial communication, they overcame these issues and worked together productively. This observation is an indicator of the importance of training employees in using virtual technology before initiating virtual teams in a virtual world. Secondly, some of the participants reported that it took more time to make a decision in the virtual world than when meeting face-to-face and the results suggest that this is true, although they are not significantly different. Again, these comments emphasize the importance of training and education with virtual worlds before companies utilize the technology for virtual teams. Lastly, to successfully implement a virtual team using virtual world technology, practitioners should consider clearly setting out the expectations and requiring participants to script the steps that are need to follow in order for the team project to be successful. Since 20 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

studies have indicated that the productivity of teams is threatened by a lack of common ground and lack of awareness among team members (Beers et al., 2005; Makitalo et al., 2005; Dillenbourg, 2002; Barab, MaKinster, and Scheckler, 2004), scripting out steps to follow could greatly help a virtual team’s performance in a virtual world environment. For example, when group members know more about their working processes, they can plan their team work and individual work more effectively. Rummel et al. (2003), who investigated how collaboration scripts affect medical and psychology students, showed that those dyads of students who had followed process script during the shared working processes produced significantly better results than the dyads in a non-scripted condition. Given the newness of virtual world environments, scripting the processes that virtual team members should follow could greatly improve their efficiency and their outcomes. In addition, it could increase their knowledge and learning about operating in a virtual team that they could then transfer to other virtual teaming arrangements.

7.0 Summary and Contribution Virtual worlds are interesting new technology that permits teams to communicate and meet across physical boundaries more realistically than more traditional methods such as phone or video conferencing. The virtual environment can be made to emulate a real location, and participants can create avatars that look like themselves and use their own voice. As virtual teams become more utilized in business, practitioners can potentially increase the effectiveness and efficiency of these teams. This study has contributed to the literature by assessing the impact of a virtual world on team decision making compared to face-to-face teams. The study has provided quantitative and qualitative data that suggest that virtual teams operating in a virtual 21 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

world can achieve the same level of process, outcome, and decision quality as teams operating face-to-face. Our results provide support that organizations that properly train and educate their employees could employ virtual teams without negatively impacting the decision making process or the quality of the outcome reached. Future research should investigate other possible outcomes such as user satisfaction and creating teams from individuals who do not have a prior relationship. We did not directly test for satisfaction; however, qualitative comments from participants suggest that this could have impacted our results. In addition, all of our participants are from the same organization and may know each other so that trust and other social interactions are established. Results may different with teams of people who have not met or are from different companies.

8.0 References Anderson, F., and Shane, H. (2002). The Impact of Netcentricity on Virtual Teams: The New Performance Challenge. Team Performance Management, 8 (1/2), 5-7. Barab, S., MaKinster, S., and Scheckler, R. (2004). Designing System Dualities: Characterizing an Online Professional Development Community. In S. Barab, R. Kling, and J. Gray (eds.): Designing for Virtual Communities in the Service Learning. New York: Cambridge University Press, 307-359. Barron, B. (2003). When Smart Groups Fail. The Journal of The Learning Sciences, 12 (3), 307359. Beers, P., Boshuizen, H., Kirschner, P., and Gijselaers, W. (2005). Computer Support for Knolwedge Construction in Collaborative Learning Environments. Computers in Human Behavior, 21 (4), 623-643. Bell, M. A. (2005). Virtual Hybrid Workgroups Are Critical to Successful Offshore Sourcing, Stamford, CT: Gartner. Biocca, F. and Harris, C. and Burgoon, J. K. (2003). Toward a more robust theory and measure of social presence: Review and suggested criteria. Presence. 12 (5), 456-480. Blascovich, J. (2002). A theoretical model of social influence for increasing the utility of collaborative virtual environments. 4th International Conference on Collaborative Virtual Environments, Bonn, Germany, 2002, 25-20.

22 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Blascovich, J., Loomis, J., Beall, C., and Swinth, K. R. (2002). Immersive virtual environment technology as a methodological tool for social psychology. Psychological Inquiry. 13 (2), 103-124. Boulos, M.N., Hetherington, L., and Wheeler, S. (2007). Second Life: An overview of the potential of 3-D virtual worlds in medical and health education. Health Information and Libraries Journal, 24(4), 233- 245. Boyd, J. (1950) “A Discourse on Winning and Losing,” unpublished collection of lecture slides. A few copies are available via Interlibrary Loan from such sources as the Marine Corps University Library, the depository for Colonel Boyd’s papers. Brennan, S. (1998). The Grounding Problem in Conversation with and through Computers. In S.R. Fussel and R.J. Kreuz (eds.): Social and Cognitive Approaches to Interpersonal Communication. Mahwah, NJ: Erlbaum. 201-225. Bromme, R. (2000). Beyond One’s Own Perspective. In P. Weingart and N. Stehr (eds.): Practicing Interdisciplinarity. Toronto Canada: University of Toronto Press. 115-133. Chase, N. (1999). Learning to Lead a Virtual Team. Quality, 38, 76-78. Cooke, N. J., Kiekel, P. A., and Helm E. (2001). Measuring team knowledge during skill acquisition of a complex task. International Journal of Cognitive Ergonomics: Special Section on Knowledge Acquisition, 5, 297-315. Cousins, K. C. and Robey, D. (2005). Human agency in a wireless world: Patterns of technology use in nomadic computing environments. Information and Organization. 15, 1151-180. Davis, A., Murphy, J., Owens, D., Khazanchi, D., and Zigurs, I. (2009). Avatars, People, and Virtual Worlds: Foundations for Research in Metaverses. Journal of the Association for Information Systems. 10 (2), 91-117. Dillenbourg, P. (2002). Over-Scripting CSCL: The Risks of Blending Collaborative Learning with Instructional Design. In P.A. Kirschner (ed.); Three Worlds of CSCL Can We Support CSCL. Heerlen: Open University Nederland, 61-91 Dube, L. and Pare, L. (2004). The Multi-faceted Nature of Virtual Teams. In D.J. Pauleen (Ed.) Virtual teams: Projects, protocols, and processes: Idea Group Publishing. Fjermestad, J. and Hiltz, S. (1999). An assessment of group support systems experimental research: methodology and results. Journal of Management Information Systems, 15 (3), 7149. Flor, N. and Hutchins, E. (1991). Analysing Distributed Cignition in Software Teams: A Case Study of Team Programming during Adaptive Software Maintenance. In R. Baecker (ed.): Reading in Groupware and Computer Supported Cooperative Work. San Mateo,CA: Morgan-Kaufman. Forgionne, G. (1999). An AHP model of DSS Effectiveness. European Journak of InformationSystems, 95-106. Gallupe, R. B., and DeSanctis, G. (1988). Computer-based support for group problem-finding: An Experimental Inverstigation. MIS Quarterly, 12(2), 277-296. Gazzard, A. (2009). The Avatar and the Player: Understanding the Relationship Beyond the Screen. Conference in Games and Virtual Worlds for Serious Applications (VS-Games). IEEE Piscataway, NJ Coventry, UK. 190-236.

23 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Gross, T., and Prinz, W. (2003). Awareness in Context: A Light-Weight Approach. In Kuutii, K., Karsten, E., Fizpatrick, G., Dourish, P., and Schmidt, K. (eds.) ECSCW 2003: Proceedings of the Eight European Conference on Computer Supported Cooperative Work, 14-18 September 2003, Helsinki, Finland. 295-314. Hacker P. (1988). The Art and Science of Decision Making: The Analytic Hierarchy Process. Working Paper 88-06-03, Decision Science Department, The Wharton School, University of Pennsylvania, Philadelphia, PA. Hardin, C., and Higgins, E. (1996). Shared Reality: How Social Verificiation Makes the Subjective Objective. In R.M.Sorrentino and E.T. Higgins (eds.): Handbook of Motivation and Cognition: Foundation of Social Behavior. New York: Guilford. 28-84. Herman, S. (2001). Counterpoints: Notes on OD for the 21st century – part II. Organizational Development Journal, 19(1), 115-118. Hewitt, J. and Scardamalia (1998). Design Principles for Distributed Knowledge Building Processes. Educational Psychology Review, 10 (1), 75-96. Higgins, E. (2000). Social Cognition: Learning About What Matters in the Social World. European Journal of Social Psychology, 30, 3-39. Jin, L. and Robey, D. (2008). Bridging social and technical interfaces in organizations: An interpretive analysis of time-space distanciation. Information and Organization, Kerr, N., and Bruun, S. (1983). Disspensibility of Member Effort and Group Motivation Losses: Free Rider Effects. Journal of Personality and Social Psychology, 44, 78-94. Kezsbom, D. (2001). Creating Teamwork in Virtual Teams. Cost Manufacturing, 42 (10), 33-37. Leinonen, P., Jarvela, S., and Hakkinen, P. (2005). Conceptualizing the Awareness of Collaboration: A Qualitative Study of a Global Virtual Team. Computer Supported Cooperative Work, 14, 301-322. Leinonen, P., Jarvela, S., and Lipponen, L. (2003). Individual Students’ Interpretations of their Contribution to the Computer-mediated Discussions. Journal of Interactive Learning, 14 (1), 99-122. Lombard, M. and Ditton, T. (1997). At the heart of it all: The concept of presence. Journal of Computer-Mediated Communication. (3) 2. Majchrzak, A., and Malhotra, A. (2003). Deploying Far Flung Teams: A Guidebook for Managers. Chicago, IL: Society for Information Management. Makitalo, K., Weinberger, A., Hakkinen, P., Jarvela, S., and Fischer, F. (2005). Epistemic Cooperation Scripts in Online Learning Environments: Fostering Learning by Reducing Uncertainty in Discourse?. Computers in Human Behavior, 21 (4), 603-622. Maznevski, M., and Chudoba, K. (2000). Bridging Space Over Time: Global Virtual-team Dynamics and Effectiveness. Organization Science, 11 (2), 473-492. Mehrabian, A. (1971). Silent messages. Belmont, CA: Wadsworth. Paul, S., Samarah, I., Seetharaman, P, and Mykytyn, P. (2004). An Empirical Investigation of Collaborative Conflict Management Style in Group Support System-Based Global Virtual Teams. Journal of Management Information Systems, 21 (3), 185-222. Phillips-Wren, G., Mora, M., Forgionne, G. and Gupta, J. (2009). An integrative evaluation framework for intelligent decision support systems. European Journal of Operational Research, June 16, 195 (3), 642-652. 24 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

Pinsonneault, A., and Caya, O. (2005). Virtual Teams: What We Know, What We Don’t Know. International Journal of e-Collaboration, 3 (1), 1-16. Prasolova-Forland, E., Sourin, A., and Sourina, O. (2005). Place Metaphors in Educational Cyberworlds: a Virtual Campus Case Study. Proceedings 2005 International Conference on Cyberworlds, CW2005. 221-228. Prince, C., Stout, R. and Salas, R. (1994). Team situational awareness preliminary lessons learned: Identification of skills, behaviors, design guidelines, and training strategies from interviews and research. Technical Report, FAA and Naval Air Warfare Center, Training Systems Division. Powell, A., Piccoli, G., and Ives, B. (2004). “Virtual teams: team control structure, work processes, and team effectiveness,” Information Technology and People, 17 (4), 359-379. Rockett, L., Valor, J., Miller, P, and Naude, P. (1998). Technology and Virtual Teams: Using Globally Distributed Groups in MBA Learning. Campus-Wide Information Systems, 15 (5), 174-182. Robey, D., Schwaig, K. S., and Jin, L. (2003). Intertwining material and virtual work. Information and Organization, 13, 111-129. Rummel, N., Spada, F., Casper, F., Ophoff, J. and Schornstein, K. (2003). Instructional Support for Computer-Mediated Collaboration. In B. Wasson, S. Ludvigsen and U. Hoppe (eds.): Designing for Change in Networked Learning Environments. Proceedings of the International Conference on Computer Support for Collaborative Learning 2003. Dordecht: Kluwer Academic Publishers, 199-208. Saaty, T. L. (1977). A Scaling Method for priorities in Hierarchical Structures. Journal of Mathematical Psychology. 234-281. Saaty, T., and Vargas, L. (1994). Decision Making in Economic, Political, Social and Technological Environments with the Analytic Hierarchy Process. RWS Publications. Pittsburgh, PA. Schmidt, K. (2002). The Problem with “Awareness”. Computer Supported Cooperative Work: Journal of Collaborative Computing, 11 (3-4), 285-298. Siebdrat, F., Hoegl, M., and Ernst, H. (2009). How to Manage Virtual Teams. MIT Sloan Management Review, 50 (4). Simon, H., 1977. The new science of management decision. 3rd ed. Englewood Cliffs, NJ: Prentice-Hall. Sotto, R. (1997). The virtual organization. Accounting, Management, and Information Technologies, 7 (1), 37-51. Turban, E., and Aronson, J. (1998). Decision Support Systems and Intelligent Systems. Upper Saddle River, NY: A. Simon and Schuster Company. Valacich, J. S, Dennis, A. R., and Nunamaker, J. F., Jr. (1992). Group Size and Anonymity Effects on Computer-Mediated Idea Generation. Small Group Research, 23, 1, 49-73. Wilson, S. (2003). Forming Virtual Teams. Quality Progress, 36 (6), 36-41. Wind, Y., and Saaty, T. L. (1980). Marketing Applications of the Analytic Hierarchy Process. Management Sciences. 26(7), 641-658. Zigurs, I. (2003). Leadership in Virtual Teams: Oxymoron or Opportunity?, Organizational Dynamics. 31 (4), 339. 25 | P a g e

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

1)

2) 3)

4) 5) 6) 7) 8) 9) 10) 11)

12)

13)

14)

15)

16) 26 | P a g e

Appendix. Data Collection questionnaire. How was the team exercise performed? a. Face-to-face b. Virtually What time did you start discussing the problem? Start Time:__________________ What time did you start discussing which items are most important to solve the problem? Start Time:__________________ What time did your team start discussing the #1 ranked item? Start Time:__________________ What time did your team start discussing the #2 ranked item? Start Time:__________________ What time did your team start discussing the #3 ranked item? Start Time:__________________ What time did your team start discussing the #4 ranked item? Start Time:__________________ What time did your team start discussing the #5 ranked item? Start Time:__________________ What time did your team start discussing the items ranked 6 - 10? Start Time:__________________ What time did your team start discussing the items ranked 11 - 15? Start Time:__________________ How did your team decided on the rankings? a. Vote b. One person decided c. Consensus d. Majority How hard was it to discuss the problem with your team? 1 2 3 4 5 Very Easy Easy Very Hard How hard was it to discuss which items were the most important with your team? 1 2 3 4 5 Very Easy Easy Very Hard How hard was it to discuss the item rankings with your team? 1 2 3 4 5 Very Easy Easy Very Hard How hard was it for your team to agree on the ranking order? 1 2 3 4 5 Very Easy Easy Very Hard Describe your experience of working as a team to perform a team exercise.

Reference: Pridmore, J. and Phillips-Wren, G. (2011). Assessing decision making in face-toface teams versus virtual teams in a virtual world, Journal of Decision Systems, 20(3), 283-308.

17) 18)

27 | P a g e

What were the strengths of working as a team for completing this exercise? What were the weaknesses of working as a team to complete this exercise?