Effects of Cross-Training on Team Performance ...

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 56th ANNUAL MEETING - 2012

1581

Effects of Cross-Training on Team Performance, Communication, and Workload in Simulated Air Battle Management Adam J. Strang Consortium Research Fellows Program, Alexandra, VA Gregory J. Funke, Benjamin A. Knott, Scott M. Galster, & Sheldon M. Russell Air Force Research Laboratory, Wright-Patterson AFB, OH Team cross-training is used to improve team interpositional knowledge (IPK). IPK is thought to promote adaptability, allowing teams to maintain coordination and performance when faced with challenges, such as increased task demands and role reconfiguration. The current experiment examined the effects of experiential cross-training, a form of training where teammates practice each other’s tasks and duties, in 5person teams performing a command and control (C2) air battle management (ABM) simulation over a 5day training period. Results indicated that under some conditions, cross-training resulted in slightly diminished team performance relative to control teams. Cross-trained teams also reported higher levels of workload throughout training. However, teams who underwent cross-training were better able to maintain team communication when faced with increased task demands.

Not subject to U.S. copyright restrictions. DOI 10.1177/1071181312561315

INTRODUCTION Modern combat operations frequently involve ill-defined problems in situations of uncertainty, critical time constraints, and high stakes outcomes (Marks, Sabella, Burke, & Zaccaro, 2002). Under these circumstances, effective teamwork and communication are vital for mission success. Contemporary combat operations are often performed by action teams, i.e., teams where expertise, information, and tasks are distributed, and where achieving objectives depends on coordinating team behaviors (Marks et al., 2002). It has been proposed that interpositional knowledge (IPK), i.e., the information team members retain regarding the roles and tasks of fellow teammates, is critical for action teams because it enhances teammates’ ability to anticipate the needs, constraints, and actions of fellow team members with minimal coordination and communication (Volpe, Cannon-Bowers, Salas, & Spector, 1996). Cross-training is an instructional method posited to promote IPK by providing team members with instruction and/or experience with the tasks, duties, and responsibilities of fellow teammates (Cannon-Bowers, Salas, Blickensderfer, & Bowers, 1998; Salas, Nichols, & Driskell, 2007; Volpe et al., 1996). McCann and colleagues (2000) have articulated four “types” of cross-training, which differ primarily in degree of cross-role experience – positional clarification, positional modeling, positional rotation, and experiential cross-training (McCann, Baranski, Thompson, & Pigeau, 2000). Due to specific goals of the experiment outlined later in this introduction, the current study focused on experiential crosstraining, which is perhaps the most intense form and involves teammates performing the roles and duties of fellow team members during training – this is opposed to the next most intense form of cross-training, positional rotation, in which team members train to a criterion knowledge level on the roles of teammates, but do not subsequently perform those roles.

Seminal research on positional rotation was conducted by Cannon-Bowers and colleagues (1998) who observed triads performing a naval command and control (C2) computer simulation. Their results indicated that positional rotation increased levels of IPK and allowed teams to maintain performance under high task demands. Researchers posited that this latter effect was due to the development of more effective implicit coordination strategies in cross-trained teams, which allowed them to maintain performance when communication bandwidth was constrained by increased task difficulty. McCann and colleagues (2000) examined experiential cross-training in a similar naval simulation and found that it was associated with a slower rate of performance improvement during initial training, but later enabled teams to perform the task more effectively following a team perturbation (spontaneous role reconfiguration). Their findings suggest that experiential cross-training may hinder development of appropriate role-specific task knowledge, but may also instill adaptive advantages (e.g., robustness to role reconfiguration). In order to expand on previous research, one goal of the current study was to examine how cross-trained teams are influenced by access to collaboration technologies. While the past two decades have seen rapid advances in development of such technologies (e.g., instant messaging, electronic whiteboards, video conferencing, etc.), researchers are still exploring the impact of these tools on team performance, communication, and workload in military environments. As described, under conditions of high task difficulty, crosstrained teams are better able to coordinate activities with minimal communication, leading to enhanced team performance (Cannon-Bowers et al., 1998). However, researchers have not addressed how access to methods of communication beyond those afforded orally may interact with cross-training. Arguments could be made that introducing


alternnative collaboraation technolog gies during cro oss-training couldd hinder (becau use of the need to develop fam miliarity with these technologies while w simultaneously learning g new team roles and duties) or facilitate (due to an increased d number and bandw width of comm munication outlets) team deveelopment. It also remainss unclear how cross-training c and a collabboration techno ologies affect team t developm ment, performance, and co oordination over longer trainiing periods. The ccross-training experiments e deescribed above were single sessioon experimentss, meaning thatt participants arrived a at the laboraatory, were asssigned to teamss, underwent trraining, and engagged in performance trials in a single day. Ho owever, training for most miilitary vocation ns takes place over o days, weekks, months, or even e years. In such s contexts, one might posit that the detrim mental effects of o cross-training g on initial team performance reported by MccCann et al. (20 000) might be quickkly mitigated, allowing a this efffect to be conssidered a “low--cost” disadvan ntage for pragm matic applicatio on. Finally, no stu udy has describ bed the effects of o crosstraining on perceiveed workload – even though workload w is an imporrtant practical concern when considering th he true utility of crooss-training. It is possible thaat cross-training g may result in n higheer workload, ow wing to increasses in cognitivee effort assocciated with learrning and perfo orming new teaam roles. Basedd on these conccerns, the curreent study was conceived c to exam mine the effects of team cross--training and co ollaboration technnologies on team m performancee, communicatiion, and perceeived workload d across severall experimental training sessioons.

1582

Appaaratus Workstationss. Five computeer workstationss (one for each h team m member) weree equipped witth LCD monitoors and standdard mouse/keyyboard. Oral coommunication was transsmitted and reccorded using raadio headsets. T To initiate a transsmission, particcipants pressedd a foot pedal uunderneath each workstation. IInstantiation annd relief of each foot press was uused to compuute the number and length of rradio transsmissions. Tactical Simu ulation Enviroonment. A simuulated Air Battlle Managementt (ABM) task w was created using Aptima, Inc.’ss Distributed D Dynamic Decission-making (D DDD) softwaree (verssion 3.0). The ddetails of this ssimulation havee been reported d elsew where (i.e., Straang, Knott, et aal., 2011), but in brief, team mmates were asssigned to one oof three team rooles. Two particcipants served as weapons diirectors (WDs)), who comm municated actioon plans, e.g., scheduling reffueling and targeeting enemy airrcraft, to the team. Two particcipants acted as sw weep operatorss, controlling thhe actions of offfensive team aircraaft. Finally, onne team membeer was assignedd as a tanker operaator, who conttrolled the actioons of two refuueling tankers. The oobjectives of thhe ABM task w were to eliminaate enemy aircraaft before theyy entered frienddly airspace (thhe yellow zone)), protect frienndly assets and aircraft from eenemy attack, and m maintain fuel aand weapons reesources (Figurre 1).

Note:: The current experiment e wass conducted as part of a larger study examin ning the effects of collaboration technnologies on team m performancee and training. The results reporrted here repreesent one facet of that researcch. Readers intereested in broadeer outcomes off this research are a directed to o Stranng, Funke, et all. (2011) and Strang, S Knott, et e al. (2011). METHO ODS Partiicipants Twenty men and a ten women (ages 18-29; M = 22.13 years, SD = 3.04) participated in this t experimentt. Participants comppleted the experriment in five-p person, same-ssex teams. Each team was rand domly assigned d to cross-training (N = 3) or controol (N = 3) cond ditions. All parrticipants proviided written inform med consent prrior to the expeeriment and experimental proceedures were app proved by an Institutional Reeview Board prior to data collectiion. Expeerimental Desiign Reflecting the design of the larger l parent sttudy, a 2 × 5 × 2 × 2 × 2 mixed dessign was emplo oyed. The singlle betweensubjects factor was team type (cross-trained, con ntrol). Within subjects factors werre training day (1-5), task dem mands (standdard, high), colllaboration tecchnology (radio o, augmented), and resource display ay (tabular, grap phical). Depen ndent measuress includded indices of team performaance, team com mmunication, and suubjective work kload.

nshot of the ABM M tactical displayy provided to Figure 1. Screen participa ants.

Team Commu unication. In thhe radio condittion, particcipants commuunicated verballly using radioo headsets. In the au augmented com mmunication coondition, particcipants could convverse using radiio or via two coollaboration toools: text-chat and a virtual whitebboard. Resource Disp splays. The tabuular display prrovided sweep and ttanker operatorrs with access tto team weapoons and fuel statu s information iin a digital form mat. The graphhical resource displlay included sim milar information that was arrranged in analoog format and aavailable to alll team memberrs (Figure 2).


Meassures Team Perform mance. Prosecu ution time (PT)) was defined as thee average amou unt of time (in seconds) it too ok a team to interccept enemy airccraft once they y entered the sim mulated battleespace. Friendly air-space pen netration (FAP P) was defined as thee number of en nemy aircraft (p per trial) that en ntered the yellow w “friendly” zo one. Low PT and a FAP indicaated high team performance.

1583

trainning day are preesented. As noted earlier, reaaders are referrred to a set of pprevious reporrts for thoroughh assessment of the oother factors inccluded in this eexperiment (Sttrang, Funke, et al., 22011; Strang, K Knott, et al., 2011). Team perform mance and com mmunication meeasures were exam mined using mixed model AN NOVAs (α = .05). Following the suuggestion of G Grieve and Ag ((1984), the Box/GeisserGreeenhouse correcttion was applieed to all analysses that incluuded a repeatedd measures facttor with three oor more levels, regarrdless of the prresence/absencce of violationss of the spherricity assumptiion. Prosecution T Time (PT). A m main effect of ttraining day, F (2.344, 9.36) = 13.400, p < .05, and a team type × resource displlay interaction,, F (1, 4) = 8.333, p < .05, werre detected. The mainn effect of trainning day indicaated that PT deecreased across trainiing sessions foor both team typpes (Figure 3).. 145 Team Type

Workload. Folllowing each trrial, participants completed the N NASA Task Loaad Index (TLX X- Hart & Staveeland, 1988). This ssurvey providees a global indeex of perceived d mental worklload on a scalee of 0 to 100 by y combining th he contriibutions of six workload sourrces: mental deemand, tempooral demand, physical p deman nd, performancce, effort, and frustrration. Per the suggestion s of MacMillan M et al. a (2005), the physiical demand co omponent was dropped d from global g estim mates because th he ABM simullation required little overt physiical activity. Proceedure Participants co ompleted an 8-hour informatiion and task familiarization sessiion prior to beg ginning trainin ng sessions. ms then returned d for five consecutive days of simulation Team training. Each sessio on included 16 6 trials – 2 trials in each k demands, colllaboration tech hnology, and combbination of task resouurce display con nditions. Acrosss teams and trraining sessioons the order of trial conditions was counterrbalanced to reducce order effectss, though due to o the large num mber of factors a com mplete counterb balance was no ot possible. Exp periential cross--training was implemented by y randomly asssigning each team member to a new n role at the start of each trraining day. he same roles th hroughout the Contrrol teams were assigned to th experriment. Each ex xperimental triial lasted 10 miinutes and particcipants were giiven a rest perio od (up to 20 minutes) m each time tthey completed d a set of four trials. t Experim mental sessions were completed in approximately a eight hours. LTS RESUL Analyysis Strategy To maintain th he focus of thiss manuscript on n crosstraining, only signifficant results reelated to team type t and

Cross-Trained

135

Control

130 125 120 115 110 105 100 0

1

2

3

4

5

Training Day

Fig gure 3. Mean pro osecution time (P PT) for cross-train ned and control tteams over five ttraining days. Errror bars are stan ndard errors. 135 ay Resource Displa 130

Mean Prosecution Time (sec.)

Figu ure 2. Tabular (le eft) and graphicall (right) resource displays. Both displa ays included info ormation concerning remaining fuel and weapons la abeled by assets s’ call signs.

Mean Prosecution Time (sec.)

140

Tabular Graphical

125 120 115 110 105 100 0

Cross-Trained

Control Team Type

Figu ure 4. Mean prossecution time (PT T) as a function o of team type and resource displayy conditions. Errror bars are stand dard errors.

Visual inspecttion of Figure 4 suggests thatt the team type × ressource display interaction waas driven by a ddecrease in PT for coontrol teams, aas compared too cross-trained teams, when the ggraphical resouurce display waas provided, acccompanied by relatiive little differeence between tteam types wheen the tabular displlay was provideed. Friendly Air--space Penetraation (FAP). A main effect off trainning day, F (2.442, 9.69) = 10.227, p < .05, teaam-type × task


demands, F (1, 4) = 21.79, p < .05, and team type × team communication × resource display × task demands interactions, F (1, 4) = 12.86, p < .05, were detected. The main effect of training day revealed a decrease in FAP with training for both team-types (Figure 5).

detected. The main effects of team type revealed that crosstrained teams reported higher workload (M = 64.53, SE = 2.51) than control teams (M = 49.77, SE = 2.51). 600 Cross-Trained Teams

13

Control Teams

550

Team Type

12

Cross-Trained 11

Control

10 9 8

Mean Cumulative Duration (sec.)

Mean Number of Enemy Aircraft

1584

500 450 400 350 Task Demands

300

Standard High

7 250

6

05

2000 Radio

1

2

3

4

5

Training Day

Figure 5. Mean friendly air-space penetration (FAP) for cross-trained and control teams over five training days. Error bars are standard errors.

Follow-up post hoc analyses of the four-way interaction revealed no further differences due to team type or training day (all p > .05), and therefore, those analyses were not pursued further. Team Communication Cumulative Frequency. No changes related to team type or training day were detected (all p > .05). Cumulative Duration. Analysis revealed team type × task demands, F (1, 4) = 10.60, p < .05, and team type × collaboration technology × task-demands, F (1, 4) = 15.38, p < .05, interactions. Visual inspection of the three-way interaction (Figure 6) indicated that for both control and cross-trained teams the augmented collaboration technology condition drove a decrease in cumulative communication duration, which supports the developmental intent of those technologies (Strang, Knott, et al., 2011). In addition, for control teams, a small but consistent increase in cumulative duration was detected in the high task demands condition, but cross-trained teams showed a slight decrease in cumulative duration under high task demands in the radio condition paired with nearly equivalent durations between task demand levels in the augmented collaboration technology condition. Together, these findings support an interpretation that cross-trained teams were better able to maintain communication when faced with high task demands compared to control teams. Workload NASA-TLX. NASA-TLX ratings were examined using a 3 (team role) × 2 × 5 × 2 × 2 × 2 mixed ANOVA (α = .05). A main effect of team type, F (1, 24) = 22.93, p < .05, and a team type × training day × team communication × taskdemands interaction, F (3.75, 53.05) = 3.00, p < .05, were

Augmented

Radio

Augmented

Collaboration Technology

Figure 6. Mean team communication cumulative duration depicting a team-type × collaboration technology × task demands interaction. Error bars are standard errors.

As was the case with FAP, follow-up post hoc assessment of the observed four-way interaction revealed no further statistically significant differences due to team type or training day (all p > .05), and therefore, those analyses were not examined further. DISCUSSION The purpose of the current experiment was to explore the effects of team cross-training and collaboration technologies on team performance, communication, and perceived workload across several experimental training sessions. Results from team performance metrics suggest several interesting findings. First, team performance improved with training for both team types across training sessions. While this finding itself is not surprising, what is revealing is that no differences were detected in the rate of improvements between the two types of teams. This suggests that experiential cross-training did not result in a lag in performance improvement during initial training in this experiment, in contrast to the findings of McCann and colleagues (2000). This effect may be due to employing a longer, and more realistic, training-period (five days) compared to the single-day training session employed by McCann et al. In short, it may be that the extended training period enabled teams to overcome some of the negative influences on team performance associated with experiential cross-training. In addition, the current experiment employed larger teams (five-person teams) and a degree of role redundancy (i.e., two WDs and two Sweep operators). While the size of the team likely resulted in an increased need for team communication and coordination, role redundancy may have provided opportunities for teammates to effectively share task


responsibilities, supporting team adaptability across team types. Second, the results of the current experiment suggest that control teams were better able to prosecute enemy aircraft than were cross-trained teams when the graphical resource display was provided, but only slightly better than cross-trained teams when the tabular display was provided. As described in the introduction, an interpretation of this effect may be that experiential cross-training reduced team members’ ability to effectively incorporate strategies for using this tool while they were also learning new team roles in each session. Control teams, on the other hand, had more opportunity to become comfortable with their roles and the technologies, perhaps allowing them to develop more effective strategies for communication when the graphical display was available. Assessment of team communication cumulative frequency revealed no difference between team type or change over training day. This seems to indicate that the internal constraints of the simulation task required a relatively consistent number of communications regardless of experimental manipulation or training. The same was not true, however, for cumulative duration, where a complex interaction was detected. With specific regard to crosstraining, what this effect indicated was that control teams tended to increase their overall communication duration under high task demands, while cross-trained teams either slightly decreased (radio condition) or were able to maintain (augmented condition) communication when faced with high task demands. These findings seem to parallel previous research indicating that cross-training promotes the ability to maintain coordination when faced with task constraints (e.g., Cannon-Bowers et al., 1998; McCann et al., 2000). Unfortunately, the ability to maintain consistent levels of communication duration did not seem to lead to improved performance for cross-trained teams under high task demands in this experiment. As expected, teams that underwent cross-training reported higher levels of workload compared to control teams. This finding aligns with speculation that experiential crosstraining is an effortful and demanding training approach. However, the fact that the workload for cross-trained teams did not diminish with prolonged training is interesting, since it implies that there was no mitigation of this effect with continued practice. With respect to this observation, it is important to note that cross-trained teams were placed in each of the 5 team positions only once. Thus, given a second round of cross-training (equating to a second week of training in this context), participants might gain more familiarity and comfort with each of the team positions, resulting in a decrease in perceived workload. Further research will be needed to examine this conjecture. Overall, the current and previous research examining cross-training suggests a continuum of returns associated with this training approach. Factors that must be balanced by trainers include time necessary or available for cross-training, the degree of required IPK, and the degree of required team inter-positional adaptability and role redundancy. As Marks et

1585

al. (2002) noted, more complex forms of cross-training may lead to greater IPK, but lower levels of IPK may be sufficient to promote team coordination and positively influence performance. Depending on the training context, crosstraining (of any type) may or may not be worth the “cost” of potential additional training to result in improved team performance. ACKNOWLEDGEMENTS The authors would like to thank Brent Miller, Allen Dukes, and James Hyson for their role in developing the ABM task and data management systems. REFERENCES Blickensderfer, E., Cannon-Bowers, J.A., & Salas, E. (1998). Cross-training and team performance. In J.A. Cannon-Bowers & E. Salas (Eds.), Making decisions under stress: Implication for individual and team training (pp. 299-311), American Psychological Association: Washington, DC. Cannon-Bowers, J.A., Salas, E., Blickensderfer, E., & Bowers, C.A. (1998). The impact of cross-training and workload on team functioning: a replication and extension of initial findings. Human Factors, 40, 92101. Grieve, A.P., & Ag, C.-G. (1984). Tests of sphericity of normal distributions and the analysis of repeated measures designs. Psychometrika, 49, 257267. Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (task load index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human Mental Workload (pp. 139183). Amsterdam, The Netherlands: Elsevier Science/North Holland. MacMillan, J., Paley, M.J., Entin, E.B., & Entin, E.E. (2005). Questionnaires for distributed assessment of team mutual awareness. In N.Stanton, A. Hedge, K. Brookhuis, E. Salas, & H. Hendrick (Eds.), Handbook of human factors and ergonomics methods (pp. 484-494). Boca Raton, FL: CRC Press. Marks, M.A., Sabella, M.J., Burke, C.S., & Zaccaro, S.J. (2002). The impact of cross-training on team effectiveness. Journal of Applied Psychology, 87, 3-13. McCann, C., Baranski, J.V., Thompson, M.M., & Pigeau, R.A. (2000). On the utility of experiential cross-training for decision-making under time stress. Ergonomics, 43, 1095-1110. Salas, E., Nichols, D.R., & Driskell, J.E. (2007). Testing three team training strategies in intact teams: A meta-analysis. Small Group Research, 38, 471-488. Strang, A.J., Funke, G.J., Russell, S.M., Miller, B.T., & Knott, B.A. (2011). Collaboration technologies decrease reliance on radio communication in simulated air battle management. Proceedings for the 16th International Symposium on Aviation Psychology, Dayton, OH, 191196. Strang, A.J., Knott, B.A., Funke, G.J., Russell, S.M., Miller, B.T., Dukes, A.W., Hyson, J., Courtice, A.M., Brown, R., Lyons, J., & Bolia, R.S. (2011). Collaboration technologies improve performance and communication in air battle management. Military Psychology, 23, 390-409. Volpe, C.E., Cannon-Bowers, J.A., Salas, E., & Spector, P.E. (1996). The impact of cross-training on team functioning: An empirical investigation. Human Factors, 38, 87-100.