Assessing ambiguity tolerance in staffing software development teams by analyzing cognitive maps of engineers and technical managers Corrado lo Storto University of Naples Federico II, DIEG (
[email protected]) ABSTRACT This paper presents a methodological framework to explore the cognitive processes implemented by members of a software development team to manage ambiguous situations at the stage of product requirements definition. Fuzzy Cognitive Maps are used in the framework to elicit cognitive schemes and develop a measure of individual ambiguity tolerance. Keywords: ambiguity, software development, cognitive maps, team staffing 1. INTRODUCTION Ambiguity management is a major concern in the development of a software product. Compared to other products, software development needs a great conceptual work and effort by people who have to work almost in an artisanal way. Moreover, many software applications are designed to execute automatically manual tasks not yet fully understood in detail. Consequently, when the development of software proceeds, new requirements and characteristics are added or substituted to the old ones, and “in a sense, the creation of software requirements is like hiking in a fog that is gradually lifting. At first only the immediate surroundings within a few feet of the path are visible, but as the fog lifts, more and more of the terrain can be seen” [1, p. 47]. In more than 90% of software development the requirements are modified during development, and the analysis and definition of specifications is one of the more critical tasks of software development. Indeed, reference [2] underlines how the most difficult thing in software development is to decide precisely what to do. No other phase or activity can have such a relevant impact on the success of the final product if carried over so inefficiently. Furthermore, for no other phase it is so difficult to remedy if errors were done. Two are the reasons why specification definition has a great impact on product development [3]. Firstly, requirements that originate specifications are vague and not precise in their nature. On the contrary, software development requires an explicit formalization of semantics. Hence, the need for software developers of filling the gap existing between the not precise requirements and formal specification methods. Second, different requirements are often conflicting and many of the conflicts are not easy to identify and resolve. Further, practices followed in the formalization of a business contract with a customer for software development are frequently ambiguous as to the determination of the size of what has to be delivered, as to the development plans, and other quantitative issues. Very frequently, it happens that business agreements
and/or contracts consider only more general issues of a deal. Removing ambiguity or effectively managing ambiguous contexts when requirements and specifications are developed and communicated is thus a major concern of most software customers, managers, and requirements engineers. Ambiguity management inside software development teams and organizations is fundamentally related to how people share knowledge and build an organizational collective memory [4] [5]. Indeed, complex software development projects tend to require thoughtful cooperation and careful interaction with numerous individuals. By using empirical data processed through computer simulation, this paper presents a methodological framework to explore the cognitive processes implemented by members of a software development team in order to manage ambiguity situations generated at the stage of product requirements definition. The study is primarily focused on the following issues: a) the investigation of the cognitive schemes (or frames) adopted by the individuals to manage ambiguous situations, and b) the evaluation of the degree of tolerance for ambiguity of team members. Fuzzy Cognitive Maps (FCMs) are used in the framework to elicit cognitive schemes and develop a measure of ambiguity tolerance during technical problem-solving occurred in the stage of requirements analysis. The methodological framework was implemented using as a case study a software development team of 7 individuals: 1 project manager, 2 analysts, 1 analystprogrammer, 3 programmers. The team was staffed within a medium sized software company with the aim to deliver a customized software product (the rejuvenation of a software product used by a large national telecommunication service company for managing billing procedures).
2. AMBIGUITY, KNOWLEDGE AND ORGANIZATIONS Ambiguity appears in the organizations when knowledge becomes more tacit or less articulated [6] [7] [8]. If knowledge cannot be articulated, technical problems can be defined only vaguely, and engineers and technicians cannot easily address problem-solving. A decreasing level of articulability of knowledge is associated with a growing difficulty of search, transfer of requested information, and contextualization of knowledge [9]. The problem of ambiguity dissolution can therefore be modelled as a shift from an ill-structured problem, that is to say a problem in which one of the state characterizing its life cycle is unknown, to a well-structured problem, that is to say a problem to which an action scheme is associated. When the organization is able to build an action scheme to deal with a problem then it has “solved” the problem of ambiguity. That does not mean it has eliminated ambiguity, but that the organization processes can live together with (and tolerate) residual ambiguity. Natural language, communication and more generally social interaction have a fundamental role in the process of knowledge creation, and - as organizing tools - they enable individuals to deal with conflicts, coordinate divergent activities, balance and share contrasting objectives, and manage effectively organizational paradoxes, and as a consequence, effectively manage ambiguity during technical problem-solving. 3. THE FUZZY COGNITIVE MAPS Fuzzy cognitive maps are a useful tool to elicit individual and organizational knowledge linking causal events, values, goal in a complex fuzzy feedback dynamic system [10] [11] [12]. The concept and method of cognitive maps as a tool of analysis was introduced the first time by reference [13]. The cognitive maps, through the graphic representation of events-cognitive states (nodes) and events-causes that determine the cognitive states (linkages between nodes) make it possible to model in an intuitive and natural way the human thinking [11]. In the traditional cognitive maps the information structure is extremely simplified. Indeed, the variables associated to each event-cognitive states can take the two values of the scale [0,1], respectively 0 when the variable is not activated (non-existence of the event), and 1 when the variable is activated (existence of the event). A similar limitation exists for the variables associated as weights to the cause-effect linkages between two cognitive states. These variables can take one of the three values of the scale [-1, 0, 1]. Particularly, the variables take the value -1 when there is a feedback effect, 0 when there is a lack of effect, and 1 when the effect exists. Summing the vector of linkage weights directed into a node-event for each event of the map it is possible to
determine through a course of iteration the dynamic evolution of the map, that is to say the new value taken by each variable associated to each event. But, the adoption of such a cognitive map has an intrinsic contradiction. If, on the one hand, it has the goal to model the human thinking, on the other hand, it neglects an aspect that strongly characterizes the human thinking, that is to say the inaccuracy, and, more exactly, the ambiguity underlying the definition of the logical categories that individuals commonly use to communicate. Therefore, the impossibility to take into account the ambiguity of situations is a limit of the adoption of conventional cognitive maps. In order to overcome this limit, reference [10] introduced the concept of fuzzy cognitive map. The geometric model of a fuzzy cognitive map is similar to a traditional cognitive map. But, the information structure is much richer. Indeed, the variables associated to each event can take all the values of the continuous scale {0, 1}. In the same way, the weights associated to each cause-effect linkage between two events can take each value of the scale {-1, 1}, or, sometimes, can vary in the scale {-∞, +∞}. Let us suppose to have a fuzzy cognitive map in which, at the time tk, the variables associated to the eventscognitive states take the values Ci=Ci(tk), for i=1, n. The state of the map at time tk remains thus defined by the vector c=(C1, C2, ..., Cn). At each time tk+1 the value of the variable associated to each state is assumed to be either constant or modified to take into account the influence of other events-cognitive states. The events-cognitive states are connected through the cause-events eji(tk), where j is the index of the event-cognitive state, i is the index of the induced cognitive state, and eji is the weight of the causeeffect event at tk. The new value of the variable associated to the event-cognitive state at time tk+1 is obtained summing the vector Cj=Cj(tk) modified by the squashing function S ⎛ n ⎞ Ci ( tk +1 ) = S ⎜ ∑ e ji ( tk ) C j ( tk ) ⎟ (1) ⎝ j =1 ⎠ 4. METHODOLOGY A software development team was considered to develop the field analysis. The team was staffed in a mediumsized software company located in the South of Italy to carry on a specific project, i.e. the maintenance of a customized software product used by a large national telecommunication service company to implement the billing process. The team includes: 1 project manager, 2 analysts, 1 analyst-programmer, 3 programmers. Figure 1 reports the model adopted in the construction of the software development team members cognitive maps. When at time tk the team member perceives a certain
amount of ambiguity generated by the source ASi he adopts a set of organizational mechanisms and behaviours OMj to alleviate his cognitive discomfort. At time tk+1 he will assess if the organizational solution implemented was effective to reduce ambiguity allowing him to proceed with his task (event eji). If the amount of ambiguity he perceives at time tk+1 is still too high, he will activate again some organizational mechanisms (OMj) to lower ambiguity. This process will come to an end when the individual perceives an amount of ambiguity that he believes is not an obstacle to perform his task. Thus, as a consequence, when the amount of perceived ambiguity fits the effectiveness of the organizational mechanism selected to reduce ambiguity the process will converge. high
ASi(t1) OMj(t1)
organizational mechanisms and behaviours implemented to alleviate the amount of ambiguity (organizational mechanisms). To reduce bias, information was interpreted and codified through content analysis techniques [14] [15] [16]. This step was particularly critical, as either the aggregation in the same category or the separation in different categories of concepts remain a subjective choice of the text analysts. A new session of interviews (IS2) was administered to members of the software development team who were separately invited to approve both lists of categories providing a solid argumentation for their validation or rejection. In this way, categories were better refined both removing overlapping and redundant information, and standardizing language. Finally, 17 categories were identified as sources of ambiguity (ASi) and 17 categories as organizational mechanisms (OMj) aimed at reducing ambiguity during requirements analysis (see Table 1). Table 1 Sources of ambiguity and organizational mechanisms implemented to alleviate ambiguity
ASi(t2) OMj(t2) ASi(tn) OMj(tn) low
Figure 1 The model adopted to construct cognitive maps The generation of the cognitive maps followed three steps: Step 1) The collection of rough information The collection of rough data included a number of stages. Indeed, shared arguments, terms and action domains (a collective memory) had to be elicited to compare cognitive maps belonging to different team members. Henceforth, a first session of interviews (IS1) was administered to team members to collect information useful to identify a set of shared concepts that could be used in the next interview sessions (IS2 and IS3) as anchor-concepts (categories) and cause-effect relationships (linkages) between categories. Interviews were administered avoiding, however, to give personal judgments, keeping the conversation neutral to not influence answers with any preconceived ideas of the interviewer. A tape-recorder was used during interviews and conversations were fully written out to analyze text. Step 2) The codification of categories In the next step, two lists of categories were preliminarily assembled, the first related to concepts associated to situations identified as generators of ambiguity (sources of ambiguity), and the second associated to
sources of ambiguity
organizational mechanisms implementing organizational mechanisms and behaviours well proved
AS1
limited or no familiarity with technical problems
OM1
AS2
project typology
OM2
developing missing documents
AS3
size of the development project
OM3
suggesting alternative solutions to the customer
AS4
conflicting requirements
OM4
defining the requirements together with the customer
AS5
missing or unclear functional requirements
OM5
managing the project without leveraging on previous experience
AS6
missing or unclear procedural requirements
OM6
adopting an unusual approach to manage the project
AS7
missing or unclear performance requirements
OM7
using automatic support tools
AS8
missing or unclear intermediate outputs
OM8
suggesting how/what to do to the customer
AS9
lack of useful documents
OM9
doing reverse engineering of existing software applications
AS10
scarce collaboration with the customer
OM10
examining the documents at the customer site
AS11
scarce collaboration with other suppliers
OM11
meeting the customer engineers and technical managers
AS12
lack of information
OM12
meeting engineers and technical managers of other suppliers
AS13
low quality of available information OM13
AS14
redundant information
OM14
implementing continuous change that had an impact on the performed activities
AS15
lack of vision or control of the company relative to project
OM15
developing and using standardized modules
AS16
lack of consistency between project estimated cost and target performance
OM16
optimizing the single activities without reviewing the overall project
AS17
frequent and unexpected change of target goals usually not communicated soon to the team
OM17
defining when necessary the intermediate output at project advancement
organizing internal meeting
Step 3) The cause-effect linkages generation In the last step, a new interview session (IS3) was administered and each member of the software development team was requested to rank categories associated to the ambiguity sources, adopting as a
criterion the amount of perceived ambiguity. A five-level ordinal scale was used. Every team member was also individually requested: a) to link categories identifying organizational mechanisms that he implemented in order to alleviate ambiguity to the specific source that generated ambiguity during requirement analysis for the development project under examination. The number of cause-effect linkages between sources of ambiguity and organizational mechanisms was variable, depending on the team member’s attitude and preference; b) to assess how effective was the organizational mechanisms to lower the amount of ambiguity on a five-level scale. Finally, for each member the following set of information was available: a) one map associating sources of ambiguity (ASi) and corresponding organizational mechanisms (OMi) adopted to alleviate ambiguity; b) the perceived effectiveness of the ASi←OMi linkages; c) the rank of sources of ambiguity. This set of information was used as an input to simulate the dynamic behaviour of team members cognitive maps, using the Fuzzy Systems Engineering software to process data. A unipolar logistic function was used as a squashing function 1 S ( Ci ) = (2) -( g ( Ci + B ) ) 1+ e where g (>1) is the gain, Ci is the input, and B is the bias added component. The input Ci is the result of the vectorial summation and it can take all the real values. The output S=S(Ci) is the value of the new activation state and varies in the range {0, 1}. The selected functional shape for S(x) is S ( x ) = arctg ( x, B, g ) (3)
Modifying both the g and B values, two squashing functions were built in order to simulate two different attitudes of individuals in terms of a higher or lower tolerance for ambiguity situations. In particular, the following values were used: g=5 and B= -175 for high tolerance and g=20 and B= -65 for low tolerance. Using as an input the FCM matrix data, the cognitive process of every team member relative to ambiguity management was simulated through computer simulation. Two sets of data were obtained for each member of the team, the first for high tolerance and the second for low tolerance. 5. RESULTS
For the sake of brevity detailed findings from the static analysis of data are not included in this paper. The 3-step data collection and analysis aimed at the identification of the team members’cognitive maps made it possible to elicit the team organizational memory related to the management of ambiguity. As to the sources of ambiguity (ASi) during requirement analysis, only 4 main sources were indicated by most team members: lack of
useful documents, lack of information, project typology and size of the software development project. Similar findings were found for the organizational mechanisms and behaviours adopted to alleviate ambiguity (OMj).
Figure 2 FCM simulation for A2 (high tolerance)
Figure 3 FCM simulation for A2 (low tolerance) The shared organizational memory related to the management of ambiguity during requirement analysis is thus apparently limited. It was also found that the role that the member has within the team and the task to be executed affects the perception he develops about ambiguous situations. When the ASi←OMj linkages are examined in cognitive maps, the association between role and cognitive map typology becomes even more evident. Moving downward along the organizational chain of the software development process (project manager, analyst, programmer), the cognitive map configuration is generally more simplified (with lower density, extension, and ramification). However, one of the programmers of the team (codified as Programmer 3, P3) was an exception to this common behaviour; on the contrary,
strong similarities clearly emerged for the cognitive maps of the other two programmers (Programmer 1, P1 and Programmer 2, P2).
The cognitive map of Programmer 3 was extremely wide and ramified. As a consequence, the role may not be the main variable that affects how team members think and behave in ambiguous situations. The dynamic analysis of team members fuzzy cognitive maps through computer simulation showed that Programmer 3 was less tolerant for ambiguity. Figures from 2 to 7 plot simulation results for the Project Manager (PM), Analyst 2 (A2) and Programmer 3 (P3). For the sake of brevity, results for 3 team members only are reported. With the exception of P3, simulation results are similar both for high and low ambiguity tolerance. For the PM and the A2, after 4 or 5 iterations the process converges and variables take values close to zero. Vice versa, the dynamic behaviour of the P3 fuzzy cognitive map is different when moving from high to low ambiguity tolerance. Indeed, when it is assumed a low tolerance ambiguity threshold for programmer P3, his cognitive process does not converge, and as it appears from Fig. 7, the perceived ambiguity generated by some sources increases with the number of iterations, with a substantial lack of fit between a low tolerance attitude for ambiguity and the structure of the cognitive map of P3.
Figure 4 FCM simulation for the PM (high tolerance)
Figure 6 FCM simulation for P3 (high tolerance)
Figure 5 FCM simulation for the PM (low tolerance)
Figure 7 FCM simulation for P3 (low tolerance) 6. CONCLUSION
This study, even though exploratory in its nature, provided insights to investigate the role of ambiguity in the individual cognitive processes and its impact on the organizational processes with particular reference to software development and team staffing. There is no doubt that findings presented here suggest indications that remain still too vague and the picture that emerge is still excessively fragmented. It has the merit to have showed how both the fuzzy cognitive mapping tool and the concept of organizational memory can be used together to study the cognitive processes and elicit knowledge useful to manage ambiguous situations more efficiently. Particularly, it has suggested a methodological framework and the usage of the squashing function S as a tool for measuring an individual tolerance for ambiguity, and, as a final analysis, the fit between his mental model to manage ambiguity and his tolerance for ambiguity. REFERENCES [1] Jones, C. “Conflict and Litigation Between Software Clients and Developers”, IEEE Engineering Management Review, Fall, pp. 46-54, 1998. [2] Brooks, F.P. “No Silver Bullet: Essence and Accidents of Software Engineering”, Computer, Vol. 15, No. 1, pp. 1018, 1987. [3] Yen, J., Liu X., and Teh, S.H. “A Fuzzy Logic-based Methodology for the Acquisition and Analysis of
imprecise requirements”, Concurrent Engineering: Research and Applications, Vol. 2, pp. 265-277, 1994. [4] March, J.G. “Bounded Rationality, Ambiguity and the Engineering of Choice”, The Bell Journal of Economics, Vol. 9, No. 2, Autumn, 1978. [5] March, J.G., and Olsen, J.P. “The Uncertainty of the Past: Organizational Learning Under Ambiguity”, Europen Journal of Political Research, Vol. 3, pp. 147-171, 1975. [6] Daft, R.L., and Lengel, R.H. (1986) "Organizational Information Requirements, Media Richness and Structural Design", Management Science, Vol. 32, 554-571, 1986. [7] Polanyi, M. Personal Knowledge: Toward a Post-Critical Philosophy, Chicago, Illinois: University of Chicago Press, 1958. [8] Weick, K. Sensemaking in Organizations, SAGE Publications, 1995. [9] Winter, S.G. “Knowledge and Competence as Strategic Assets”, in D.J. Teece (ed.) The Competitive Challenge. Strategies for Industrial Innovation and Renewal, Cambridge, Massachusetts: Ballinger, pp. 159-184, 1987. [10] Kosko, B. “Fuzzy Cognitive Maps”, International Journal Man-Machine Studies, Vol. 24, pp. 65-75, 1986. [11] Laukkanen, M. Comparative Cause Mapping of Management Cognitions, Helsingin Kauppakorkeakoulun Julkaisuja, D-154, 1992. [12] Taber, R. “Fuzzy Cognitive Maps Model Social Systems”, AI Expert, July, 1994. [13] Axelrod, R. Structure of Decision: the Cognitive Maps of Political Elities, Princeton, NJ: Princeton University Press, 1976. [14] Berelson, B. “Content Analysis”, in Lindzey G. (ed.), Handbook of Social Psychology: Theory and Method, Vol. 1, Addison-Wesley, Reading, MA, pp. 488-522, 1954. [15] Holsti, O.R. “Content Analysis”, in Lindzey G. and Aronson, E. (eds) The Handbook of Social Psychology: Research Methods, Vol. 2, Addison-Wesley, Reading, MA, pp. 596-692, 1968. [16] Kolbe, R.H., and Burnett, M.S. “Content Analysis Research: An Examination of Applications with directives for Improving Research Reliability and Objectivity”, Journal of Consumer Research, Vol. 18, pp. 243-250, 1991.
