development of a questionnaire to measure critical

0 downloads 0 Views 238KB Size Report
Moreover, recommendations for actions can be derived to facilitate the ... G. Schuh, A. Bräkling, A. C. Valdez, A.-K. Schaar, and M. Ziefle, “Using Liferay as an.
QUANTIFYING INTERDISCIPLINARITY IN COMPLEX RESEARCH NETWORKS – DEVELOPMENT OF A QUESTIONNAIRE TO MEASURE CRITICAL INCIDENTS Sarah L. Müller1, Sebastian Stiehm1, Sabina Jeschke1, Anja Richert1 1

Institute of Information Management in Mechanical Engineering (IMA), Center for Learning and Knowledge Management (ZLW), Associated Institute for Management Cybernetics e.V. (IfU), RWTH Aachen University, (GERMANY)

Abstract Through the last years, interdisciplinary research has increased in prevalence and it is expected that this trend will continue. As it becomes more and more evident that interdisciplinary science has a positive impact on knowledge production and innovation, it is of special importance to analyse how the interdisciplinary processes can be further supported. The current scientific discourse on cooperation includes only a few studies which incorporate the stakeholders’ perspective. To close this research gap, Jooß [1] used a qualitative method mix to establish a dataset of 30 critical incidents (CI) for interdisciplinary cooperation in the German DFG Cluster of Excellence “Integrative production Technology for High-Wage Countries”. These CIs are clustered into three patterns: 1.) integration and allocation of time, 2.) integrated knowledge management as regards the common interdisciplinary vision as well as 3.) recursive and process-related support and participation of stakeholders. Based on this exploratory groundwork, the present research project aims at developing a standardized measurement for examining the given CIs in a quantitative way. In a first step, the CIs have been operationalized as a questionnaire. To find a valid set of items and a selected initial factor structure, an exploratory factor analysis has been accomplished with the collected data of 59 interdisciplinary employees of the Cluster of Excellence. After systematic reduction of items, six factors have been extracted and were named environmental factors for networking, internal networking, interdisciplinary cooperation and tolerance, knowledge transfer and cooperation, dissemination of research and personal development and awareness about goals. This initial questionnaire – as a first attempt to measure critical incidents of interdisciplinarity quantitatively – enables a cost-effective evaluation and facilitates the support of cooperation. Keywords: Interdisciplinary Development.

1

Cooperation,

Critical

Incidents,

Quantification,

Questionnaire

INTRODUCTION

Through a continuously growing knowledgebase in traditional research fields, a disciplinary division of labour and specialization is inevitable [2]. The ability of single researchers to synthesize deep and vast knowledge of different disciplines is limited. Hence, interdisciplinary teams increase in prevalence to solve the increasingly complex problems with which our society is being confronted [3] and to create a high level of innovation which only can be attained at the interfaces of traditional subject boundaries [4]. Germany as a leading country in production faces the challenge to ensure the production and to develop new production strategies, precisely in the context of the dynamic framework of a world affected by globalization. Approaches for the implementation of new technologies, especially in respect of individualization, virtualization, hybridization and self-optimizing production have to be developed. Such a holistic approach can only be developed considering various disciplines, wherefore the RWTH Aachen University bundles 25 institutes of several areas such as production, material science, economics, social and natural sciences and cooperates with companies of various branches [5] within the Cluster of Excellence “Integrative Production Technology for High-Wage countries” (CoE). As part of the nationwide excellence initiative of the federal government of Germany, the CoE has been initiated in 2006 to develop sustainable solutions for production in the future. Although there is no generally approved definition of a cluster of excellence [6] it can be defined as (locally) bundled research activities of a group of actors with the aim at gaining a competitive advantage [7]. In general,

a cluster is characterized by high density of relational structures and therefore connected with network adequate processes [8]. An interdisciplinary cluster faces some challenges which have to be overcome to ensure knowledge production and innovation. On the one hand, content-related challenges occur due to the actors’ different disciplinary and biographical backgrounds such as different theoretical background, thinking, terminologies, and (scientific) methods [9]. On the other hand, organizational challenges such as the integration of heterogeneous organizational cultures of disciplines and institutes, different hierarchical levels in a cluster, the range of age, the surplus of men and a general high fluctuation in scientific research need to be mastered [5]. To foster the interdisciplinary cooperation purposeful and efficient, it is important to get a deeper knowledge of the interdisciplinary processes to find the right adjustment levers. The current scientific discourse on cooperation focuses mainly on bibliometric or citation data [10], only a few studies which incorporate the stakeholders’ perspective. To close this research gap, Jooß [1] used a qualitative method mix to establish a dataset of 30 critical incidents (CI) for interdisciplinary cooperation in the CoE within a timespan of five years (see Tab. 1).

Tab 1. Critical Incidents of Interdisciplinary and resulting patterns [1]. Interviews

Employee Surveys

CI-1: Exchange and terminology

CI-8: Development of a common language

CI-2: Co-creation of the involved researchers

CI-9: Reflection of the research findings

CI-3: Exchange of methods CI-4: Key persons and networking CI-5: Interdisciplinary qualification program CI-6: Visualization of the vision CI-7: Opportunity for continuous exchange

Evaluations CI-15: Exchange and networking

Observations CI-18: Iteration and participation

CI-16: Sensitization and CI-19: Key persons for tolerance towards other networking disciplines CI-10: Identification and CI-20: Identification and CI-17: Reflection and visualization of incentives interfaces terminologies CI-21: Virtual CI-11: Exchange and temporal freedom CI-12: Interdisciplinary competences CI-13: Interdisciplinary integration CI-14: Communication and cooperation platform

communication CI-22: Handling of staff turnover CI-23: Integration of external persons CI-24: Research contribution CI-25: Temporal freedom CI-26: Measures for further training CI-27: Disciplinary advances in knowledge CI-28: Personal benefit CI-29: Public relations CI-30: Publications

Resulting Patterns

Pattern 1: Integration and allocation of time

Pattern 2: Integrated knowledge management as regards the common interdisciplinary vision

Pattern 3: Recursive and Process-related support and participation of stakeholders

After theoretical saturation, these CIs have been clustered into three patterns [1]. 1. The first pattern “Integration and allocation of time” describes that constant communication between the actors is crucial for a successful cooperation regarding the interdisciplinary integration of knowledge and methods. The researchers have to develop an awareness for each other’s competences and to develop common language or rather common concepts. Furthermore, it is necessary to work out the interfaces and limits of the involved disciplines to cooperate successfully and to learn from each other. This complex negotiation process needs time and organizational support. For this purpose, a sufficient organizationally established allocation of time for constant exchange is needed and should be provided. Through spatial proximity, this process can be further enhanced. 2. The second pattern “Integrated knowledge management as regards the common interdisciplinary vision” points out that a common vision and a continuous knowledge management with regard to the realization of the vision is necessary for successful cooperation. A common vision is the base for the identification with and the motivation for interdisciplinary cooperation. The vision needs to be established and the researchers have to be aware of it. In terms of the vision, a continuous support of the knowledge exchange should take place. Therefore, regular networking activities should be fostered and key persons for networking should be initiated. 3. The third pattern summarizes CIs in the fields of “Recursive and Process-related support and participation of stakeholders”. The participation of the involved actors and a recursive and process-accompanying support are of special importance which means that all actors of the CoE should be involved in building the vision and the scientific contents of the cluster. Furthermore, a continuous support of the interdisciplinary, communicative and team competences of the researchers is needed and should be supported through adequate trainings. Based on the exploratory qualitative work of Jooß [1], a questionnaire has been developed to measure the CIs of interdisciplinary cooperation on a quantitative level. The quantitative measurement enables an economical evaluation, allows a comparison of different clusters and makes a conduction of further statistical analysis possible.

2

METHOD

Fifty-nine subjects (seven women, 49 men, three without gender details) have been polled with a range of age from 25 to 46 years (M = 29.92 years, SD = 3.72). The data collection has taken place online during an event of the cluster. Therefore, all hierarchical levels of the CoE, except the professors, have been involved (project leaders, management and researchers) and all projects have been represented. The sample consists of interdisciplinary participants in the fields of engineering, informatics, communication sciences, psychology, and cultural studies. Hence, it can be regarded as a representative sample of the CoE (which has around 80 members in total). The questionnaire has been conducted via SoSci Survey. At the beginning of the questionnaire, an instruction has explained the background of the testing and mentioned the confidentiality of the data. Subsequently, demographic data has been collected, followed by the main exploratory questions regarding the CIs. A deductive approach was utilized for generating items, and each CI was operationalized in a consistent way: To inquire the perceived quantitative importance of each CI, participants had to rate its significance on a 5-point Likert scale (1 = very important, 5 = very unimportant). If possible, CIs have been taken over almost verbatim. If they addressed more than a single issue, they were split, e.g. CI23 “Integration of external persons” was broken into “Integration of external scientists in the research process” and “Integration of industry and practice partners in the research process”. A second part has gathered data about the subjective quantitative characteristics of the CIs regarding the CoE, an example for CI-1 is “Within the entire CoE, a common understanding of terminologies was developed”. For the initial analysis, which is reported in this paper, this second part has not been considered.

3

RESULTS

In order to obtain an initial factor structure, the principal component analysis has been used for factor extraction applied with Varimax rotation. In this way, a single structure can be achieved, which facilitates the interpretation of content. The Kaiser-Meyer-Olkin criterion is KMO = .69, confirming that the data is suitable for factor analysis. According to Kaiser [11], this value is middling. Bartlett test of sphericity χ2 (190) = 459.912, p 0.5), a six-factor model has been developed. All of their eigenvalues are greater than one and thus meet the Kaiser criterion. Tab. 2 depicts all initial factors and their eigenvalues and highlights the final six factors.

Tab 2. Eigenvalues of all initial factors, the final six factors are highlighted. Factor 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Total Sum 5.936 1.882 1.746 1.465 1.401 1.067 .941 .807 .658 .639 .513 .433 .386 .292 .268 .210 .168 .110 .078

Initial Eigenvalues % of Variance 31.244 9.907 9.191 7.713 7.376 5.614 4.951 4.247 3.462 3.361 2.698 2.277 2.030 1.536 1.411 1.108 .883 .580 .411

Accumulated % 31.244 41.151 50.343 58.056 65.497 71.045 75.996 80.243 83.705 87.066 89.764 92.041 94.070 95.606 97.017 98.125 99.009 99.589 100

The communalities of variables range from .46 to .85, showing that 46% to 85% of the variance of the variables can be attributed to the factors. The factors explain 71.05% of the total variance. Factor 1 (eigenvalue = 5.93) is able to explain 31.24% of the variance, factor 2 (eigenvalue = 1.88) 9.91%, factor 3 (eigenvalue = 1.75) 9.19%, factor 4 (eigenvalue = 1.47) 7.71%, factor of 5 (eigenvalue = 1.40) 7.38% and factor 6 (eigenvalue = 1.07) explains 5.61% of the variance. In Tab. 3 the rotated component matrix is depicted. The primary factor correlations range from .53 to .89 for factor 1, from .60 to .80 for factor 2, from .59 to .83 for factor 3, from .73 to .79 for factor 4, from .77 to .84 for the fifth factor, and from .68 to .85 for the sixth factor. All items have only one high factor loading, on all other factors their loadings were below .5. According to Guadagnoli and Velicer [13] a factor may be interpreted when at least one variable has a minimum factor loading of .6. The mathematical reduction of factors shows that six factors are sufficient for explaining more than 70% of the total variance. With regard of the items, factor names have been formulated:      

The first factor is named “Awareness about Goals”, the second “Dissemination of Research and Personal Development”, the third one “Knowledge Transfer and Cooperation”, the name of the fourth factor is “Interdisciplinary Cooperation and Tolerance”, that of the fifth is “Internal Networking” and the sixth factor was named “Environmental Factors for Networking”.

Tab 3. Rotated component matrix as result of the principal component analysis. Item

Factor 1

Identification and Display/Depiction of Interfaces Sufficient time for exchange Existence of exchange-networks Key person who promotes networking/interconnectedness Interdisciplinary comprehensible communication of facts Periodic reflection of experiences with interdisciplinary cooperation Tolerance for other disciplines Participation of all employees in relevant decision processes Continuous cooperation in scientific routine Knowledge transfer in case of departure of employees Management of employee turnover Personal enlargement of scientific network Needs-oriented training offering for improvement of interdisciplinary cooperation Sustaining utilization of research results for society and economy Sustaining utilization of research results for science Clarification of each employee’s contribution to the goals of the whole excellence cluster Willingness and motivation to work interdisciplinary Awareness about one’s own contribution to the overall goal Awareness about one’s project’s contribution to the overall goal

4

2

3

4

5

6 .676 .853

.766 .839 .728 .793 .793 .588 .634 .710 .826 .602 .677 .749 .803 .532 .594 .847 .893

DISCUSSION AND OUTLOOK

The number of factors is smaller than the original number of CIs of the qualitative study. This was to be expected because the original CIs show content-related overlaps and similarities as the described patterns already displayed. Therefore, the aggregated CIs are more selective than the previous 30 CIs. To verify the factor structure, the model should be tested via a confirmatory factor analysis as a next step as recommended by Gerbing and Hamilton [14]. The six factors match the three described patterns, 

where factors 4 and 6 correspond the first pattern “Integration and allocation of time”,



factors 1 and 2 equal the second pattern “Integrated knowledge management as regards the common interdisciplinary vision”



and factors 3 and 5 fit the third pattern “Recursive and process-related support and participation of stakeholders”.

This supports the assumption of how important it is to actively support the cooperation process by providing time and assistance through cluster-internal events. Within the CoE, for example a biannual, full-day so-called colloquium of employees is conducted to actively support the networking and knowledge exchange [15]. For an efficient research process, the factors point out the importance of a proper knowledge management which starts with a definition of common aims. Within the CoE, a collaboration platform for example is used to support the digital exchange and storage [16].

Furthermore, within an interdisciplinary environment a participative management is appropriate, so that the teams can control and organize themselves to a higher degree than in classic teams. For this purpose, the management of the CoE uses for example formative evaluations of events and the employee survey to get a direct feedback from the actors [17]. The quantification of the CIs allows the analysis of interdependencies between the CIs, which will be tested in a follow-up study using structural equation modelling. Structure equation models allow the estimation and testing of correlative relationships between latent variables. Furthermore, interactions with performance measures, like the cluster-internal annual employee survey, can be analyzed to identify critical control levers for the cluster management. This initial questionnaire is a first attempt to measure CIs of interdisciplinarity quantitatively. Beside the practical aspect that it makes an economical evaluation possible, it allows further empirical research as the analysis of interdependencies. Moreover, recommendations for actions can be derived to facilitate the interdisciplinary cooperation due to the empirical conclusions or through benchmarking with other research networks. Nevertheless, through the specific and interdisciplinary structure of the CoE, it is questioned whether these findings are generalizable. To examine this, the questionnaire has to be tested in other research networks.

ACKNOWLEDGEMENTS The authors would like to thank the German Research Foundation DFG for the kind support within the Cluster of Excellence “Integrative Production Technology for High-Wage Countries” at RWTH Aachen University.

REFERENCES [1]

C. Jooß, Gestaltung von Kooperationsprozessen interdisziplinärer Forschungsnetzwerke. Dissertation, 1st ed. Aachen: Books on Demand, 2014.

[2]

V. Fuest, "Alle reden von Interdisziplinarität aber keiner tut es." - Anspruch und Wirklichkeit interdisziplinären Arbeitens in Umweltforschungsprojekten. [Online] Available: http://www.heidelberger-lese-zeiten-verlag.de/archiv/online-archiv/fuestneu.pdf.

[3]

A. Kieser, Wissenschaft und Beratung: Vorgetragen am 14. Juli 2001. Heidelberg: Universitatsverlag C. Winter, 2002.

[4]

C. Schmickl and A. Kieser, “How much do specialists have to learn from each other when they jointly develop radical product innovations?,” Research Policy, vol. 37, no. 3, pp. 473–491, 2008.

[5]

C. Brecher, Integrative production technology for high-wage countries. Berlin, New York: Springer, 2012.

[6]

B. Alecke and G. Untiedt, Zur Förderung von Clustern: "Heilsbringer" oder "Wolf im Schafspelz"? Available: http://doku.iab.de/veranstaltungen/2005/gfr_2005_alecke_untiedt.pdf.

[7]

R. Vossen, Ein Verfahren zur Ermittlung und Bewertung des intellektuellen Kapitals von wissenschaftlichen Exzellenzclustern, 1st ed. Norderstedt: Books on Demand, 2012.

[8]

R. Häussling, Grenzen von Netzwerken, 1st ed. Wiesbaden: VS Verlag für Sozialwissenschaften / GWV Fachverlage GmbH, Wiesbaden, 2009.

[9]

J. Heilbron, “Das Regime der Disziplinen: Zu einer historischen Soziologie disziplinärer Wissenschaft,” in Interdisziplinarität als Lernprozess: Erfahrungen mit einem handlungstheoretischen Forschungsprogramm, H. Joas and H. G. Kippenberg, Eds., Göttingen: Wallstein, 2005, pp. 23–45.

[10]

F. J. van Rijnsoever and L. K. Hessels, “Factors associated with disciplinary and interdisciplinary research collaboration,” Research Policy, vol. 40, no. 3, pp. 463–472, 2011.

[11]

H. F. Kaiser, “An index of factorial simplicity,” Psychometrika, vol. 39, no. 1, pp. 31–36, 1974.

[12]

M. S. Bartlett, “Tests of significance in factor analysis,” British Journal of statistical psychology, vol. 3, no. 2, pp. 77–85, 1950.

[13]

E. Guadagnoli and W. F. Velicer, “Relation to sample size to the stability of component patterns,” Psychological Bulletin, vol. 103, no. 2, pp. 265–275, 1988.

[14]

D. W. Gerbing and J. G. Hamilton, “Viability of exploratory factor analysis as a precursor to confirmatory factor analysis,” Structural Equation Modeling: A Multidisciplinary Journal, vol. 3, no. 1, pp. 62–72, 1996.

[15]

S. L. Müller et al., “Managing interdisciplinary research clusters,” pp. 606–610.

[16]

G. Schuh, A. Bräkling, A. C. Valdez, A.-K. Schaar, and M. Ziefle, “Using Liferay as an Interdisciplinary Scientific Collaboration Portal,” vol. 9742, pp. 405–414.

[17]

S. L. Müller, T. Thiele, C. Jooß, A. Richert, and S. Jeschke, “Continuous Formative Evaluation,” in Integrative Production Technology: Theory and Applications, C. Brecher and D. Özdemir, Eds.: Springer Verlag, 2017.