A Model to Measure the Business Value of Information Technology ...

11 downloads 608702 Views 271KB Size Report
The impact of information technology (IT) on improving the productivity of information ... technologies on firm performance based on two types of outcomes: (a) .... Efficiency refers to the degree to which an organization is able to more cost- and ..... we regressed the relevant financial performance measure in year t (FPt) ...
A Model to Measure the Business Value of Information Technology: The Case of Project and Information Work Indranil Bardhan The University of Texas at Dallas [email protected] Vish Krishnan The University of Texas at Austin [email protected] Shu Lin The University of Texas at Dallas [email protected] Comments on an earlier version of this paper from Gregory Dess, V. Sambamurthy, participants at the 2004 INFORMS and DSI conferences, and seminar participants at the University of Texas at Dallas and the University of Texas at Austin are gratefully acknowledged. _________________________________________________________________________ ABSTRACT The impact of information technology (IT) on improving the productivity of information work has been mixed. While some firms have realized gains, many others have found the benefits to be elusive. Prior research has primarily focused on the impact of IT spending on firm-level performance. To further this line of research, it is necessary to understand the operational and process-level changes and isolate the impact of specific types of IT applications on organizational dynamic capabilities which mediate the impact of IT on firm performance. We measure the impact of different types of enterprise information technologies on firm performance based on two types of outcomes: (a) business process-level measures such as project/program quality, cycle times, cost, and completion rates, and (b) firm-level financial performance measures such as gross margin, return on assets, and return on equity. The central research contribution is the development of a research framework to improve our understanding of the operational impact of information work on the development of dynamic organizational capabilities which explains differences in processand firm-level performance across a cross-section of companies.

Keywords: Information work productivity, Dynamic capabilities, Effectiveness, Efficiency, Project outcomes, Firm performance.

1.0

INTRODUCTION Improving information and knowledge work productivity is the next frontier for

information technology (IT) offerings and applications. Technology has delivered significant and documented productivity improvements in the manufacturing sector of the economy (Weill, 1992; Barua et al., 1995). Information work – defined as the “creation, coordination, integration and management of information within a firm and its value network (Conner and Prahalad, 1996)” - is in its rudimentary stages of being mapped, measured, and managed. The intangibility and diversity of information work and its outputs presents unique challenges in its description, measurement, and improvement. Prior research has focused primarily on the aggregate impact of IT spending on firm performance (Barua et al., 1995; Brynjolfsson and Hitt, 1996; 1998; Kohli and Devaraj, 2003). The deeper, operational and

process-level impact of software applications are only currently being examined in detail. To develop a finer and a more nuanced understanding of the business value of IT, we focus on the improvement in project work productivity and performance. In the information economy, projects are important vehicles for firms to complete work and realize value. Tom Peters (Peters, 1999) goes as far as to state that, “In the new economy, all work is project work.” Focusing on projects or programs (i.e., collection of projects) allows us to study the impact of IT on information work in multiple firms while using a common vocabulary and drawing on a large body of knowledge on project management. Most prior studies, dealing with relationships between IT and organizational performance, evaluate the business value of IT by lumping IT-related spending under one category termed “IT capital.”

While IT capital is a useful measure of firm-level IT

investments, it needs to be complemented with additional insights into the usage and impact

1

of different types of IT, including software capabilities, at the business process level. Since IT investment decisions are often made at the application level, it is important to trace the impact of different classes of IT applications on organizational capabilities and firm performance (Mukhopadhyay et al., 1997). We focus on the role of information work, the relationships between three classes of major IT applications used by managers to plan, control, and manage projects, and their impact on organizational capabilities and performance. We draw on the theory of dynamic capabilities to explain how firms create competitive value by creating relevant organizational routines to leverage their IT assets (Zhu and Kraemer, 2002). We propose and validate a model that enhances our understanding of the impact of IT applications on project and firm performance. Our research questions can be summarized as follows: (a) How do we measure the value of information work defined in terms of organizational capabilities that are enabled by IT?, (b) what are the mechanisms by which IT has an impact on project and process-level outcomes? and (c) what is the impact of IT-enabled capabilities on firm performance? We validate our research using survey data from project and program managers and executives drawn from a large cross-section of US firms as well as financial data for a subset of the publicly traded firms that participated in the survey. Our results suggest that ITenabled dynamic capabilities have a significant impact on improvement in four project-level outcomes (cycle time, cost, quality, and on-time completion rate), and two of the four firmlevel financial measures. Our research integrates two distinct bodies of literature: one that draws on the strategic management literature, dealing with the relationships between IT usage and dynamic capabilities, and the other that draws on the IT productivity literature dealing with the impact of IT on information work.

2

2.0

BACKGROUND One of the difficulties in managing information work productivity is the challenge in

measuring the effects of implementing IT. In industrial work, the effect of technology was to increase the output and decrease the inputs needed for processing. IT may have more intangible effects on the output than merely increasing the quantity produced (Fichman, 2004). A close case-based examination of the literature and industrial practice reveals that besides improving output, technology also can help improve quality and consistency of information work and help the organization create better products/services through improved market intelligence and customer insight. The networked digital environment enables a firm to keep the customer engaged throughout the project, learn third-party customer information from online media, leverage the work of global suppliers, and reuse knowledge assets from other organizational entities (Barua et al., 2004). We draw on the theory of dynamic capabilities which explains how firms create value by creating relevant organizational processes and routines to leverage their IT assets (Teece et al., 1997). Rooted in the resource-based view (RBV) of the firm, dynamic capabilities represent the organizational and strategic routines that help firms respond to changing market conditions and create sustainable advantage by assembling inimitable resources to create organizational capabilities (Wernerfelt, 1984; Barney, 1991, Eisenhardt and Martin, 2000, Iansiti and Clark, 1994). Dynamic capabilities encompass different types of identifiable routines that have been the subject of extensive empirical research (Pavlou and El Sawy, 2004). For example, effective product development projects are characterized by the use of cross-functional teams

13

The four dimensions of effectiveness are highly correlated, with pairwise correlations exceeding 0.75. Hence, our approach to combine them into one meta-construct is validated by the results of the EFA.

3

that leverage different sources of expertise which enhances the breadth of available information, and enable coordination and concurrency of manufacturing, marketing, and design activities (Eisenhardt and Martin, 2000). Similarly, strategic decision making is an example of a dynamic capability for integrating project resources wherein project managers pool their diverse expertise to make project investment choices that are aligned with the strategic direction of the firm (Eisenhardt, 1989). Other dynamic capabilities associated with project management include effective external communication capabilities (Clark and Fujimoto, 1991), knowledge transfer and reuse capabilities which include routines for replication and brokering (Hansen, 1999, Hargadon and Sutton, 1997), and patching or continual realignment of business processes, resources, and organization to changing market conditions and customer needs (Eisenhardt and Brown, 1999). Dell’s constant segmentation of its operating units to better align with shifting customer requirements, organizational efforts to leverage supplier and internal knowledge assets, and practices to capture and use customer feedback are examples of these dynamic capabilities (Magretta, 1998). Effective alliance and acquisition processes for bringing new resources into the firm also represent important dynamic capabilities, since they enable managers to assemble advanced technical know-how for superior performance (Zollo and Winter, 2002). IT assets can provide the building blocks for business processes to enhance their dynamic capabilities (Amit and Schoemaker, 1993; Tallon and Kraemer, 2004). Although IT assets may be viewed as tangible resources, which are mobile and imitable, they enable unique organizational routines and are often bundled with an organization’s commitment to specific business processes (Wade and Hulland, 2004). Hence, firms can develop these ITenabled capabilities through multiple paths, and independently of other firms. Eisenhardt

4

and Martin (2000) argue that, even though there may be commonalities in the key features of effective dynamic capabilities, they can actually be different in terms of many details related to the resources used and trajectory of their adoption paths. Therefore, we draw upon dynamic capabilities theory to provide the theoretical foundation for studying the context in which information technologies may lead to better project and firm performance. 3.0

RESEARCH FRAMEWORK Project and program management work may typically involve a broad range of

activities, including product management, coordination and integration management, and process management (Harter and Slaughter, 2003). Also, a broad range of information technologies are used in a variety of industries to improve project performance. To be able to measure information work productivity, we first develop a typology to distinguish among different types of information technologies that are typically used for project/program management. This typology helps us relate various technology types to different kinds of improvement possible in project and process performance. 3.1

Technology Typology Our classification is driven by the nature of interdependencies that exist in different

projects. While there is enormous diversity in the types of information technologies observed in organizations, the nature of task interdependencies inherent in different types of project management activities entails usage of different types of IT. There are different levels of task interdependencies (Thompson, 1967).

Pooled interdependencies are common in

activities that are coordinated by common standards and/or rules-based mechanisms and where project teams share common IT resources but are otherwise independent (Kumar and van Dissel, 1996).

Sequential interdependencies are common where the tasks of each

5

module are distinct and serially structured so that the output of one unit is likely to affect subsequent downstream tasks. Documentation tasks in project and product management activities are sequential since they receive their primary inputs from the development teams (Harter and Slaughter, 2003). Reciprocal interdependencies occur when there is mutual exchange of information and tasks require ongoing adjustments and adaptation between work units. Project teams that are engaged in tasks with reciprocal interdependencies are tightly coupled to the timing of others. Hence, the nature of interdependencies drives the level of usage of different types of IT, since tasks that are tightly coupled may require a specific class of collaboration technologies compared to others that are more independent. Building upon this body of prior research, we now classify the information technologies typically used for project/program management into three groups: Core Communication Technologies (CCT), Enterprise Computing Technologies (ECT), and Group Collaboration Technologies (GCT). CCTs, which include basic technologies, such as e-mail and web portals, support tasks that entail pooled interdependencies where IT resources are shared by all entities. ECTs institutionalize sequential interactions between work units and support structured sequential interactions between users which enable them to access and exchange data in a structured format. ECTs encompass technologies, such as enterprise resource planning (ERP) software, which facilitates information exchange where output from one functional area serves as the input to other functional areas. GCTs support reciprocal interdependencies where communication involves information that is exchanged, processed, and adapted by users. These systems involve collaboration technologies such as groupware, online teamspaces and discussion databases, instant messaging and video-conferencing, which enable teams to communicate in real time (Carte and Chidambaram, 2004).

6

3.2

Efficiency versus Effectiveness of Information Work An examination of the research literature and field studies of industrial practice

suggests that the impact of IT on information work can be classified based on the benefits within the process and the interface with the customers, suppliers, and the extended organization. While firm productivity is generally defined as a ratio of firms’ outputs divided by their inputs, the measurement of information work productivity is complicated by the fact that both the inputs and outputs associated with information work are often intangibles.

Most of the prior IT literature has focused on the efficiency aspects of

information work productivity which studies the quantity of outputs produced and the costs associated with this effort. We propose a broader definition of information work productivity that captures the impact of IT on the effectiveness and efficiency dimensions of information work (Tallon et al., 2004). While efficiency refers to “doing things right,” effectiveness, which refers to “doing the right things,” signifies achievement of specific organizational objectives and is manifested in the attainment of sustained competitive advantage (Ray et al., 2004; Melville et al., 2004). We now discuss these capabilities in greater detail. 3.2.1

Efficiency Efficiency refers to the degree to which an organization is able to more cost- and

time-effectively achieve its targets by automating, streamlining, and accelerating its business processes. IT improves efficiency in a number of ways including re-engineering of business processes, reducing the latency and wait times of project tasks, and in the time spent in saving/retrieving business information. Hence, the impact of IT on project efficiency can be measured based on several factors including the time spent in searching and waiting for information, project task durations, and elimination of rework and non-value added tasks.

7

3.2.2

Effectiveness Effectiveness encompasses dynamic capabilities that pertain to the organization’s

ability to achieve more and better customer-perceived outcomes with fewer and lower-cost resources. Specifically, effectiveness can be measured in terms of the following indicator variables: (a) quality-consistency of the project outcome, (b) customer and market relevance of the outcome (Zeithaml and Bitner, 1996), (c) degree to which the project effort is structured to leverage the efforts of suppliers and other value chain partners (Yu et al., 2003), and (d) the alignment of the effort with the firm’s and business unit’s strategy (McGrath and Iansiti, 1999; Tallon et al., 2004; Nambisan, 2002). Quality, a key component of effectiveness, is particularly difficult to measure for information work, in contrast to manufacturing where it is measured as conformance to specifications. One measure of quality we use is consistency, which refers to the extent to which IT users are able to achieve targeted business results in a predictable manner and achieve their stated goals. Consistency is also an indicator of the visibility the organization has into its business results and improvement efforts. By helping to aggregate business information across the enterprise, IT should improve a firm’s ability to track progress, spot and correct deviations, and consistently execute on business plans and initiatives. Alignment denotes the organization's ability to streamline its multiple functions, departments, and initiatives towards a set of common business plans and goals. Centralized IT infrastructure and communication technologies helps align projects and programs with the company's business strategy, rationalizing project and product portfolio, and eliminate redundant initiatives (McGrath and Iansiti, 1999). Another dimension of the effectiveness of information work is the potential for IT to improve project/program relevance, which

8

measures the degree to which the users and their organizations are customer- and marketoriented in terms of their ability to meet current and future customer needs (Barua et al., 2004). Used appropriately, IT should help the firm build “higher bandwidth” communication channels with its lead users and mainstream customers and capture tacit and emerging customer information (Zeithaml and Bitner, 1996). 3.3

Research Model The term “IT business value” refers to the impact of IT on organizational

performance, which is typically measured using intermediate process-level measures as well as firm-level measures that represent both financial and operational metrics.

The dynamic

capabilities theory provides a valid framework to analyze the impact of IT on organizational performance due to its focus on resource attributes and its relevance for measuring IT and non-IT resources (Peteraf and Barney, 2003; Wheeler, 2002).

Improvements in firm

capabilities, in turn, drive improvements in business process performance as well as firmlevel measures such as profit margins. In our research context, the impact of IT-enabled dynamic capabilities on improvements in business process performance is measured by studying their impact on project outcomes, such as reduction in project cycle time, cost, and on-time completion rates. Since projects represent important vehicles for firms to realize business value, we also examine the impact of IT-enabled dynamic capabilities on firm financial measures, such as gross margins and return on assets (Bharadwaj, 2000). Our conceptual research model is shown in Figure 1. Our model examines the role of dynamic capabilities in mediating the impact of different types of IT resources on projectand firm-level measures of performance. We integrate the information processing view of IT resources with the dynamic capabilities perspective to provide a sharper understanding of (a)

9

the differential impact of different types of IT and their usage, and (b) the role of effectiveness and efficiency in mediating the impact of IT on process outcomes and firm financial measures. Specifically, we incorporate the role of different types of software when conceptualizing the IT constructs which has been not been addressed in prior studies (Melville et al., 2004, pp. 312). Our model provides a sharper distinction of effectiveness and efficiency, which enables us to measure the differences in their impact on performance. 3.4

Research Hypotheses

Impact of IT on Efficiency and Effectiveness While the impact of aggregate IT resources on process efficiency has been extensively studied in prior research, less attention has been given to studying the differential impact of specific types of IT resources (Barua et al., 1995; Brynjolffson and Hitt, 1998). We measure the impact of IT on process efficiency based on several factors, including the time spent in searching and waiting for project information, task durations, and elimination of rework and non-value added tasks. In a project/program management setting, basic communication technologies (CCTs), such as e-mail and web portals, can improve efficiency by reducing the latency of project tasks and in the time spent in searching for retrieving information. Similarly, GCTs such as instant messaging and vide-conferencing technologies enable teams to exchange project information in real-time and thereby reduce the wait times involved in information transfer and making project decisions (Carte and Chidambaram, 2004). ECTs, such as document management systems and knowledge management systems, provide efficient electronic storage and enable quick information retrieval which improves process efficiency.

Hence, we expect that usage of IT resources is associated with

improvements in the efficiency of organizational capabilities.

10

H1:

Greater usage of IT resources is associated with an improvement in the efficiency of organizational capabilities.

H1a: Greater CCT usage is associated with improvements in project/program efficiency. H1b: Greater ECT usage is associated with improvements in project/program efficiency. H1c: Greater GCT usage is associated with improvements in project/program efficiency. We argue that measuring the effectiveness of organizational capabilities may be a better approach to empirically test resource-based logic since it is possible for firms to appropriate the economic profits generated by a firm’s processes before these profits are reflected in the firm’s gross profit margins (Ray et al., 2004).

ECTs, such as project

management software and business intelligence systems, enable firms to improve the consistency of project execution by facilitating greater visibility to project data, allowing managers to track progress more easily and identifying project risks. Similarly, customer relationship management (CRM) software enable managers to identify and disseminate customer requirements in a timely manner and allow greater customer involvement in project decisions.

Knowledge management systems and business intelligence software enable

project managers to identify emerging market trends and develop relevant product enhancements and new market opportunities. CCTs, such as project portals and e-mail communication technologies, enable project teams to solicit customer and supplier involvement in key project decisions which can lead to improvements in the quality of project decisions. They also facilitate re-use of product data and other knowledge assets within and across project teams. Similarly, GCTs such as mobile computing technologies (handheld computers, PDAs) facilitate quicker exchange of missioncritical data, and thereby enable more effective alignment of distributed project teams. Other

11

technologies, such as groupware and instant messaging software, enable easier tracking of projects and improve project visibility even though project teams may not be co-located. Hence, we hypothesize that IT usage is associated with an improvement in the effectiveness of organizational capabilities. H2:

Greater usage of IT resources is associated with an improvement in the effectiveness of organizational capabilities.

H2a: Greater CCT usage is associated with improvements in project effectiveness. H2b: Greater ECT usage is associated with improvements in project effectiveness. H2c: Greater GCT usage is associated with improvements in project effectiveness. We control for project characteristics, such as team size, extent of co-location of project teams, project duration, and a firm’s propensity to use IT, that may also have an impact on the development of firm dynamic capabilities.

Team size dictates the extent of

information processing that is required to disseminate information across project teams. The increasing use of virtual project teams, where team members may be distributed across different offices in different countries, also has an impact on organizational capabilities. A firm’s propensity to use IT, as measured by the level of adoption of emerging technologies, may influence the types of IT resources that are used and thereby impact the effectiveness and efficiency dimensions of organizational capabilities. Impact of Human IT Capital The “human IT capital” component of IT resources plays an important role in determining the impact of IT on organizational performance (Bharadwaj, 2000; Ross et al., 1996). It includes technical expertise skills, such as application development, maintenance, and integration, as well as managerial expertise such as the ability to provide project

12

leadership, and provide the necessary financial and human resources (Melville et al., 2004). Here, we focus on the managerial expertise component since prior researchers have argued that successful application of IT needs to be accompanied by complementary organizational changes such as organizational culture and workplace practices (Mata et al., 1995; Brynjolffson and Hitt, 2000). We draw upon the concept of organizational IQ, a construct that was proposed by Mendelson (2000) to measure human IT capital within organizations. It measures the extent of adoption of advanced management practices for project planning, execution and management, which include competitive benchmarking of best practices, senior executive leadership, and appropriate training in the use of work methods, processes and IT. Hence, human IT capital represents the degree to which management facilitates knowledge transparency, information awareness and decision-making capabilities within the firm (Boynton et al., 1994; Armstrong and Sambamurthy, 1999). We expect that human IT capital will be associated with improvements in dynamic organizational capabilities. H3: Greater levels of human IT capital are associated with an improvement in firm dynamic capabilities. H3a: Greater human IT capital is associated with improvements in project effectiveness. H3b: Greater human IT capital is associated with improvements in project efficiency. Organizational Performance Since a limitation of RBV is that it assumes that resources are applied in their best uses, saying little about how this is done, we expand our research framework to better understand how IT is applied to improve process performance. Although the impact of IT on operational efficiency and competitive advantage has been supported in the IT business value

13

literature (Melville et al., 2004; Barua et al., 1995; Soh and Markus, 1995), the research question that is not well-understood is “what are the mechanisms by which IT enables firms to achieve efficiency and competitive advantage?” Recent work by Ray et al. (2004, pp. 35) suggest that it is important to develop a better understanding of the relationship between a firm’s resources and the effectiveness of its organizational routines.

We argue that IT

resources enable improvements in dynamic capabilities which, in turn, lead to improvements in process-level outcomes and firm-level performance. Improvements in process efficiency, due to a reduction in project latency (wait times) and ability to undertake concurrent development of multiple project tasks, is associated with a reduction in project cycle times and improved project completion rates. Similarly, a reduction in rework associated with projects work units/modules and removal of non-value added tasks is associated with improvements in project quality and costs, since defects/errors that are identified and corrected in later stages of the project life-cycle are costlier to fix. Improvements in process effectiveness capabilities have an impact on project outcomes. Improved consistency of project execution, through better transparency and risk management, can reduce operating costs and increase project completion rates. Improvements in a firm’s ability to leverage its existing network partners to manage its sourcing processes is associated with reduction in project cycle times and costs. Similarly, achieving better alignment through closure of functional gaps and distributed project teams, can lead to improved project quality and cycle times. Hence, we expect that improvements in firm effectiveness capabilities are associated with improvements in project outcomes. H4:

Improvements in firm dynamic capabilities are associated with improvements in project-level outcomes.

14

H4a: Improvements in process efficiency are associated with improved project outcomes. H4b: Improvements in the effectiveness of organizational capabilities are associated with improvements in project outcomes. Improvements in dynamic capabilities not only drive project outcomes, but are associated with improvements in firm-level performance, since these capabilities proliferate across projects and project teams.

Improvements in a firm’s capabilities to effectively

leverage its suppliers to improve its sourcing processes, and thereby reduce its cost of sales, is associated with improvements in gross margins. By improving their customer focus, firms can extract higher premiums and create new markets, which are associated with improvements in firm margins and return on assets (ROA). Similarly, improvements in firms’ consistency of execution may be associated with better capabilities to leverage their investments into more productive assets, which in turn drive higher returns on equity (ROE). H5:

Improvements in dynamic capabilities are associated with improvements in firm financial performance. We control for the impact of other organizational characteristics on project- and firm-

level performance. For instance, firm size may have an impact on the ability of firms to realize improvements in process-level outcomes, since smaller firms may be more agile and better able to respond to shifting market trends compared to larger firms. We also control for the impact of project autonomy, process maturity, and the number of projects that are concurrently managed, on organizational performance. Since organizational dynamic capabilities link the impact of IT resources to improvements in project- and firm-level performance, we frame our hypothesis as:

15

H6:

Dynamic capabilities mediate the impact of technological IT resources on improvements in project outcomes and firm performance

4.0

RESEARCH DATA

4.1

Data Collection A cross-sectional survey was employed for data collection. We designed a research

survey to capture the organizational work characteristics, information technology usage, and project performance data, as shown in the Appendix. The survey was administered online in June 2004, through an independent, professional survey firm, to a random sample of project and product managers drawn from US private- and public-sector organizations. A first phase of pilot data was followed by a full-fledged data collection effort with 780 responses. The final dataset of 625 respondents represents completed surveys where the respondents met the survey selection criteria of “being a project/program/business manager in their firm.” Table 1 shows the profiles or respondents based on firm and project characteristics. About 42% of the survey respondents were from publicly traded firms while the remainder belonged to not for profit organizations, state or federal government agencies, and privately held firms. About a third of the respondents were responsible for managing 4-5 projects concurrently, while 15% responded that they managed between 6-10 projects and 12% managed more than 10 projects concurrently. While 36% of the respondents indicated that their projects lasted less than one year on average, an equal number responded that their projects lasted between one and five years. 4.2

Construct Measurement Constructs that represent usage of different types of IT resources and the

organizational dynamic capabilities have been discussed in the previous section.

16

Effectiveness is measured using a twelve item-scale where each of its dimensions are represented by three indicator variables. We have proposed several variables to control for the impact of project and firm characteristics on the measurement of IT productivity. Project duration is measured as the average length of a typical project/program (measured in years), while Team size represents the size of the project team measured as the number of full-time equivalent (FTE) of project staff. Co-location measures the extent of dispersion among team members. Higher values represented dispersed project teams while lower values represent teams that are located in one location. Autonomy measures the extent to which project teams have autonomy from management in making project decisions (Mendelson, 2000). Process maturity represents the degree of implementation of advanced process capabilities, and includes practices such as cross-functional teams, robust project management processes, and customer feedback for continuous process improvement (Harter and Slaughter, 2003). This construct was adapted from the Capability Maturity Model–Integrated Product Development (IPD) framework. The propensity of a firm to use IT is an important indicator of a firm’s willingness to deploy emerging technologies to its business, and we use this variable to evaluate whether firms are early adopters, mainstream users or late adopters of IT. We also collect data on the number of concurrent projects managed since this is an important indicator of managers’ roles and their dedication to current projects. Firm size was used as a control variable to study the impact of IT on firm-level performance. We measured firm size in two ways: the number of firm employees and the natural logarithm of the dollar value of firm assets for the publicly traded firms in our sample.

17

We measure organizational performance at two levels: change in project-level outcomes and firm performance. We define four project outcomes which represent changes in project cycle time, cost, quality, and on-time completion rate, during the prior one-year period at the time of the survey. ∆(Cycle Time) represents the change in project cycle times, defined as the total time elapsed from project initiation to completion. ∆(Cost) represents the change in project costs, defined as the total program/project lifecycle cost incurred from initiation to completion. ∆(Quality) represents the change in project quality, measured as the total number of errors, defects, and rework associated with the project. Finally, ∆(On-time completion rate) represents the change in the projects completed on time. These perceptual measures represent outcomes for a typical (or average) project in the manager’s portfolio. The three firm financial measures that we examined are: (a) Gross Margin, which is defined as (Sales minus Cost of Sales) / Sales, (b) ROA, defined as (Net Income / Assets), and (c) ROE, defined as (Net Income / Common Equity).

Gross profit margins are

influenced by two factors: the price premium that a firm’s products command in the marketplace, and the efficiency of the firm’s procurement and production processes. ROA measures how much profit a firm is able to generate for each dollar of assets invested (Palepu et al.,1997; Santhanam and Hartono, 2003).

On the other hand, ROE is a more

comprehensive indicator of firm performance because it measures how well managers deploy the resources invested by shareholders to generate returns.

Hence, it provides a good

indicator of the extent of financial leverage of the firm. 4.3

Construct Validity and Reliability Exploratory factor analyses (EFA) was first conducted to check if the proposed

factors are consistent with our survey data. The factor structures suggested by EFA are

18

consistent with the factors identified in our research model and they account for 62% of the variance in the data. Since our survey data are self-reported, we performed a Harmon’s onefactor test to check for common methods bias (Podsakoff and Organ, 1986). The items measuring the independent variables and those measuring project outcomes loaded on different factors, which do not indicate common methods bias. The Cronbach alpha values for these factors range from 0.68 to 0.94 which indicates good internal consistency.3 We also performed a confirmatory factor analysis (CFA) to establish the reliability of our constructs. Composite reliability reflects the internal consistency of the indicators and it exceeded the recommended threshold of 0.7 for six of the seven constructs shown in Table 2 (Werts et al., 1974; Nunnally, 1978). The t-statistics for all factor loadings were significant and suggest that our indicator measures satisfy convergent validity (Phillips and Bagozzi, 1986). To establish the discriminant validity of the constructs, we calculated the average variance extracted (AVE), a ratio of the construct variance to the total variance among indicators, which exceeds the threshold of 0.5 for all factors (Fornell and Larcker, 1981). We calculated several goodness-of-fit statistics which confirm that the measurement model provides a satisfactory explanation of the observed data. 5.0

EMPIRICAL RESULTS Since our research framework represents a recursive model, we estimated the model

using ordinary least squares (OLS) regressions which provide consistent and efficient estimates (Johnston, 1984, pp. 468-469). We also estimated the model using a structural equation model (SEM) and these results are generally consistent with our OLS estimates.

19

5.1

Impact of IT Resources on Dynamic Capabilities The impact of IT on the effectiveness and efficiency dimensions of dynamic

capabilities is shown in Table 3. The results indicate that there is a distinct divide in the impact of information technologies on these two dimensions of information work. Both ECT and CCT have a positive impact on the effectiveness of information work, in terms of their ability to improve the consistency of project execution and leverage knowledge assets. Although the impact of GCT on project effectiveness is positive, it is not statistically significant. Hence, our results support hypotheses H1a and H1b, but do not support H1c. On the other hand, GCTs have a significant impact on the efficiency component of information work. GCTs support real-time collaboration among project teams, and provide a communications infrastructure for project teams to collaborate across multiple locations. By enabling concurrent work among project teams, GCTs improve project efficiency by reducing the time searching/waiting for information and eliminating non-value added tasks. Hence, our results support hypothesis H2c, but do not support hypotheses H2a and H2b. Our results indicate that IT human capital has a positive impact on improvements in project effectiveness and efficiency. Although, the impact of IT human capital on efficiency is positive, it is statistically significant at a p-value < 0.10. These results suggest that managerial IT expertise plays an important role in improving business process capabilities, in terms of their effectiveness and efficiency. Hence, our results support hypothesis H3. Our results suggest that a firm’s propensity to use IT plays an important role in terms of its impact on organizational capabilities. Firms which are more likely to adopt emerging technologies are likely to realize improvements in their process effectiveness, and to a lesser extent, on process efficiency.

20

5.2

Impact of IT on Project Performance Table 4 shows the impact of IT and IT-enabled capabilities on the four project

performance measures: change in cycle time, cost, quality, and on-time completion rate. Odd-numbered columns represent regressions where we treat the performance variable as dependent on the effectiveness and efficiency capabilities and the firm controls, which include project autonomy, process maturity, firm size, and the number of concurrent projects managed. Hence, the regression model is specified in equation (1) as: Project performance = f (Effectiveness, Efficiency, Firm controls)

(1)

Equation (1) is consistent with our research framework which suggests that the impact of IT on project performance will be completely mediated through organizational dynamic capabilities. We observe that project effectiveness capabilities and process maturity dominate other variables in terms of their ability to explain the variation in project outcomes. Effectiveness has a significant impact on all four project outcomes while efficiency has a significant impact all outcomes except on-time completion rates. Hence, our data support hypothesis H4a and H4b. To validate whether our model provides a satisfactory explanation of the relationship between IT and project performance, we test the mediation model against a competing model which included direct paths from the IT variables to the four project performance variables. Hence, each direct model has three paths more than its corresponding mediation model, which represents the direct links between the three IT variables and the performance variable. Therefore, our regression model is specified in equation (2) as Project performance = f (Effectiveness, Efficiency, ECT, GCT, CCT, Human IT capital, Firm controls)

(2) 21

These regression results are provided in the even numbered columns of Table 4. To compare the two models, we compared the R2 values which represent the percentage of variance explained by the predictor variables. Since the models are nested, we can examine whether the mediation model provides an adequate explanation of the variance in project outcomes, through an F-test (Subramani, 2004; Venkatraman, 1988). We performed an Ftest on the difference in R2 and report these F-statistics at the bottom of Table 4. We observe that differences in R2 values are very small for each of the four pairs (direct and mediation) of regression models.

We note that the F-test is statistically

significant for the ∆(Cycle Time) and ∆(Quality) models, while it is not significant for the ∆(Cost) and ∆(On-time Completion Rate) models. Hence, our findings suggest that dynamic capabilities completely mediate the impact of technological IT resources (ECT, GCT, CCT) on project cost and completion rates, while they partially mediate IT impact on project cycle times and quality. The results imply that dynamic capabilities may mediate the impact of IT on different types of project outcomes in different ways, and may serve only as partial mediators in some cases. A closer look at Table 4 suggests that higher levels of ECT usage is associated with improvements in project on-time completion rates, which implies that usage of enterprise systems and project management software have a tangible impact on project timeliness. Our results also indicate that GCT usage is associated with improvements in project cycle time, which implies that collaboration technologies are useful in coordinating project schedules and resources. We observe that the IT human capital and process maturity constructs are associated with improvements in all project performance measures, which is consistent with findings from prior IS research (Mendelson, 2000; Harter and Slaughter, 2003).

22

5.3

Impact of IT on Firm Performance A unique characteristic of this study is our ability to go beyond project performance

measures as reported by the survey respondents. We also examined objective financial measures of the public firms to which respondents belong. Financial data for a subset of 147 companies was collected from the Standard & Poors Compustat database (S&P). We regressed the firm-level financial measures against dependent variables such as IT usage, the dynamic capabilities of effectiveness and efficiency, and organizational and process variables. Our regression results are shown in Table 5. In a manner similar to our approach described in the previous section, the odd-numbered columns denote results where we regressed the relevant financial performance measure in year t (FPt) against the dynamic capabilities, firm control variables, and past firm performance in year t-1 represented by “Fin. Perft-1”.4 The mediation model is specified as, FPt = f (Effectiveness, Efficiency, Firm controls, Fin. Perft-1)

(3)

The even-numbered columns represent regressions of FPt against the dynamic capabilities as well as the IT and control variables. That is, FPt = f (Effectiveness, Efficiency, ECT, GCT, CCT, IT Human Capital, Firm controls, Fin. Perft-1)

(4)

The results indicate that effectiveness has a significant impact on firms’ gross margins while efficiency has a significant impact on firms’ ROE. These results indicate that quality of information work, measured in terms of a firm’s ability to improve its consistency, leverage, alignment and relevance, is associated with improvements in gross margins as firms are able to do a better job of reducing the cost of sales and improve price premiums by making their

4

Our approach is similar to the one used by Zhu and Kraemer (2002) to control for the effect of prior performance.

23

product/services more relevant to customer needs. In a similar vein, we note that efficiency is associated with improvements in ROE as it enables firms to leverage their shareholders investments into valuable assets which generate profits. To compare the two models, we once again compare the R2 values of the mediation and direct models. Clearly, the F-test results in Table 5 show that the incremental direct impact of IT on firm performance is not statistically significant.

These results confirm that

the impact of IT on firm performance is mediated through effectiveness and efficiency, but through firm-level data. Hence, our results support hypothesis H5 and partially support H6, since dynamic capabilities partially mediate the impact of IT on project outcomes. 6.0

DISCUSSION AND CONCLUSIONS Improving information work (IW) productivity has been a major challenge due to the

intangibility of organizational outcomes.

As a first step in gaining a more nuanced

understanding of the role and business value of IT, we focused on information work in a project context.

Our research objectives were to understand (a) how different types of IT

resources impact dynamic capabilities within organizations, and (b) how to gauge and measure the value of IT in an information work setting. We have extended prior research by (a) improving our understanding of the differential impact of alternate types of IT resources as well as their usage, (b) incorporating the role of software when conceptualizing the IT resource, which has been missing from most IT productivity research, (c) operationalized two salient dimensions of dynamic capabilities, effectiveness and efficiency, which explain a significant portion of the variation in firm performance, and (d) provided empirical support to test the dynamic capabilities framework with data collected from an information work setting

24

where technologies are used to manage projects, programs, and people. Furthermore, our model measures a key construct, the level of IT usage, and links it to dynamic capabilities which mediate the impact on firm performance (Devaraj and Kohli, 2003). Our observed results using perceptual measures of project outcomes were further enhanced through the collection of financial data for a subset of 147 public firms. Indeed, our results suggest that prior studies that have attempted to measure the productivity of information work may be inadequate since they mainly capture the efficiency dimension of information work. Our results indicate that quality of firm outputs (effectiveness) may be even more important in terms of its impact compared to the quantity of outputs (efficiency). In this manner, our research framework supports the arguments for extending IT innovation research as proposed by Fichman (2004). Our study has several limitations. First, it does not account for the use of information technologies by individual contributors or non-managerial personnel who may have different IT usage patterns from project/program managers. Second, while our study contains a large number of data points, it can be enhanced further by including respondents in non-US organizations to account for country- or culture-specific differences in their IT usage. Third, our survey findings must be validated with more field studies and additional data collected in industry-specific settings which will allow us to examine the impact of industry characteristics and IT usage on the development of specific dynamic capabilities in different environments. Future research should also include data on project complexity, team diversity and uncertainty which are important project management variables.

25

The overall results suggest that IT needs to be implemented with the intent of maximizing project and financial outcomes, in a manner that fundamentally improves the dynamic capabilities of the firm, which, in turn, will improve project and firm-level performance. Organizations will not realize significant performance improvements if IT is used to only improve quantity of outputs and inputs, ignoring the effectiveness or quality of the outcomes. Effectiveness should be as much of a focus as efficiency, which has hitherto driven IT investment decisions. Our study represents an early effort to study the impact of IT specifically focused on project and information work (Nambisan, 2003).

Our findings

amplify the need for firms to strengthen their organizational dynamic capabilities after making investments in information technology (Dorgan and Dowdy, 2004). While we have experienced many anecdotally-driven discussions regarding whether IT matters or not, our data shows that IT matters to the extent that it can extend organizational dynamic capabilities.

REFERENCES Amit, R. and P.J.H. Schoemaker, “Strategic assets and organizational rent,” Strategic Management Journal 14(1), 1993, 33-46. Armstrong, C.P., and V. Sambamurthy, “Information technology assimilation in firms: the influence of senior leadership and IT infrastructure,” Information Systems Research 10(4), 1999, 304327. Barney, J.B., “Firm resources and sustained competitive advantage,” Journal of Management 17(1), 1991, 99-120. Barua, A., C. Kriebel, and T. Mukhopadhyay, “Information technology and business value: An analytic and empirical investigation,” Information Systems Research 6(1), 1995, 3-23. Barua, A., P. Konana, A. Whinston, and F. Yin, “An empirical investigation of net-enabled business value,” MIS Quarterly 28(4), 2004, 585-620. Bharadwaj, A.S., “A resource-based perspective on information technology capability and firm performance: An empirical investigation,” MIS Quarterly 24(1), 2000, 169-196. Boynton, A. C., R.W. Zmud, and G.C. Jacobs, “The influence of IT management practice on IT use in large organizations,” MIS Quarterly 18(2), 1994, 299-318.

26

Brynjolfsson, E., and L. Hitt, “Paradox lost? Firm-level evidence on the returns to information systems spending,” Management Science 42(4), 1996, 541-558. Brynjolfsson, E., and L. Hitt, “Beyond the productivity paradox,” Communications of the ACM 41(8), 1998, 49-55. Brynjolfsson, E., and L. Hitt, “Beyond Computation: Information technology, organizational transformation, and business performance,” Journal of Economic Perspectives 14(4), 2000, 2348. Carte, T., and L. Chidambaram, “A capabilities-based theory of technology deployment in diverse teams: Leapfrogging the pitfalls of diversity and leveraging its potential with collaborative technology,” Journal of the Association for Information Systems 5(11-12), 2004, 448-471. Chatterjee, D., R. Grewal, and V. Sambamurthy, “Shaping up for e-commerce: Institutional enablers of the organizational assimilation of web technologies,” MIS Quarterly 26(2), 2002, pp. 65-89. Clark, K., and T. Fujimoto, Product Development Performance: Strategy, Organization and Management in the World Auto Industries, 1991, Harvard Business School Press, Boston, MA. Conner, K.R., and C.K. Prahalad, “A resource based view of the firm: Knowledge versus opportunism,” Organization Science 7(5), 1996, 477-501. Devaraj, S., and R. Kohli, “Performance impacts of information technology: Is actual usage the missing link?” Management Science 49(3), 2003, pp. 273-289. Dorgan, S.R., and J.J. Dowdy, “When IT lifts productivity,” The McKinsey Quarterly (4), 2004. Eisenhardt, K., “Making fast strategic decisions in high-velocity environments,” Academy of Management Journal 32(3), 1989, 543-576. Eisenhardt, K., and S.L. Brown, “Patching: Restitching business portfolios in dynamic markets,” Harvard Business Review 77(3), 1999, 72-82. Eisenhardt, K., and J. Martin, “Dynamic capabilities: What are they?” Strategic Management Journal 21, 2000, 1105-1122. Fichman, R.G., “Going beyond the dominant paradigm for information technology innovation research: Emerging concepts and methods,” Journal of the Association for Information Systems 5(8), 2004, 314-355. Fornell, C., and D. Larcker, “Evaluating structural equation models with unobservable variables and measurement errors,” Journal of Marketing Research 18(1), 1981, pp. 39-50. Gefen, D., D.W. Straub, M.C. Boudreau, “Structural equation modeling and regression,” Communications of the Association for Information Systems 4(7), 2000, 1-70. Hansen, M.T., “The search-transfer problem: The role of weak ties in sharing knowledge across organizational subunits,” Administrative Science Quarterly (March) 44, 1999, 82-111 Hargadon, A, and R.I. Sutton, “Technology brokering and innovation in a product development firm,” Administrative Science Quarterly 42(4), 1997, 716-749. Harter, D.E., M.S. Krishnan, S. Slaughter, “Effects of process maturity on quality, cycle time, and effort in software product development,” Management Science 46(4), 2000, 451-466. Harter, D.E., S. Slaughter. 2003. Quality improvement and infrastructure activity costs in software development: A longitudinal analysis. Management Science. 49(6), 84-800. Hitt, L.M., D.J. Wu, X. Zhou, “Investment in enterprise resource planning: Business impact and productivity measures,” Journal of Management Information Systems 19(1), 2002, 71-98.

27

Iansiti, M., and K.B. Clark, “Integration and Dynamic Capability: Evidence from Product Development in Automobiles and Mainframe Computers,” Industrial and Corporate Change 7(3), 1994, pp. 557-605. Johnston, J., Econometric Methods, 1984, Third Edition, McGraw-Hill, Inc., New York, NY. Kohli, R., and S. Devaraj, “Measuring information technology payoff: A meta-analysis of structural variables in firm-level empirical research,” Information Systems Research 14(2), 2003, 127-145 Kumar, K., and H.G. van Dissel, “Sustainable collaboration: Managing conflict and cooperation in inter-organizational Systems,” MIS Quarterly 20(3), 1996, 279-300. Magretta, J., “The power of virtual integration: An interview with Dell Computer’s Michael Dell,” Harvard Business Review 76(2), 1998, 72-84. Mata, F.J., W.L. Fuerst, and J. Barney, “Information technology and sustained competitive advantage: A resource-based analysis,” MIS Quarterly 19(4), 1995, 487-505. McGrath, M., and M. Iansiti, “Envisioning IT-enabled innovation” Insight Magazine, 1998, 2-10. Melville, N, K.L. Kraemer, and V. Gurbaxani, “Information technology and organizational performance: An integrative model of IT business value,” MIS Quarterly 28(2), 2004, 283-322. Mendelson, H., “Organizational architecture and success in the information technology industry,” Management Science 46(4), 2000, 513-529 Mukhopadhyay, T., S. Rajiv and K. Srinivasan, “Information technology impact on process output and quality,” Management Science 43(12), 1997, pp. 1645-1659. Nambisan, S., “Designing virtual customer environments for new product development: Toward a theory,” Academy of Management Review 27(3), 2002, 392-413. Nambisan, S., “Information systems as a reference discipline for new product development,” MIS Quarterly 27(1), 2003, 1-18. Nunnally, J.C., and I.H. Bernstein, Psychometric Theory, 1994, McGraw Hill, New York, NY. Palepu, K.G., V.L. Bernard, and P.M. Healy, Introduction to Business Analysis and Valuation. 1997, South-Western Publishing, Cincinnati, OH. Pavlou, P.A., and O.A. El Sawy, “Understanding the ‘black box’ of dynamic capabilities: A missing link to the strategic role of IT in turbulent environments,” Working Paper, 2004, University of California at Riverside, CA. Peteraf, M.A., and J.B. Barney, “Unraveling the resource-based tangle,” Managerial and Decision Economics 24(4), 2003, 309-323. Phillips, L., and R. Bagozzi, “Assessing measurement error in key informant reports: A methodological note on organizational analysis in marketing,” Journal of Marketing Research (18), 1986, pp. 395-415. Podsakoff, P.M., D.W. Organ. 1986. Self-reports in organizational research: Problems and prospects. Journal of Management. 12(4), pp. 531-544. Peters, T., “The Wow Project”, Business 2.0, May 1999, Issue 24, Page 116. Ray, G., J.B. Barney, and W.A. Muhanna, “Capabilities, business processes, and competitive advantage: Choosing the dependent variable in empirical tests of the resource-based view,” Strategic Management Journal 25, 2004, 23-37.

28

Ross, J.W., C.M. Beath, and D.L. Goodhue, “Develop long-term competitiveness through IT assets,” Sloan Management Review 38(1), 1996, 31-42. Sambamurthy, V., “Business Strategy in Hyper-competitive Environments: Rethinking the Logic of IT Differentiation,” in Framing the Domains of IT Management: Projecting the Future through the Past, Zmud, R.W., (ed.), 2000, Pinnaflex, Cincinnati, OH, 245-261. Santhanam, R., and E. Hartono, “Issues in linking information technology capability to firm performance,” MIS Quarterly 27(1) 2003, 125-153. Soh, C., and L.M. Markus, “How IT creates business value: A process theory synthesis,” in Proceedings of the 16th International Conference on Information Systems, J.I. DeGross, G. Ariav, C. Beath, R. Hoyer, and C. Kemerer (eds.), Amsterdam, the Netherlands, 1995, 29-41. Subramani, M., “How do suppliers benefit from information technology use in supply chain relationships?” MIS Quarterly 28(1), March 2004, 45-73. Tallon, P.P., and K.L. Kraemer, “The impact of IT capabilities on firm performance: Perspectives on the mediating effects of strategic alignment,” Working paper, 2004. Tallon, P.P., K.L. Kraemer, V. Gurbaxani, and J. Mooney, “A multi-dimensional assessment of the contribution of information technology to firm performance,” CRITO Working Paper #ITR-105, 2004, University of California at Irvine, Irvine, CA. Teece, D.J., G. Pisano, A. Shuen, “Dynamic Capabilities and Strategic Management,” Strategic Management Journal 18(7), August 1997, 509-533. Thompson, J.D., Organizations in Action, 1967, McGraw Hill, New York, NY. Venkatraman, N., “The concept of fit in strategy research: Toward verbal and statistical correspondence,” Academy of Management Review 14(3), 1989, 423-444. Wade, M., and J. Hulland, “The resource-based view and information systems research: Review, extension, and suggestions for future research,” MIS Quarterly 28(1), 107-142. Wernerfelt, B., 1984, “A resource-based view of the firm” Strategic Management Journal 5(2), pp. 171-180. Werts, C., R. Linn, and K. Joreskog, “Interclass reliability estimates: Testing structural assumptions,” Educational and Psychological Measurement 34(1), 1974, pp. 25-33. Weill, P. "The relationship between investment in information technology and firm performance: A study of the valve manufacturing sector," Information Systems Research 3(4) 1992, pp 307- 333. Wheeler, B.C., “NEBIC: A dynamic capabilities theory for assessing Net-enablement” Information Systems Research 13(2), 2002, pp. 125-146. Yu, Seunghee, T. Mukhopadhyay, and S. Slaughter, “Organizational capabilities and the assimilation of electronic procurement in service industries,” Proceedings of the Twenty-Fourth international Conference on Information Systems, 2003, 975-981. Zeithaml, V., and M.J. Bitner, Services Marketing, 1996, McGraw-Hill, New York, NY. Zhu, K., and K.L. Kraemer, 2002, “e-Commerce metrics for net-enhanced organizations: Assessing the value of e-Commerce to firm performance in the manufacturing Sector,” Information Systems Research 13(3), 275-295. Zollo, M., and S.G. Winter, “Deliberate learning and the evolution of dynamic capabilities,” Organization Science 13(3), 2002, 339-351.

29

Figure 1:

Conceptual Research Model Project Controls

Organizational Performance

• Team Size Information Technologies • Enterprise Computing Technologies (ECT) • Group Collaboration Technologies (GCT) • Core Communication Technologies (CCT)

Project Outcomes

• Co-location • Propensity to Use IT • Project Duration

Change in • • • •

Project Cycle Time Project Cost Project Quality On-time Completion Rate

• Effectiveness • Efficiency Firm Financial Measures

Organizational Capabilities

• Gross Margin • Return on Assets • Return on Equity

Human IT Capital • Autonomy • Process Maturity • Number of Concurrent Projects • Firm Size

Organizational Controls

Table 1: Survey Respondent Firms’ Profiles Firm Revenues $10 million to $49.9 million $50 million to $99.9 million $100 million to $499.9 million $500 million to $999.9 million $1 billion to $9.9 billion $10 billion or more Not for profit Don’t Know or not at liberty to disclose Industry Communications Energy Financial Services Healthcare Public Sector Manufacturing Retail/Wholesale Travel and Transportation Other Project Team Size 1 - 10 people 11 - 20 people 21 - 50 people 51 - 100 people > 100 people TOTAL

Number of Respondents 90 62 74 48 103 103 64 81

Percent (%) 14.4 9.92 11.84 7.68 16.48 16.48 10.24 12.96

51 18 77 67 104 121 54 25 108

8.16 2.88 12.32 10.72 16.64 19.36 8.64 4.00 17.28

354 127 82 30 32 625

57% 20% 13% 5% 5%

30

Table 2: Confirmatory Factor Analyses of the Model Variables Construct

ENTERPRISE COMPUTING TECHNOLOGIES (ECT)

GROUP COLLABORATION TECHNOLOGIES (GCT)

CORE COMMUNICATION TECHNOLOGIES (CCT) EFFECTIVENESS

Indictor Variable

Standardized Loading

t-statistic

Composite Reliability

Knowledge Management Software Document Management Software Enterprise Application Software CRM Software Content Management Systems Project Management Software Business Intelligence Software Video Conferencing and Voice over IP Groupware and Online Teamspaces Mobile Computing Instant Messaging Software Internet Search Engines and Web Portals Email Mobile Communication Technologies EFFECT-1 EFFECT-2 EFFECT-3 EFFECT-4 EFFECT-5 EFFECT-6 EFFECT-7 EFFECT-8 EFFECT-9 EFFECT-10 EFFECT-11 EFFECT-12

0.789

23.29

0.89

Average Variance Extracted (AVE) 0.73

0.660

18.20

0.681

18.97

0.762 0.788

22.11 23.25

0.717

20.33

0.681

18.95

0.693

19.00

0.78

0.70

0.722

20.09

0.718 0.624

19.92 16.61

0.657

15.09

0.64

0.61

0.503 0.657

11.37 15.10

0.745 0.740 0.719 0.725 0.689 0.760 0.747 0.754 0.736 0.768 0.692 0.703

21.69 21.47 20.63 20.90 19.50 22.29 21.78 22.06 21.33 22.65 19.58 20.02

0.94

0.73

(continued on the next page)

31

EFFICIENCY HUMAN IT CAPITAL

PROCESS MATURITY

EFFICIENCY-1 EFFICIENCY-2 EFFICIENCY-3 ORGIQ-1 ORGIQ-2 ORGIQ-3 ORGIQ-4 ORGIQ-5 ORGIQ-6 ORGIQ-7 PROCM-1 PROCM-2 PROCM-3 PROCM-4 PROCM-5

0.737 0.800 0.750 0.784 0.782 0.706 0.662 0.655 0.592 0.718 0.634 0.741 0.781 0.802 0.717

19.72 21.82 20.13 22.98 22.91 19.82 18.20 17.96 15.81 20.30 17.04 20.96 22.57 23.48 20.04

0.81

0.76

0.88

0.70

0.86

0.74

Table 3: Impact of IT on Organizational Dynamic Capabilities Independent Variable Intercept ECT GCT CCT Human IT Capital Project Controls Project Duration Team Size Team Co-location Propensity to Use IT Adjusted R2

Dependent Variables Effectiveness Efficiency 0.14 (0.31) 0.30*** (0.0001) 0.05 (0.19) 0.23*** (0.0001) 0.25*** (0.0001)

0.15 (0.33) 0.03 (0.31) 0.17*** (0.0001) 0.05 (0.17) 0.08* (0.06)

-0.004 (0.89) -0.01 (0.67) 0.01 (0.61) 0.07** (0.05)

0.03 (0.44) 0.03 (0.37) -0.03 (0.34) 0.07** (0.05)

0.28

0.07

*** indicates p < 0.01, ** indicates p < 0.05, and * indicates p < 0.10.

32

Table 4. Independent Variable

Impact of IT and Dynamic Capabilities on Project Outcomes ∆(CYCLE TIME ) (1)

Intercept

3.37***

ECT

(0.000) 0.34*** (0.000) 0.15*** (0.000) _

GCT

_

CCT

_

Effectiveness Efficiency

Human IT Capital Autonomy Process Maturity Firm Size (Employees)

0.05 (0.172) 0.01 (0.789) 0.11*** (0.001)

0.00 (0.895)

Number of Concurrent Projects Managed Adjusted R2 ∆(Adj. R2) F-statistic#

#

0.02 (0.430) 0.245

(2) 3.39*** (0.000) 0.32*** (0.000) 0.13*** (0.000) 0.05 (0.129) 0.09*** (0.006) 0.05 (0.108) 0.02 (0.496) 0.00 (0.907) 0.09** (0.012) 0.00 (0.997) 0.02 (0.427) 0.254

0.009 3.59** (0.014)

∆(COST) (3)

(4) 3.60*** (0.000) 0.33*** (0.000) 0.12*** (0.000) 0.01 (0.708) 0.06 (0.063) -0.02 (0.505) 0.13*** (0.000) -0.02 (0.597) 0.13*** (0.001) -0.00 (0.816) -0.05 (0.052) 0.276

3.57*** (0.000) 0.33*** (0.000) 0.13*** (0.000) _ _ _

0.14*** (0.000)

-0.01 (0.695) 0.14*** (0.000)

-0.00 (0.994)

-0.05 (0.055) 0.274 0.001 1.39 (0.24)

∆(QUALITY) (5) 3.69*** (0.000) 0.37*** (0.000) 0.10*** (0.001) _ _ _ 0.12*** (0.001) 0.00 (0.940) 0.16*** (0.000) -0.00 (0.825) -0.00 (0.986) 0.315

(6) 3.67*** (0.000) 0.34*** (0.000) 0.09*** (0.004) 0.02 (0.570) 0.02 (0.545) 0.09*** (0.006) 0.10*** (0.003) -0.00 (0.955) 0.15*** (0.000) -0.00 (0.992) -0.00 (0.990) 0.320 0.005 2.64** (0.05)

∆(ON-TIME COMPLETION RATE) (7) (8) 3.57*** 3.59*** (0.000) (0.000) 0.33*** 0.30*** (0.000) (0.000) 0.02 0.01 (0.452) (0.707) _ 0.07** (0.043) _ 0.05 (0.136) _ 0.03 (0.355) 0.18*** 0.16*** (0.000) (0.000) 0.02 0.01 (0.547) (0.606) 0.22*** 0.20*** (0.000) (0.000) -0.00 -0.01 (0.854) (0.740) 0.00 0.00 (0.985) (0.946) 0.317 0.320 0.003 2.03 (0.11)

*** indicates p < 0.01 and ** indicates p < 0.05. p-values are shown in parentheses. The F-test was computed based on the ∆(R2) values of the complete mediation and full models with df1 = 3 and df2 = 617.

Table 5.

Impact of IT and Dynamic Capabilities on Firm Financial Performance

Independent Variable

GROSS MARGIN

ECT

(1) -0.057 (0.057) 0.013** (0.03) -0.007 (0.20) -

GCT

-

CCT

-

Intercept Effectiveness Efficiency

Propensity to use IT Human IT Capital

0.001 (0.89) -

Process Maturity

-0.006 (0.36) 0.000 (0.95) 0.006*** (0.01) 0.891*** (0.000) 0.780

Autonomy Log(Assets) Fin. Perft-1 Adjusted R2

(2) -0.054 (0.074) 0.013* (0.08) -0.008 (0.14) 0.002 (0.69) 0.007 (0.20) -0.002 (0.72) 0.000 (0.98) -0.006 (0.37) -0.008 (0.15) -0.001 (0.88) 0.006** (0.02) 0.881*** (0.000) 0.784

ROA

(3) -0.018 (0.58) 0.007 (0.33) 0.004 (0.46) 0.007 (0.22) 0.001 (0.84) 0.004 (0.51) 0.004 (0.11) 0.343*** (0.000) 0.235

(4) -0.012 (0.71) 0.006 (0.47) 0.005 (0.37) 0.006 (0.37) -0.006 (0.35) -0.002 (0.75) 0.009 (0.14) -0.007 (0.28) 0.000 (0.99) 0.004 (0.49) 0.004 (0.13) 0.346*** (0.000) 0.241

ROE

(5) -0.091 (0.67) -0.002 (0.95) 0.068* (0.09) 0.059 (0.13) 0.008 (0.86) 0.025 (0.49) 0.015 (0.38) 0.773*** (0.000) 0.324

(6) -0.058 (0.79) -0.004 (0.93) 0.083** (0.04) 0.033 (0.45) -0.082* (0.05) -0.010 (0.82) 0.079** (0.05) -0.029 (0.48) -0.003 (0.94) 0.031 (0.40) 0.017 (0.35) 0.767*** (0.000) 0.333

∆(Adj. R2)

0.004

0.006

0.009

F-statistic

0.685

0.664

1.657

p-values are shown in parentheses. *** indicates p < 0.01 and ** indicates p < 0.05.

34

APPENDIX: SURVEY QUESTIONNAIRE6 The objective of this research survey is to develop an understanding of the business processes, management practices and information technologies (IT) in use in your firm. Your responses will help the research team understand the impact of IT and organizational processes on information work productivity. The first section asks for specific information about your role, work characteristics and your firm. In the second section, data is collected about the business impact you have observed. In the third section, data about project performance outcomes are measured. 1. Select the generic role below that best describes your role within your current company: 1=Individual Contributor, 2=Project Manager, 3=Program Manger, 4=Product Manager, 5=Business Manager 2. Is your company…? (Please select only one response.) 0=Publicly traded, 1=Privately held. Provide your company name. 3. FIRM SIZE: How many regular full-time employees are in your company? (Please select only one) 1 = < 100 employees (TERMINATE); 2 = 100 to 249; 3 = 250 to 499; 4 = 500 to 999; 5 = 1,000 to 4,999; 6 = 5,000 to 9,999; 7 = 10,000 to 24,999; 8 = 25,000 or more; 9 = Not Sure 4. NUMBER OF CONCURRENT PROJECTS MANAGED: What is the typical number of projects you manage concurrently (at any one time)? 1 = 1 Program/project; 2 = 2 - 3 Programs/projects; 3 = 4 - 5 Programs/projects 4 = 6 - 10 Programs/projects; 5=More than 10 Programs/projects 5.

PROJECT DURATION: How long does your typical program or project last?) 1 = Less than 1 Year; 2 = 1 - 2 Years; 3 = 3-5 Years; 4 = 6 - 10 Years; 5 = > 10 Years

6. TEAM SIZE: How many people are typically involved in your programs or projects? 1 = 1 - 10 People; 2 = 11 - 20 People; 3 = 21 - 50 People; 4 = 51 - 100; 5 = > 100 People 7.

CO-LOCATION: Describe the extent of co-location of your team. Most of the program/project team is in the: 1 = Same Building; 2 = Same City; 3 = Same Time Zone; 4 = Different Time Zones; 5 = Different Country and Time Zone

8.

PROPENSITY TO USE IT: How would you describe your organization's use of IT? 1 = Late Adopter; 2 = Conservative Adopter; 3 = Mainstream User; 4 = Early User; 5 = Leading Edge User

9.

AUTONOMY: Project/program targets/goals are primarily set by (choose one): 1= Business Unit Head; 2 = Program Sponsor; 3 = Project Manager; 4 = Project Team

10. HUMAN IT CAPITAL Please comment on the following program/project related statements as they relate to your organization. (Please provide a rating for each) 1 = Strongly Disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly Agree i. We benchmark our performance against best-practices from other industries ii. Project personnel are well-trained in the use of work methods, and practices iii. Information technology training is a high priority

6

The question headings (in bold) were not shown on the survey questionnaire and questions were arranged in random order. They are shown here to improve the clarity of the model variables and constructs to the reader.

iv. New or updated methods, practices and Information systems are used to keep project teams in touch with customers v. Information systems are used to support communications between project teams vi. Senior management is involved in championing my programs/projects vii. Senior management provides the necessary financial and human resources to ensure project success [USE THE FOLLOWING SCALE FOR question 11] 1 = Never; 2 = Rarely; 3 = Sometimes; 4 = Frequently; 5 = All the Time 11.

PROCESS MATURITY Please comment on the following program/project related statements as they relate to your organization. i. We use cross-functional project teams ii. Projects are structured to enable knowledge transfer among project teams iii. We have well-defined processes for program/project execution iv. Quantitative targets are used to manage program/ projects v. Metrics are automatically collected for process management vi. Customer feedback is collected, tracked, and used for continuous process improvement

12. Please rate the following IT applications on their degree of usage in your current job/role: 1 = No Use; 2 = Limited Use; 3 = Medium Use; 4 = High Use; 5 = Heavy Use CORE COMMUNICATION TECHNOLOGIES (CCT) i. Email ii. Internet Search Engines and Web Portals iii. Mobile Communication (WLAN, Voice, Cell phones, Wireless laptops) GROUP COLLABORATION TECHNOLOGIES (GCT) i. Instant Messaging Software ii. Mobile Computing (e.g. Tablets, PDA, Blackberry hand-helds) iii. Video-Conferencing Technologies (e.g., Webex) iv. Groupware and Online Teamspaces (e.g. Sharepoint, LiveLink, eRoom) ENTERPRISE COMPUTING TECHNOLOGIES (ECT) i. Enterprise Application Software (ERP, Supply chain management software) ii. Knowledge Management Software iii. Customer Relationship Management Software (CRM, Collaboration Tools) iv. Project Management Software v. Business Intelligence (e.g. SAS, Hyperion, Cognos) vi. Content Management Systems/Project Portals vii. Document Management Solutions/Systems 13.

Please evaluate the impact of IT applications for each statement listed below. Please respond based on what you would consider your typical program/project over the past 12 months. 1 = Significant Negative Effect; 2 = Moderate Negative Effect; 3 = Mild Negative Effect; 4 = No Effect; 5 = Mild Positive Effect; 6 = Moderate Positive Effect; 7 = Significant Positive Effect EFFICIENCY i. Time spent searching and waiting for information ii. Program/project task durations and enabling concurrent work iii. Rework and non-value added tasks

36

EFFECTIVENESS i. Making current program/project information (schedule, goals) visible throughout the team ii. Ease of tracking and monitoring programs/projects throughout the company iii. Identifying and mitigating program/project and business risks iv. Identification and facilitation (management) of qualified suppliers and partners v. Ability to lower procurement cost of goods vi. Re-use of information and corporate knowledge assets across the enterprise vii. Making the program/project fit the firm's strategic direction and project/business portfolio viii. Identification and closure of functional gaps within and across program/project teams ix. Alignment of virtual and distributed program/project teams x. Identifying and disseminating customer needs and requirements to the program/project teams xi. Continual involvement of customers in programs/projects xii. Creating new markets, product enhancements, and business opportunities 14.

PROJECT OUTCOMES Evaluate the change in the following project outcomes in the previous one year (2003 to 2004): 1 = High level of negative benefit; 2 = Some negative benefit; 3 = No positive or negative benefit; 4 = Moderate level of benefit; 5 =High level of benefit i.

Change in project cycle/completion times, defined as the total time elapsed from project initiation to completion. ii. Change in program or project costs, defined as the total program/project cost incurred from initiation to completion (including labor, equipment, services) iii. Change in program or project quality, measured as the total number of errors, defects, and rework associated with the project work * iv. Change in % of programs or projects completed on-time, measured as the deviation between actual and expected project completion time *

37