Benchmarking Business Processes in Software ... - CiteSeerX

0 downloads 0 Views 230KB Size Report
Dec 6, 2001 - Department of Management, Monash University ..... individuals, … or used to allocate blame for performance problems' were not an issue,.
Benchmarking Business Processes in Software Production: a Case Study*

Megan Seen School of Computer Science and Software Engineering Monash University Nicholas Beaumont 61 3 9903 2371 (Phone) 61 3 9903 2718 (Fax) [email protected] Department of Management, Monash University PO Box 197 Caulfield East, Vic 3145 Australia Christine Mingins School of Computer Science and Software Engineering Monash University

*

All correspondence to Dr N. B. Beaumont.

Benchmarking Business Processes: a Case Study Page 1

This printout: 12/06/01

Benchmarking Business Processes in Software Production: a Case Study Abstract Organisations constantly seek improvement of organisational practices or market positions by implementing improvement programs and quality initiatives such as ISO certification. An improvement project is any project that aims to make a business process more efficient, flexible, reliable or responsive; or raise the quality of working life. We note that few organisations make the effort to evaluate the effectiveness of such improvement programmes, explore why this might be so, and describe some of the benefits of undertaking evaluation activities. We present a methodology for selecting Key Process Indicators most appropriate to selected business processes and assessing an improvement program, and describe the methodology’s implementation in a small Australian software company. We stress that measurement of performance before and after ostensible improvements is needed for objective assessment of change processes. Keywords: Benchmarking, small business, quality systems, monitoring, implementation, Australia. INTRODUCTION Background Organisations undertake change to improve internal process or are impelled by market forces (exemplified by customers deciding to purchase only from ISO certified suppliers (Struebing, 1996; Weston, 1995)). When business is tough, there may be pressure to focus on the core activities of the business and the improvement project itself rather than spend resources on apparently secondary activities such as evaluation of completed projects (Gray, 1998). This may be especially true in small organisations where every activity must show a return on investment, managerial time is a very scarce resource, and cash flow constraints limit spending on activities that do not contribute directly to profit. Too little emphasis has been placed on measuring the changes brought about by implementing an improvement initiative. Our industry contacts suggest that, despite the benefits of doing so, few organisations conscientiously measure performance both before and after process improvements are implemented. The literature on benchmarking and other change processes seems to neglect, or at best leave implicit, the issue of measuring prior and post performances. Zairi (1994, pp 52-63) summarises benchmarking methodologies used by 14 organisations. Few explicitly mention measurement of existing operations (baselining) or post implementation reviews of improvement projects. Exceptionally, Owen and Rogers (1999) give several different approaches to evaluating the merit of proposed changes and effects of already implemented programs. These range from strategic (Does a proposal support strategy?), through effectiveness (Does the implemented proposal do something useful?) to efficiency (Does the implemented proposal use minimal resources?). Part of the problem is that quantitative performance indicators may not perfectly reflect performance. It is hard to unambiguously trace the effects of an internal change or initiative on customers who react to a “basket” of changes, not individual changes. Customers’ opinions and needs are not homogeneous; and, in the short-term, improvements such as Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

product reliability may not be noticed by customers. It is difficult to measure changes in intangible variables (such as customer satisfaction) reliably (Merget and Weaver, 1999). The process of identifying and developing appropriate performance indicators is therefore problematic. There are a number of benefits associated with performing evaluation activities that may have been overlooked: •

Pre and post measurement and analysis of Key Performance Indicators (KPIs) identify the most effective components of an improvement process. Where improvements have eventuated, regular monitoring can help routinise them and make them part of the company’s operating procedures. Even when improvements have not materialised, data gathered can be input to diagnostic methodologies such as the Deming ‘Plan-Do-StudyAct’ cycle (Evans and Lindsay, 1996).



Establishing and monitoring KPIs can be politically useful. Organisational change can be protracted and painful and ISO certification is usually a long and expensive process. Documented improvements in important aspects of the organisation’s performance can help top managers resist pressure to slow down or abandon change programs. Cynics often opine that ‘nothing ever really changes’; that ‘Wheels spin, reams of paper pile up, and the work goes on in spite of the evaluation process and almost unaffected by it’ (Gray, 1998: xv). Winston (1991) alludes to the scant evidence for the successful application of performance indicators in aspects of government. Documentary evidence of improvement can overcome cynicism.

Selecting appropriate KPIs can illuminate other, sometimes endemic, problems. For manufacturers, emphasising low prices implies long production runs and little product variety whereas emphasising flexibility and customer service implies short production runs and much product variety. Although such conflicts ought to have been resolved when implementing the organization’s strategic plan, the development of an appropriate set of KPIs forces such conflicts (perhaps already manifest in tension between different departments) to be addressed. If ostensibly intangible variables such as customer satisfaction are important, efforts should be made to measure them by using surveys or recording sales peoples’ experiences. Research Objectives and Methodology It is possible to identify and measure KPIs that focus on key processes or outcomes without creating excessive paperwork. One such methodology was introduced at an Australian software development company as part of a larger action research project that is investigating the role of information technology in facilitating quality systems management (QSM). The objectives of the main project included evaluating the changes wrought by introducing of ITsupported QSM. Establishing a process that could measure the effects of this type of improvement project was therefore a key objective. The project’s first cycle involved the researcher (the first author) helping to design and implement a process to help Paradigm Software Pty. Ltd., in which the first case study took place, gain ISO certification using QSM software (Seen and Mingins, 2000). This paper describes Paradigm, gives a conceptual framework for benchmarking, describes and analyses the methodology and process used to establish the measuring system at Paradigm, gives an interim evaluation of the continuing project and suggests further research. BACKGROUND The Company Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

Paradigm is a small software development company based in Melbourne, Australia. It sells software supporting quality management processes in Australia and overseas. The first version of its principal software product Paradigm Quality was designed to help organisations manage information pertaining to customers, suppliers, personnel, quality, equipment, documents and images, workflows, scheduling, audits, problem tracking and resolution, and meetings. Paradigm Quality was described as ‘quality management software’. Its ability to help manage integrated business activities (rather like an Enterprise Resource System) is now emphasised, especially in the updated product Paradigm II. In Paradigm itself, Paradigm Quality was used for a number of separate and isolated business functions. An objective was for Paradigm Quality to replace several internal processes used to make critical decisions, provide an integrated business management system, and thereby facilitate Paradigm’s ISO certification. Motivation for undertaking ISO 9001 certification Paradigm had two motives for seeking ISO 9001 certification: •

After a period of restructuring, the managing director wanted to re-focus the organisation; adopting ISO 9001 would force systematic review and improvement of the organisation’s practices. ISO 9001 certification was a qualifier for some overseas customers. It was embarrassing for a company producing software to help organisations administer their businesses not to itself have a formal business management system.



Paradigm had no objective evidence that Paradigm Quality was useful. It relied on commonsense arguments (e.g. better record keeping must generate improvement) and customers’ anecdotes. It was thought likely that the outcomes from implementing ISO 9000 in-house using Paradigm Quality would be positive and could be used to market Paradigm Quality, and that any negative outcomes could be used to improve Paradigm Quality or implementation processes. Having readily available quantifiable data that reflected organisational performance – especially in nebulous areas such as software development and customer interaction – appealed to the managing director (MD).

WHAT IS BEST PRACTICE? Relatively few writers have defined Best Practice (BP), perhaps because the term's meaning is ostensibly obvious. Almost all writers define BP in terms of components (summarized in Table 1). A more fundamental examination of BP entails examining its definition, its implementation and criteria for assessing BP programs. The following discussion draws especially on Fitz-enz (1993).

Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

Table I Components of Best Practice Component A focus on continuous, improvement in cost, quality and delivery. Closer links with suppliers and customers. Effective use of technology for strategic advantage. Flatter organisation structures, greater flexibility and preparedness to adopt structures and processes to changed circumstances. Human resource policies promoting continuous learning, teamwork, participation and flexibility. Comparison of one's performance with similar firms. Managerial leadership

Authors (Australian Department of Industrial Relations and Australian Manufacturing Council, 1992; Australian Manufacturing Council, 1994; Davis, 1995; Dertouzos, 1990; Ewer, 1993; Golovin, 1996; Griffin, Gleason et al., 1995; Taninecz, 1997). (Australian Department of Industrial Relations and Australian Manufacturing Council, 1992; Davis, 1995; Dertouzos, 1990; Ewer, 1993; Griffin, Gleason et al., 1995; Taninecz, 1997) (Australian Department of Industrial Relations and Australian Manufacturing Council, 1992; Davis, 1995; Dertouzos, 1990; Ewer, 1993; Taninecz, 1997) (Australian Department of Industrial Relations and Australian Manufacturing Council, 1992; Davis, 1995; Dertouzos, 1990; Ewer, 1993).

(Australian Department of Industrial Relations and Australian Manufacturing Council, 1992; Australian Manufacturing Council, 1994; Davis, 1995; Dertouzos, 1990; Ewer, 1993; Griffin, Gleason et al., 1995; Taninecz, 1997). (Davis, 1995; Voss, 1995). (Australian Department of Industrial Relations and Australian Manufacturing Council, 1992; Australian Manufacturing Council, 1994).

Fitz-enz notes that BP is not unique, and that different practices can work well in different contexts, cultures and organizations (Thompson, 1967). There are many kinds of work; manual work is usually easier to measure than intellectual work or work involving people. Creating quality software is a difficult and demanding process. Amongst the difficulties are: communications amongst users and project team members; a chronic shortage of resources; estimation and scheduling and the intrinsic difficulties of programming, testing and implementation. There is a tendency, especially in popular literature, to ignore stake-holding issues and to assume that the interests of the firm and workers are at least similar; Wright (1996) provides an interesting counter-example. Most writers stress tangible measures but intangible measures may be very important (the development of pervasive quality programs or long term relationships with customers may be valuable long-term investments). Financial measures of performance may be distorted by inappropriate transfer prices or the way overheads or depreciation are distributed amongst time periods or departments. Assessment of a unit's performance may be affected by external and transitory factors (about half the "excellent" companies identified in Waterman (1982) have since suffered major reversals) or simply luck. Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

These problems create methodological difficulties: some astonishing "proofs" of BP are based on small samples (Fitz-enz, 1993). An improvement in results following the implementation of BP does not prove that the latter caused the former. A firm may be consistently successful, not because it uses BP, but because it has intrinsic advantages such as economies of scale, some kind of monopoly power or effective management. A typical definition of BP is: Best Practice is the co-operative way in which firms and their employees undertake business activities in all key processes: leadership, planning, customers, suppliers, community relations, production and supply of products and services, and the use of benchmarking. These practices, when effectively linked together, can be expected to lead to sustainable world-class outcomes in quality and customer service, flexibility, timeliness, innovation, cost and competitiveness” (Australian Manufacturing Council, 1994). There can be no universally agreed operational definition of BP; every organization will operationalise BP to suit its own circumstances and preconceptions. Defining BP simply as a list of components is unsatisfying, but the elements can be classified in various ways: e.g. as primarily internal and external (relations with customers and suppliers); as direct and indirect (training) or as pertaining to people or processes. We opine that BP is best treated as a mechanism for expressing and implementing a strategy. Motivated by the strategic context, we identify three aspects of BP: Operational Best Practice: optimises production of goods and services. Internal Best Practice: optimises structure, staffing, systems and culture so that strategy is optimally expressed. If, for example, rapid fulfilment is emphasised, customers will be able to order goods and services through web pages, these orders will be made available to factory schedulers instantly, and there will be sophisticated scheduling and order tracking systems and flexible production systems. External Best Practice: (a) optimises relations with and from external parties, especially customers and suppliers but also with legislators, regulators, communities and labour; (b) obtains required resources (e.g. raw materials and labour) on the best possible terms and conditions; and (c) sells finished goods on the best possible terms and conditions. Operational BP methodologies should be applied to software creation. As noted in the case study, Paradigm opined that applying BP methodologies to software creation was premature; it was better to develop internal best practices. The emphasis was on ascertaining internal and external customers’ requirements; internal best practice methodologies were therefore appropriate. THE CASE STUDY Establishing Benchmark Measures Before the process of formalising the business management system began, Paradigm had established benchmark measures in three key areas: customer satisfaction, organisational culture and internal operations. The current performance in the first two areas was measured using surveys and interviews based on established instruments: Likert’s (1967) organisational culture survey and Rouse, Watson and Jian-Xiang’s Softqual (1997), a survey based on Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

ServQual (Parasuraman, Zeithaml et al., 1988) designed to measure customer satisfaction in the software development industry. Using the Key Performance Indicators Methodology Initiating Between April and May 1999 Baker’s (1995) methodology (see Table II) was used to help identify the most relevant KPIs. Identifying and using KPIs was a necessary preliminary to systematic and continuous measurement of improvement and performance. This process (as explained at a staff meeting) would cumulate in a more formal business management system and ISO 9001 certification. Staff feedback was invited, Baker’s (1995: 31) concerns that ‘Employees are often concerned that performance information will be collected on individuals, … or used to allocate blame for performance problems’ were not an issue, because the company enjoyed a culture of trust and open communication. Table II: A methodology for identifying, implementing and monitoring KPIs (Baker, 1995) Initiating

Building Implementing Reviewing

1 2 3 4 5 6 7 8

Aligning KPI development with other change and improvement strategies Explaining to staff the purpose of the development and use of KPIs Establishing an agreed process for KPI development and use Identifying critical success factors for the organisation Selecting KPIs (done at a team level) Developing display, report and review frameworks at all levels Facilitating the use of KPIs to assist performance improvement (Best Practice) Refining and modifying KPIs to maintain their relevance

Building The next stage (4) involved the MD and researcher identifying Critical Success Factors (CSFs) for customer service and satisfaction, financial performance, human resource development, workforce contribution and involvement, key internal processes, and innovation, learning and growth. Seven CSFs critical to Paradigm were identified: • • • • • • •

Customer satisfaction (for new and repeat business) Profitability for long-term survival A harmonious culture Committed, professional staff Research and development (improvements in old products and creation of new products) Improvement in processes Commercialisation of new products, concepts and ideas

Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

Stage 5 (selecting the KPIs best reflecting each CSF) was laborious; it entailed identifying and prioritising KPIs measuring each business processes that might affect a CSF. The MD and researcher identified the 31 processes listed in the third column of Table III. Because monitoring all 31 processes was impractical, each process’s importance was assessed by asking the questions (Baker, 1995) listed below. Answers (tick or cross) were recorded in columns 4 to 8.

Benchmarking Business Processes: a Case Study Page 9

This printout: 12/06/01

Table III: Paradigm’s processes and activities that affect critical success factors Aspect of CSFs Organisational Performance

Customer focus

Financial

People

Customer satisfaction and repeat business

Customer satisfaction and new business Profitability for long-term survival

A harmonious culture

Committed and professional staff

Processes/Activities that affect CSFs

Process directly affects CSF

Weights Technical support Enhancement inclusion Requirements elicitation User group meetings Training Invoices and renewal Communication – correspondence/phone Distributors well trained Advertising, site visits, presentations, referrals Long-term view of managing resources Cashflow management Managing operating costs Developing business strategies Recruiting Participative management style Ethics/morals Staff reviews Recognition/reward Skills maximization

5 D D D D D U

3 D D D U U U

2 D D U D D U

D

D

D

Benchmarking Business Processes: a Case Study Page 15

Process Process involves occupies cost resources blowouts

Process is problematic

Weighted Total

Ranking

5 U D ? U U D

Process affects ability to do work 3 D D U U U U

13 18 10 7 7 5

4 1 5 9 9 11

D

D

D

18

1

D

D

D

U

15

2

D

U

D

U

U

7

9

D

D

D

D

U

15

2

D D

U D

D D

U D

U U

7 15

9 2

D

D

D

U

U

10

5

D

D

D

U

U

10

5

D

U

D

U

D

10

5

D D D D

U U ? U

U D U U

U U U U

D U D D

8 7 9 8

7 9 6 7

This printout: 12/06/01

Aspect of CSFs Organisational Performance

Innovation/ Process improvement

Research and development Improvements in old products Creation of new products

Improvement in processes

Commercialisation of new products, concepts and ideas

Processes/Activities that affect CSFs

Process directly affects CSF

Weights Work systems & implementation. User group feedback Customer feedback Understanding IT trends Capture of customer expectations User guide updates* Regular management reviews, meetings brainstorming/planning Keeping in touch with changing market Good communication Identification of root problems Weekly meetings with staff Converting ideas to reality through programming* Project management Marketing – packaging, positioning; assessing, choosing, training, and motivating distributors

5

3

2

D

D

D D D

Benchmarking Business Processes: a Case Study Page 14

Process Process involves occupies cost resources blowouts

Process is problematic

Weighted Total

Ranking

5

Process affects ability to do work 3

D

U

D

13

4

D D D

D D U

U U D

U U U

10 10 13

5 5 4

D

D

D

U

U

10

5

U

U

D

D

U

7

9

D

D

D

U

U

10

5

U

D

D

U

D

8

7

D

U

U

?

D

7

9

D

U

D

U

D

10

5

D

U

D

U

D

10

5

D

D

D

D

D

18

1

D

D

D

D

D

18

1

D

D

?

D

U

14

3

This printout: 12/06/01

Aspect of CSFs Organisational Performance

Processes/Activities that affect CSFs

Process directly affects CSF

Process Process involves occupies cost resources blowouts

Weights Production of CDs/kits*

5

3

2

D

D

D

* Denotes activity affects more than one CSF – usually also customer satisfaction

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Process is problematic

Weighted Total

Ranking

5

Process affects ability to do work 3

D

U

15

2

• • • • •

What products or services are directly and materially affected by this process? If this process is not done well, will it tend to cause subsequent cost blowouts? Does this process require many resources? Is this process problematic or not under control? Does this process adversely affect staff morale and staff ability to do required work?

A useful innovation was weighting the above factors to obtain a composite measure of the importance of and hence a ranking of each process (see Table III columns 9 and 10). The company’s staff were given the results of the process and asked to provide feedback; communication activities were consequently added. The availability of the data needed to measure the effectiveness of the identified activities was next considered. The questions in the first row of Table IV were asked of each identified process. Answers, reported in Table IV Availability of Data, helped determine the final selection of KPIs to be implemented. The final step entailed working out how to obtain data not already available (this often related to intangible aspects of operations, such as the effectiveness of software development processes).

Benchmarking Business Processes: a Case Study Page 15

This printout: 12/06/01

Table IV Availability of Data Key Activity

Potential KPIs

Enhancement inclusion

Length of time to develop enhancement Number of complaints after release Number of technical support requests after release Length of time from request to release Number of requests for modification

Software development process

Software development process continued

External

Ask questions in BRM re SD process Number of post version release fixes Customer

Is the data available already?

How much effort will it take to collect the data? Yes – R&D Already Task sheet collected

Collection frequency

Is it reliable and accurate?

On completio n Ongoing

What would be done with this data once it was collected?

Yes

Who will Is time or collect it? resources available to collect data? MD Yes

Yes

TM

Yes

AO/TW to collate in Excel weekly; R to analyse monthly; review at BRM AO/TW to collate in Excel weekly; R to analyse monthly; review at BRM

R to analyse; review at BRM

Yes – Help Desk Query form Yes – Help Desk Query form

Already collected Already collected

Ongoing

Yes

TM

Yes

Yes – Help Desk Query form Yes – Enhancement Request form No

Not much

Ongoing

(1)

MD

Yes

R to analyse; review at BRM

Already collected

Ongoing

Yes

TM

Yes

AO/TW to collate in Excel weekly; R to analyse monthly; review at BRM

Get info from BRM

Monthly

Based on perception s

Yes

R to analyse monthly; review at BRM

Yes -HD Already Query form collected

Ongoing

Yes

MS, MD & developm ent staff TM

Yes

AOO/T to collate in Excel weekly; review at BRM

No

Monthly

Based on

MS, TM

Yes

AO/TW to collate; TW to

Benchmarking Business Processes: a Case Study Page 15

Unclear

This printout: 12/06/01

Key Activity

Potential KPIs

communicati on Project management

satisfaction

Is the data available already?

How much effort will it take to collect the data?

Estimated time to No complete project compared to actual

Collection frequency

Is it reliable and accurate? perception s Yes

Who will Is time or collect it? resources available to collect data? & MD

What would be done with this data once it was collected? analyse; review at BRM

Moderate: Ongoing SD staff; Yes AO/TW to collate in Access; analysis of MD MD to analyse; review at Project Time BRM Sheet MD – Managing Director; TM – Technical Manager; AO – Administrative Office; TW – Technical Writer; SD – Software Development; R – Researcher; BRM - Business Review Meeting Notes: (1) Low priority requests not dealt with promptly may skew results.

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Table V: KPIs selected as a result of the process Key Activity Enhancement inclusion Software development process

External communication with customers Project management

Key Performance Indicator Length of time to develop enhancement Number of complaints after release Number of technical support requests after release Length of time from request to release Number of requests for modification Ask questions in business review meeting – relate to steps in software development process* Number of post version release fixes Ask communication question in BR meetings re user group meeting, newsletter, individual clients Estimated time to complete project compared to actual time

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Table VI Updated Organisational Performance Indicators δ (3)

Customer Focus 1 Customer satisfaction and repeat business.

1 Technical support

3

2

5

3

13

A

2 Post release Development inclusion 3 User group meetings 4 Training 5 Communication correspondence and telephone 1 Distributors well trained and informed

3

2

5

0

10

B

0 0 3

2 2 2

0 5 5

0 3 3

2 10 13

G B A

3

2

5

0

10

B

2 Advertising, promotion 3 3 Site visits, 3 presentations, referrals 4 Product education 0 0 1 Profitability for 1 Long-term view of long term survival managing resources 2 Cashflow management 0 3 Managing operating 0 costs 4 Developing business 3 strategies 5 Invoices and renewal 0 3 1 A harmonious 1 Recruiting culture 2 Participative 0 management style 3 Ethics/morals 0 0 2 Committed and 1 Staff reviews professional staff 2 Recognition/reward 0 3 Skills maximization 0 4 Work systems and 0 implementation

2 2

5 0

0 0

10 5

B E

2 2

5 5

0 0

7 7

D D

2 2

0 5

0 0

2 7

G D

2

5

0

10

B

2 2

0 5

0 3

2 13

G A

2

0

3

5

E

0 2

0 0

3 0

3 2

F G

0 0 2

5 0 5

3 3 3

8 3 10

C F B

2 Customer satisfaction and new business

Financial

People

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Focus

α (3) β (2) γ(5)

Total

Aspect of Paradigm’s Processes/Activities Organisational Critical Success that affect CSFs Performance Factors

Table VI (Continued) Updated Organisational Performance Indicators δ (3)

Innovation and 1 R&D Process Improvements in Improvement existing products 2 R&D creation of new products

0 0

2 2

0 5

0 0

2 7

G D

1 Incorporating IT 3 trends 2 Capture of customer 3 expectations 3 User Meeting updates* 0 4 Regular management 0 reviews, meetings, brainstorming and planning 1 Keeping in touch with 3 3 Internal changing market processes 2 Identification of root 3 problems 3 Staff: interactions and 0 consultations with staff 3 4 Commercialisat 1 Converting ideas to reality through ion of new programming* products, 2 Project management 3 concepts and 3 Marketing - packaging, 3 ideas positioning; assessing, choosing, training, and motivating distributors 4 Production of CDs and 0 kits*

2

5

0

10

B

2

5

0

10

B

2 2

0 5

0 3

2 10

G B

2

5

0

10

B

2

5

3

13

A

2

5

3

10

B

2

5

3

13

A

2 2

5 5

3 0

13 10

A B

2

0

0

2

G

1 User group feedback 2 Customer feedback

 * = activity affects more than one CSF - usually also customer satisfaction α = Process can involve cost blowouts β = process occupies resources γ = Process is problematic δ = Process affects staff's ability to do work. Implementation The next step was to implement the KPIs, ‘…the value of KPIs is almost completely derived from their consistent use in assisting performance improvement’ (Baker, 1995: 47). Existing data pertaining to calls to the Help Desk and other historical data from paper records were transferred to Excel spreadsheets. Each week’s data was subsequently collected and entered. Summaries of performance data could be viewed in numeric or graphical format. Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Focus

α (3) β (2) γ(5)

Total

Aspect of Paradigm’s Processes/Activities Organisational Critical Success that affect CSFs Performance Factors

In June 1999 the first Business Review Meeting (BRM), comprising the MD, technical manager, and researcher was held. A standard agenda, driven by KPIs, was established. Thereafter, BRMs were held every six to eight weeks. Attendees reviewed the data presented for each of the KPIs, compared current data with data previously presented, interpreted findings, sought the causes of any unusual results, and decided whether action was necessary. Step 7 of the methodology entails the use of KPIs to help improve performance and achieve Best Practice. KPIs should become increasingly integrated into ‘…broader organisational improvement initiatives such as strategic planning, enterprise bargaining, benchmarking, and associated workplace change strategies.’ (Baker, 1995). Reviewing The final step (8) of the Baker methodology comprises review and possible modification of the selection and use of KPIs to better track changing business objectives. Several adjustments were made between May and October 2000. The MD requested more information and different analyses of data (exemplified by a finer analysis of calls to the Help Desk); new analyses and graphs were developed. A review of the choice of KPIs themselves was scheduled as an audit item reminding top management to analyse annually the relevance of the KPIs to the business’ CSFs. Analysis of implementation Gray (1998: xvii) lists five characteristics underpinning the successful implementation of evaluation programmes: •

Asking cogent questions, gathering and sharing relevant information and making informed decisions. Assessing progress and changing through continual organizational learning.



Involvement of all stakeholders.The creation of trust.



Using simple, cost-effective, user-friendly evaluation methods that can be adapted to meet the organization’s particular needs and idiosyncrasies.

The project at Paradigm partly exemplified each of these characteristics. Questions were designed to extract information relevant to the CSFs of the business. Regular meetings were used to assess and communicate progress. Staff were not inhibited when proposing new ideas. The process of selecting appropriate KPIs and developing relevant measurements was simple, inclusive and cost-effective. Neither the process of identifying KPIs nor generating requisite data had a substantial impact on staff workloads; most data needed was already being generated. KPIs as catalysts for improvement The BRMs became less important than anticipated because: •

The managing director made decisions and took action on a daily basis.

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Consequently, the BRM’s emphasis changed to reviewing, not making, decisions and ensuring that “important but not urgent” issues were considered. •

The release of a new version of Paradigm Quality (named Paradigm II) was imminent and resources were devoted to developing and testing it. Minimal maintenance was done on the to-be-superseded version. The strategy for Paradigm II will be very different: BRMs will be more frequent and the KPI data relating to the new product will be used proactively to help prioritise post-release support of the new version. (Refer to postscript)

CONCLUSIONS AND FURTHER RESEARCH Conclusions After Paradigm’s CSFs had been identified, Baker’s eight-step methodology enabled KPIs and associated measuring systems to be selected and implemented straightforwardly within this small and entrepreneurial software company. Paradigm’s experience shows that the process described in this paper can be highly effective in helping an organisation select relevant KPIs and that it is possible to overcome some of the barriers associated with implementing a quality measurement system. The methodology was simple, robust and flexible; weighing criteria helped rank KPIs, and staff feedback on the set of KPIs initially nominated by management was fruitful. Most measures used extant data that was input to and stored in Excel spreadsheets. A modest programming effort yielded analyses and graphical views of the data relevant to management and line staff. Regular formal management meetings at which KPIs were reviewed demonstrated to staff the importance management placed on the KPIs and supporting systems. At present, the KPIs enables the company to quantify the effectiveness of individual work processes; in future, the KPIs and supporting systems will also be used to measure the impact of implementing ISO 9001 and formalising the organisation’s business management system. Further Research Most Best Practice literature pertains to manufacturing. This emphasis is outdated. In modern economies, less than 20% of GNP is attributable to manufacturing; most people are employed in service industries. Studying BP in services is more difficult: the criteria relevant to services are more numerous and less tangible. Software production is ostensibly manufacturing but has peculiarities: The implementation of software packages such as Paradigm II can have pervasive effects on business processes and jobs; the criteria for quality software and the software creation process are different from those pertaining to most manufactured products and manufacturing processes. The production of good software is notoriously difficult. In the best of circumstances designing, writing and implementing software is a task demanding unremitting effort and intellectual skill. Circumstances are rarely the best: users may not make their requirements clear and may make conflicting or impossible requests; users may fear the change that a new computer system will impel; business requirements are constantly changing and project champions may quit. Software project managers often face notoriously difficult estimation problems and unrealistic deadlines with Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

inadequate resources. Benchmarking should be used to establish standards for some kinds of computing tasks. Although each software project is unique, all have common components on which the performance of different groups could be prepared. For example, the emergence of standard times and demonstrably best methods for some programming tasks would greatly enhance software project scheduling. A particular feature of software production is that potential users should be intimately involved in its design. Software is never delivered to the purchaser in finished form; in almost all cases there is a cycle of delivery. Version 1 of the software is delivered, its inevitable bugs are gradually fixed. Customers’ requests for changes and enhancements accumulate and the supplier releases Version 2. The relationship between the supplier and customers is not arm’s length but is co-operative (the customers involuntarily undertake some of the testing work that cannot be done inhouse). Best Practice techniques should be extended to recognise this relationship and contribute to an explicit recognition of each party’s costs. This paper has described one software development company’s efforts to create a framework that measures the outcomes of an improvement initiative. Its experience has shown that criteria for establishing BP can be applied to service industries despite their less clear-cut criteria. POSTSCRIPT Since this article was written, there have been several significant changes at Paradigm Software. In June 2000, Paradigm replaced Paradigm Quality with Paradigm II (a new business management package incorporating many major changes) and incorporated the KPI outcomes and its unique Business Risk Management Rating System into its ISO9001 management system. The KPI's initially implemented (see Table V) have been reviewed and revised as shown in Table VI, a consequence of continuous adjustments accommodating customers’ changing requirements. It was simultaneously decided that the phrase Organisational Performance Indicator (OPI) was more meaningful to staff than and would replace the phrase Key Performance Indicator (KPI). Notice that the last column of Table VI is headed Focus (stressing continuing attention) rather than Priority. On 23 April 2001 Paradigm Software was independently audited and certified as having attained ISO9001: 2000 standard. This was the culmination of Paradigm’s two-year internal implementation of quality management systems. Points not brought out in the narrative are the use of the balanced score card approach and the integration of OPIs into daily operations, for example, business review meetings are organised around OPIs and implementation of decisions throughout the firm is carefully monitored. ISO9001: 2000 encourages a top down management style and action is driven by Paradigm’s objectives and strategies. Paradigm expresses its objectives and strategies in all aspects of its operations by monitoring and improving OPIs. Paradigm’s management firmly believe that its systems and techniques are key factors Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

to the company being able to develop new and proven software quickly despite having only a small software development team. Two critical success factors are being able to quickly test and (possibly) implement customers’ suggestions (or fix their bugs) and running the system in-house (a useful test bed and expression of confidence). Management opines that it is a leader in its speciality, an opinion confirmed by the ISO auditor’s comments. Certainly, Paradigm has sold Paradigm II to some household names. Acknowledgements The authors would very much like to thank the staff of Paradigm Software Pty. Ltd. www.paradigmsoftware.com.au, especially managing director Mr Peter Meinig, for their help and co-operation in our undertaking this research and writing this paper. This project exemplifies the benefits of academics working in close co-operation with industry.

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

REFERENCES Australian Department of Industrial Relations and Australian Manufacturing Council (1992), International best practice : report of the Overseas Study Mission, February 1992, Dept. of Industrial Relations, Canberra. Australian Manufacturing Council (1994), Leading the way: a study of best manufacturing practices in Australia and New Zealand, Australian Manufacturing Council., Melbourne. Baker, T., Ed. (1995). Key performance indicators manual: a practical guide for the best practice development, implementation and use of KPIs. Pitman Publishing, South Melbourne. Davis, F. E. (1995), Best practice awareness: workplace reform, quality management, getting started in quality, Rollocorp, Cabarita, N.S.W. Dertouzos, M. L. (1990), Made in America : regaining the productivity edge, Harper Perennial, New York. Evans, J. and Lindsay, W. (1996), The Management and Control of Quality, West Publishing Company. Ewer, P. (1993), International best practice? : a critical guide, Pluto Press, Leichhardt, N.S.W. Fitz-enz, J. (1993) "The truth about "best practice"”, Human Resource Planning, Vol. 16 No. 3, pp. 19-26. Golovin, J. (1996) "'Best practices' makes perfect”, Manufacturing Systems, Vol. 14 No. 4, pp. 74-75. Gray, S. T. (1998), Evaluation with Power: A New Approach to Organizational Effectiveness, Empowerment, and Excellence, Jossey-Bass Publishers, San Francisco. Griffin, A., Gleason, G., Preiss, R., et al. (1995) "Best practice for customer satisfaction in manufacturing firms”, Sloan Management Review, Vol. 36 No. 2, pp. 87-98. Likert, R. (1967), The Human Organisation: Its Management and Value, McGrawHill Book Company, New York. Merget, A. and Weaver, E. (1999), Organizational Behavior and Policy, in S. Gray ( ed.), Evaluation with Power, Jossey-Bass, San Francisco, pp 63-73. Owen, J. and Rogers, P. (1999), Program Evaluation: Forms and Approaches, Allen and Unwin, Sydney. Parasuraman, A., Zeithaml, V. A. and Berry, L. L. (1988) "SERVQUAL: A Multiple Item Scale for Measuring Customer Perceptions of Service Quality”, Journal of Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01

Retailing, Vol. 64 No. pp. pp.12-40. Rouse, A., Watson, D. and Jian-Xiang, X. (1997) Development of Softqual: A Scale to Measure the Service Quality of Software Package Vendors. 8th Australasian Conference on Information Systems, Adelaide, University of South Australia. Seen, M. and Mingins, C. (2000), A Groupware Case Study: Process and Product to Implement a Business Management System, Monash University, Caulfield East. Struebing, L. (1996) "Survey Finds ISO Registration is Market Driven”, Quality Progress, Vol. 29 No. 3, p. 23. Taninecz, G. (1997) "Best practices & performances”, Industry Week, Vol. 246 No. 22, pp. 28-43. Thompson, J. D. (1967), Organizations in action: social science bases of administrative theory, McGraw-Hill,, New York. Voss, C. A. (1995) "Alternative paradigms for manufacturing strategy”, International Journal of Operations & Production Management, Vol. 15 No. 4, pp. 5-16. Waterman, R. H. and Peters, T. (1982), In search of excellence: Lessons from America’s best run companies, Harper and Row, New York. Weston, F. (1995) "What do Managers Really Think of the ISO 9000 Registration Process?”, Quality Progress, Vol. Oct No. pp. 67-73. Winston, J. (1991) Linking Evaluation and Performance Management. Conference of the Australasian Evaluation Society, Adelaide. Wright, C. and Lund, J. (1996) "Best-Practice Taylorism: 'Yankee Speed-Up' in Australian Grocery Distribution”, Journal of Industrial Relations, Vol. 38 No. 2, pp. 196-212. Zairi, M. and Leonard, P. (1994), Practical benchmarking : the complete guide, Chapman & Hall, London.

Benchmarking Business Processes: a Case Study Page 14

This printout: 12/06/01