Protocol for Software Process Improvement of Small & Medium Web ...

Download PDF
10 downloads 132 Views 518KB Size Report
Comment
Protocol for Software Process Improvement of Small &. Medium Web ..... 3.3.4 Conferences: ✍ International Conference on Software Engineering (ICSE).
Protocol for Software Process Improvement of Small & Medium Web Development Organizations Muhammad Sulayman

Table of Contents 1. Rationale of the Research ......................................................................................... 3 2. Research Questions ................................................................................................... 4 3. Identification of Relevant Literature ........................................................................ 5 3.1 Searching Procedure .............................................................................................. 5 3.2 Searching Stages ................................................................................................... 7 3.3 Primary Search Phase ............................................................................................ 7 3.3.1 Online Databases ................................................................................................ 7 3.3.2 Online Search Engines.................................................................................... 7 3.3.3 Individual journals.......................................................................................... 7 3.3.4 Conferences:................................................................................................... 8 3.3.5 PhD Dissertation and Theses .......................................................................... 8 3.4 Secondary Search Phase ........................................................................................ 9 3.5 Search Process Record Keeping ............................................................................. 9 4. Study Quality Assessment Criteria ........................................................................... 9 4.1 Inclusion/Exclusion Criteria .................................................................................. 9 4.2 Hierarchy of Evidence ......................................................................................... 10 5. Study Quality Assessment Checklist....................................................................... 10 5.1 Checklist for Quantitative studies ........................................................................ 10 5.2 Checklist for Qualitative studies .......................................................................... 11 6. Data Extraction ....................................................................................................... 11 7. Data Synthesis ......................................................................................................... 14 7.1 Question 1 ......................................................................................................... 104 7.2 Question 2 ......................................................................................................... 115 7.3 Question 3 ......................................................................................................... 115 7.4 Question 4 ......................................................................................................... 106 7.5 Question 5 ......................................................................................................... 116 8. Systematic Literature Review Schedule ............................................................... 166 References.................................................................................................................... 20

2

1. Rationale of the Research Software processes play an important role, in helping project teams in large organizations use similar and sound practices. Ideally, these processes should combine the need for rigor and discipline with the need for flexibility and creativity, but that balance is hard to achieve [1]. Formal processes emphasize the explicit command-and-control side of the organization due to concrete nature, while informal team practices emphasize the mutual adjustment and explorations needed to accomplish tasks successfully. [2] [3] Almost all Modern software organizations operate in a competitive market, under tight time and cost constraints [4]. As an answer to this modern era’s needs, organizations have started to undertake software process improvement (SPI) initiatives (see [5] for an overview of different approaches) aimed at increasing the maturity and quality of their software processes [6]. Investment in process improvement has had various business benefits i.e. to improve the product quality, reduce time to market, result in better productivity [6], to increase organizational flexibility and customer satisfaction [7] [8]. Many researchers (e.g. [9]) are focusing their attention to define the process and its relation to the quality of the products [10]. While this remains important, many researchers have been exploring the people issues that inherently play major roles in adoption of new processes by software organizations [8]. It is observed that in recent years small and medium sized software development companies have emerged very swiftly and they are mostly working in the domain of web development [11]. Web systems engineering is different from traditional software development and it requires agile process models, so the development methodologies are very different [12][13]. There is also a very strong focus on rapid application development and agility concepts in web software process [14][15][16]. The methodology for development is tried to be user centered due to rapid change of content and flexible nature [17]. The purpose of this research will be the investigation of software process improvement initiatives observed by small or medium sized web organizations. The focus is on understanding of the overall software process improvement progression. Investigation of best models, goals or practices and pitfalls of others are also major considerations. To gain effective insight into the problem and understand the existing approaches a systematic literature review is required. It will be used as a mean to evaluate and interpret all the existing research available [18].The aim is to gain a thorough understanding of the existing software process improvement activities aligned currently with the small and medium web companies and their short comings as well.

3

2. Research Questions Identifying the valid research questions are most important component of any valid systematic literature review [18]. For the formulation of my research questions I would like to use the PIOC (Population, Intervention, Outcome, and Context) criteria defined by Petticrew and Roberts [19] as described in table 1. Table 1: Summary of PIOC Population Intervention

Outcomes Context

Small & Medium Web Development Organizations Using Software Process Improvement Various Practices of Software Process Improvement, Corporate Software Process Improvement Models i.e. CMMI, ISO, SPICE etc., Quality Assurance Activities Used by Small and Medium Web Organizations Measure of success showing how successful SPI approaches have been to small and medium Web organizations Industry, practitioners and consultants from Small & Medium Web Development Organizations, small & large-scale tasks

The question identification process can be iterative. On my current and very limited knowledge I have analyzed the following research questions that may require further review and revisions. Question 1 Which software process improvement models/techniques are followed by small and medium Web development organizations? Question 2 Which software process improvement activities/practices were successful to small and medium Web development organizations? Question 3 Are there any software process improvement frameworks that have been specifically proposed to support the infrastructures of small and medium Web development organizations? Question 4 What are the important indicators of success for small and medium Web development organizations that adopt software process improvement measures? Question 5 What constitutes a small or medium Web organization for the studies investigated?

4

Question 6 What are the characteristics of small and medium Web development organizations that have successfully used software process improvement?

3. Identification of Relevant Literature Identification of relevant literature requires exhaustive, rigorous and thorough searching of the relevant material. With its help gap in the existing literature can be found and provides a context for the placement of new research [20]. The data sources to be used by this study may include online databases, research journals, conferences, and data from industry and might be some unpublished research data or literature. 3.1 Searching Procedure Constructing the relevant search terms is a very important task and the adopted approach is summarized as under: i. ii. iii. iv. v.

Major search terms can be derived from the research questions i.e. PICOC Keywords from the found articles can also become relevant terms Synonyms of key terms can be found by using thesaurus. Boolean OR can be used with relevant terms Boolean AND can be used to restrict the search.

Table 2: Terms Derived from PIOC Population

Small & Medium Web Development Organizations Using Software Process Improvement Intervention Software Process Improvement, Software Quality Assurance, Small and Medium Web Development Organizations/Companies, Software Process improvement Models, Software Process Improvement Frameworks Used by Small and Medium Web Organizations Outcomes Measure of success showing how successful SPI approaches have been to small and medium Web organizations Context Industry, practitioners and consultants from Small & Medium Web Development Organizations, small & large-scale tasks Similarly key terms can also be searched across various relevant articles. The sample is given below. Table 3: Terms Derived from Synonyms Basic Term Software Process Improvement

Alternate Term Software Process Enhancement, software Process Enrichment, Software Maturity Attitude

5

Quality Organization

Excellence Setting / Business / Organisation/ Enterprise/Company Assessment Evaluation, Appraisal, Review Measurement Estimation, Capacity, Capability CMM / SW-CMM CMMI, SPICE, ISO-15504, PRINCE II Web Internet / WWW / World Wide Web Small & Medium Web Development Small & Medium Web Companies/ Internet Organizations Companies Table 4: Concatenation of Similar Terms with Boolean OR (Software Process Improvement OR Software Process Enhancement OR Software Process Enrichment) (Software Process Improvement OR Software Maturity Attitude) (Software Process Assessment OR Software Process Evaluation OR Software Process Appraisal OR Software Process Review) (Software Process Estimation OR Software Process Measurement OR Software Process Capacity OR Software Process Capability) (Software Process Improvement Goals OR Software Process Improvement Objectives OR Software Process Improvement Targets OR Software Process Improvement Purpose) (Software Process Improvement Goals OR Software Process Improvement Objectives OR Software Process Improvement Targets OR Software Process Improvement Purpose ) (CMMI OR SW-CMM SPICE OR ISO-15504 OR PRINCE II) (Web OR Internet OR WWW OR World Wide Web)) (Small and Medium Web Development Organizations OR Small & Medium Web Companies OR Internet Companies) Table 5: Concatenation of Similar Terms with Boolean AND (Software Process Improvement OR Software Process Enhancement OR Software Process Enrichment OR Software Maturity Attitude) AND (Software Process Assessment OR Software Process Evaluation OR Software Process Appraisal OR Software Process Review) AND (Software Process Estimation OR Software Process Measurement OR Software Process Capacity OR Software Process Capability) AND (Software Process Improvement Goals OR Software Process Improvement Objectives OR Software Process Improvement Targets ) AND (Measurement OR Estimation OR Assessment) (CMMI OR SW-CMM) AND (Web OR Internet OR WWW OR World Wide Web)) AND (Small & Medium Web Development Organizations OR Small & Medium Web Companies OR Small & Medium Web Development Organisations)

6

3.2 Searching Stages In order to retrieve the relevant literature there is a proper mechanism required. The following flow chart elaborated the basic searching process which is to be carried out. The overall search phase will be consisting to two components i.e. ? Primary Search Phase ? Secondary Search Phase 3.3 Primary Search Phase The primary search phase will comprise the search of the relevant literature in Online Databases, Search Engines, Individual Journals, Conferences, PhD/MS Dissertations and Theses. Details of the sources are listed below. 3.3.1 Online Databases ? ? ? ? ? ? ? ? ? ? ?

ACM Digital library Software Engineering Institute(SEI) Resources EBSCO host Electronic Journal Service IEEE Xplore ISI Web of Science INSPEC Sage Full Text Collections Science Direct Springer Link SCOPUS The IEEE Computer Society Digital Library

3.3.2 Online Search Engines ? Google scholar ? CiteSeer ? Agile alliance 3.3.3 Individual journals ? ? ? ? ? ? ? ? ?

Journal of Systems and Software (JSS) Empirical Software Engineering (EMSE) IEEE Transactions on Software Engineering (TSE) Journal of Software Process Improvement and Practice\ Information and Software Technology Journal of IEEE Software Software Quality Journal Journal of Software Maintenance and Evolution (SME) ACM Transactions on Software Engineering and Methodology (TOSEM)

7

3.3.4 Conferences: ? ? ? ? ? ?

International Conference on Software Engineering (ICSE) IEEE International Symposium on Empirical Software Engineering WOSQ (International Workshops on Software Quality) EuroSPI (European Conferences on Software Process Improvement) International SPICE Conferences on Process Assessments and Improvement Australian Software Engineering Conferences

3.3.5 PhD Dissertation and Theses ? ProQuest Dissertations & Theses ? Auckland University Library Fig. 1 Process of Identifying Relevant Literature from Primary Search Phase

8

3.4 Secondary Search Phase In the secondary search phase various companies and organizations as well as individuals will be contacted for data and unpublished technical works. It might include grey literature and technology white papers as well. 3.5 Search Process Record Keeping Search process record keeping is essential to discipline the review, future reference and keeping it unbiased. Strategy of record keeping will be the same as suggested by Kitchenham [21]. Table 6: Procedure for Search Process Record Keeping Data Source Digital Library

Search Engines

Individual Journals Conference Proceedings PhD Dissertation and Theses Unpublished Studies

Others

Record Keeping Database Name Search Strategy Date of Search Years Covered Engine Name Search Criteria Date of Search Journal Name Years Covered Conference Name Date of Conference Journal Name (If Published) Name of Theses Publication Year Name of University Software Companies Individuals Web Links Emails Web Reference

4. Study Quality Assessment Criteria All the considered studies then require the assessment for quality. Following are the important steps that need to be focused. 4.1 Inclusion/Exclusion Criteria In this regard we are considering the studies that focus on software process improvement activities in small and medium Web enterprises. The studies that do not explicitly focus on small or medium Web organizations, or do not investigate the use of SPI techniques 9

and frameworks will be excluded. More focus is to be put on the papers that are describing techniques or models particular to the subject. We may further exclude the studies that do not explicitly focus on small and medium enterprises or do not significantly provide details of software process improvement process for small and medium Web enterprises. The above mentioned research questions relevance to the context will be our basic consideration. Those studies definitely have more weight that provides empirical evaluations supported with case studies. It is not that we will be rating any work; it is just that we will try to find the relevance of the work according to our domain. Once again it is reiterated that only those research publications are to be categorized as relevant literature that have a focus on software process improvement in relevance to the small and medium organizations and are very specifically to be Web companies. 4.2 Hierarchy of Evidence Quality assessment is to be performed in two steps. Selection process for the literature found is going to be iterative i.e. Primary and final selection process. During the initial phase the search on all the sources is performed and screening is done on the names and abstracts of the papers. Relevant literature is to be included in the reference library and the rest is to be discarded. In the final selection process full papers are to be read that are the result of the initial phase and if found relevant they are to be included in the final reference library and the rest are to be discarded. The aim is to discard bias and improve internal as well as external validity. [22].

5. Study Quality Assessment Checklist Two brief checklists are developed to refine the study selection as well as assessment of quality and quantity for our selected literature. On the basis of two checklists we can develop data extraction forms which will be used for mining of selected literature. The checklists are made by extraction of some questions from Petticrew [19], Crombie [23] and Fank [24]. Some of the questions are modified for the convenience of this research and some of them are new. The checklists are presented in the following table. 5.1 Checklist for Quantitative studies Following checklist is to be considered if the review contains the qualitative studies. Table 7: Quality Assessment Checklist for Quantitative Studies No. 1 2 3 4

Question Are the aims of the research clearly stated? Is the research methodology used suitable to address the research questions? Does the sample match the ideal population? Was the sample used Random?

Answer Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially

10

5 6 7 8 9 10 11 12 13 14 15 16 17 18

Was the SPI Technology used clearly defined? Did the study account for confounding factors? Are the measures used in the study fully defined? Are the measures used in the study relevant to answer the research questions? Are the data collection methods adequately defined? If different groups are treated, are they treated equally in the study? Was only relevant data used in the study? Were any statistical methods used for analysis of data described? Has the use of statistical methods been motivated? Were all main findings relevant to answer the research questions? Were the negative findings presented? Has the research ignored any significant factors, either methodology or measures? Were the results compared with previous results? Does the result adequately answer the research questions?

Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially

5.2 Checklist for Qualitative studies Following checklist is to be considered if the review contains the qualitative studies. Table 8: Quality Assessment Checklist for Qualitative Studies No. 1 2 3 4 5 6 7 8 9

Question Does the article use a suitable research methodology? Does the article target the ideal population? Is the methodology used suitable to address the stated research questions? Does the article discuss any of the previous work/literature? Is the study process specified in the article repeatable? Is the article biased towards one SPI framework model or technology? Do the findings address the original research questions? Does the article document any assumptions taken? Does the article document the procedure used to validate its findings?

Answer Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially Yes/No/Partially

Score of 1 may be assigned for a yes, 0.5 for partially and 0 for a no for both qualitative and quantitative studies. Maximum a paper can score is 18 if it is a quantitative study and 9 if it is a quantitative one. Higher a paper scores more the quality of paper has according to our assessment. It is once again reiterated that this assessment is to rate articles that meet our criteria and are relevant for us to do further investigations. This is not a mean to criticize anybody’s work.

6. Data Extraction Purpose of data extraction is to store the extracted results from the primary studies. Data extraction is performed on the basis of the following extraction forms. There are two forms in number. One is supposed to store the results for the qualitative studies and other is for quantitative studies. The structure of both the forms is described as under: 11

Table 9: Data Extraction form for Quantitative Studies Data Item

Value

Identification of Study Year Author/s Title Reference Type Publisher Country of Study Setting Type of Study

Unique Number

Additional Information

Journal/Conference/Report

University/Industry Experiment/Case Study/Survey Article Peer Reviewed? Yes/No Questions to Help Answer the Study’s Research Questions What SPI model/technique is used by the organization? What SPI activities are considered successful to the organization being studied? How does the organization measure success and what are its indicators? How the article does categorizes small and medium Web companies and differentiates them from large companies or organizations? What are the important characteristics of small and medium web organizations? Does the article propose any framework or model or technique that is specific to small and medium Web development organizations? Quantitative Study Analysis Are the aims of the research clearly stated? Does the study successfully relate to its research questions? Does the population match with the desired population? Was the method used to obtain samples unbiased i.e. random samples or convenience samples? Was the SPI Technology used clearly defined? Did the study account for all the relevant factors i.e. confounding factors? Are all measures used in study are fully defined to answer the research questions? Are all measures used in the study are relevant to answer our research questions?

12

Are data collection methods adequately defined and documented? If different groups are treated, are they treated equally in the study? Has only relevant data being used in thee study? Are any statistical methods used for analysis of data described? Has the use of statistical methods been motivated? Are all main findings relevant to answer the research questions? Are the negative findings presented? Has the research ignored any significant factors, either methodology, confounding factors or measures? Are the results compared with previous results? Does the result add significantly to answer research questions?

Table 10: Data Extraction form for Qualitative Studies Data Item

Value

Identification of Study Year Author/s Title Reference Type Publisher Country of Study Setting Type of Study

Unique Number

Additional Information

Journal/Conference/Report

University/Industry Experiment/Case Study/Survey Article Peer Reviewed? Yes/No Questions to Help Answer the Study’s Research Questions Did the article appear in any International Journal/Conference of repute which cannot be ignored? Does the article targets the required population of small and medium Web companies and how it is justified? Which of the study questions presented in the study are related to the research of small and medium Web companies and how? Does the article support any of the case studies on the topic? If the articles support case studies, Are the results of case studies or experiments are documented then how? If the article supports any existing technology/model of software process

13

improvement improve small and medium Web companies then how? Qualitative Study Analysis Does the article use a suitable research methodology? Does the article targets ideal population? Is the methodology used suitable to address the stated research questions? Does the article discuss any of the previous work/literature? Is the study process specified in the article repeatable? Is the article biased towards one SPI framework model or technology? Do the findings address the original research questions? Does the article document any assumptions taken? Does the article document the procedure used to validate its findings? Does the article use a suitable research methodology? Does the article targets ideal population?

7. Data Synthesis In the data synthesis portion results from all the findings will be integrated and each question will be assessed individually against the findings. The summaries will also be created for all analysis performed through extraction forms and tabulated results will be used for further investigations. The process for each question can be formulated as under briefly 7.1 Question 1 It states that “Which software process improvement models/techniques are followed by small and medium Web development organizations?” After data extraction we may be able to answer this question as we will come to know about any existing models, techniques or may be frameworks that there for software process improvement of small and medium web companies. If there will be any such existing techniques, they will be investigated in greater detail and there case studies and practical applications will be given more focus and data will be collected. Following tables will be used for the synthesis of results obtained from question 1. Sr. No.

Process Technique

Model/

No. of Companies Using

Cost

Time

Salient Features

Drawbacks

Table 11: Data Synthesis format for Question 1 14

7.2 Question 2 It states that “Which software process improvement activities/practices were successful to small and medium Web development organizations and how success is being measured?” By the selected literature it is going to be reviewed that what are practices or initiatives of software process improvement that are very special and are worthy for small and medium web development organizations. It will also be further investigated that how and why they were more successful and which factors are there behind their success. Following table will be used for the synthesis of results obtained from question 2. Sr. No.

SPI Activities/Practices

No. of Companies Using

No. of Projects Used

Time

Cost

Benefits Obtained

Drawbacks

Table 11: Data Synthesis I format for Question 2 Sr. No.

Company

Project

Success Measure/Metrics

Table 12: Data Synthesis II format for Question 2 7.3 Question 3 It states that “Are there any software process improvement frameworks that have been specifically proposed to support the infrastructures of small and medium Web development organizations?” We will also investigate in the literature that is there any typical frameworks residing for software process improvement that is targeting small and medium web development organizations. The question may be addressed by tabulating the summaries of the frameworks as illustrated in table 13. Sr. No.

Framework

Executive Summary

Quality Assessment Score

Table 13: Data Synthesis format for Question 3 Data obtained from this question may further be investigated in further depth in table 11, table 12 and table 13. 7.4 Question 4 It states that “What are the important characteristics of small and medium Web organizations that follow software process improvement activities and practices?” This question will also have a major focus in investigation as we will looking for the unique

15

characteristics of small and medium organizations due to which there is a need of special software process improvement initiatives for them and typically they are web development organizations. The question may be addressed by tabulating the summaries of the frameworks as illustrated in table 14 Sr. No.

Web Projects Type

Company’s Age

Target Market/s

Total Turnover

No. of Employees

Average Project Duration

Average Project Cost

SPI Model Used

SPI Activities Followed

Process Model (s) Used

Table 14: Data Synthesis format for Question 4 7.5 Question 5 It states that “What constitutes a small or medium Web organization for the studies investigated?” It is very important to know from the relevant literature that how small, medium and large organizations are categorized and what is the basis for categorization. Initially small and medium organizations will be put in one group while large will be put into another one. During the second phase small and medium will be differentiated among themselves and all the factors will be highlighted that cause the differentiation. This differentiation will be synthesized with the help of table 14. Sr. No.

Small Company

Medium Company

Table 15: Data Synthesis format for Question 5

8. Systematic Literature Review Schedule For this specific research we have allocated around nine months time for systematic literature review and time is allocated for every single task. Systematic literature review is detailed process that may be divided into different tasks and activities that are listed as under: ? ?

Systematic Literature Review Study and Understanding: This Phase helps in developing and understanding of review concepts and we have allocated a time span of four weeks for this phase. Development of a Study Protocol: This phase is very rigorous and iterative as well as it covers the overall plan for the systematic literature review and time allocated for this task is four weeks.

16

Other(s)

? ? ? ? ? ?

Identification of Relevant Literature: This phase surrounds the identification of primary and secondary studies and is a search phase. So time allotted for this phase is again six weeks. Selection of Studies: For the identified literature very carefully some studies are selected and are further drilled down. This includes both primary and secondary studies. Time allocated for this phase is again three weeks. Study Quality Assessment: Both qualitative and quantitative studies are assessed for quality in this phase based on the developed checklists and proper weights are assigned to each study and two and a half weeks are allocated to this phase. Data Extraction: Data is to be very carefully extracted from each study and it is also weighed upon the research questions. This phase is again rigorous so a time of four weeks is allocated to it. Data Synthesis: After extraction data is aggregated, integrated and summarized for the further clarity and it is ordered to answer the research questions. A time slot of three weeks is allocated to it. Review Report Write Up: A very important concluding phase that summarizes the results and findings of the overall systematic literature review process comes at last. All previous phases contribute to it and a time of five weeks is allocated for this task.

This overall process is further elaborated in the following Gantt chart for the purpose of clarity.

17

Fig. 2 Systematic Literature Review Gantt chart

18

Fig. 3 Systematic Literature Review Process Flow Chart

19

References [1] R.L. Glass Software Creativity, Englewood Cliffs, New Jersey: Prentice-Hall (1995). [2] Lasse Harjumaa et. al. Improving Software Inspection Process with Patterns Proceedings of the Fourth International Conference on Quality Software, 2004 [3] Marion Lepassaar, Timo Makinen, Integrating Software Process Assessment Models using a Process Meta Model, Proceedings of IEMC 2002: Volume I, IEEE, 2002. [4] Cugola, G. and Ghezzi, C. Software Processes: A Retrospective and a Path to the Future. Software Process Improvement and Practice, 1998, 4, 101-123. [5] Thompson, H.E. and Mayhew, P. Approaches to Software Process Improvement. Software Process:Improvement and Practice, 1997, 3 (1), 3-17. [6] Zahran, S. Software process improvement: practical guidelines for business success. Addison-Wesley Pub. Co., Reading, Mass., 1998. [7] Florac, W.A., Park, R.E. and Carleton, A.D. Practical Software Measurement: Measuring for Process Management and improvement, CMU/SEI-97-HB-003, The Software Engineering Institution, Pittsburgh, 1997. [8] Abrahamsson, P. Rethinking the Concept of Commitment in Software Process Improvement, Scandinavian Journal of Information Systems Volume 13, 2001. [9] Kuilboer, J.P. and Ashrafi, N. Software process and product improvement: an empirical assessment. Information and Software Technology, 2000, 42 (1), 27-34. [10] Tortorella, M. and Visaggio, G. Empirical Investigation of Innovation Diffusion in a Software Process. International Journal of Software Engineering and Knowledge Engineering, 1999, 9 (5), 595-621. [11] Allen, P.; Ramachandran, M.; Abushama, H., PRISMS: an approach to software process improvement for small to medium enterprises, Proceedings of Third International Conference on Quality Software, 2003. [12]Ahamd R. Web engineering: a new emerging discipline, Proceedings of the IEEE Symposium on Emerging Technologies, 2005. [13] A.Ginige & S.Murugesan, "Web Engineering: An Introduction," IEEE MultiMedia, vol. 8, no. 1, Jan.-Mar. 2001. [14] Deshpande Y, Murugesan S, Ginige A, Hansen S, Schwbe D, Gaedke M & White B "Web Engineering" Journal of Web Engineering, 1(1), 2002

20

[15] Pressman, R S 'Software Engineering: a Practitioner's Approach (6th ed.)', McGraw-Hill Publishing Company, 2005. [16] Yogesh Deshpande, Martin Gaedke "Web Engineering: Developing Successful Web Applications In A Systematic Way" 14th International World Wide Web Conference, 2005, Chiba, Japan [17] Gnaho, C. and F. Larcher, “A User Centered Methodology for Complex and Customizable Web Applications Engineering,” Proceedings of First ICSE Workshop on Web Engineering, ACM, Los Angeles, 1999. [18] Barbara Kitchenham, Guidelines for Performing Systematic Literature Review in Software Engineering, EBSE Technical Report, Keele University, 2007 Version 2.3. [19] Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide, Blackwell Publishing, 2005. [20] E. Mendes, A Systematic Review of Web Engineering Research, International Symposium on Empirical Software Engineering, 2005 [21] B. A. Kitchenham, S. Charters, Procedures for Performing Systematic Literature Reviews in Software Engineering, EBSE Technical Report, Software Engineering Group, School of Computer Science and Mathematics, Keele University, UK and Department of Computer Science, University of Durham, UK, 2007. [22] Cochrane Collaboration. Cochrane Reviewers’Handbook. Version 4.2.1. December 2003. [23] Crombie, I.K. The Pocket Guide to Appraisal, BMJ Books, 1996. [24] Fink, A. Conducting Research Literature Reviews. From the Internet to Paper, Sage Publication, Inc, 2005.

21