Counting Comments: A critical appraisal of the evaluation of social media campaigns by UK digital marketing practitioners.
Brendan J. Keegan Submitted to: RIBM Doctoral Symposium 2011 30th-31st March
Director of Studies: Prof. Jenny Rowley
Brendan J. Keegan Manchester Metropolitan University Business School Aytoun Street Aytoun Building Manchester M1 3GH Tel: +44(0)161 247 3835 E-Mail:
[email protected] 1 Brendan J. Keegan
Abstract: This research aims to explore the evaluation approaches by digital marketing practitioners when employing social media as part of multichannel campaigns. The inclusion of social media within the integrated marketing communications paradigm suggests the traditional evaluation dogma can be transposed onto this relatively new approach. The dearth of work performed on practitioner approach in this area will be supplanted by previous work into traditional evaluation in marketing communications. Through engaged action research, this work will examine the challenges encountered with a view to conceptualising research implications. The absence of universally agreed evaluatory criteria within social media campaigns has been identified as a gap in academic knowledge and is hence offered up for academic scrutiny. It is envisaged that work will seek to understand further how practitioners evaluate campaigns and the levels of innovation within these processes. Keywords: Social Media, Digital Marketing, Marketing Communications, Evaluation, Measurement
Introduction to the research Modern web innovation and technological prowess has been well documented in recent years as being the catalyst for modern digital marketing techniques and approaches (Moran & Gossieaux 2010; Kim 2008; Riegner 2007; Ferguson 2008; Trusov et al. 2009). The emergence of social media as a constituent channel of Pickton's (1999) integrated marketing communications paradigm has also led to the need for an investigation into whether traditional techniques can be applied to this new practice. (Mangold & Faulds 2009) The crux of the research problem at hand lies within the level of inconsistency in the evaluation approaches employed upon social media campaigns by the practitioner. Considering the rate that social media is being utilised in multi-channel campaigns coupled with the immense number of participants, it would serve industry and academia well to investigate these practices.
2 Brendan J. Keegan
Aim: The overall aim of the research is to perform a critical appraisal of the evaluation approaches employed by digital marketing practitioners within social media campaigns by way of action research in order to understand the process involved. Objectives: To investigate social media evaluation approaches in order to; 1. develop an understanding the process of evaluation at pre-planning and postcampaign stages 2. assess the challenges involved with the broad range of campaigns 3. analyse and conceptualise findings from 1 and 2 in order to assist future practitioners Literature Review Fill et al (2008) argues that within modern digital marketing research, the development of commercial practice will be supersede academic material due to the rate of innovation. A number of commercial reports (e.g. Roy 2009; Saleem 2008) come to the fore when searching for literature encompassing social media evaluation, whereas their bias towards their own commercial gain is clearly apparent. These reports tended to be biased towards the writer and ignore the other side of the argument, specifically; they may be incorrect in their assumption. Circumventing the difficultly in locating apt literature was achieved by examining literature from a similar vein but not necessarily within the social media remit. Social media, while clearly a marketing endeavour, is essentially a composite of other disciplines including Information Communications Technology (ICT), Marketing Communications and Digital Marketing and these subsets were explored. Social Media Evaluation Whereas widespread adoption of Web2.0 technologies and social media platforms into contemporary multi-channel campaigns as a viable channel. Despite the widely acclaimed affirmation of the importance of Web2.0 technologies, (Riegner 2007; Pelsmacker & Neijens 2009) the scant amount of material written on the very practitioners who implement it as a marketing platform is noteworthy. Literature which is available on this topic, as highlighted by Ivanauskas (2009), tends to lean towards the US market with disregard for the EU and, applicable to this study, UK domains. The predominantly reported set of evaluatory factors of a social media campaign are a viable increase in site-specific traffic, conversion or engagement rates and finally reach. Murdough (2010) outlines a schematic (Figure 1) which is a reasonable methodical approach while also highlighting the need for further scrutiny in this area. The ‘pillars’ are particularly relevant as they are comprised of evaluation criteria, 3 Brendan J. Keegan
although at a macro-level, which are specific to each campaign. When addressing the full extent of these variable at the micro-level a numerical challenge incurs what the author refers to as a ‘flotilla of variables’. These hitherto unchartered variables are a large proponent of the primary contribution of this research and the genesis of a model, ideologically similar to Murdough’s (2010) Figure 1: The Social Media Measurement Process
Source: (Murdough 2010: 96)
Traditional marketing communications theory and practice have long been asserted within social science by established authors in the field (Fill, 2002; Pickton, 1999; Yeshin, 2003b) with evaluation methodologies taking its rightful place within the chapters. Fendwick (cited by Fill 2008) refers to the ‘nirvana stage’ of advertising effectiveness whereby through computer influence, tracking and evaluation can be performed with the greatest of ease to enable quick and efficient modification of messages. The underlying component of Web2.0 applications is the ability to manipulate a tremendously large range of data which is involved with campaigns. It is commonplace for practitioners to measure every aspect of behaviour as part of their top line budget. Edelman explores this point and refers to the ‘commensurate increases in effectiveness’ when considering modern campaign measurement. (Edelman 2010: 401) Additionally, the myriad of critical success factors (CSF) and key performance indicators (KPI) as indicated by Chaffey & Smith (2008: 445) tend to be case specific and extremely diverse as investigated by Fill (2008: 753) in his work on viral marketing campaigns (Figure 2). This culture of measurement is exemplified by a host of authors (Fill et al 2008; Pergelova et al. 2010; Pfeiffer & Zinnbauer 2010) and has been alluded to as having redefined return on investment (ROI) to the extent that modern marketing budgets has been reshaped to include the smallest element of customer behaviour within the campaign. (Hoffman & Fodor 2010)
Figure 2: Concordance of evaluation criteria terminology
Source: Fill et al (2008: 753)
4 Brendan J. Keegan
By considering this culture of measurement within the discipline, it seems feasible to be able to apply this to all facets of digital marketing, specifically social media. Recent schools of thought have sought to establish the link between relationship marketing and the technical prowess digital marketing measurement affords the modern campaign. Within this notion, Rowley (2004) referred to the creation of mutual value, relationships and presence whereas traditional marketing communications focus has been solely on promotion. The potential (as realised by Fill (2008)) for encouraging a relationship marketing approach through this medium has become apparent. Rowley’s (2004) shift in power to the consumer can enable this approach through customisation, personalised communication and an integrated marketing communications network by organisations. Thus the emphasis is on the practitioner to appreciate this shift and utilise it through their campaigns. Recent and progressive works have tackled the issue of bringing established marketing communications theory and practice into line with innovative technology, whereas at the time of writing, only a few (Fill et al 2008; Ivanauskas 2009; Lindberg 2010; Murdough 2010; Bughin et al. 2010) have attempted to link the traditional evaluation approaches to the digital marketing. Conversely, Weber (2007) champions the contrary and asserts that social media has re-written the traditional marketing methods due to the creation of the social web; whereas Hemerking (2006) pointed out that the global spread of ICT is epitomic of postmodernist globalisation and is in line with natural progression. To combine traditional measurement with digital marketing has been explored to some degree by Pfeiffer & Zinnbauer (2010), however, it is noteworthy that they focus on digital display advertising and neglect to explore the social media aspect and display lacks the key engagement element present in social media. Chaffey (2009) asserts that digital marketing allows for an increased level of personalisation of the marketing message by E-mail, SMS text messages and even websites although the drawbacks indicated are key aspects we need to consider when combining the old and new media. Discussion The purpose of this literature review is to explore what state the existing literature within the area of digital marketing with particular reference to evaluation within social media campaigns. The initial research problem as indicated by Fill (2008) was that current practice, at the time of writing, was to measure ‘hits, or impressions’ and these did little to satisfy qualitative evaluation of viral marketing campaigns. While this practice has changed somewhat as Edelman (2010) pointed out, the problem still remains that the current practice leans towards a quantitative end of the spectrum. This reliance on numerical data juxtaposes the very basis of social media mechanics where semantic elements of human behaviour are prevalent. The literature has also identified considerations when combining the traditional approaches in marketing communications and modern social media practice as well as the potential to enable a richer approach to relationship marketing. 5 Brendan J. Keegan
Methodology In this study, a qualitative approach will be applied. Through a deductive research approach, a literature review will be created from which hypotheses will be generated. This research employs active research approaches in line with empirical research techniques as outlined by Saunders (2009) and Fischer (2010). In addressing this type of project it is envisaged that an engaged action research approach would suit the nature of the topic. The original Lewinian style of action research comprises of planning, action and fact-finding however, considering the nature of the typical social media campaign this set format will have to be manipulated to facilitate suitable data collection. (Coughlan et al, 2010) Figure 3: Action Research Cycles
Source: 1 Coughlan (2002: 233)
Primary data will be collected by periodic ‘interventions’ upon agreement with participant practitioners, which shall be further directed by the clients who have requested a marketing action of this type. It is anticipated that this series of interventions will distort the traditional action research to facilitate multiple cycles with a view to obtaining richer veins of data from the campaigns themselves. (Figure 3) A daily journal will be utilised while engaging with the campaigns for primary data. Supporting these cycles will be a retrospective campaign review with the practitioner which shall be directed by semi-structured interview coupled with an iterative approach to analyse the variables outlined by the action research. The active research shall involve the author actively participating in work with the practitioners adding while actively observing the running of the campaign from preplanning stage. To further clarify the nature of the work being performed, a clear division will be made to isolate commonality in the subject campaigns. The paradigm for these campaigns will range from social (e.g. brand awareness) to economic (e.g. commercial) and both will be scrutinised during the active research with the relevant 6 Brendan J. Keegan
and associated variables as identified in the literature review. In keeping with qualitative research the analysis of the data will be inextricably linked with the data collection method. The timescale of the research will also be considered as the short to medium term goals, for example, direct sales may be investigated with greater ease than for example, brand equity, which may demand a longer time frame. e.g. E-Bay (Lindstrom: 2000: 131) Admittedly, this is a speculative attempt to counter such issues, however contingencies shall be employed in the interest of risk assessment. Ethical consideration: Social media practitioners will be approached with a speculative request to gauge their propensity to include the action researcher as a bona fide employee within the organisation. Treatment of the data Electronic journals will be kept and backed up regularly. Post campaign meetings will periodically examine the subject campaigns from pre-planning, through to post campaign evaluation. These retrospective sessions will be of a qualitative nature analysing aspects of the campaign and how it was approached. Expected contribution This paper contributes by proposing the use of action research as a methodology to empirically examine the integral details of the digital marketing practitioners approach to evaluating social media campaigns. It is the researcher’s intention to apply an academic perspective to an industry problem to further the understanding of evaluation within social media practice. The contribution will be envisaged as a piece of work which addresses and delimits the boundaries of the research problem within a UK context, with the possibility of extending the study onto M.Phil and PhD level with appropriate effort. Risk analysis: Importance Risk 1 Participants do not agree to allow active research
2
Incorrect time scale estimated
3
Data loss
4
Technology Failure
Contingency Organise ‘pool’ of practitioners to call upon with time consideration to delay the completion of the research. Regular discussions with the practitioner will identify feasible endpoints for the active research. Regular back-ups will be kept periodically. Paper journals will be kept in line with electronic receptacles. 7
Brendan J. Keegan
Bibliography: Bughin, J., Doogan, J. & Vetvik, O.J., (2010). A new way to measure word-of-mouth marketing. McKinsey Quarterly, (2), pp.113-116. Chaffey, D., Ellis-Chadwick, F., Mayer, R. &Johnston, K. (2009) Internet Marketing: Strategy, Implementation and Practice, 4th Ed.: Prentice Hall, Harlow Chaffey, D., Smith, P.R. (2008) E-Marketing Excellence: Planning and Optimising Your Digital Marketing, 3rd Ed., Butterworth-Heinemann, Oxford. Coughlan, D. & Brannick, T. (2010) Doing Action Research in your own organization, 3rd Ed. SAGE Publications, London Coughlan, P. & Coghlan, D., (2002). Action research for operations management. International Journal of Operations & Production Management, 22(2), pp.220-240. E-Consultancy (2010) Agency Rate Survey, [Online] [Accessed on 01/12/10] http://econsultancy.com/uk/reports/Agency-rate-card-survey Edelman, B. (2010) Commentary: Who Owns Metrics? Building a Bill of Rights for Online Advertisers. Journal of Advertising Research, 49 (4), p.p.401. Ferguson, R., (2008). Word of mouth and viral marketing: taking the temperature of the hottest trends in marketing. Journal of Consumer Marketing, 25(3), pp.179-182. Available at: http://www.emeraldinsight.com/10.1108/07363760810870671. (accessed 19/12/10) Fill, C. (2009) Marketing Communications; Interactivity, Communities and Content.5th Ed., Prentice Hall, Harlow. Fill, C. & Cruz, D. (2008) Evaluating Viral Marketing: Isolating the key criteria, Journal of Marketing Practice: Applied Marketing Science, pg. 742-756 Fisher, C. (2010) Researching and Writing a Dissertation, 3rd Ed., Prentice Hall, Harlow.
Hermeking, M. (2006), Culture and Internet Consumption: Contributions from Cross-Cultural Marketing and Advertising Research, Journal of Computer-Mediated Communications, Vol 11, No. 1, pp 192-216 Hoffman, D. & Fodor, M. (2010) Can You Measure the ROI of Your Social Media Marketing? MIT Sloan Management Review, 52 (1). Ivanauskas, G., (2009). The evaluation of social media effects on marketing communications: The UK consumers perspective. International Business. Kim, S., 2008. A Framework for Advertising in the Digital Age. Journal of Advertising Research, (August), pp.23-32. Mangold, W.G. & Faulds, D.J., (2009). Social media: The new hybrid element of the promotion mix. Business Horizons, 52(4), pp.357-365. Moran, E. & Gossieaux, F., (2010). Marketing in a Hyper-Social World: The Tribalization of Business Study and Characteristics of Successful Online Communities. Journal of Advertising Research, 50(3), p.232.
8 Brendan J. Keegan
Murdough, C., (2010). SOCIAL MEDIA MEASUREMENT : IT ’ S NOT IMPOSSIBLE. , Journal of Interactive Advertising, 10(1), pp.94-100. (accessed 19/12/10) Pelsmacker, P.D., Peter C Neijens (2009) Journal of Marketing Communications:Special Issue on New Advertising Formats. Journal of Marketing Communications, 15 (2), p.pp.205-206. Pergelova, A., Prior, D. & Rialp, J. (2010) Assessing Advertising Efficiency. Journal of Advertising, 39 (3), p.pp.39-54. Pfeiffer, M. & Zinnbauer, M., (2010). Can Old Media Enhance New Media?: How Traditional Advertising Pays off for an Online Social Network. Journal of Advertising Research, 50(1), p.42. Pickton, B.H.D., (1999). Integrated marketing communications requires a new way of thinking. Journal of Marketing Communications, 5(2), pp.97-106. Riegner, C., (2007). Word of Mouth on the Web: The Impact of Web 2.0 on Consumer Purchase Decisions. Journal of Advertising Research, 47(4), pp.436-447 ST - Word of Mouth on the Web: The Impact. Rowley, J., (2001). Remodelling marketing communications in an Internet environment. Internet Research, 11(3), pp.203-212. Rowley, J. (2004) Just another channel? Marketing communications in e-business. Marketing Intelligence & Planning, 1, p.pp.24-41. Roy, J. (2009). Marketing Metrics and ROI: How to set up a Measurement System that Can Double Your Profitability [Online] Available at http://www.marketing-metrics-made-simple.com/supportfiles/marketingmetricsandroi-smallbusinessedition-20091007.pdf Accessed 28/02/11 Saunders, M. Lewis, P. & Thornhill, A (2009), Research Methods for Business Students, 5th Ed., Prentice Hall, Harlow. Slaleem, M, (2008). Social Media Marketing ROI-Metrics and Analytics [Online] Available at http://searchengineland.com/social-media-marketing-roi-metrics-and-analysis-14630 Accessed 28/02/11 Trusov, M. Bucklin, R. E., Pauwels, K. (2009) Effects of Word-of-Mouth Versus Traditional Marketing: Findings from an Internet Social Networking Site, Journal of Marketing, Vol. 73 (September 2009), 90–102. Vilpponen, A., Winter, S. and Sundqvist, S. (2006), “Electronic word-of-mouth in online environments: exploring referral network structure and adoption behaviour”, Journal of Interactive Advertising, Vol. 6 No. 2, pp. 71-86. Webber, L. (2009). How digital customer communities build your business. John Wiley & Sons. Yeshin, T., (2003). Integrated Marketing Communications. The Holistic Approach, ButterworthHeinemann, Boston, MA.
9 Brendan J. Keegan
