Enhancing Online Community Activity - Wiley Online Library

Journal of Computer-Mediated Communication

Enhancing Online Community Activity: Development and validation of the CA framework F.J.M. van Varik H. van Oostendorp Institute of Information and Computing Sciences, Utrecht University, Princetonplein 5, 3584 CC, Utrecht, The Netherlands

Factors contributing to development of active communities are identified and combined into the Community Activity framework, which is useful in setting up new, or revitalizing inactive, communities. Found factors include: notifying members of new messages by e-mail, having a news section, and ability to add pictures to member profiles. During application of the framework to an inactive community, changes have been made to privacy options, polls, activity notifications, and other areas. Significant positive effects have been found in the number of visits, volume of posted messages, and number of topics. Interest of community members in both user profiles and the message board increased significantly. We conclude that the Community Activity framework is able to contribute in developing active online communities. Key words: Community Activity Framework, community success, community activity, interactivity, online communities. doi:10.1111/jcc4.12020

Problem definition By using the Internet, people can form communities with members who are spread all over the world. It is also easier than ever before to find people with overlapping interests, questions, and goals. One of the ways in which people can communicate over the Internet, is by using online communities. They allow people to share both information and feelings with one another. An example of an online community is a message board for people with a similar illness or medical problem. By sharing information and supporting one another, these online communities become valuable to the people that use them. However, many cases can be identified in which new online communities only receive a handful of both visitors and messages. If we define community success as the amount of activity within a community (e.g. number of active visitors), then many communities are unsuccessful. Success of online communities forms the basis for discussion in this paper, in which we focus on methods to increase the amount of active visitors in online communities. Therefore, our research question is as follows: ‘‘Which factors contribute to the development of an active and successful online community and how can we stimulate the interactivity?’’ The answer to this question will be based on both a literature study and our own empirical research (also see Figure 1). First, possible factors that contribute to the success of online communities have been identified from the literature. Next, the influence of these factors is examined by statistically comparing a set of online communities. From this 454

Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

set of factors, the Community Activity (CA) Framework has been created. This CA framework can be used as an aid in setting up new online communities, and enhancing the interactivity in current ones. The latter has been done in a Case Study, in which a community with low amounts of activity has been changed according to the CA framework. Subsequently, activity in the resulting community has been compared with the original community. The final section contains conclusions and both implications for the application of the CA framework and ways to extend the framework.

Related work Many definitions of online communities may be found when examining current literature. The definition utilized in this paper is that of Leimeister, Sidiras and Krcmar (2004): ‘‘A virtual community [also known as an online community] consists of people who interact together socially on a technical platform. The community is built on a common interest, a common problem or a common task of its members that is pursued on the basis of implicit and explicit codes of behavior. The technical platform enables and supports the community’s interaction and helps to build trust and a common feeling among the members.’’ This definition was chosen for its combination of both social and technical aspects. The definition includes the people who visit an online community, and the reasons for which they may do so. It also includes a technical aspect, and the goal for which the technology is utilized within online communities. The combination of message boards (also known as forums or bulletin boards) and member profiles are widely used as a technical platform for online communities. The message boards allow people to reply to, or start new topics of discussion (also known as threads). The profile section allows people to disclose personal information and pictures with other people in the community. By incorporating interactive features into a website like message boards and member profiles, visitors are able to have an equal say in the content of the website. With static websites, all content is supplied by the webmaster. However, by adding interactive features, visitors are able to influence the content of a website and use this influence to fulfill their informative needs (Arguello et al., 2006).

Community success The success of an online community may be defined in different ways, depending on the stakeholders that are involved with the community, and the goals of the community managers. For example, definitions used in earlier research revolved around member loyalty (Lin & Lee, 2006) when focusing on the importance of retaining community members, members’ perceived professional development (Hew, 2008) in profession-based communities or community activity in terms of topic response rate (Arguello et al., 2006; Burke et al., 2007; Rafaeli & Sudweeks, 1997). In this research, we will use member activity (e.g. posting of messages and creating topics) as our measure of community success. As stated in the first section of this paper, many online communities are unsuccessful in acquiring and retaining both active members and consecutive posts. Even though many new online communities

Figure 1 Steps in the development of the CA framework


455

are formed each year, only a few turn out to be a viable platform for both discussion and support. Butler (1999) shows that this problem is widespread: Over 50% of all e-mail based communities did not receive any message in a time span of 4 months. After installing one of the many online community software packages that are available (or building a platform from scratch), activity within the online community may be limited, even though the community manages to get a steady stream of visitors. Nielsen (2006) shows that only a small portion of the people that visit an online community become active: 9% of all visitors become moderately active, and only 1% of all visitors becomes very active. Himelboim (2008) shows a similar skewed distribution of member activity. This presents the danger that the small part of the visitors who post messages become demotivated by a shortage of replies to their messages (Williams, Cheung & Choi, 2000) and cease to visit the community, as the community is not able to fulfill their needs (Sangwan, 2005). The success of online communities is dependent on the overall level of community activity (Holtzblatt & Damianos, 2011). They showed that active communities were more likely to be rated as successful. To increase the success-rate of new online communities, it is important to both raise the amount of visitors that become active members and increase the amount of actions each member performs (e.g. reading and posting messages, opening new topics.) Inactive members As Nielsen has shown, up to 90% of the visitors of an online community do not open topics and do not post new messages: They are inactive. The only activity of this group within the community is reading the messages other people have posted, which may also be recorded by the message board software to determine which messages have been read. As this is a passive activity, this group has been defined by Kollock and Smith (1996) as ‘lurkers’ because these people do not contribute to the community. Preece, Nonnecke, and Andrews (2004) define lurkers less negatively, as they say that while lurkers do not post now, they might posts messages in the future and convert to active members. Lurking is not necessarily bad behavior, as it might be used to learn the rules of participation within a community, or to check if a community fulfills required needs (Walther & Boyd, 2002; Maloney-Krichmar & Preece, 2005). Nonnecke and Preece (2000) show that the amount of lurkers within a community depends on the theme of the community, and differ from 46% for health related communities to 82% for communities related to technology. Fisher, Smith, & Welser (2006) see similar results, with a high number of replies to topics in social support newsgroups, and a low amount of replies in technical newsgroups. However, a certain number of active users are required in every community to be successful. People only want to invest in communities of which they know that they are able to fulfill social and emotional needs (Levine & Moreland, 1994). So if the majority of a community lurks and only few people participate actively, the few active members are likely to leave the community.

Interactivity Another way to describe the success of a community is by measuring interactivity. Rafaeli and Sudweeks (1997) define interactivity as the manner in which messages relate to one another. As face-to-face communication becomes interactive when people respond to their conversation partner, the same is true for online communication. On message boards, interactivity may be seen as the rate to which people reply to each other’s messages. The higher the measure of interactivity, the more people reply to messages of others. Interactivity is an essential part in the ability for online communities to fulfill the needs of its users. Without the possibility of getting answers to questions and replies to inquiries, users abandon the community and look for other avenues that are better suited to fulfilling their needs. Another reason for community abandonment also involves interactivity: If people do not receive a reply to their question 456


or inquiry (i.e. a low degree of interactivity), they may feel ignored or ostracized (Williams, Cheung & Choi, 2000). An important factor in the development of a high degree of interactivity is the number of active members within a community. Markus (1987) refers to this construct as her ‘Critical Mass Theory.’ This theory states that as more members are available within a community, the chance that one of these members can answer the question of a fellow member is heightened. Upon receiving a helpful answer, members will invest more time and energy in the community, increasing the amount of knowledge that is available within the community. This is supported by Joyce and Kraut (2006), who found that newcomers to a community have a 12.4% higher chance to post a second message, when they received a reply to their first message. Work by Arguello et al. (2006) shows that the degree of interactivity is not constant for a community as a whole, but that it differs from topic to topic, based upon differences between different topic starts (also known as first posts). They found that when topic starts contain a clear request or question, chances for a reply are increased by 6%. A short introduction with some elements of disclosure (e.g. age, history and current situation) increases the chance for a reply by 10%. The increase in chances for a reply upon disclosing background information is also supported by Collins and Miller (1994), who found that people get a more positive image of others who have disclosed some information about themselves. Within support focused online communities, people receive more supportive messages when they have self-disclosed some information (Pfeil, Zaphiris & Wilson, 2010). The increase in chances for a reply upon posting a request or question are also supported by Joyce and Kraut (2006), who found an increase of 16.4% when posting a request or question, when compared to posting information, advice or opinions.

Methodology of framework development As it was our goal to develop a framework with factors that lead to successful online communities, two constructs needed to be formed. At first, we needed to define how community success can be measured, to be able to compare the success of different online communities. Secondly, the properties (or factors) of online communities to be used in this study needed to be defined. After defining the metrics of community success and properties of online communities, the relations between these two constructs are discovered by comparing the metrics of success of 58 online communities in a statistical analysis. The communities that were used for this comparison had either Dutch or English as the official discussion language, and were all related to health. Furthermore, the communities could be labeled as social communities, placing focus on members and their feelings, problems and questions (Porter, 2004). All of the properties that have been used in this study were binary. E.g. a community either does or does not have message board access for guests. For each property, the set of 58 communities were split up into two groups. The communities in the first group did not have the property; the communities in the second group did. Both groups were then compared with the Independent Samples Mann-Whitney U-Test for each metric of community success. If this test yielded a significant result, we can state that communities with and without the corresponding factor differ significantly from one another for a metric of community success.

Community Success Different metrics may be utilized to express community success. As we place our focus on the activity within a community, metrics related to activity have been selected. The selected metrics have been placed into four distinct categories, which can be found below. The metrics themselves are displayed in Table 1. The last three columns of this table also state the groups in which these metrics have been used. Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

457

Table 1 Activity metrics and groups of properties in the CA Framework (A checkmark denotes which metrics are used in which parts of the CA Framework)

Community activity metrics Category

Variable

Absolute community size

Relative community size

Interactivity Member activity

Groups of properties in the CA Framework Communities overall

Message board topics

Member profiles

Total number of members Total number of created topics Total number of posted messages Number of new members per year Number of new topics per year Number of new messages per year Average number of messages per topic Average number of topics per member Average number of messages per member Average number of replies in own topics Average number of posted messages

Absolute community size The success of a community can be defined by its size. The bigger the community, the more successful it is. Three metrics have been selected to measure the size of a community (Schoberth, Preece & Heinzl, 2003; Butler, 1999). Relative community size While these metrics provide an overview of the size of a community, they do not adjust for the age of a community. Older communities have had more time than younger communities to acquire their members, topics and messages. Therefore, normalized metrics also need to be considered. Interactivity With the metrics from the first two categories, a view of the overall activity within a community can be generated. To be able to acquire the activity per topic, a measure of interactivity needs to be used. This measure can be generated by dividing the total number of messages by the total number of topics (Rafaeli & Sudweeks, 1997). 458


Member activity Based on the number of members, topics, and posts within a community, the average activity per member can be generated. The higher the figures for these metrics are, the higher the activity of the community members is. These activity metrics help to compare communities with a different amount of members. For example: A community with 100 members may have 500 posts, and a community with 20 members may have 400 posts. While the first community has more posts, the members of the second community are more active, as they each post more messages than the members of the first community. Community properties By defining a set of properties of online communities, we are able to compare communities to one another and identify the properties that relate to community success. These properties have been split up into three distinct groups: 1) properties of communities overall, which involve the interaction aspects, 2) properties of the message board topics, which concern the topics themselves that are posted on the message board, and 3) properties of the member profiles are the properties of the members that are defined by their profiles. The main points from each of these groups are defined below. 1 Communities overall The overall properties of communities entail the first of three groups defined in Table 1. The properties from this group have been split in several categories. Below, the definition of these categories can be found. Table 2 contains the corresponding properties and an explanation for each property.

Accessibility of data: How much of the (private) data within a community is visible to guests, who aren’t members? Is it possible to see data about members? With the answers to these questions, insight in the ‘openness’ of a community can be found. Also, influence of openness on community activity may be determined (Kollock, 1997). Registration On closed communities, a guest needs to register and become a member, before he or she is able to post messages. Registration forms are used to become a member. These forms may be a barrier when they are not easy to use (Preece et al., 2004). CAPTCHAs, or ‘‘Completely Automated Public Turing Test to Tell Computers and Humans Apart’’ (Von Ahn, Blum & Langford, 2004), can be used to check if the registration form for a community is filled out by a human, instead of a software program. The latter mainly wish to gain access to post spam-messages, deteriorating the community. However, CAPTCHAs may also be a hindrance for registering guests (Yan & El Ahmad, 2008). Profiles By adding member profiles to a community, members are able to learn more about their fellow members, without the need to browse all earlier posted messages (Preece et al., 2004). The ability to add free text and photos to member profiles is also measured. Posts When a member posts a message within the community, display of additional member information may lead to an increase in liking (Collins & Miller, 1994). Furthermore, a post may contain emoticons, which can increase member satisfaction and reduce miscommunication (Riviera, Cooke & Bauhs, 1996). Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

459

460


Posts

Sideline activities

Connections with other applications

Free text in profile Photos in profiles Profile picture with post

Meetings Advertising

Connections with social networks News section Information portal

Graphical emoticons in posts E-mail notification

Display of post count

Account activation Profile functionality

Guest access to posts Guest access to profiles Posting as guest Number of mandatory fields during registration Displaying house rules during registration CAPTCHA during registration Validation of e-mail address

Property

Profiles

Registration

Accessibility of data

Category

Table 2 Measured community properties

Are the house rules of a community presented to the user during the registration procedure? Does a CAPTCHA need to be completed during the registration procedure? Are e-mail addresses verified by sending an activation link to the address supplied by the user? Do new accounts need to be activated by a moderator? Do community members get a profile, on which they can display information about themselves? Are members able to display a text (about themselves) in their profiles? Are members able to add photos to their profiles? Is the profile picture of a member placed next to the posts the member has made? Is the total number of posts a member has made placed next to the posts the member has made? Can users add graphical emoticons to their posts? Can members be notified by e-mail, when other members reply to their messages? Is the community connected with social networks (Twitter and/or Facebook)? Does the community website host regularly posted news articles? Does the community website host a section with relevant information and/or links? Are face-to-face meetings organized for members of the community? Are advertisements displayed on the community website?

Can guests read posts of members? Can guests view the profiles of members? Are guests able to post messages, without being a member? How many fields need to be filled to complete the registration procedure?

Explanation


461

Moderation

Ownership and target group

Supply of information

Member appreciation

Category

Table 2 continued

Message posting by moderators

Selective target group

Private / organizational initiative

Frequently asked questions

Quantitative appreciation Qualitative appreciation House rules after registration

Property

Are members appreciated in a quantitative way for their contributions? Are members appreciated in a qualitative way for their contributions? Can members find the house rules of a community easily after the registration procedure has been completed? Is a document with answers to frequently asked questions available to members? Is the community the initiative of an organization, or is it a personal initiative of members? Is the target group of the community clearly defined and displayed within the community? Do moderators of a community post messages themselves within the community?

Explanation

Connections with other applications The community may be connected with other applications, like e-mail and social networks. By adding e-mail functionality, members may be notified when other members reply to their messages. These e-mails also serve as an invitation to visit the community again (Woodall et al., 2007). Sideline activities Next to being used for textual communication, communities may act as a hub for other services towards members. Examples include posting of relevant news and providing links to other relevant websites (Leimeister et al., 2004). Member appreciation The commitment of active members may be appreciated by positive feedback (Preece et al., 2004). This feedback may be quantitative or qualitative. Quantitative appreciation also provides a competitive element (Liu, Geng & Whinston, 2007). Supply of information Within communities, house rules provide a guideline for member behavior and discussions. A document with frequently asked questions (also known as a FAQ) may also be supplied to members. With these documents, members know what is and is not allowed within the community and the documents enable members to learn about community protocols (Preece et al., 2004). Ownership and target group People may set up communities to fulfill their own needs or those of others, or act on instruction of an organization. This results in private initiatives (member-initiated communities) and initiatives of organizations (organization-sponsored communities) (Porter, 2004). Porter states that these two groups of communities mainly differ with regard to the types of relationships that are formed. Private initiatives focus on personal and nonprofessional relationships, while initiatives of organizations focus on shared professional interests and knowledge-sharing. The amount of resources available to personally and organizationally backed initiatives can also differ, bringing different degrees of success. Furthermore, when a community is set up, a target group may be chosen. Selecting a target group is advisable (Preece et al., 2004), as it reduces wrong expectations. Moderation Next to members, another group of people that is present within communities are the moderators. The main goal of moderators is to monitor the activity within a community and penalize people who break the community’s house rules. Furthermore, they may also take part in discussions by posting messages and opening new topics (Preece et al., 2004; Williams et al., 2000). 2 Message board topics The topics that have been posted on message boards encompass the second group of community properties, as found in Table 1. These properties have been split into five distinct categories. These categories and the properties that they entail can be found below. Table 3 lists the properties that belong to these categories. 462


Table 3 Measured properties of message board topics Category Greetings and closure

Property Greeting Closure

Disclosure

Introduction or disclosure

Activation of readers

Request or question in title Request or question in body

Length

Poll Length of topic start Average sentence length

Social topics

Social topic

Explanation Does the first message within a topic contain a greeting? Does the first message within a topic contain a closure? Does the topic starter introduce him- or herself in the first message? Does the title of the topic contain a request or question? Does the first message within a topic contain a request or question? Is a poll available within the topic? How many characters does the first message within a topic contain? How many words does each sentence from the first message of a topic contain? Is the topic a social-oriented topic?

Greetings and closure Each message within a topic may contain a greeting at the start of the message and a closure at the end. The greeting and closure of the first message within a topic may influence the number of replies that follow, as readers experience polite e-mails in a more positive manner than rude ones (Jessmer & Anderson, 2001). Disclosure When a new member of a community wants to discuss a certain subject, he or she may open a new topic on the message board of this community. Next to introducing the subject the member wanted to discuss, additional information may be given as to why this subject is important to the member. This may be done by posting about the relationship a member has with the group (‘Group introduction’), or share a personal connection (‘Topic introductions’), as described by Burke et al., 2007. Both Burke et al. and Arguello et al. (2006) see a positive relation between member disclosure and the number of replies in Usenet newsgroups. The same relation may be found within message boards. However, as message boards often provide members with the ability to create member profiles, which can also be used to disclose personal information, the positive effect of disclosure in topics may be less pronounced. Activation of readers Members may add a request or question to their topics to activate members that read them (e.g. ‘Fill out our survey!’ or ‘What do you think about the current changes in healthcare plans?’). Earlier research by Joyce and Kraut (2006) and Burke et al. (2007) focused on requests and questions in the messages within the topic. Burke et al. show an increase in reply rate, when posted messages contain a request. Joyce and Kraut found the same result for newcomers (i.e. people who post their first message within a Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

463

community). However, the title of the topic is also important, as it needs to be selected by the user for the topic to display. Another way in which readers may be activated is by adding polls, as voting for a poll requires fewer actions than writing a reply yourself. Furthermore, the poll results give an easy to read overview of distribution of opinions within the community, which may spark further discussion.

Length The length of a message may also influence the number of replies the first message within a topic may receive. Message length may be expressed in different ways, e.g., the number of lines in a message (Arguello et al., 2006) or the number of words in a message (Jones, Ravid & Rafaeli, 2004). However, as the number of lines within a message presented to the member of a community may vary due to resolution and text size, amongst others, we found the number of words a more viable approach. Furthermore, when users cannot reply to all of the messages they want to reply to, they will pick the smaller ones (Jones et al., 2004), showing that shorter messages may receive more replies. Social topics Finally, a topic may be a ‘‘social topic.’’ This kind of topic is likely to be more approachable and we observed that the purpose often seems to be to relate to other persons in a social way. The subject is chosen in such a way that discussion of previous materials is not necessarily required and replies can be formulated with ease. It often concerns social small talk. Examples of social topics include: ‘‘What is your favorite band?’’ and ‘‘What is your next holiday destination?’’ 3 Member profiles The last of the three groups of properties, as mentioned in Table 1, focuses on the profiles members can create when joining a community. The categories and corresponding properties may be found in Table 4. Table 4 Measured member profile properties Category

Property

Relations between members

Number of messages received on profile Number of messages sent to other profiles Number of own friends

Disclosure via user profile

Number of times friend of other member Filled user profile

Added profile picture

464

Explanation How many messages has the owner of the current profile received on his profile? How many messages has the owner of the current profile sent to the owners of other profiles? The number of members marked as friend by the profile owner The number of members that marked the profile owner as a friend Has the profile owner filled out at least half of the questions in the profile? (e.g. age, gender, location and favorite pastimes) Did the profile owner upload at least one picture to his or her profile?


Relations between members Next to posting on the community’s message board, the community may support the option of messaging other people directly and placing messages on their community profile (e.g. Facebook’s ‘wall’). Furthermore, it may be possible for members to mark other members as friends, and list these friends on their profile. Disclosure via user profile People who introduce themselves in face-to-face meetings, or tell about their past (i.e. self-disclosure), are liked more than people who do not (Collins & Miller, 1994). The same is true in Usenet newsgroups (Arguello et al., 2006; also see ‘disclosure’ in the previous group). In this case, we aim to see which approaches to disclosure are effective: filling at least half of the questions in the user profile and adding at least one profile picture.

Community Activity Framework The online community properties or factors that had a significant effect (p < .05) on measures of success have been combined into the Community Activity framework (Figures 2, 3 and 4), which currently consists of 24 factors and 10 measures of activity. We want to stress that the included factors are based on this statistical analysis of 58 online communities. This CA framework contains both positive and negative relations between community factors and measures of success, and can be used during the development of new communities and redesigning current ones. The depicted relations have to be interpreted with care because other, unknown factors could be involved in explaining the relations that were found. Each of the figures focuses on one aspect of online communities: overall factors, message board topics, and member profiles. The framework is laid out in three columns of two different types: one for measures of success and two for community factors, for which the latter both hold the same meaning. Between these columns, lines are displayed, indicating a significant (p < .05) relationship. Positive (+) and negative (-) signs are used to indicate the kind of relation. An example can be found in the upper right-hand corner of Figure 2, with the community factor ‘Guest access to posts’ and the measure of success ‘Average number of topics per member.’ Between these two items, an arrowed line with a negative sign (-) can be found, showing a negative relation. This means that as guests are allowed to have access to posts, the average number of topics per member decreases. All of the relations between community factors and measures of success that were found within the Community Activity framework have also been combined in a set of guidelines, which can be used to increase community activity. The first group is based on the overall community factors, and contains the following 11 guidelines: 1. Create a document with frequently asked questions, but do not build a large information portal. 2. Do not be too selective when defining the target group of the community, so interested visitors will not be excluded. 3. Provide members of the community with a section that is not accessible to guests. 4. Provide members with the possibility to add one or more photo’s to their user profile. 5. Create a set of house rules, and let new members read and accept these during the registration procedure. 6. Let members use graphical emoticons in the messages that they post. 7. Notify members by e-mail of new posts in the topics they have subscribed to. Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

465

Figure 2 Community Activity Framework—Communities overall (Relations based on the statistical analysis; numbers in parentheses refer to the corresponding guideline)

Figure 3 Community Activity framework—Message board topics (Relations based on the statistical analysis; numbers in parentheses refer to the corresponding guideline)

8. Give members insight in the activity of others by displaying the total number of posts next to their messages. 9. Add a CAPTCHA to the registration procedure. It helps to stop automated (fake) account creation, but won’t influence the activity within a community. 10. Set up a section with periodical news postings. It may serve as a daily attractor of visitors. 11. Acquire organizational backing if possible, as it influences the total number of members positively. The second part of the framework, focusing on message board topics, consists of six relations between topic factors and measures of success. These relations have been combined in the following five guidelines: 12 Members do not need to add greetings and closures to each of their messages. 466


Figure 4 Community Activity framework—Member profiles (Relations based on the statistical analysis; numbers in parentheses refer to the corresponding guideline)

13 14 15 16

Put a request or question in the title, as it triggers people to react to the corresponding topic. Give members the option to add a poll to their topics, as it increases the average number of replies. Write in short and simple sentences, as long and complex sentences decrease the number of replies. Allow a few social topics, which are easily approachable.

The third and final part of the framework, focusing on member profiles, consists of six relations between member profile factors and measures of success. These relations have been combined in the following three guidelines: 17 Motivate members to fill out their profile and add a profile picture. 18 Give members the option to place messages on others’ profiles (e.g. ‘walls’ or guestbooks). 19 Guide members into befriending each other, as members with more friends receive more replies in their topics.

Case Study In the previous sections, the Community Activity framework has been introduced. The relations and guidelines of this framework will be used in this section to rejuvenate an inactive community on basis of the Community Activity framework and accompanying guidelines. By implementing guidelines 3 (profile access for guests), 4 (displaying active members), 6 (graphical emoticons), 7 (e-mail notification), 8 (insight in member activity) and 14 (polls) (also see the section Changes in the Leefwijzer Community) we aim to increase member interest in the interactive parts of a community (message board and member profiles) and increase user visits and activity. The guidelines other than those mentioned above were already present within the community. The community that has been selected for this case study is Leefwijzer (www.leefwijzer.nl), a Dutch community for people with a chronic illness and/or handicap. Following definitions of Jin, Park, and Kim (2010) it can be characterized as a noncommercial, organization-sponsored social community. The website is owned by the Dutch National Council of Health. While the website of the community is actively visited because of the regular posting of news and articles by moderators, the message board got only a few messages per week. To enhance the activity in the interactive parts of the community, the Community Activity framework has been applied. The most important changes in the website are described in the sections below. Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

467

Methodology Two methods were used to validate the changes made by the Community Activity framework: member surveys and usage statistics. The member surveys were used to get an insight to member interest in the interactive parts of the community. Logs with usage statistics were used to track user visits and activity. Both methods have been used twice: one time before renewing the community on 10 August 2010, and one time afterwards. Member survey As stated, two surveys have been conducted with the members of the Leefwijzer community. Goal of this survey was to measure the interest users had in the interactive parts of the community. The first of these surveys was conducted before changes were made to the community, from 2 February through 8 March 2010 (35 days) and had 60 participants; the second survey was conducted after changes had been made, from 27 through 30 August 2010 (4 days) and had 46 participants. The lower amount of participants in the second survey may be due to the shorter timeframe that was available to conduct this survey. Both surveys were held online. Members of the online community were notified of the survey by e-mail (in case the member was on the newsletter mailing list) and through an article on the front page of the community. This way, both frequent and infrequent members and guests of the community could be approached. Even when approaching community members, the responses to the surveys were anonymous. In both surveys, respondents have been asked for their interest in both the member profiles and the messages board on a 7 point Likert scale, where 1 equaled ‘not interested’ and 7 equaled ‘very interested.’ Because the survey was accessible in multiple ways, for both members and visitors, a response rate couldn’t be determined. Usage statistics next to the surveys, a statistical analysis of the usage statistics has been conducted for both the old and renewed community, which have been retrieved from the Leefwijzer community. The data for this analysis has been acquired in two intervals of 91 days each. The interval before the community renewal lasted from the 10 May through the 8 August 2010. The interval after the renewal of the community lasted from the 16 August through the 14 November 2010. Between the two intervals, the renewed community was launched on the 10 August 2010. Data collection was paused for 4 days after the launch of the new community, to let the members acclimatize to the new design. Based on the data that was recorded in both the old and new website, four metrics have been constructed, which will be used in the comparison of both logs with usage statistics. These metrics will be used to compare the activity within both intervals. These metrics are: ‘Number of visits per day,’ ‘Number of new topics per week,’ ‘Number of new messages per week,’ and ‘Number of sent private messages.’ Changes in the Leefwijzer Community This section contains the most important changes that have been made to the Leefwijzer community, based on the Community Activity framework and the corresponding guidelines. With each change, a reference is made to the guidelines of the CA framework in previous section. Profile access for guests (based on guideline 3). the CA framework shows a negative relation between guest access and the average number of posted topics and messages per member. Therefore, a set of privacy controls have been implemented in the website. With these controls, users can open or close any 468


Figure 5 Some of the user controlled privacy settings (Translations may be found on the right)

Figure 6 Display of active members in the old (top) and new (bottom) community (Translation of top image: ‘‘Recently online,’’ bottom image: ‘‘Members introduce themselves’’)

part of their profile to guests and members. A part of the implemented privacy settings can be found in Figure 5. Each member can specify who has access to the different parts of their user profile, e.g. access to the real name, birthdate, and pictures can be given to: everybody, members of the Leefwijzer community or nobody. This allows each member to adjust user profile access to personal needs.

Displaying active members (based on guideline 4) the homepage of the Leefwijzer community contained a bar with the names of recently active (logged in) members, without additional information (Figure 6). The CA framework shows however, that a positive relation exists between being able to add pictures to member profiles and the number of new topics and messages per year. Therefore, profile pictures have been added to the names shown in the bar with active members (Figure 6). Graphical emoticons (based on guideline 6) the CA framework shows a positive relation between the ability to use graphical emoticons in messages and the number of new messages per year. However, the community only supported textual emoticons. Therefore, support for graphical emoticons has been added (Figure 7). Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

469

Figure 7 Examples of posts with textual (top) and graphical (bottom) emoticons (Translation: ‘‘I completely agree with you!’’)

Figure 8 Part of member profiles (left) and posted messages (right) (Translations are placed next to the figures)

E-mail notification (based on guideline 7) the Leefwijzer community did not support the sending of e-mail based notifications on changes in topics. However, the CA framework shows that the total number of posted topics and messages within a community may profit from such notifications. Therefore, this functionality has been added to Leefwijzer. Now, users can subscribe to certain topics and receive updates by e-mail. Insight in member activity (based on guideline 8) on the old Leefwijzer community, little was known about the activity of other members. While member profiles were available, they did not contain information about past and current activity. This has been changed in the renewed community. Now, member profiles contain information about the total number of posts members have made (post count), the date of their last login and the date on which people became a member of the community (Figure 8, left). Furthermore, both the post count and joining date are shown next to messages of members (Figure 8, right), giving members direct access to some measures of member activity. Polls (based on guideline 14) next to typing posts on the message board, polls are another way in which members may have their opinion heard. The framework shows that on average, topics that have a poll, receive more posts 470


Figure 9 Poll on the new Leefwijzer homepage, showing different opinions a member may have on statement (Translations may be found on the right)

Table 5 Comparison of member surveys Interest in . . . Member profiles ∗∗ Message boards ∗∗

Website

Average (higher is better)

Std. dev.

Old New Old New

3.08 3.65 3.22 3.67

1.239 1.479 1.121 1.399

With ∗∗ for p < .01 than topics without a poll. Therefore, support for polls has been implemented in the new Leefwijzer community in two ways. First of these is the ability to add a poll to a topic on the message board. Secondly, moderators may select one of these polls (or add one themselves) and display it on the homepage of the community (Figure 9). By doing so, we aim to lower the threshold to reply to the poll and other discussions on the message board and increase the activity within the community.

Results of Changes in the Leefwijzer Community As described above, both member surveys and analysis of usage statistics have been used to validate the effectiveness of the changes made in the website. The results of both methods can be found below. Member survey The results from these surveys can be found in Table 5. Statistical analysis of these results shows that the interest among members for the profiles and message boards had risen significantly. An Independent Samples Mann-Whitney U-test shows a significant difference in both cases (z = -2.836, p < .01 and z = -2.612, p < .01 respectively). Usage statistics after finding that the data follow a normal distribution (by utilizing One-Sample Kolmogorov-Smirnov tests for both intervals), Independent Samples T-tests have been used to see if metrics differ significantly between both intervals. The results can be found below, in Table 6. Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

471

Table 6 Comparison of usage statistics Metric Number of visits per day ∗∗ Number of new topics per week ∗ Number of new messages per week ∗ Number of sent private messages With

∗

for p < .05 and

∗∗

Website

N

Average (higher is better)

Std. dev.

Old New Old New Old New Old New

91 91 13 13 13 13 13 13

423.18 477.81 1.00 5.69 4.38 22.46 43.62 51.00

88.36 112.72 1.16 5.20 4.99 10.38 16.73 29.24

for p < .01

From the results above, we can see that the number of visits per day differs significantly between both intervals (t(180) = -3.447, p < .01), from which we can conclude that the new website receives significantly more visits than the old site. A significant difference can also be found with the second metric, number of new topics per week, when comparing both intervals (t(13.180) = -3.175, p < .01). This shows that the number of new topics per week has increased significantly. A third significant effect can be found in the metric ‘number of new messages per week’: a comparison between both intervals shows significant results (t(17.269) = -5.658, p < .01), from which we can conclude that the number of new messages per week has increased significantly. The fourth metric, however, does not significantly differ between the two intervals (t(24) = -0.790, p > .05), from which we can conclude that the changes did not influence the number of sent private messages.

Conclusion and discussion In this study we have aimed to identify factors of online communities that had either a positive or negative relation within these communities. We have done so in order to find a solution for the inactivity that occurs in many online communities. To answer our research question, several steps have been taken. At first, a literature study has been done to identify possible factors that relate to activity in online communities. After comparing these factors within a set of online communities via statistical analysis, the factors that related significantly with the community activity metrics have been incorporated into the Community Activity framework. Of course we have to realize that the set of 58 communities that were analyzed can be characterized as social communities, focusing on health (issues) and related social support. The CA framework currently consists of 24 factors and 10 measures of community activity. Factors include, for instance, the positive effects of displaying house rules during registration, restricting community access for guests, notifying members about activity by e-mail and giving insight in member activity. Negative effects for private community initiatives, information portals and selective target groups have also been found. During the case study of an inactive online community, several changes have been made based on application of the CA framework and the corresponding guidelines. These focused changes include the addition of access control for members, the ability to receive e-mail notifications, allowing the usage of graphical emoticons and the display of members and their activity. Based on the positive results of both the member surveys and community usage statistics, we found that both the interest in the interactive 472


parts of the community and the actual usage of these parts have increased significantly, even after there had been some delay. Therefore, we can state that the CA framework has been applied successfully to this community, which gives it the potential to boost the success of other communities as well. As stated, we have been able to apply the CA framework successfully in the case study. It appeared possible with adding a restricted set of functionalities on basis of the CA framework to enhance interest and activity in this online community. However, to validate this positive effect more robustly, the CA framework will need to be applied to other communities as well. We restricted ourselves to a noncommercial, organization-sponsored social community. During this validation, a distinction could be made between different kinds of communities (e.g. social and technical online communities, cf. Porter, 2004; Jin, Park & Kim, 2010), to see if the effectiveness of the framework depends on the kind of community. Furthermore, the application of the CA framework does not guarantee community success. Although the framework does provide positive changes, the importance of other factors does not decrease. Communities still need, for instance, effective moderators to moderate posts by members and post messages themselves. The extension of the framework also provides future opportunities. An aspect not examined yet because it is more difficult to analyze, concerns the value of a community for its members. One could think of measures like content quality, response times, etc. For instance, Lin and Lee (2006) showed as can be expected that information quality has a significant influence on user satisfaction and behavioral intention to use the online community (see also Hew, 2009). While the framework currently harbors 24 different factors, more may be identified in future research. As the number of factors grows, it may be beneficial to weigh them, to be able to find the factors that influence community activity the most and to make a sensible decision to which guidelines to implement, when many are available. Related to the preceding remark, we purposely grouped our factors and guidelines into three groups: factors related to the interaction with the system, user-related factors (profiles of members) and factors concerning the message (topics) themselves. In our opinion we covered a great part of the information processing that is going in using systems that support online communities. However, we think it can be worthwhile to make more explicit and visualize important conditions and phases in using online community systems and connect these to the distinguished factors and guidelines, particularly those that appeared to be significant.

References Arguello, J., Butler, B. S., Joyce, E., Kraut, R., Ling, K. S., Rosé, C. & Wang, X. (2006, April). Talk to me: Foundations for successful individual-group interactions in online communities. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada), (pp. 959–968). NY: ACM Press. Burke, M., Joyce, E., Kim, T., Anand, V. & Kraut, R. (2007). Introduction and requests: Rhetorical strategies that elicit response in online communities. Proceedings of the Third Communities and Technologies Conference, 21–40. Butler, B. (1999). The dynamics of cyberspace: Examining and modeling online social structure. PhD thesis, Carnegie Mellon University, Pittsburgh, PA. Collins, N. & Miller, L. (1994). Self-disclosure and linking: A meta-analytic review. Psychological Bulletin, 116(3), 457–475. Fisher, D., Smith, M. & Welser, H. T. (2006). You are who you talk to: Detecting roles in usenet newsgroups. Proceedings of the 39th Hawaii International Conference on System Sciences. Journal of Computer-Mediated Communication 18 (2013) 454–475 © 2013 International Communication Association

473

Hew, K.F. (2009). Determinants of success for online communities: An analysis of three communities in terms of members’ perceived professional development. Behaviour & Information Technology, 28(5), 433–445. Himelboim, I. (2008). Reply distribution in online discussions: A comparative network analysis of political and health newsgroups. Journal of Computer Mediated Communication, 14, 156–177. Holtzblatt, L.J. & Damianos, L.E. (2011). Measuring the success of on-line communities in an enterprise networking tool. In A.A. Ozok and P. Zaphiris (Eds.), Online communities. HCII 2011, LNCS 6778, (pp. 188–196). Berlin, Heidelberg: Springer Verlag. Jessmer, S. & Anderson, D. (2001). The effect of politeness and grammar on user perceptions of electronic mail. North American Journal of Psychology, 3(2), 331–346. Jin, B., Park, J.Y., & Kim, H.-S. (2010) What makes online community members commit? A social exchange perspective. Behaviour & Information Technology, 29, 587–599. Jones, Q., Ravid, G. & Rafaeli, S. (2004). Information overload and the message dynamics of online interaction spaces: A theoretical model and empirical exploration. Information Systems Research, 15(2), 194–210. Joyce, E. & Kraut, R. (2006). Loyalty: Predicting continued participation in newsgroups. Journal of Computer Mediated Communication, 11, 723–747. Kollock, P. & Smith, M. (1996). Managing the virtual commons: cooperation and conflict in computer communities. In S. Herring (Ed.), Computer-mediated communication: Linguistic, social, and cross-cultural perspectives. Amsterdam: John Benjamins. Kollock, P. (1997). Design principles for online communities. The Internet and Society: Harvard Conference Proceedings. Boston: O’Reilly. Leimeister, J., Sidiras, P. & Krcmar, H. (2004). Success factors of virtual communities from the perspective of members and operators: An empirical study. Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS’04). Levine, J. & Moreland, R. (1994). Group socialization: Theory and research. In W. Hewstone (Ed.), European review of social psychology (pp. 283–308). NY: John Wiley & Sons. Lin, H.F. & Lee, G.-G. (2006). Determinants of success for online communities: An empirical study. Behaviour & Information Technology, 25(6), 479–488. Liu, D., Geng, X. & Whinston, A. (2007). Status seeking and the design of online entertainment communities. Annals of Information Systems, 1(4), 281–304. Maloney-Krichmar, D. & Preece, J. (2005). A multilevel analysis of sociability, usability and community dynamics in an online health community. Transactions on Human-Computer Interaction, 12(2), 201–232. Markus, M. L. (1987). Toward a ”critical mass” theory of interactive media: Universal access. Communication Research, 14(5), 491–511. Nielsen, J. (2006). Jakob Nielsen’s Alertbox: Participation inequality: Encouraging more users to contribute. Retrieved 2 October 2009 from http://www.useit.com/alertbox/participation_inequality.html. Nonnecke, B. & Preece, J. (2000). Lurker demographics: Counting the silent. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 73-80. Pfeil, U., Zaphiris, P. & Wilson, S. (2010). The role of message-sequences in the sustainability of an online support community for older people. Journal of Computer Mediated Communication, 15, 336–363. Porter, C.E. (2004). A typology of virtual communities: a multi-disciplainary foundation for future research. Journal of Computer-Mediated Communication, 10(1). 474


Preece, J., Nonnecke, B. & Andrews, D. (2004). The top five reasons for lurking: improving community experiences for everyone. Computers in Human Behavior, 20(2), 201–223. Rafaeli, S. & Sudweeks, F. (1997). Networked interactivity. Journal of Computer-Mediated Communication, 2(4). Riviera, K., Cooke, N. & Bauhs, J. (1996). The effects of emotical icons on remote communication. Proceedings of the CHI ’96 (pp. 99-100). New York, NY: ACM Press. Sangwan, S. (2005). Virtual community success: A uses and gratifications perspective. Proceedings of the 38th Hawaii International Conference on System Sciences. Schoberth, T., Preece, J. & Heinzl, A. (2003). Online communities: A longitudinal analysis of communication activities. Proceedings of the Hawaii International Conference on System Sciences (HICSS’03). Von Ahn, L., Blum, M. & Langford, J. (2004). Telling humans and computers apart automatically. Communications of the ACM, 47(2), 56–60. Walther, J. & Boyd, S. (2002). Attraction to computer-mediated social support. In C. Lin, & D. Atkin (Eds.), Communication technology and society: Audience adoption and use (pp. 153–188). Cresskill, NJ: Hampton Press. Williams, K. D., Cheung, C. K. T. & Choi, W. (2000). Cyberostracism: Effects of being ignored over the internet. Journal of Personality & Social Psychology, 79(5), 748–762. Woodall, W., Buller, D., Saba, L., Zimmerman, D., Waters, E., Hines, J., Cutter, G. & Starling, R. (2007). Effect of emailed messages on return use of a nutrition education website and subsequent changes in dietary behavior. Journal of Medical Internet Research, 9(3). Yan, J. & El Ahmad, A. (2008). Usability of CAPTCHAs, or usability issues in CAPTCHA design. Proceedings of the 4th Symposium on Usable Privacy and Security. 337. New York, NY: ACM Press.

About the Authors Ferdy van Varik received his Masters degree from Utrecht University, after graduating from both a Bachelor and Master program in Information Science. He is currently pursuing a PDeng degree from the Eindhoven University of Technology, through the User System Interaction program. Current research interests include Human Computer Interaction and Computer Mediated Communication. Email: [email protected] Herre van Oostendorp is Associate Professor Human-Media Interaction. His background is Cognitive Science. His research interests are cognitive principles in Serious Games, and cognitive modeling of Web Navigation. Email: [email protected]


475