Reliability and Validity of Community Power Questionnaire Instrument (CPQI)
Mohamed Ismail Mohideen Bawa Senior Lecturer in Management Department of Management Faculty of Management and Commerce South Eastern University of Sri Lanka Oluvil
[email protected] 0094 77 69 444 77.
1
Abstract Researchers face difficulties in correctly knowing reliability and validity of their questionnaires. Reliability and validity can be statistically calculable for reliability and validity. Research may be questionable without reliability and validity of a questionnaire. On this base, this study attempts at knowing reliability and validity of a questionnaire instrument of community power. 5 types of powers such as legitimate, reward, coercive, referent and expert powers discussed by Burton and Thakur (1995) are applied for community managers or leaders. Researcher uses a non- probability technique for selecting sample size. A sample size of 100 community leaders was studied. Data are collected during the period of 2012 and 2013 using primary source of data collection. Questionnaire is used as an instrument of data collection. Data are presented in pie chart. Reliability is analysed by Cronbach alpha. Validity is analysed by convergent and discriminant validity with the aid of statistical software such as SPSS and Excel. Study found that legitimate power, reward power, coercive power, referent power and expert power have Cronbach's Alpha value of 0.856, 0.694, 0.116, 0.963 and 0.557. All powers except coercive power have high reliability. All powers except coercive power have higher convergent validity and lower discriminating validity. Questionnaire has high construct validity.
Keywords: Community Power Instrument, Reliability, Validity.
Introduction
Mason and Bramble (1989) studied about reliability and validity. Reliability determines how consistently a measurement of skill or knowledge yields similar results under varying conditions. If a measure has high reliability it yields consistent results. There are four principal ways to 2
estimate the reliability of a measure. Validity or face validity is defined as the degree to which the instrument measures what it’s supposed to measure. If an instrument is not reliable over time, it cannot be valid, as results can vary depending upon when it is administered. An instrument can be neither reliable nor valid, be reliable and invalid or be both reliable and valid. However, an instrument must be reliable in order to be valid. Afzalur (1989) studied about the relationships of leader power to compliance and satisfaction with supervision with the evidence from a national sample of managers. They investigated the effectiveness of the bases of leader power, such as coercive, reward, legitimate, expert, and referent in influencing behavioral compliance with superior's wishes and satisfaction with supervision. Regression analyses of data from a national random sample of managers (N = 476), after demographic, job-related, and other extraneous variables were controlled for, showed that expert and referent power bases were positively associated with compliance and satisfaction and legitimate power base was positively associated with compliance but negatively associated with satisfaction. Study reveals that managers have different types of powers. It can be argued that not only organizational leaders but also community leaders also have these powers. Community power is the power that is held by community leaders who carry out different enterprises in their community. Community leaders are coconut farm owners, paddy land lords, livestock owners, other food crop land owners, fishing boat owners, quarry owners, cottage industrialists, housing contractors, wholesalers, retail traders, hoteliers, restaurant operators and land transactors.
Statement of the problem
Foa, et. al., (1993) studied about reliability and validity of a brief instrument for assessing posttraumatic stress disorder (PTSD). This study examines the psychometric properties of two
3
versions of the PTSD symptom scale (PSS). The scale contains 17 items that diagnose PTSD according to DSM-III-R criteria and assess the severity of PTSD symptoms. An interview and self-report version of the PSS were administered to a sample of 118 recent rape and non-sexual assault victims. The results indicate that both versions of the PSS have satisfactory internal consistency, high test-retest reliability, and good concurrent validity. The interview version yielded high inter-rater agreement when administered separately by two interviewers and excellent convergent validity with the SCID. When used to diagnose PTSD, the self-report version of the PSS was somewhat more conservative than the interview version. Scahill, Riddle, McSwiggin-Hardin, Ort, King, Goodman, Cicchetti and Leckman (1997) studied about children's yale-brown obsessive compulsive scale [CY-BOCS] using reliability and validity. 25 female and 40 male children having the age of 8 to 17 years with obsessive-compulsive disorder were assessed with the said scale. Inter-rater agreement was assessed by 4 raters in a subsample (n = 24). Discriminant and convergent validity were assessed by comparing CY-BOCS scores to selfratings of depression, anxiety, and obsessive-compulsive symptoms. Results show that internal consistency was high, measuring 0.87 for the 10 items. The intra-class correlations for the CYBOCS total, obsession, and compulsion scores were 0.84, 0.91, and 0.66, suggesting good to excellent inter-rater agreement for subscale and total scores. The CY-BOCS Total score showed a significantly higher correlation with a self-report of = obsessive-compulsive symptoms (r.62 for the Leyton survey) compared = with the Children's Depression Inventory (r.34) and the Children's = Manifest Anxiety Scale (r = .37) (p.02 and .05, respectively). The CY-BOCS yields reliable and valid subscale and total scores. Reliability and validity appear to be influenced by the sample’s age and the hazards associated with integrating data from parental and patient sources. Questionnaire has to be tested with for its internal consistency using
4
measures such as Cronbach alpha. However, reliability for a questionnaire is not enough. It has to be tested with its validity.
Research gap Foa, et. al., (1993) studied about reliability and validity. Few studies have been carried out in post-traumatic stress disorder (PTSD) using interview and questionnaire considering a sample size 118. Year Scahill, Riddle, McSwiggin-Hardin, Ort, King, Goodman, Cicchetti and Leckman (1997) studied reliability and validity. Some other studies have been carried out in Children's Yale-Brown Obsessive Compulsive using scale considering 25 females and 40 males. These studies were conducted in different contexts in different countries. There are few research studies that detail about reliability and validity of a questionnaire instrument. Most of the researches have been carried out in superficially or on the surface of validity and reliability. Therefore, this present study analyses reliability and validity of community power questionnaire instrument in Ampara district, Eastern Province of Sri Lanka.
Research question and objectives Previous findings confirm that research issue rest on two concepts such as reliability and validity. This study attempts at knowing reliability and validity of a community power questionnaire instrument (CPQI). Significances of the study
This study is significant in several ways. First, Colin and Julie (2005) stated that reliability is the degree to which an assessment tool produces stable and consistent results. Second, validity is the
5
strength of our conclusions, inferences or propositions. Third, reliability and validity of questionnaire has been tested in different parts of the Word. For instance, Ware, Kosinski and Keller (1996) studied about a 12-item short-form health survey with regard to construction of scales and preliminary tests of reliability and validity in US. Keane, Caddell and Taylor (1988) studied about reliability and validity in Vietnam. This study is important to academics, practitioners, researchers, and industries to know clarity about reliability and validity. Reliability and validity can be tested by statistical concepts such as Cronbach alpha and correlation value. Non- statistical academicians can get familiarity about these terminologies for the betterment of their research works. Reliability and validity would be beneficial for contextualizing and customizing the questionnaire to a domestic country profile.
Review of Literature
Berstein and Putnam (1996) studied about development, reliability and validity of a dissociation scale. Item-scale score correlations were all significant, indicating good internal consistency and construct validity. Ware, Kosinski and Keller (1996) studied about a 12-item short-form health survey with regard to construction of scales and preliminary tests of reliability and validity. Testretest (2-week) correlations of 0.89 and 0.76 were observed for the 12-item physical component summary and the 12-item mental component summary in the general US population (n = 232). Validity tests involving physical criteria, relative validity estimates for the 12-item physical component summary and mental component summary ranged from 0.43 to 0.93 (median = 0.67) and 0.60 to 1.07 (median = 0.97). Bernstein, Fink, Handelsman and Foote (1994) studied about initial reliability and validity of a new retrospective measure of child abuse and neglect using the reliability and validity of the childhood trauma questionnaire (CTQ) for 286 drug- or alcohol-
6
dependent patients (aged 24–68 yrs). The CTQ demonstrated high internal consistency and good test-retest reliability over an interval of 2–6 months. The CTQ also demonstrated convergence with the childhood trauma interview, indicating that samples reports of child abuse and neglect based on the CTQ were highly stable, both over time and across types of instruments.
Yukl and Falbe (1991) studied the types of power used to influence subordinates and peers. The study involved 2 taxonomies for conceptualizing power, 2 questionnaires for measuring power, 2 outcome criteria, 2 levels of management, and 2 respondent samples. Results indicate that managers had more position power over subordinates than over peers. Downward reward and coercive power were greater for middle managers than for lower level managers. Legitimate power, expert power, and agent persuasiveness were the 3 most important reasons reported for doing what a peer or boss requested. Personal power was more important than position as a determinant of task commitment and ratings of managerial effectiveness. Hakan and Jamel (2006) studied about the relationship between leadership power bases and job stress of subordinates from the example from boutique hotels. A total of 400 subjects (20 managers and 380 non-managerial employees) participated in this study from 20 boutique hotels. Participants were told that the study was designed to collect information on the leadership power bases used by first line managers and on the job stress levels of employees in the hospitality workforce. The Rahim Leader Power Inventory and Spielberger and Vagg's Job Stress Survey were used to assess leadership power bases and job stress, respectively. Research found that there are significant relations between leader power bases and subordinates’ job stress. The findings support the suggestion in the literature that positional power bases stimulate job stress in the hospitality industry. Taggart and Farzad (1992 studied that the relationship between leader power 7
base and influence. One of the most significant factors influencing the quality of leader-member relations and leader effectiveness is trust. The purpose of this study is to examine the moderating role of trust between a leaders’ power base and their ability to influence their subordinates. The sample consisted of 99 participants in which they responded to a questionnaire that measured 1) the leader’s power base (legitimate, coercive, reward, expert, referent, and information) according to French and Raven’s (1959) research, 2) the leader’s level of influence, and 3) the leader-member level of trust. The results indicate that all bases on power have significant correlations with the trust variable. The trust-influence relationship and the power influence relationship all showed a significant and positive correlation. The results also indicate that the trust variable significantly affects the influence-power base relationships. Conceptual Model and Operationalisation Burton and Thakur (1995) discussed 5 types of powers such as legitimate, reward, coercive, referent and expert powers that are necessary for organisational managers or leaders. These five types of powers needed for managers or leaders are combined and applied to community leaders. Therefore, this conceptual model deviates from previous models in the way of community leadership and combination of powers termed as “total power”. Total power is combination of all 5 types of powers denoted by Burton and Thakur (1995). Adapted conceptual model is shown in figure 1. Figure 1: Total power
8
Legitimate power
Reward power
Coercive power
Total power
Referent power
Expert power (Source: Adopted from Burton and Thakur, 1995) Based on the above conceptualization, operationalisation is tabulated in table 1: Table 1: Operationalization Concept
Power
Construct/variable/factor Measure/ item Legitimate power 1. Rules 2. Policies 3. Regulations Reward power 4. Wage 5. Recommendations 6. Praise Coercive power 7. Layoff 8. Fire 9. Verbal warning 10. Terminations Referent power 11. Identification 12. Loyalty 13. Charisma Expert power 14. Information 15. Expertise 16. Specialism
Source
Burton and Thakur (1995); Yukl and Falbe (1991); Hakan and Jamel (2006); Taggart and Farzad (1992); (Khan and Kram, 1994).
(Source: Adopted from Burton and Thakur, 1995) Methodology
9
Researcher could not get sampling frame of community leaders. Researcher uses a nonprobability technique for selecting sample size. A sample size of 100 community leaders was studied. Data are collected during the period of 2012 and 2013 using primary source of data collection. Questionnaire is used as an instrument of data collection. Data are presented in pie chart. Reliability is analysed by Cronbach alpha. Validity is analysed by convergent and discriminant validity with the aid of statistical software such as SPSS and Excel. Results and discussion of findings Pie chart Community leaders such as wholesalers, retail traders, hoteliers, and land lord owners occupy first four places in community. Those of fishing boat owners, cottage industrialists, restaurant operators and coconut garden owners occupy the second four places in society. Those of livestock owners, other food crop, housing contractors and land transactors occupy the third four places. Quarry owners occupy the least among community. They are depicted in figure 2. Figure 2: Community leaders
10
community leaders
8, 8.0%
2, 2.0%
7, 7.0% 11, 11.0%
12, 12.0%
6, 6.0%
4, 4.0% 13, 13.0%
Category Coconut garden owners Paddy land lords Liv estock owners Other food crop land owners Fishing boat owners Quarry owners Cottage industrialist Housing contractors Wholesalers Retail traders Hoteliers Restaurant operators Land transactors
10, 10.0% 1, 1.0% 14, 14.0% 3, 3.0%
9, 9.0%
(Source: survey data) Reliability Legitimate power is measured by 3 items such as organisational rules, policies and regulations. Value of Cronbach's Alpha is 0.856. Reward power is measured by 3 items such as wage, recommendations and praise. Value of Cronbach's Alpha is 0.694. Coercive power is measured by 4 items such as verbal warning, fire, terminations and layoff. Value of Cronbach's Alpha is 0.116. Referent power is measured by 3 items such as identification, loyalty and charisma. Value of Cronbach's Alpha is 0.963. Expert power is measured by 3 items such as information, expertise and specialism. Value of Cronbach's Alpha is 0.557. Reliability statistics for the items are shown in table 2.
11
Table 2: Reliability statistics Power Legitimate Reward Coercive power Referent power Expert power
Cronbach's Alpha 0.856 0.694 0.116 0.963 0.553
N of Items 3 3 4 3 3
(Source: survey data) Validity Convergent validity refers to how items are related to construct. Discriminant validity refers to how items are unrelated to construct. Construct validity refers to how constructs are related to each other.
Convergent and discriminant validity Command, unwritten policies and regulatory controls are converging 78%, 88% and 92% respectively with legitimate power. They are discriminating 22%, 12% and 8% respectively with legitimate power. Changing wages, recommendation and praise are converging 98%, 100% and 29% respectively with reward power. They are discriminating 2%, 0% and 71% respectively with reward power. Layoff, oral warning, firing and termination are converging 56%, 89%, 88% and 5% respectively with coercive power. They are discriminating 44%, 11%, 12% and 95% respectively with coercive power. Identification, loyalty and attraction are converging 97%, 98% and 88% respectively with referent power. They are discriminating 3%, 2% and 12% respectively with referent power. Information, expertise and specialism are converging 85%, 87% and 38% respectively with expert power. They are discriminating 15%, 13% and 62% respectively with expert power. Convergent and discriminant validity is shown in table 3. 12
Table 3: Convergent and discriminant validity
Power
Items
legitimate power
Command, unwritten policies and regulatory controls Reward Changing wages, power recommendation and praise Coercive Layoff, oral power warning, firing and termination Referent Identification, power loyalty and attraction Expert Information, power expertise and specialism (Source: survey data)
Number of items 03
Convergent Discriminant Sig. validity validity 0.78, 0.88 and 0.22, 0.12 0.000,0.000 and 0.92 and 0.08 0.000
03
0.98, 0.29
04
0.56, 0.89, 0.44, 0.11, 0.000, 0.000 0.88 and 0.05 0.12 and 0.95 and 0.636
03
0.97, 0.98 and 0.03, 0.02 0.000, 0.000 0.88 and 0.12 and 0.000
03
0.85, 0.87 and 0.15, 0.13 0.000, 0.000 0.38 and 0.62 and 0.000
1
and 0.02, 0 and 0.000, 0.000 0.71 and 0.003
Construct validity Expert power correlates 17%, 97%, 50% and 63% respectively with legitimate, reward, coercive and referent power. It is uncorrelated 83%, 3%, 50% and 37% respectively with legitimate, reward, coercive and referent power. Construct validity is shown in table 4. Table 4: Construct validity Power Expert power Sig.
Legitimate power 0.168 0.094
Reward power 0.968 0.000
Coercive power 0.504 0.000
Referent power -0.627 0.000
(Source: Construct validity)
13
Findings and conclusions
Results found that legitimate power, reward power, coercive power, referent power and expert power have Cronbach's Alpha value of 0.856, 0.694, 0.116, 0.963 and 0.557. All powers except coercive power have high reliability. Coercive power measured by 4 items such as verbal warning, fire, terminations and layoff have to be redesigned. These items have to be redefined. In terms of convergent and discriminant validity, layoff, oral warning, firing and termination are converging 56%, 89%, 88% and 5% respectively with coercive power. They are discriminating 44%, 11%, 12% and 95% respectively with coercive power. All powers except coercive power have higher convergent validity and lower discriminating validity. With regard to construct validity, expert power correlates 17%, 97%, 50% and 63% respectively with legitimate, reward, coercive and referent power. It is uncorrelated 83%, 3%, 50% and 37% respectively with legitimate, reward, coercive and referent power. It is concluded that 4 out of 5 powers are both highly reliable and valid. Questionnaire has high construct validity.
Limitations of the study and further research chances
Population frame is not accessible to the researcher to undertake a probability sampling. Sample size may or may not be enough to deal with the study. This study may be carried out using probability sampling method. Colin and Julie (2005) said that there are 4 types of reliability such as test-retest, parallel forms reliability, inter-rater reliability & internal consistency reliability and 5 types of validity such as face, construct, criterion, formative and sampling. This study considers only internal consistency of reliability and construct of validity. Other aspects of
14
reliability and validity are ignored. Researcher allows other researcher to remove these limitations and to extend this research as much as possible in different questionnaires of different research areas.
Managerial implications
This study is important to academics, researchers and others to imply the highlights about reliability and validity of questionnaire instrument. Researchers can know how items and constructs can be reliable and valid in their researches.
References
Afzalur, R, A. (1989), “ Relationships of Leader Power to Compliance and Satisfaction with Supervision: Evidence from a National Sample of Managers”, Journal of Management, Vol. 15 No.4 pp. 545-556
Berstein, E. and Putnam, F. (1996), “Development, Reliability, and Validity of a Dissociation Scale”, Journal of Nervous & Mental Disease, Iss. December.
Bernstein, D. P., Fink, L., Handelsman, L. and Foote, J. (1994), “Initial reliability and validity of a new retrospective measure of child abuse and neglect”, The American Journal of Psychiatry, Vol. 151 No. 8, pp. 1132-1136.
Burton, G. and Thakur, M. (1995), “Management today: principles and practice”, p. 356., Tata McGraw-Hill publishing company, New Delhi.
Colin, P. and Julie, W. (2005), “Exploring reliability in academic assessment”, Graduate Assistants, UNI Office of Academic Assessment (2005-06).
15
Foa, E. B., Riggs, D. S., Dancu, C. V. and Rothbaum, B. O. (1993), Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. J. Traum. Stress, 6: 459– 473.
Hakan, V. E. and Jamel, C. (2006) "Relationship between leadership power bases and job stress of subordinates: example from boutique hotels", Management Research News, Vol. 29 Iss: 5, pp.285 – 297
Ismail, M. B. M., (2013), “Role of Community Governance on Economic Development in Ampara Coastal District” A working paper.
Keane, T. M., Caddell, J. M. and Taylor, K. L. (1988), “reliability and validity”, Journal of Consulting and Clinical Psychology, Vol. 56 No. 1 pp. 85-90.
Mason, E. J., & Bramble, W. J. (1989). Understanding and conducting research (2nd ed.). New York: McGraw-Hill.
Scahill, L., Riddle, M. A., McSwiggin- Hardin, M., Ort, S. I., King, R. A., Goodman, W. K., Cicchette, D. and Leckman, J. F. (1997), “Children’s Yale- Brwon Obsessive Compulsive Scale: Reliability and Validity”, Journal of the American Academy of Child, Adolescent Psychiatry, Vol 36 No. 6, pp. 844- 852.
Taggart, F. F. and Farzad, M. (1992), “The Relationship Between Leader Power Base And Influence: The Moderating Role Of Trust”, Journal of applied business research, Vol. 8, No 4.
Ware, J., Kosinski, M. and Keller, S. (1996), “A 12-Item Short-Form Health Survey: Construction of Scales and Preliminary Tests of Reliability and Validity”, Medical Care, Vol. 34 Iss. 3, pp 220-233
16
Yukl, G. and Falbe, C. M. (1991), “Importance of different power sources in downward and lateral relations”, Journal of Applied Psychology, Vol. 76 No. 3, pp. 416-423.
17
