Measurement of information and communication technology ...

4 downloads 46523 Views 155KB Size Report
computer use, information and communication technology skills, satisfaction and ...... between associate and degree students, more practical experi- ence of ...
JAN

JOURNAL OF ADVANCED NURSING

REVIEW PAPER

Measurement of information and communication technology experience and attitudes to e-learning of students in the healthcare professions: integrative review Ann Wilkinson, Alison E. While & Julia Roberts Accepted for publication 17 November 2008

Correspondence to A. Wilkinson: e-mail: [email protected] Ann Wilkinson BA MPhil RSW Lecturer in Learning Technology King’s College London, Florence Nightingale School of Nursing and Midwifery, UK Alison E While BSc PhD RN Professor of Community Nursing King’s College London, Florence Nightingale School of Nursing and Midwifery, UK Julia Roberts PhD RN RCNT Senior Lecturer Head of Department of Specialist Care, King’s College London, Florence Nightingale School of Nursing and Midwifery, UK

W I L K I N S O N A . , W H I L E A . E . & R O B E R T S J . ( 2 0 0 9 ) Measurement of information and communication technology experience and attitudes to e-learning of students in the healthcare professions: integrative review. Journal of Advanced Nursing 65(4), 755–772 doi: 10.1111/j.1365-2648.2008.04924.x

Abstract Title. Measurement of information and communication technology experience and attitudes to e-learning of students in the healthcare professions: integrative review. Aim. This paper is a report of a review to describe and discuss the psychometric properties of instruments used in healthcare education settings measuring experience and attitudes of healthcare students regarding their information and communication technology skills and their use of computers and the Internet for education. Background. Healthcare professionals are expected to be computer and information literate at registration. A previous review of evaluative studies of computer-based learning suggests that methods of measuring learners’ attitudes to computers and computer aided learning are problematic. Data sources. A search of eight health and social science databases located 49 papers, the majority published between 1995 and January 2007, focusing on the experience and attitudes of students in the healthcare professions towards computers and e-learning. Review methods. An integrative approach was adopted, with narrative description of findings. Criteria for inclusion were quantitative studies using survey tools with samples of healthcare students and concerning computer and information literacy skills, access to computers, experience with computers and use of computers and the Internet for education purposes. Results. Since the 1980s a number of instruments have been developed, mostly in the United States of America, to measure attitudes to computers, anxiety about computer use, information and communication technology skills, satisfaction and more recently attitudes to the Internet and computers for education. The psychometric properties are poorly described. Conclusion. Advances in computers and technology mean that many earlier tools are no longer valid. Measures of the experience and attitudes of healthcare

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

755

A. Wilkinson et al.

students to the increased use of e-learning require development in line with computer and technology advances. Keywords: attitudes, e-learning, experience, health professions education, information and communication technology, literature review, measurement, nurse education

Introduction

The review

Successive reports on emerging technologies relevant to higher education (New Media Consortium, 2006, 2007, 2008) indicate that the information literacy of students entering higher education is not improving and that academics are not keeping pace with the potential of mobile technologies for education. Universities are expected to educate healthcare professionals who, at the point of registration, are computer and information literate. In the United Kingdom, the Knowledge and Skills Framework (Department of Health, 2004) requires registered healthcare professionals to be aware of and keep up-to-date with the knowledge base of their professions. Core skills include the ability to use electronic libraries, critically appraise evidence for healthcare, and provide health information for service users. Studies worldwide suggest that healthcare professionals are not confident at the point of qualification. For example, 53% of graduating Finnish nurses reported positive attitudes to using IT for nursing but inadequate IT skills for practice (Saranto et al. 1997). Similarly, a Canadian study (Balen & Jewesson 2004) showed that registered pharmacists required additional information and communication technology (ICT) skills for effective practice. Elsewhere it is reported that medical students are disadvantaged by poor access to online knowledge resources needed to improve health outcomes (Bello et al. 2004, Samuel et al. 2004). A national study of Australian nurses identified both limited preregistration training and lack of confidence with computers by Registered Nurses (Hegney et al. 2006). A UK literature review of computer-based learning (CBL) (Lewis et al. 2001) showed that there were difficulties in measuring students’ attitudes to CBL and no reliable instruments. In addition, Lewis et al. (2001) and Chumley-Jones et al. (2002) in the United States of America (USA) noted that there was no good method for differentiating between different learners and that more careful attention needed to be paid to these variables in future studies. 756

Aim The aim of the review was to describe and discuss the psychometric properties of instruments used in healthcare education settings measuring experience and attitudes of healthcare students’ regarding their ICT skills and their use of computers and the Internet for education.

Design A quantitative methodological review using an integrative approach with narrative description of findings, as described by Whitemore and Knafl (2005), was chosen to permit the inclusion of studies using a variety of methods.

Search methods The literature on computer experience and attitudes is multidisciplinary; as a consequence, a separate search strategy was formulated for each of the major bibliographic databases in health, education and social science (Table 1). There were three variables of interest: e-learning; attitude and experience; and the healthcare professions. The search was limited to papers written in English from 1995 to January 2007. This timeframe coincides with the transition from small developments of CBL to ubiquitous Internet and ICTs (Smith 2005) and worldwide expansion of higher education (Duke 2002, Cornford & Pollock 2003). Preliminary searches, Zetoc Alerts and reference tracking yielded earlier studies.

Search outcome The results (n = 2628) were imported into Endnote and initial filtering removed duplicates, non-peer reviewed papers, items about patient education, staff/educator experience, satisfaction studies and measures of learning styles, leaving 292 items for detailed appraisal (Figure 1). The main exclusion criteria were: instrument insufficiently described; experimental study

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

JAN: REVIEW PAPER

Literature review measurement tools

Table 1 Keywords used in database searches

Data abstraction and synthesis

Database Keywords

Studies were synthesized under the following subheadings: country, author and date of publication, study design, sample and response rate, data collection instrument, number of items in instrument and psychometric testing properties. Where substantial data were missing, studies were excluded. Forty-nine papers remained (Table 2).

CINAHL Internet, World Wide Web, Information Technology, Computers and Computerization’, computer literacy, exp computerized educational testing or computerized adaptive testing, Computer Environment, Computer Assisted Instruction, virtual learning, VLE, (web-based learning or web-based learning), (online adj3 learning), (e-learning or elearning), (electronic adj3 learning), (computer$ adj3 learn$), confidence, Fear, anxiety or anticipatory anxiety, Aptitude, access to information or remote access to information, expectation$, student attitudes or student satisfaction, nursing education, midwifery education, education, allied health or education, dental hygiene or education, occupational therapy or education, physical therapy or education, physician assistants or education, social work Specific search strategies were developed for BNI, Medline, ERIC, BEI, AEI Web of Knowledge (ISI), PsycINFO and these may be requested from the corresponding author.

with no measure of attitudes or experience of students; nonhealth; and unpublished. Certain key papers pre-1995 which provided evidence of a history of scale development (e.g. Stronge & Brodt 1985, Parks et al. 1986) or instruments from non-health settings which were adopted for healthcare education (e.g. Maurer & Simonson 1984) were included.

Quality appraisal Quality appraisal of individual studies was not undertaken as the main purpose of the methodological review was to determine psychometric properties of instruments. One instrument has been extensively revalidated (Table 4).

Results The studies originated from North America (n = 26), Europe (n = 15) and the rest of the world (n = 8). Although there was evidence that tool development was incremental, with researchers drawing on previous studies, there was inconsistent reporting of the research methods used and it was not always clear whether the instruments had been validated. The majority of instruments reported originated in North America and were derived from those first developed in the 1980s. Three approaches to instrumentation were identified; first, the development and validation of instruments which were subsequently used in healthcare education settings (e.g. Loyd & Gressard 1984, Maurer & Simonson 1984, Jayasuriya & Caputi 1996); second, those drawing on previous studies to reproduce or revalidate a refined or composite instrument (e.g. Agho & Williams 1995, Sinclair & Gardner 1999, Lynch et al. 2000, Billings et al. 2001, Cragg et al. 2003, DeBourgh 2003, Brumini et al. 2005, Sit et al. 2005, Maag 2006); and, third, a small number of instruments were author-created (e.g. Hollander 1999, Ray & Hannigan 1999, Dorup, 2004; Balen & Jewesson 2004, Bello et al. 2004).

Combined results of database searches (n = 2628)

Excluded (n = 2336) Patient education, Staff/Faculty experience Satisfaction studies Measures of learning styles Qualitative studies Clinical education Simulation studies

Figure 1 Selection of papers for review.  2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

Studies for evaluation (n = 292)

Excluded (n = 247) Instrument insufficiently described Experimental designs Non health discipline Unpublished

Included for review (n = 49)

757

758

Maurer (1984)

Stronge (1985)

Parks (1986)

Schwirian (1989)

Dover (1991)

Wilson (1991)

Scarpa (1992)

Agho (1995)

Stockton (1995)

Jayasuriya (1996)

US

US

US

US

CN

US

US

US

US

AU

Co.

First Author (date)

Survey

Survey

Survey

Survey

Survey

Survey

Survey

Survey

Survey

Survey

Study design

Phase 1 undergraduate nurses (n = 145) response 70% Phase 2 year 1 nursing students (n = 71) hospital nurses (n = 99)

Hospital nurses T1 (n = 391) 45% response T2 (n = 265) 36% response.

Random sample of accredited Schools. Allied health students H level (n = 377)

Hospital Nurses (n = 136)

Non-random sample Y3–4 pre and post registration nursing students (pilot n = 8; n = 73). Single university Nursing (n = 272) from 5 institutions

Nursing students undergraduate (n = 124), Master’s (n = 105), nurse academics (n = 71). Single university Registered nurses (n = 358) nursing students (n = 353)

Test–retest College students (n = 25). Survey 6 norm groups State of Iowa Nursing students (n = 48) 80% response

Sample, response rate

Nurses’ Computer Attitudes Inventory (NCATT) and Dambrots Computer Attitude Scale (CATT) to test concurrent validity Refining and assessing instrument

Nurses’ Attitudes towards Computerisation (Stronge and Brodt) questionnaire. Psychometric testing of existing tool 1. Maurer & Simonson Computer Opinion Survey (COS). 2. Modified Parks et al. Computer Knowledge Survey (CKS) to report attitudes to computers and gaps in computer literacy. Stronge & Brodt NATC questionnaire. Pre post computerisation enabled factor analysis and reliability test

22

20

T1 a = 0Æ93 T2 a = 92 Three factors identified: Computers and patient care, Computers and personal security, General attitude a = 0Æ95 Instrument reduced from 40 to 22 items following factor analysis. Construct Computer Attitudes multidimensional

1. COS a = 0Æ94, 2. CKS a = 0Æ95

1 = 26 2 = NR

20

3 Factors re anxiety: positive usefulness (12 items); expressed fear of computers (5 items); dislike and mistrust of computers (9 items) a 0Æ90. Five factors identified but 3 items did not load onto a single factor

a 0Æ95 for refined 17 item 3 factor tool with nurses. Three items did not load onto any factor Items amended for face validity Pilot with 8 students no further validation

Test–retest reliability r = 0Æ9. Internal consistency (IC) a = 0Æ94 Reduced to 20 items with index of discrimination ‡0Æ50 Spearman Brown split half r = 0Æ91 Content validity by panel (n = 5). 8 components IC a = 0Æ91–0Æ95

Psychometric testing properties

26

142

17

NKC 21 Usage 7

Nurses knowledge of computers (NKC) current and desired

Nurses’ attitudes towards computerisation (Stronge and Brodt) questionnaire. Psychometric testing of existing tool Questionnaire items drawn from prior studies. Used for clinical practice, administration and learning, comfort with and use of computers Computer Anxiety Index (Maurer and Simonson) The effect of hands-on computer experience on computer anxiety

66

26

Computer Anxiety Index (CAIN). Develop and validate a test of computer anxiety Nurses’ attitudes towards computerisation development and validation of instrument

Items

Data collection instrument

Table 2 Instruments measuring information and communication technology (ICT) skills, attitude to ICT and ICT for education

A. Wilkinson et al.

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

McBride (1996)

Williams (1996)

Saranto (1997)

Bachman (1998)

Grigg (1999)

Hollander (1999)

Ray (1999) Sinclair (1999)

CN

US

FI

US

UK

US

IE UK -NI

Co.

First Author (date)

Table 2 (Continued)

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

Survey Survey

Survey

Survey

Pre–post survey

Survey

Survey

Survey

Study design

Final year dental students 1996 (n = 44) 88% response, 1997 (n = 42) 86% response Medical students (n = 208). 60% response. Single institution over 2 years Dental students (n = 140) Diploma nurses 2 year 1 cohorts (n = 745) response 100%. Single university

RN-MSN nursing students (n = 20) Non-equivalent control (n = 23). Single university

Random sample from national cohort graduating nursing students (n = 373) response 62%

Random sample of programs (n = 5), physiotherapy students (n = 160) national survey

Registered nurses (n = 394), 42% response; nursing students (n = 299), 60% response. Hospital and university.

Sample, response rate

No evidence of validation

52

Questionnaire Computer literacy Questionnaire derived from MacMahon, unpublished data (1997) and Loyd and Gressard (1984) Self assess competence (SA), attitudes (A) against simple knowledge test (K)

E-mail questionnaire. Four cohorts perception of IT skills and ownership of computers

Questionnaire including the Stronge & Brodt Nurses’ Attitudes Toward Computerization Questionnaire used for pilot Internet course on information systems in health care Questionnaire. Perception of IT skills and attitude to IT for two cohorts

SA = NR A = 20 K = 16

NR

20

No evidence of validation Factor analysis of attitude items showed three factors; confidence 8 items, a = 9; motivation to use, 8 items a = 0Æ8, perceived career related importance 3 items a = 0Æ7

No evidence of validation

Item analysis a for groups of items Negative IT = 0Æ9; IT useful = 0Æ8; +ve attitude to studying IT = 0Æ7; ve attitude to studying IT = 0Æ6; experience of teaching = 0Æ7; ve IT for nurse practice = 0Æ6; Interest use of IT = 0Æ5 a = 0Æ90 pre test 0Æ86 post test. Perceived computer skill a = 0Æ89 and 0Æ88 respectively 15 LE 18 IT 32 HC

26 20

Two sub-scales in student sample poor Internal Consistency. Factor analysis suggested problems with construct validity. a varied between samples and sub-scales 1. COS a = 0Æ94 Single factor solution 2. CKS a = 0Æ95

20

Nurses’ Attitudes towards Computerisation (Stronge & Brodt) questionnaire. Psychometric testing of existing tool

1. Maurer & Simonson Computer Opinion Survey (COS) 2. Parks et al. Computer Knowledge Survey (CKS) Opinions about computers, self assessment of current and desired knowledge, current use 5 part questionnaire on perceptions of learning environment (LE); Learning IT (IT) and views on computerisation in health care (HC). Author developed with reference to previous research

Psychometric testing properties

Items

Data collection instrument

JAN: REVIEW PAPER Literature review measurement tools

759

760

Billings (2001)

Duggan (2001)

Atack (2002)

Curtis (2002)

Mattheos (2002)

Seago (2002) Steele (2002)

US

US

CN

IE

SE

US

Wishart (2002)

Bloom (2003)

UK

US

US

Lynch (2000)

US

Co.

First Author (date)

Survey

Survey

Survey

Survey

Survey

Survey

Survey

Survey

Pilot survey

Survey

Study design

Table 2 (Continued)

Nursing students (n = 165) students in health science (n = 206). Pilot study (n = 36) Single institution

Nursing students (n = 154) and teachers (n = 128)

Y1 medical students average response rate 81% (n = average 171 per annum) Medical students (n = 150) single university

Dental students (n = 590) sampled from dental schools (n = 16).Response 80–85%. 9 countries

Post-registration nurses. Pilot (n = 10) Main study (n = 74). Response rate 62% University

4-step sample (n = 395); scalability (n = 70); operationalisation (n = 69), pilot (n = 68); Final (n = 188). Single university Registered Nurses (n = 28) single course. Response 49%

Undergraduate and graduate Nursing students (n = 219), 3 schools of nursing

Year 1–4 medical students (n = 301), response 90%

Sample, response rate

Questionnaire including Duttweiler’s Internal Locus of Control Index (ILCI), attitudes towards computers (AC) Author developed questionnaire. Responses to technology enhanced face to face education

Items from Computer Opinion Survey (Loyd and Gressard) and Computer Attitude Scale (Maurer and Simonson) new Opinions About Computers (OAC) scale to examine preparedness for computer based testing (CBT) Evaluating Educational Uses of the Internet (EEUWIN) adapted from Flashlight programme Current Student Inventory (CSI) online Benchmarking WBL Attitudes Toward Educational Use of Internet (ATEUI) Development and validation of instrument tested with communication and health promotion students Author developed Learner Demographic (LD) Online Learner Support Instrument (OLSI) to measure Interaction, Course, Technology, Environment and Impressions. Single module web-based post diploma Amended questionnaire Nursing Students’ Experiences and Attitudes to Computers (Sinclair & Gardner) No validation. Nursing Degree programme European Dental Students Association (EDSA) Questionnaire. Computer competence and attitudes, European study. Further items (22) re dental education not discussed 10 year cycle of Medical Student Computer Experience Survey 1991–2000 Computer Attitude Survey & Rezler Learning Preferences Inventory. CAI Angiography

Data collection instrument

a = 0Æ85

Final ATEUI a = 0Æ9

40 WBL 10 demographic data 18

36 in two sections 3 unstructured

Instrument reviewed by multiple raters from institution. Items re experience and comfort with technology enhanced learning a = 0Æ85

ILC previously validated (1984) no further validation

No evidence of validation. 15 items common to all surveys. Not revalidated

No evidence of validation

10 person data and ICT

16 Year 1 23 Year 10 Cas-g 16 CAS-e10 Cas-p 20 ILCI 28 AC 7

Instrument reviewed by 2 statisticians and IT consultant prior to pilot.

50

a = 0Æ95 for 45 item OLSI without Environment subscale a = 06 work environment a = 0Æ8 home environment

OAC a = 0Æ79 Lower than the scales from which derived

OAC = 8

LD = 26 OLSI = 56

Psychometric testing properties

Items

A. Wilkinson et al.

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

Study design Sample, response rate

Data collection instrument

Items

Psychometric testing properties

CN Cragg (2003)

Survey

Sub-scales: comfort a = 0Æ74, efficacy Random sample of hospital (n = 291, Sources of Practice Knowledge (SPK); Attitudes SPK 18 Other items not a = 0Æ85, interest a = 0Æ77, utility 58%) and nursing students (n = 207) Toward Computers (AC) multidimensional; beliefs a = 0Æ72, affordability a = 0Æ64; reported Computer and Internet Confidence and Computer a = 0Æ96 and Internet Computer and Internet Comfort. confidence a = 0Æ98; Computer a = 0Æ93 Scales modified by authors. and Internet comfort a = 0Æ95 To assess barriers and facilitators of webbased education US DeBourgh CorrelatPost-graduate nurses (n = 43, 100%) Student Satisfaction Survey (SSS) adapted SSS 59 Technology and technology competence (2003) ional response. University from Telecourse Evaluation Questionnaire a = 0Æ78; between class use of research (Biner). communication technologies a = 0Æ85; design Distance module satisfaction a = 0Æ88; instructor/instruction a = 0Æ89; course management a = 0Æ41; experience with technology courses a = 0Æ11 19 some Pilot study no other validations reported Questionnaire reporting staff and student UK Walmsley Survey Years 1–3 dental students (n = 145) attitudes to the use of the Internet. Instrument qualitative (2003) 81% response. staff (n = 22) 100% published response. Single university CN Balen Survey Hospital pharmacists (n = 58) 55% Assessing baseline computer skills, use and 84 No evidence of validation (2004) response rate. Two hospitals training needs. Items drawn from literature No evidence of validation Questionnaire assessing level of training 19 computer NG Bello Survey Medical & health record staff & and use of IT knowledge (2004) students (n = 148) response 16 attitude and rate 82% Hospital utilisation Appendix 20 No evidence of validation Web-based questionnaire related to IT DK Dorup Longitudinal Year 1 medical students n = 1159 mixed items access and attitudes towards using IT (2004) survey over 5 years. 79% response. for learning. Administered in IT skills course Single university No evidence of validation Questionnaire with items from earlier studies. 36 items? Not NZ Honey Survey Post-graduate nurses (n = 146) fully reported Access, proficiency, skills, use of digital (2004) response 90%. Single school resources and barriers. of nursing Scoping feasibility of introducing flexible learning CSE a = 0Æ94; CSA a = 0Æ92; Attitude 16 Attitude Toward Behaviour, Computer FI Vuorela Pre–post Medical (n = 21) and sociology CSE 10 CSA 20 ASSI a = 0Æ52 0Æ85 Self-Efficacy Scale (CSE) part of Approaches (2004) survey (n = 21) students. ASSI 18 items and Study Skills Inventory (ASSI), Computer Response pretest 90% in subscales State Anxiety (CSA), Evaluation of theory post-test 76% of planned behaviour (TPB) as explanation of student use of WBL

First Author Co. (date)

Table 2 (Continued)

JAN: REVIEW PAPER

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

Literature review measurement tools

761

762

Rajab (2005)

Salamonson (2005)

Sit (2005)

Adams (2006)

Hegney (2006)

AU

HK

IE

AU

Masiello (2005)

SE

JO

Brumini (2005)

HR

Mathur (2005)

Billings et al. (2005)

US

CN

Wilkinson (2004)

UK

Co.

First Author (date)

Survey

Survey

Survey

Survey

Survey

Survey

Pre–post survey

Survey

Survey

Pre–post survey

Study design

Table 2 (Continued)

National survey registered nurses n = (4300). Response 43%

Post registration nurses (n = 198) response 50% Post registration nurses (n = 32). Response 73% Single university

Nurses Pilot (n = 42) main (n = 143) Single university

Years 2–5 dental students (n = 268) response 81%. single university

Qualified physical therapists (n = 732). 56% response. Two stage Pilot (n = 31), (n = 25)

Year one medical students (n = 54) 49% response rate. Single course

Registered nurses (pre n = 29, post n = 28). Single institution Graduate and undergraduate nursing students (n = 558). 6 schools of nursing Hospital nurses (n = 1081). Response 96%

Sample, response rate

Questionnaire based on Flashlight program CSI and focus group. online learning Questionnaire developed from literature to report changes in skill over time and relationships between student IT skills and a range of variables Author designed questionnaire To determine access, use and barriers to use of IT by nurses

No evidence of validation

28

77

Face and content validity via interviews and pilot

a = 0Æ86

32

27

a = 0Æ93 whole scale Factor 1 Prefers traditional format a = 0Æ87 Factor 2 Prefers hybrid format a = 0Æ91 No evidence of validation

20

NR

IT a = 0Æ73; IO concerning learning supported by technology a = 0Æ75; Blended Orientation (BO) regarding the combination IT and face to face interaction with staff/students for learning, (a = 0Æ88) 2 part pilot for face validity. No test–retest for reliability

IT 6 + ve IO 7 + ve BO 10

a = 0Æ86 for modified NATC

a = 0Æ94. Reliability of sub-scales ranged from 0Æ73–0Æ93

57

30

Alpha for sub-scales varied 0Æ7–0Æ0Æ9. Not fully validated

49 in 7 subscales

Author developed pilot questionnaire, Exploration of perceptions of four web-based learning courses EEUWIN online exploring differences between graduate and undergraduate experience Modified Stronge and Brodt Nurses’ Attitude Toward Computers (NATC) instrument used in the workplace IT learning questionnaire, based on previous and author developed items, to measure readiness and attitudes to learning. Evaluation of a learning management system technology. Subscales IT Orientation (IT); Independent Orientation (IO) Author developed questionnaire; part two surveyed interest in different modes of learning Measuring interest in computer assisted learning, distance & CD-ROM Questionnaire drawn from previous dental surveys in UK. Knowledge skills and experience of ICT Author developed Students’ Preference of Course Format (SPOCF) scale. Drawn from previously validated Students’ Evaluation of Educational Quality (SEEQ) survey item bank

Psychometric testing properties

Items

Data collection instrument

A. Wilkinson et al.

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

20 item TAS originally validated with teachers. Content validity by author. Pilot paper only a = 0Æ88. Main a = 0Æ89 Factor analysis 2 factor scale: 1. positive TA a = 0Æ91; 2. negative TA a = 0Æ88 BAC a = 0Æ92 ABWL a = 0Æ87 TAS 15

Public Health Nurses (PHN), random sample from 369 health centres (n = 329) response 84% Survey Yu (2006) TW

Convenience sample undergraduate and graduate nurse students (n = 721) 7% response. US SON (n = 21) Maag (2006) US

Survey

Author developed Basic Computer Competence (BAC) and Scale of Attitude to Web-based Learning (AWBL) Understanding attitudes to web-based learning

BAC 26 AWEU 16

3 item subscale usefulness of e-learning a = 0Æ65 Adding 2 further items reduced reliability NR Year 1 Medical students (n = 1160) response 79% single institution Link (2006) AT

Survey

Questionnaire online. Attitude and experience of e-learning, usage and private access Completed during introductory course on CBT/WBT Questionnaire paper and online. Included modified Technology Attitude Scale (TAS) and self report of computer application education

Psychometric testing properties Data collection instrument Sample, response rate Study design Co.

First Author (date)

Table 2 (Continued)

Literature review measurement tools

Items

JAN: REVIEW PAPER

The literature fell into two groups in focus and method: studies from the fields of dentistry, medicine and pharmacy and those from nursing and allied health professions.

Dentistry, medicine and pharmacy Information and communication technology skills and literacy In dental, medical and pharmacy education the majority of studies used self-report surveys of students’ ICT skills, information literacy and computer ownership (Grigg & Stephens 1999, Hollander 1999, Ray & Hannigan 1999, Lynch et al. 2000, Dorup, 2001, Mattheos et al. 2002, Seago et al. 2002, Walmsley et al. 2003, Balen & Jewesson 2004, Bello et al. 2004, Rajab et al. 2005) rather than educational use. Questionnaires, with the exception of that by Lynch et al. (2000), were author-developed, with little reported on validation. Samples were homogeneous and demographic variables other than sex were rarely reported. A USA crosssectional survey of medical students (Lynch et al. 2000) examined preparedness for computer-based testing (CBT) using eight items taken from the Loyd and Gressard (1984) Opinions about Computers Scale and Maurer and Simonson’s (1984) Computer Anxiety Scale. Internal consistency of the modified scale (a=0Æ79), although good, was lower than for the original scales (a = 0Æ94 and 0Æ95 respectively), which raises questions about the reliability of the modified scale. Lynch et al. (2000) isolated three predictive factors indicating preparedness for CBT: opinions about computers; sex and sex interacting with previous experience of CBT. A number of researchers surveyed successive cohorts of dental or medical students (Grigg & Stephens 1999, Seago et al. 2002, Dorup 2004) or, simultaneously, several yeargroups (Hollander 1999, Lynch et al. 2000, Walmsley et al. 2003, Rajab et al. 2005). Repeated surveys indicated that students’ reported that their ICT skills improved over time; their skills were not always sufficient for their course, and they needed further training (Grigg & Stephens 1999, Mattheos et al. 2002). These studies clearly built on the experiences in previous studies; for example, Rajab et al. (2005) derived their items from three previous dental surveys (Grigg & Stephens 1999, Mattheos et al. 2002, Walmsley et al. 2003) but no psychometric data were reported. E-learning in medicine Four further studies explored attitudes to e-learning among medical students. A US study of medical students (Steele et al. 2002) used a combination of two previously-published scales to create a two-part Computer Attitude Survey (CAS). Subscale CAS-g, previously validated with healthcare

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

763

A. Wilkinson et al.

professionals and students in practice (Startsman & Robinson 1972), surveyed general attitudes to computers, and CAS-e explored attitudes and comfort with computers for education. Post-learning intervention respondents also completed CAS-p evaluating the computer assisted instruction. No details of validation of the modified scales were given, which compromises the results, given the different sample and the time elapsed since some items were developed. In Finland, Vuorela and Nummenmaa (2004) explored medical and sociology students’ performance and self-confidence in a web-based collaborative learning environment, predicting that those who reported positive attitudes and high computer self-efficacy would engage with the environment. They used five scales: attitude measurement; computer self-efficacy (Compeau & Higgins 1995); approaches to learning; computer state anxiety. Vuorela and Nummenmaa (2004) found that expectation, confidence and learning approach could not be used to predict outcomes in the learning environment in their sample. A Swedish study (Masiello et al. 2005) employed an ITLearning Questionnaire, with items from two previous studies and new items. Pre-existing scales were presumed to be more reliable, although the papers cited (Dewhurst et al. 2000, Slotte et al. 2001) presented no information about validation. The internal consistency of sub-scales was satisfactory and three principal components were identified: IT Orientation, Independent Orientation and Blended Orientation. These three constructs explained 49% of the variance, which is just under the 50–60% required to agree factor structure (Netemeyer et al. 2003). Finally, an online study of Austrian medical students (Link et al. 2006) examined computer literacy, computer and Internet use and attitudes to e-learning. An index on usefulness of e-learning was found to be moderately reliable, but including two additional items reduced reliability. No further validation was reported.

Nursing and allied healthcare professions More systematic efforts to develop or adapt instruments to measure students’ attitudes to computers and experience of e-learning are found in the nursing and allied healthcare literature (Table 3). Few instruments, however, have been repeatedly used and tested. Attitudes to computers The presence of anxiety is considered likely to affect performance; thus, Maurer and Simonson (1984) sought items which would describe behaviour likely to indicate anxiety. They drew on previous measures of anxiety, and added new items generated by students; these were then reduced and modified in two pilot tests. The instrument was then tested for reliability 764

using test–retest a = 0Æ90, yielding internal consistency of r = 0Æ94 on test-retest and r = 0Æ96 using a random sample from a normative study. The new instrument was correlated against responses to a separate generalized measure of anxiety and observation of students using computers when they were assessed as anxious; neutral or comfortable. The refined instrument was then further validated in a state-wide normative study with computer professionals, computer users, school and university students, staff and a non-specific group. It has been used in studies in nursing (Wilson 1991), allied healthcare (Agho & Williams 1995, Williams et al. 1996) and medicine (Lynch et al. 2000). In a US study the unmodified Computer Anxiety Index (CAIN) (Wilson 1991) was used to test levels of computer anxiety experienced by students in five schools of nursing, whether anxiety varied in relation to level of computer experience and whether there were differences between degree and associate degree students. While no difference was found between associate and degree students, more practical experience of computers was associated with less anxiety. Wilson (1991) concluded that the CAIN was a valid measure of anxiety with computers and measured three separate factors: positive usefulness, expressed fear of computers, and dislike and mistrust of computers. Agho and Williams (1995) and Williams et al. (1996) explored US allied healthcare students’ attitudes to computers and perceptions of computer literacy using CAIN (Maurer & Simonson 1984) and a modified computer knowledge survey (Parks et al. 1986). They found the instruments to have high reliability in their study and the 26 items of Maurer and Simonson (1984) formed a single factor (Table 2). Stronge and Brodt (1985) developed the Nurses’ Attitudes Towards Computerisation (NATC). The initial 66 item selfreport questionnaire was reduced to 20 items in a pilot study in the workplace with US nursing students. They proposed the instrument as a reliable 6-factor scale but the sub-scales were challenged in later studies with larger samples. It has been adopted, adapted or revalidated (Table 4) by a number of authors for use with both Registered Nurses and nursing students (Schwirian et al. 1989, Scarpa et al. 1992, Stockton & Verhey 1995, Jayasuriya & Caputi 1996, McBride & Nagle 1996, Bachman & Panzarine 1998, Brumini et al. 2005). Schwirian et al. (1989) used the NATC in a comparative study of nursing students and Registered Nurses. While confirming that overall reliability was high, they proposed three subscales: Computers and Patient Care, Computers and Personal Security, and General Attitude. Three items, however, did not fit into any construct and there were small variations in the factor loadings between students and nurses. A replication study (Scarpa et al. 1992) with a sample of nurses with varying demographic profiles showed that only previous computer experience was related to positive

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

JAN: REVIEW PAPER

Literature review measurement tools

Table 3 Details of computer attitude self-report measures Instrument

Developed by

Used or adapted by

Used in

Computer Attitude Scale (30 items, 3 factors, overall Cronbach’s a = 0Æ95) Computer Anxiety Index (CAIN) (26 items, test retest r = 0Æ90, overall Cronbach’s a = 0Æ94) Nurses attitude toward Computerisation (NATC) (20 items, split half r = 0Æ90)

Loyd & Gressard 1984

Sinclair & Gardner 2004, Lynch et al. 2000, Curtis et al. 2002,

University & Schools; education, medicine, nursing

Maurer & Simonson 1984

Wilson 1991, Williams et al. 1996, Lynch et al. 2000, Agho & Williams 1995, Schwirian et al. (1989); Scarpa et al. (1992); Stockton & Verhey 1995, Jayasuriya & Caputi 1996, McBride & Nagle 1996, Bachman & Panzarine 1998, Brumini et al. 2005, Agho & Williams 1995, Balen and Jewesson (2004) drew some items Masiello et al. 2005,

University, allied health, medicine, nursing

Stronge & Brodt 1985

Computer knowledge survey

Parks et al. 1986

Computer self-efficacy 10 items IC of constructs reported >0Æ8 Nursing computer attitudes inventory (NCATT) Current student inventory (item bank, no psychometric information)

Compeau and Higgins (1995)

Technology Attitude Scale (TAS) Online Learner Support Instrument (OLSI) a = 0Æ95 Attitudes to the Educational use of the Internet (ATEUI)

Jayasuriya and Caputi (1996) Ehrmann, 1997

McFarlane et al., 1997 Atack & Rankin, 2002

Billings et al. 2001 Evaluating Educational Uses of the Internet (EEUWIN); Sit et al. 2005; Billings et al. 2005, modified by Maag (2006)

Duggan et al. 2001

attitude. While they established similar reliability to that in the previous study, their factor analysis identified five factors different from those of Stronge and Brodt (1985). Three items did not load on to a single factor and all positive items loaded on to one factor. Further testing with larger samples was proposed. Stockton and Verhey (1995) conducted reliability testing of the instrument using US hospital nurses in two phases over 16 months. There were personnel changes between Time 1 and Time 2, but the populations were described as demographically similar. They confirmed the instrument’s overall high reliability but expressed doubts about the number of sub-scales and, after factor analysis, questioned whether, out of three identified, the factors were independent as the

Hospital and University; nursing

Staff, students and graduate students School of Nursing Business computing, medical education Registered Nurses and nursing students Schools of Nursing

Teachers, nursing students Registered nurse students Undergraduate students (communication, health promotion)

reliability was reduced in two factors and all the positive items were located in factor 1. Jayasuriya and Caputi (1996) retrieved 72 items from international studies, including Stronge and Brodt’s (1985) instrument, and reduced the number using a review panel to yield a 40-item instrument, the Nursing Computer Attitudes Inventory (NCATT). The instrument was then administered to nursing students. Following factor analysis, five factors were identified which contributed to 57% of the variance. They subsequently used the NCATT instrument with Australian Registered Nurses in practice and nursing students at university. The student sample also completed Dambrot et al. (1985) Computer Attitude Scale (CATT), which was extracted from a 20-item tool (a = 0Æ84). The NCATT scale

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

765

766 refined scale a = 0Æ95 nurses

66 reduced to 20 (14 ve)

a = 0Æ90 r = 0Æ90

Categories proposed pre pilot themes from literature 1. Job security 2. Legal ramifications 3. Quality of patient care 4. Capabilities of computers 5. Employee willingness to use computers 6. Benefit to the institution

Number of Items

Cronbach’s a Spearman Brown

Factors (n)

a = 0Æ92

20

Scarpa et al. (1992) n = 136 (40% response) Replication study

T1 a = 0Æ93 T2 a = 0Æ92

20

Stockton and Verhey (1995) T1 n = 391 T2 n = 265 (overall response 41%) A. 40 renamed (NCATT) incl. Stronge & Brodt B. 40 reduced to 22 1. a = 0Æ95 2. a = 0Æ95

Jayasuriya and Caputi (1996) A. n = 145 (70% response) B. n = 170 nurses and students

Following Factor A. Following factor Factor analysis: Following factor Analysis (63% of analysis (57% there was analysis(57% of variance) of variance) variation variance) 1. nursing efficiency 1. Computer Anxiety between nurses 1. Computers & (5) (r = 0Æ92) 2. Computers & and students patient care (8) 2. computer Patient Care and three items (T1 & T2 a = 0Æ92) inefficiency 3. Cost & efficiency did not fit 2. Computers & (5) (r = 0Æ78) to institution 1. Computers & personal security 4. Computers & patient care (9) 3. Agency & societal (2) (T1 a = 0Æ78, impact (2) (r = 0Æ85) expansion a = 0Æ94 T2 a = 0Æ75) of interest 2. Computers & 3. General attitude 4. Limitations of 5. Patient Personal Security computers (3) (3) (T1 a = 0Æ74, Confidentiality (4) a = 0Æ78 T2 a = 0Æ78) (r = 0Æ78) B. Following factor 3. General attitude 5. Patient analysis (90% (4) a = 0Æ85 confidentiality of variance) (2) (r = 0Æ95) 1. Patient care Three items did (nurses a = 0Æ91, not load students a = 0Æ80) 2. Anxiety (nurses a = 0Æ88, students a = 0Æ92) 3. Confidentiality (nurses a = 0Æ72, students a = 0Æ71)

20 reduced to 17

Stronge & Brodt (1985) n = 48 (80% response)

Author/ sample

Schwirian et al. (1989) n = 358 nurses n = 353 student

Table 4 Nurses’ attitudes towards computerisation (NATC)

Following factor analysis (53% nurses, 49% students of variance) 1. Nurses work (nurses 7 a = 0Æ87, student 6 a = 0Æ81) 2. Organisational issues (nurses 7 a = 0Æ81, 4 students a = 0Æ74) 3. Barriers (nurses 4 a = 0Æ74, students 5 a = 0Æ65) 4. Efficiency (students 4 a = 0Æ54) Nurses: two items did not fit Students: one different item did not fit.

nurses a = 0Æ91 student a = 0Æ85

20

McBride and Nagle (1996) n = 362 nurses (42%) n = 299 student (60%)

pre test a = 0Æ90. post test a = 0Æ86 No report of factor analysis

Bachman and Panzarine (1998) n = 20 sample n = 23 nonequivalent control 20

Scree plot analysis three highly correlated factors Principal component factor analysis gave one factor solution.

30 (NATC revised 15 ve and +ve) a = 0Æ86

Brumini et al. (2005) (n = 1081)

A. Wilkinson et al.

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

JAN: REVIEW PAPER

was further reduced to 22 items following factor and item analysis and correlation with the CATT scale. Jayasuriya and Caputi (1996) identified three factors which contributed to 90% of the variance. One of the factors: Computer Anxiety correlated highly with the CATT (r = 0Æ51). They reported that none of the items were derived from the Stronge and Brodt (1985) NATC items, which may be a further indication that they had become dated. The report of a Canadian cross-sectional study of Registered Nurses and nursing students also critiqued the Stronge and Brodt instrument (McBride & Nagle 1996) following a review of previous reported uses of the scale. The authors questioned the validity of this scale since their findings could only explain 50% of the variance for the Registered Nurses and 48% for the students. This finding is congruent with that of Stockton and Verhey (1995). McBride and Nagle (1996) also found greater scale consistency for the Registered Nurses than the student sample. They suggested that the lack of consistency may in part be attributed to the age of the scale reducing item validity. The findings from this group of studies show considerable variability in the number of factors identified, which may indicate that either the researchers are using different criteria in conducting the analysis or that individual items are not valid. A US study of Registered Nurses undertaking an Internetbased course on IT (Bachman & Panzarine 1998) used the Stronge and Brodt instrument. The aim of the study was to evaluate the impact of a pilot online course. Internal consistency of the whole scale was different pre- and postintervention. Additional subscales on computer use, perceived computer skill and knowledge of the Internet were developed by the authors in the absence of appropriate measures. Content validity was assessed by raters and the reliability of the Perceived Computer Skill sub-scale was reported (pre a = 0Æ89 post a = 0Æ88). The Stronge and Brodt instrument focused predominantly on the introduction of computers to the workplace and, as well as being dated, it is also less likely to be valid in an academic setting. In a two-site workplace study with Croatian nurses, Brumini et al. (2005) used a modified version of the NATC, with new items from an unpublished study. Half the 30 items on attitudes to computers were negatively phrased. Having performed factor analysis, they decided to work with a 1-factor solution. Limitations of their study included the cross-sectional design and respondents being observed as they completed the survey. Two nursing studies (Curtis et al. 2002, Sinclair & Gardner 1999) drew on Loyd and Gressard’s (1984) Computer Attitude Scale developed for use in schools. Sinclair and Gardner (1999) surveyed diploma nursing students in North-

Literature review measurement tools

ern Ireland to measure their perceived competence, attitudes and previous IT training against a knowledge test. Three factors were identified: confidence in using computers, motivation to use, and perception of career-related importance of computers. Those with previous knowledge and training had statistically significantly higher levels of confidence (P < 0Æ001) and motivation (P < 0Æ01) but there was no statistically significant difference in their perception of career-related importance. Sinclair and Gardner (1999) recommended a change from identifying students’ basic skills to identifying their ability to apply those skills in practice. Curtis et al. (2002) used the scale with small amendments to survey Irish undergraduate Registered Nurses and found positive attitudes and moderate to high skills but did not report confidence levels or validation. A further Irish study (Adams & Timmins 2006) with Registered Nurses on a parttime access to degree programme used an author-developed instrument with items developed from the literature (a = 0Æ86) to explore access to and use of computers and the Internet and the relationship to demographic variables and usage patterns. However, as no further psychometric data were reported there was little potential to draw on or replicate these studies. A subsequent US national study (Maag 2006) drew on Sinclair and Gardner (1999) work and suggested that sound education about the use of technology would have an impact on nursing students’ future use of technology for patient care. To provide a baseline for educators, they developed a modified Technology Attitude Scale (TAS) to explore students’ attitudes and views towards using technology. In common with previous studies, the TAS addressed education to use computers and not educational use. e-Learning A small number of recent studies have used instruments to explore healthcare students’ experience of using computers specifically for education. A US study (Duggan et al. 2001) identified a lack of instruments to measure student attitudes to the educational use of the Internet and the researchers developed and validated an 18-item scale, Attitudes to the Educational Use of the Internet (ATEUI), in three stages. Educational literature provided a pool of 60 statements, subsequently reduced to 33 and administered to students to determine the scalability of items and then operationalised with a new sample when 18 items were retained. The pilot AETUI, combined with behavioural statements, was completed by a further student sample (a = 0Æ89). The final instrument was administered with a further group of students enrolled on communication and health promotion courses (a = 0Æ91). The findings of Duggan et al. (2001) suggest that students who are in control of their learning have positive attitudes,

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

767

A. Wilkinson et al.

and this theme is echoed in a UK cross-sectional study with postgraduate teacher trainees and pre and postregistration nursing students (Wishart & Ward 2002), which suggested that there was an association between attitude to computers and user sense of internal control. Wishart and Ward (2002) chose the previously-validated Duttweiler Internal Locus of Control Index (Duttwieler 1984). An additional seven items focused on computer attitudes and questions about their computer use, ownership and experience. There is no evidence that the amended instrument was validated as part of the study. Atack and Rankin (2002), in a pre and post-test evaluation of web-based learning (WBL) by Canadian Registered Nurses, developed a two-part instrument. Items on computer skills and access were combined with five sub-scales measuring interaction, design, technology, environment and impressions. The authors highlighted the variation in responses concerning interaction and learning from the work environment, which were congruent with the findings of larger studies (DeBourgh 2003, Billings et al. 2005, Sit et al. 2005). In a similar UK pre and post-test pilot study, Wilkinson et al. (2004) used an author-developed instrument,with seven subscales to explore computer use; online learning; distance learning; evaluation (post); learning outcomes; support; and utility with Registered Nurses on four courses. Respondents reported more positive orientation to online (P = 0Æ04) and distance learning (P = 0Æ02) post-course. Those who reported prior use of ICTs were more likely to complete the course. Both these studies were small, exploratory and uncontrolled and had participant attrition. Researchers have used the national evaluation programme of the American Association of Higher Education (AAHE) (Billings et al. 2001), which is focused on educational technology. In this national evaluation programme conducted by the AAHE and known as the Flashlight programme (now a non-profit Teaching Learning Technology Group) an item bank of ‘validated’ questions, the Current Student Inventory Toolkit (CSI), on educational technology was created for researchers to use (See http://www.tltgroup.org/flashlightP. htm). Billings et al. (2001) drew upon this bank of questions to develop and pilot an online 52-item instrument called Evaluating Educational Uses of the Internet (EEUWIN) to validate an educational model with nursing students across three universities. The reliability of the 52-item EEUWIN was good but no data were reported for its sub-scales. In the pilot study positive relationships were identified between student distance from the campus, older age and convenience, satisfaction and connectedness. In a later study, Billings et al. (2005) investigated generational differences among nursing students across six universities using a revised 57-item EEUWIN. 768

In a Hong Kong study with Registered Nurses taking one or more online courses researchers also used 27 items from the CSI to ask questions about overall satisfaction with online courses and the perceived barriers to online study (Sit et al. 2005). The items were reviewed by an educational team for content validity related to six themes. Internal consistency and test–retest reliability of the CSI were evaluated but the data were not reported. The researchers found seven facets which explained 54% of overall satisfaction: convenience; confidence to tackle difficult tasks; understanding concepts; taking responsibility for learning; interactivity; supplementary face-to-face interaction; and multimedia. Convenience was the most important factor in satisfaction but a major hindrance was lack of face-to-face interaction. A US study of Registered Nurses (DeBourgh 2003) had similar findings using a 59-item Student Satisfaction Survey derived from an existing instrument, with added items on computer-mediated communication identified from literature searches on student satisfaction with distance education. DeBourgh (2003) tested reliability coefficients for eight facets (five student and three teacher characteristics) and these varied from poor to good. Technical aspects cited as the most frequent source of negative attitudes in the literature. DeBourgh (2003), however, concluded that the human interface issues represented by quality of the instructor remained the most important factor in satisfaction with distance learning. This study was limited by its small homogeneous sample. The ability of teachers to apply educational principles to the use of technology was a major influence on students’ satisfaction in another US crosssectional study (Bloom & Hough 2003) exploring nurse and health science students’ engagement with technologyenhanced learning before and during their course and the quality of the experience. The author-developed instrument was validated by a rating panel and pilot test. Reliability for the second part of the instrument concerning experience and comfort with technology-enhanced learning was a = 0Æ85. New studies are emerging from other regions of the world. In a Canadian/Chinese study of Registered Nurses, Cragg et al. (2003) combined and made small changes to existing scales to measure Attitudes Toward Computers (ATC) The authors developed a new scale Attitudes Towards the Internet based on the ATC scale which, except for the affordability sub-scale (a = 0Æ53), had similarly good to excellent reliability. The scales were translated into Chinese and validated. In New Zealand, Honey (2004) surveyed Registered Nurses using an author-developed instrument incorporating items from previous studies in a cross-sectional study to scope the feasibility of introducing online learning. There was no

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

JAN: REVIEW PAPER

description of instrument development or psychometric data, although reference was made to previous research. A Taiwanese study (Yu & Yang 2006) explored the attitudes of a random sample of public health nurses to WBL using an author-developed questionnaire with items generated from review of the literature, interview and expert panel resulting in two sub-scales: Basic Computer Competence and Attitude toward WBL. Overall, nurses were positive about WBL but rural nurses were statistically significantly more positive than those in urban settings. Those with computer skills and access to computers in the workplace were more positive. There were, however, indications that respondents had concerns about the quality of learning materials and being isolated, similar themes to those identified by DeBourgh (2003) and Sit et al. (2005). A Canadian study with physiotherapists (Mathur et al. 2005) employed an author-developed instrument previously used by one of the authors and amended for face validity and pilot tested but not further validated. Level of education, level of access to the Internet, frequency of use, skill and practice area were positively related to interest in CAI. In contrast to the work of Billings et al. (2001)and Yu and Yang (2006), distance from education centres was not related to interest in CAI. There is some emerging work on the development and validation of instruments to explore student responses to use of computers and the Internet for education (Table 3) but currently no single instrument has been developed and refined using large samples across a variety of settings. Furthermore, few such instruments have been either reused or revalidated.

Discussion The literature search identified instruments used to collect data on four main topics: assessing ICT and information literacy skills (e.g. Parks et al. 1986, Maag 2006); measuring attitudes to computers or computerisation in practice (e.g. Stronge & Brodt 1985, Jayasuriya & Caputi 1996); attitude and access to computers and the Internet (e.g. Mattheos et al. 2002, Cragg et al. 2003); and exploring attitudes to computers and the Internet for education (e.g. Duggan et al. 2001, Atack & Rankin 2002, Bloom & Hough 2003). All the studies were cross-sectional or pre and post- test studies, with no longitudinal studies with repeated measures, and the pre and post-test intervention studies were of single units or modules of WBL with consequential small sample sizes (Atack & Rankin 2002). More recent studies have focused on attitudes to web-based and distance education and have explored interrelationships between all the above topics. Methodological details of many studies were inadequately described, creating concerns about instrument validity and

Literature review measurement tools

reliability (e.g. Grigg & Stephens 1999, Mattheos et al. 2002, Seago et al. 2002, Hegney et al. 2006). In descriptions of the instruments, face and content validity were more consistently discussed than other aspects of validation and included the use of subject experts, reference groups or themes derived from the literature (e.g. Mathur et al. 2005, Hegney et al. 2006). Since the 1980s there have been substantial changes in the capacity and incidence of ICT use in higher education, and therefore items designed to report their usage, responses to and comfort with ICT may have lost their validity over time. The only instrument that was repeatedly tested in healthcare education and practice (Stronge & Brodt 1985) is now dated and proved more reliable with nurses working in hospitals than with nursing students. The variation in factors identified by successive investigators also limits comparison across studies. Earlier instruments which were rigorously developed had higher reliability than those described more recently, but this may be the result of rapidly-changing ICT. Methodological detail was reported less in the medical literature than in the nursing and allied healthcare studies included in the review. A number of reports, the majority medical, gave limited information on instrument design, pilot work or psychometric properties (Hollander 1999, Ray & Hannigan 1999, Steele et al. 2002, Walmsley et al. 2003, Bello et al. 2004). While items were often derived from previous measures, it was not always clear where individual items originated, and the final instrument was rarely published (Curtis et al. 2002, Honey 2004). There are few studies where the samples were derived from more than one institution and/or discipline group, so that there are limits to the transferability of findings. Replication of studies is difficult as the work of Scarpa et al. (1992) demonstrated, since prior reports did not give complete psychometric information for useful comparative analysis. There was often no consistency in the demographic sections of instruments, and authors presented very varied and often limited information about study samples. This may be a result of homogeneous populations in many studies, but this is not always made explicit; therefore, issues remain concerning differentiating between groups of learners. To date, no instrument has demonstrated high reliability in measuring student attitudes to e-learning over more than one group. However, studies are emerging from outside the US which address the use of ICT for learning by different populations (e.g. Sit et al. 2005, Link et al. 2006, Yu & Yang 2006). Online learning has been targeted by universities as an area of growth, but there have been examples where such initiatives have failed, for example the UK e-University (House of Commons Education & Skills Committee 2005). This review suggests that little is known about either the

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

769

A. Wilkinson et al.

What is already known about this topic • Healthcare professionals are expected to be computer literate at the point of registration. • There are few rigorous quantitative studies on the attitudes and experience of healthcare professionals regarding computers and use of computers for learning. • Consistent methods of measuring attitudes of healthcare professionals’ use of computers for learning have not been established.

What this paper adds • The majority of studies are cross-sectional and therefore the development of students’ attitudes and experience with information and communication technology and online learning over time has not been explored. • Future measurement tools need to address the information literacy and preparedness for online education of healthcare students, as well as their basic information and communication technology skills. • The quality of instruments is variable, with little evidence of progressive development and validation.

for contemporary healthcare professionals. The requirement for nurses, for example, to access information, critically appraise and then synthesise it for others indicates that instruments which survey students’ ability and confidence to make more complex use of ICTs are required. ICT use is becoming ubiquitous in higher education and the workplace, and so educators need to know whether students on healthcare programmes are equipped with the requisite skills. While there have been a number of studies exploring the skills, attitudes and experience of healthcare students regarding ICTs and ICTs for education, there have been no large scale or longitudinal studies exploring whether attitudes change during and after education. Continued expansion of web-based learning at all levels, the mobility of the healthcare workforce and the need for this workforce to have flexible modes of continued education mean that it is imperative to develop and validate instruments to explore students’ experiences with e-learning and to develop models for engaging students in e-learning.

Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Implications for practice and/or policy • Information literacy is a key aspect of nursing practice and nurses’ careers and therefore gaps in knowledge of nursing students’ capabilities and attitudes are of concern. • Future research should focus on the impact of demographics on attitudes and experience of nursing students. • Further work is required to establish a valid instrument to explore the attitudes and experience in relation to information and communication technology of nursing students.

preparedness or views of healthcare students in relation to online education. Furthermore, the items in early instruments no longer reflect terminology used in connection with current ICT skills and rarely consider the use of communication other than e-mail.

Conclusion Further research is needed to identify what should be measured. Previous tools used to gather reports of respondents’ ICT skills need to be updated if they are to be relevant and valid 770

Author contributions AW, AEW & JR were responsible for the study conception and design. AW performed the data collection. AW performed the data analysis. AW was responsible for the drafting of the manuscript. AEW & JR made critical revisions to the paper for important intellectual content. AEW & JR supervised the study.

References Adams A. & Timmins F. (2006) Students views of integrating webbased learning technology into the nursing curriculum. Nurse Education in Practice 6(1), 12–22. Agho A.O. & Williams A.M. (1995) Actual and desired computer literacy among allied health students. Journal of Allied Health 24(2), 117–126. Atack L. & Rankin J.A. (2002) A descriptive study of registered nurses’ experiences with web-based learning. Journal of Advanced Nursing 40(4), 457–465. Bachman J.A. & Panzarine S. (1998) Enabling student nurses to use the information superhighway. Journal of Nursing Education 37(4), 155–161. Balen R.M. & Jewesson P.J. (2004) Pharmacist computer skills and needs assessment survey. Journal of Medical Internet Research 6, 1. Retrieved from: http://www.jmir.org/2004/1/e11/ on 17 March 2008.

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

JAN: REVIEW PAPER Bello I.S., Arogundade F.A., Sanusi A.A., Ezeoma I.T., Abioye-Kuteyi E.A. & Akinsola A. (2004) Knowledge and utilization of information technology among health care professionals and students in Ile-Ife, Nigeria: A case study of a university teaching hospital. Journal Of Medical Internet Research 6(4), 23. Billings D., Connors H. & Skiba D. (2001) Benchmarking best practices in web-based nursing courses. Advances in Nursing Science 23, 41–52. Billings D.M., Skiba D.J. & Connors H.R. (2005) Best practices in web-based courses: Generational differences across undergraduate and graduate nursing students. Journal of Professional Nursing 21(2), 126–133. Bloom K.C. & Hough M.C. (2003) Student satisfaction with technology enhanced learning. CIN: Computers, Informatics Nursing 21(5), 241–246. Brumini G., Kovic I., Zombori D., Lulic I. & Petrovecki M. (2005) Nurses’ attitudes towards computers: Cross sectional questionnaire study. Croatian Medical Journal 46(1), 101–104. Chumley-Jones H.S., Dobbie S. & Alford C.L. (2002) Web-based learning: sound educational method or hype? A review of the evaluation of literature. Academic Medicine 77(10), S86–S93. Compeau D.R. & Higgins C.A. (1995) Computer self-efficacy development of a measure and initial test. MIS Quarterly 19, 189–211. Cornford J. & Pollock N. (2003) Putting the University Online: Information Technology and Organisational Change. OUP, Buckingham. Cragg C.E., Edwards N., Yue Z., Xin S. & Hui Z.D. (2003) Integrating web-based technology into distance education for nurses in China: Computer and Internet access and attitudes. CIN: Computers Informatics Nursing 21(5), 265–274. Curtis E., Hicks P. & Redmond R. (2002) Nursing students experience and attitudes to computers: A survey of a cohort of students on a Bachelor in nursing studies course. ITIN 14(2), 7–17. Dambrot F.H., Watkins-Malek M.A., Silling S.M., Marshall R.S. & Garver J.A. (1985) Correlates of sex differences in attitudes toward and involvement with computers. Journal of Vocational Behavior 27(1), 71–86. DeBourgh G.A. (2003) Predictors of student satisfaction in distancedelivered graduate nursing courses: what matters most? Journal of Professional Nursing 19(3), 149–163. Department of Health (2004) The NHS Knowledge and skills framework and the development review process. HMSO, London. Retrieved from http://www.dh.gov.uk/assetRoot/04/09/08/61/ 04090861.pdf on 11 May 06. Dewhurst D.G., Macleod H.A. & Norris T.A.M. (2000) Independent student learning aided by computers: an acceptable alternative to lectures? Computers & Education 35, 223–241. Dorup J. (2004) Experience and attitudes toward information technology among first year medical students in Denmark: longitudinal questionnaire survey. Journal of Medical Internet Research 6(1). Retrieved from http://www.jmir.org/2004/1/e10/ on 17 May 2006. Dover L.V. & Boblin S. (1991) Student nurse computer experience and preferences for learning. Computers in Nursing 9(2), 75–79. Duggan A., Hess B., Morgan D., Sooyeon K. & Wilson K. (2001) Measuring students attitudes towards educational use of the Internet. Journal of Educational Computing Research 25(3), 267–281.

Literature review measurement tools Duke C. (2002) Managing the Learning University. OUP, Buckingham. Duttwieler P.C. (1984) The internal control index: a newly developed measure of locus of control. Educational and Psychological Measurement 44, 209–221. Ehrmann S.C. & Zu´n˜iga R.E. (1997) The Flashlight Evaluation Handbook TLT group. Retrieved from http://www.tltgroup.org/ Flashlight/Handbook/Toc.htm on 5 December 2008. Grigg P.A. & Stephens C.D. (1999) A survey of the IT skills and attitudes of final year dental students at Bristol University in 1996 and 1997. European Journal of Dental Education 3(2), 64–73. Hegney D., Eley R., Buikstra E., Fallon T., Isoar J. & Gilmore V. (2006) Australian nurses access and attitudes to information technology. Studies in health technology and informatics 122, 688–692. Hollander S. (1999) Assessing and enhancing medical students’ computer skills: a two-year experience. Bulletin of the Medical Library Association 67(1), 67–73. Honey M. (2004) Flexible learning for postgraduate nurses: a basis for planning. Nurse Education Today 24, 319–325. House of Commons Education & Skills Committee (2005) UK e–University: Third Report of Session 2004–05. The Stationery Office Limited, London. Jayasuriya R. & Caputi P. (1996) Computer attitude and computer anxiety in nursing: validation of an instrument using and Australian sample. Computers in Nursing 14(6), 340–345. Lewis M.J., Davies R., Jenkins D. & Tait M.I. (2001) A review of evaluative studies of computer-based learning in nursing education. Nurse Education Today 21, 26–37. Link T.M., Marz R., Link T.M. & Marz R. (2006) Computer literacy and attitudes towards e-learning among first year medical students. BMC Medical Education 6, 34. Loyd B.H. & Gressard C. (1984) Reliability and Factorial validity of computer attitude scales. Educational and Psychological Measurement 44, 501–505. Lynch D.C., Whitley T.W., Emmerling D.A. & Brinn J.E. (2000) Variables that may enhance medical students’ perceived preparedness for computer-based testing. Journal of American Medical Informatics Association 7, 469–474. Maag M.M. (2006) Nursing students attitudes toward technology: a national study. Nurse Educator 31(3), 112–118. Masiello I., Ramberg R. & Lonka K. (2005) Attitudes to the application of a web-based learning system in a microbiology course. Computers & Education 45(2), 171. Mathur S., Stanton S. & Reid W.D. (2005) Canadian physical therapists’ interest in web-based and computer-assisted continuing education. Physical Therapy 85(3), 226. Mattheos N., Nattestad A., Schittek M. & Attstro¨m R. (2002) Computer literacy and attitudes among students in 16 European dental schools: current aspects, regional differences and future trends. European Journal of Dental Education 6, 30– 35. Maurer M.M. & Simonson M.R. (1984) Development and Validation of a Measure of Computer Anxiety. Ames, Iowa. McBride S. & Nagle L. (1996) Attitudes toward computers: a test of construct validity. Computers in Nursing 14(3), 164–170. McFarlane T.A., Hoffman E.R. & Green K.E. (1997) Teachers’ Attitudes towards Technology (A survey). Paper presented at the

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd

771

A. Wilkinson et al. American Educational Research Association, March 24th–28th, Chicago, IL (ERIC #ED411279). Netemeyer R.G., Bearden W.O. & Sharma S. (2003) Scaling Procedures: Issues and Applications. Sage, Thousand Oaks. New Media Consortium (2006) The Horizon Report. The New Media Consortium and Educause Learning Initiative. Retrieved from http://www.nmc.org/pdf/2006_Horizon_Report.pdf on 30 July 2007. New Media Consortium (2007) The Horizon Report. The New Media Consortium and Educause Learning Initiative. Retrieved from http://www.nmc.org/pdf/2007_Horizon_Report.pdf on 30 July 2007. New Media Consortium (2008) The Horizon Report. The New Media Consortium and Educause Learning Initiative. Retrieved from http://www.nmc.org/pdf/2008_Horizon_Report.pdf on 30 July 2007. Parks P.L., Damrosch S.P., Heller B.R. & Romano C.A. (1986) Faculty and student perceptions of computer applications in nursing. Journal of Professional Nursing 2, 104–113. Rajab L.D., Baqain Z.H., Rajab L.D. & Baqain Z.H. (2005) Use of information and communication technology among dental students at the University of Jordan. Journal of Dental Education 69(3), 387–398. Ray N.J. & Hannigan A. (1999) A survey of the computer literacy of undergraduate dental students at a University Dental School in Ireland during the academic year 1997–98. European Journal of Dental Education 3(2), 56–63. Salamonson Y. & Lantz J. (2005) Factors influencing nursing students’ preference for a hybrid format delivery in a pathophysiology course. Nurse Education Today 25(1), 9–16. Samuel M., Coombes J.C., Miranda J.J., Melvin R., Young E.J.W. & Azarmina P. (2004) Assessing computer skills in Tanzanian medical students: an elective experience. MBC Public Health 4: 37. Retrieved from http://www.biomedcentral.com/1471-2458/4/37 on 28 October 2007. Saranto K., Leino-Kilpi H. & Isoaho H. (1997) Learning environment in information technology the views of student nurses. Computers in Nursing 15(6), 324–332. Scarpa R., Smeltzer S.C. & Jasion B. (1992) Attitudes of nurses toward computerisation: a replication. Computers in Nursing 10(2), 72–80. Schwirian P.M., Malone J.A., Stone V.J., Nunley B. & Francisco T. (1989) A comparison of the attitudes of nurses and nursing students. Computers in Nursing 7(4), 168–177. Seago B.L., Schlesinger J.B. & Hampton C.L. (2002) Using a decade of data on medical student computer literacy for strategic planning. Journal of Medical Library Association 90(2), 202– 209.

Sinclair M. & Gardner J. (1999) Planning for information technology key skills in nurse education. Journal of Advanced Nursing 30(6), 1441–1450. Sit J.W.H., Chung J.W.Y., Chow M.C.M. & Wong T.K.S. (2005) Experiences of online learning: students’ perspective. Nurse Education Today 25, 140–147. Slotte V., Wangel M. & Lonka K. (2001) Information Technology in medical education: a nationwide project on the opportunities of new technology. Medical Education 35, 990–995. Smith J. (2005) From flowers to palms: 40 years of policy for online learning. ALT-J 13(2), 93–108. Startsman T.S. & Robinson R.E. (1972) The attitudes of medical and paramedical personnel toward computers. Computers and Biomedical Research 5, 218–227. Steele D.J., Johnson J.E.J., Palensky Lynch T.G., Lacy N.L. & Duffy S.W. (2002) Learning preferences, computer attitudes, and student evaluation of computerisation instruction. Medical Education 36, 225–232. Stockton A.H. & Verhey M.P. (1995) A psychometric examination of the Stronge-Brodt Nurses’ attitudes towards computers questionnaire. Computers in Nursing 13(3), 109–113. Stronge J.H. & Brodt A. (1985) Assessment of nurses’ attitudes towards computerization. Computers in Nursing 3(4), 154–158. Vuorela M. & Nummenmaa L. (2004) How undergraduate students meet a new learning environment? Computers In Human Behavior 20(6), 763. Walmsley A.D., White D.A., Eynon R. & Somerfield L. (2003) The use of the Internet within a dental school. European Journal of Dental Education 7(1), 27–33. Whitemore R. & Knafl K. (2005) The integrative review: updated methodology. Journal of Advanced Nursing 52(5), 546–553. Wilkinson A., Forbes A., Bloomfield J. & Gee C.F. (2004) An exploration of four web-based open and flexible learning modules in post-registration nurse education. International Journal of Nursing Studies 41(4), 411–424. Williams A.M., Agho A.O. & McCloy C. (1996) Attitudes and perceptions of computer applications among baccalaureate physical therapy students. Journal of Physical Therapy Education 10(1), 14–17. Wilson B.A. (1991) Computer anxiety in nursing students. Journal of Nursing Education 30(2), 52–56. Wishart J. & Ward R. (2002) Individual differences in nurse and teacher training students’ attitudes toward and use of information technology. Nurse Education Today 22(3), 231–240. Yu S. & Yang K.-F. (2006) Attitudes toward web-based distance learning among public health nurses in Taiwan: A questionnaire survey. International Journal of Nursing Studies 43, 767– 774.

The Journal of Advanced Nursing (JAN) is an international, peer-reviewed, scientific journal. JAN contributes to the advancement of evidence-based nursing, midwifery and health care by disseminating high quality research and scholarship of contemporary relevance and with potential to advance knowledge for practice, education, management or policy. JAN publishes research reviews, original research reports and methodological and theoretical papers. For further information, please visit the journal web-site: http://www.journalofadvancednursing.com

772

 2009 The Authors. Journal compilation  2009 Blackwell Publishing Ltd