A rapid evidence-based service by librarians ... - Wiley Online Library

DOI: 10.1111/j.1471-1842.2009.00861.x

A rapid evidence-based service by librarians provided information to answer primary care clinical questions Jessie McGowan*,†,‡,§,–, William Hogg*,†,**, Tamara Rader†, Doug Salzwedel††,‡‡,§§, Danielle Worster†, Elise Cogo†,–– & Margo Rowan*,*** *Faculty of Family Medicine, †Institute of Population Health, University of Ottawa, Ottawa, Canada, ‡Department of Information Studies, University of Wales, Aberystwyth, UK, §Faculty of Medicine, University of Ottawa, –Ottawa Health Research Institute, **C.T. Lamont Primary Health Care Research Centre, E´lisabeth Bruye`re Research Institute, Ottawa, ††Child and Family Research Institute, ‡‡Canadian Institute for the Relief of Pain and Disability, §§University of British Columbia, Vancouver, ––Faculty of Information and Media Studies, University of Western Ontario, London and ***Rowan Health Policy Consulting, Ottawa, Canada

Abstract Background: A librarian consultation service was offered to 88 primary care clinicians during office hours. This included a streamlined evidence-based process to answer questions in fewer than 20 min. This included a contact centre accessed through a Web-based platform and using hand-held devices and computers with Web access. Librarians were given technical training in evidence-based medicine, including how to summarise evidence. Objectives: To describe the process and lessons learned from developing and operating a rapid response librarian consultation service for primary care clinicians. Methods: Evaluation included librarian interviews and a clinician exit satisfaction survey. Results: Clinicians were positive about its impact on their clinical practice and decision making. The project revealed some important ‘lessons learned’ in the clinical use of hand-held devices, knowledge translation and training for clinicians and librarians. Conclusions: The Just-in-Time Librarian Consultation Service showed that it was possible to provide evidence-based answers to clinical questions in 15 min or less. The project overcame a number of barriers using innovative solutions. There are many opportunities to build on this experience for future joint projects of librarians and healthcare providers.

Key Messages Implications for Practice d

d d

Health librarians can be trained to provide a question and answering service using clearly outlined procedures for retrieving evidence-based answers; Web access is essential but hand-held devices are not necessary for service delivery; Synthesising research to create context-specific answers can aid knowledge translation for clinicians.

Implications for Policy d d

d d

Librarians can deliver high-quality clinical information in less than 15 min; Librarians can collaborate effectively with physicians, nurses and other allied health professionals, as well as other librarians in a team-based approach to clinical reference services at point of care; A question and answering service helps clinicians’ with their patient-related decision making; Clinicians will use such a question and answering service as part of a multidisciplinary team.

Correspondence: Jessie McGowan, Institute of Population Health, University of Ottawa, 1 Stewart Street, Ottawa, ON, Canada K1N 6N5. E-mail: [email protected]

ª 2009 The authors Journal compilation ª 2009 Health Libraries Group. Health Information and Libraries Journal, 27, pp.11–21

11

12

Evidence-based service by librarians, Jessie McGowan et al.

Introduction As information experts, we know that receiving high-quality answers as quickly as possible is a priority for our clients. We developed a ‘Just-In-Time’ (JIT) Librarian Consultation Service for primary care clinicians (physicians, nurses and residents) to save them time in addressing clinical questions as well as to improve the quality and efficiency of their clinical decision making. This project was developed in response to the Government of Canada’s Primary Health Care Transition Fund, which was created to improve primary care in Canada and which highlights the importance of multi-disciplinary teams.1 We defined ‘Just-In-Time’ information as the delivery of answers to questions of primary care clinicians in 20 min or less. If we could provide an answer within this time period, then a clinician could potentially use this information during that patient’s office visit. We defined the type of question as ready reference, where the answer to a question is usually found in a standard reference work.2 Our librarians had electronic access to reference works, databases and journals of the Library Network at the University of Ottawa, which supports a medical school, nursing school and health sciences programme (http://www.biblio. uottawa.ca/index-e.php). Our service was provided to 88 primary care clinicians (82 physicians, 5 nurses and 1 resident) in Ontario, Canada, from September 2004 to March 2006. The service was tested using a randomised controlled trial (RCT) and results are reported elsewhere.3 The unit of randomisation in the RCT was the clinical question. Questions allocated to the intervention group were answered by the service, while no answer was provided for control group questions (and a message stating this was sent immediately) and therefore the clinicians had to find the answers themselves. The average amount of time for librarians to answer clinical questions was 13.68 min per question (95% CI, 13.38–13.98); or less than 15 min. This was a 25% reduction in time of what we had anticipated we could provide.3

Objectives The objectives of this paper were to describe the process and lessons learned from developing and

operating the JIT reference service. We hope to inspire other health information experts to pursue the further development of point-of-care information provision by librarians. Methods This paper discusses the issues encountered in the process of developing and operating the JIT service. The reasons why decisions regarding particular issues were made are analysed. Lessons learned about the project are synthesised from information identified in the project evaluation, as well through librarian interviews and clinician surveys. Design requirements and review of existing evidence The first challenge in designing the service was identifying the technology to sustain a streamlined evidence-based process to answer questions in fewer than 20 min. Additionally, we wanted to ensure that librarians did not feel pressured by the time limit and were confident in the process. Consequently, the main requirements for the design of our service included simple and clearly outlined procedures for retrieving evidence-based answers, easy to use technology, fast delivery of reliable electronic receipts and information, and an accurate and reliable repository for data collection and storage. The literature was reviewed for similar librarian consultation services to see if we could learn from others. Similar services were found both within and outside the library context, using either a clinical question and answer (Q&A) service4–11 or virtual reference service.12,13 All of the Q&A services were in primary care and while the virtual reference services included all types of health specialties. None of the Q&A services were delivered by librarians; however, both of the virtual services were. One of the Q&A services had librarian assistance for literature searching.11 The turn-around times for these services to answer clinical questions ranged from 2.5 h11 to greater than 1 week.9 We also found the Turning Research Into Practice (TRIP) database (http://www.tripdatabase.com), which was developed as a result of a clinical ques-



tions service.5 This is an immensely valuable database and became a primary resource. Many useful criteria were compiled by Rockliff to evaluate messaging software solutions, such as the time-stamping of questions and replies.13 Despite the positive features of the services reviewed, we chose a customised solution to fit the research requirements of our project. This solution involved two main components, technology and training. Technology Contact centre approach. A contact centre approach is one in which a client sends a request to a centralised service where the request is processed and the client assisted. This approach was chosen because of its successful implementation in information technology (IT) help desks to track and record technical problems.14 Furthermore, it would provide us with flexibility to record data in many fields and more importantly, it would allow us to ‘time and date’ stamp each question and answer to ensure that we could accurately report turn-around times. Web-based contact centre. We chose to deliver our contact centre through a Web-based platform that supported the use of both hand-held devices and desktop or laptop computers with Web access. The data were stored off-site for added security. Librar-

ians with clinical hand-held devices and computers linked directly to the contact centre software. The first step for the clinician was to send a question either by their hand-held device or using the website (Fig. 1). Automatically after the question was sent, the librarians’ computers would receive a pop-up notification in less than 60 s. The librarian then searched for information and once an answer was formed, this information with reference citation details was entered into the software also using a Web form. This information was sent to the requesting clinician and stored in the database, enabling librarians to search and retrieve previous questions or answers via a keyword search at any time. The resulting ticket included the date, time, a brief description of the action taken and the user identifier (ID) for the librarian or clinician who performed the action (Fig. 2). Additionally, a ‘notes’ field was populated with keywords for each question to assist in later retrieval. Using hand-held devices or Web form to ask questions. Clinicians were encouraged to use hand-held devices but they could also use the website to submit questions. Requirements for the hand-held devices and contact centre technology included the following key capabilities: d track incoming requests, response times, service usage, etc.; d support an frequently asked questions database; d monitor usefulness of response data;

Figure 1 JIT clinician web portal interface (TrialStat Corporation) ª 2009 The authors Journal compilation ª 2009 Health Libraries Group. Health Information and Libraries Journal, 27, pp.11–21

13

14


Figure 2 Ticket web portal activity log (TrialStat Corporation)

integrate response system with the email tracking system; d provide a scalable system to allow for additional clinicians or librarians in the future. Two hand-held devices were researched at the outset of the project, the BlackBerry (RIM Researsh in Motion, Ontario, Canada) and Palm Tungsten (Palm Inc, California, USA). Palm hand-helds, including the Tungsten, were the most frequently used type of hand-held by physicians at the time of the study and were favoured for a variety of activities, including recording patient interactions, drug reference and scheduling.15,16 The BlackBerry was selected because it did not require wireless base stations and associated hardware at clinical sites in order to send and receive email. This allowed clinicians to submit questions from anywhere in the project’s geographical region or from various locations around the office. The BlackBerry also used ‘push’ technology, enabling primary care clinicians to be automatically notified when project librarians had sent replies to clinical questions by email. d

Ethical and privacy concerns. A common concern of the development team was the potential liability of providing clinical information for use in a patient encounter. Accordingly, a disclaimer was developed in collaboration with the University of Ottawa’s legal department to emphasise JIT’s role as an information service, not a medical advice service. The disclaimer was included as a footnote to all answers sent by the librarians, which

allowed the librarians to execute their tasks within clear and safe parameters. Ensuring privacy of both patient data and clinician identity were also significant considerations during the project’s planning and operational phases. The consent form for entry into the project included a statement reminding clinicians that they were responsible for ensuring no data were sent that might identify a patient. Training librarians Four librarians (2.26 full-time equivalents), each with a Masters degree in Library and Information Science (MLIS) and experience in health information, were intensively trained to deliver the service. In addition to the training described below, project-specific training was provided for understanding primary care service delivery by a nurse practitioner. The Web portal developers also offered step-by-step technical training to the project librarians. A librarian project co-investigator provided training in understanding evidence-based philosophies. Librarians had specific training in evidencebased medicine to help them understand how to interpret a well-built clinical question, which was the format that we used to train clinicians to pose questions to the service.17 Unlike a traditional reference service, our service was designed so that the librarian would receive only a clinical question and to save time no reference interview would be performed. The clinician used the PICO (Popula-



tion, Intervention, Comparator ⁄ control, Outcome) structure to fill in the blanks beside each element in the reference form. This allowed the librarian to easily break the question down into manageable elements. Librarians further categorised all clinical questions by type. Additionally, the librarian needed a good understanding of medical terminology used in questions. Where necessary, approved resources (e.g. MEDLINEPLUS, Taber’s Medical Dictionary) were used to identify or clarify clinical terminology or drug names. A master clinical terminology file was created for the project and updated regularly. Librarians were given advanced evidence-based training to search secondary (i.e. pre-appraised) resources, such as reviews produced by The COCHRANE Collaboration, BMJ CLINICAL EVIDENCE; reputable practice guidelines (e.g. National Institute for Health and Clinical Excellence, NICE guidelines); and health technology assessment reports known to follow a rigorous process. The training involved using unfamiliar resources, which was self-directed but supplemented with collaborative learning among the librarians through journal club-type sessions. Also librarians were provided training in how to appraise the literature, recognise the study design of the searched information and identify the best level of evidence available from a list of search results. Librarians used existing critical appraisal guides to learn this aspect of the process and consulted with one another in uncertain cases.18 Answer preparation training. A process was developed for delivering consistent results to clinicians that included selecting an excerpt from the text of the information source that best represented the answer, thus avoiding the need to re-write the answer, and presenting the answer in a neutral manner. Full bibliographic citation details and, where possible, a link to the full text were provided. Where conflicting evidence was found, the librarian clearly indicated this and gave a summary of the differing viewpoints, with links to the full text for both answers where possible. A day in the life of JIT. This section provides an example of the process that the librarians used on a daily basis (Fig. 3). Each morning, librarians

opened their Web-based portal. The question and answer process was initiated each time a query was submitted by a clinician. After a question was sent, the software interface sent a pop-up notice to the service and an audio chime sounded. A librarian then took ownership of the question. If the question was considered ‘out-of-scope’ (i.e. if it was a patient-specific drug dosage question or not a practice-oriented question), it was entered into the database but not answered. The librarian then sent a reply to the clinician using a template informing them that the question was out of the scope, and then closed the ticket. If multiple questions were asked, librarians were instructed to answer only the first question. The next step was to categorise the question by level of complexity from one (a simple question with no modifiers) to four (a question with many modifiers). The question type (diagnosis, aetiology, therapy, prognosis, prevention or other) was also recorded.19 Next, the clinician’s question was copied into the librarian answer template and the question was edited as necessary for clarity. The librarian then selected a resource to assist in answering the question. A checklist was developed for answering questions (Table 1). Librarians consulted the list of quality-approved resources to select the best resource for the type of question. These included a wide variety of information sources such as systematic reviews, evidence summaries or guidelines from government or other health agencies, major RCTs with commentary, position statements from professional organisations and excerpts from medical textbooks. Electronic resources were used as primary sources, e.g. MEDLINE, TRIP and BMJ CLINICAL EVIDENCE. Table 2 shows the list of the most used resources that were used during the RCT. As well, the database of archived clinical answers was consulted whenever it was perceived that a similar question had been previously asked. Once the librarian located appropriate evidencebased information, he or she would select and summarise the information and add it into the answer field of the librarian answer template. If it was appropriate for the type of question, a ‘yes ⁄ no’ answer was preferred and it would be entered at the top of the answer field and ⁄ or a qualifying statement. The reference citation was


15

16


A clinician asks a question using the handheld, just in time (JIT) webform, phone, or in person

The Librarians are signed in to the just in time dashboard

All librarians receive a new question notification

One librarian takes ownership (or assigns ownership) and looks at the question

Librarian sends the out of scope message to clinician

Librarian sends incomplete message to clinician

Step 1: categorize No

No

Librarian looks up medical terms

No

Is it within scope?

Is it complete?

Do you understand the question?

Yes

Yes

Yes Step 2: search

Categorize the question. Search pre-appraised, evidence-based sources according to the question type:

Therapy

Diagnosis

Prognosis

Prevention

Etiology/ Risk

Other

Cochrane clinical evidence TRIP database

Guidelines Harrison’s online TRIP database

Evidencebased journals TRIP database

Cochrane guidelines TRIP database clinical evidence

TRIP database Harrison’s online

Attract TRIP database medline

Figure 3 JIT librarian process flowchart

included along with a link to the full text when possible. If the website was not available to the clinician due to being password protected or copyrighted, a note was made that it was a ‘subscription’ resource. Once the question was completed, the librarian put a check in the ‘close

ticket’ box and sent the response to the clinician. The service received approximately 51 questions per week. Evaluation. The librarians kept track of all their problems and successes during the project. At the



Step 3: appraisal

Librarian appraises the search results

Results are found

Was a relevant answer found?

Yes Librarian searches again or scans the search results

No

1. Is a systematic review or randomized controlled trial available?

Yes

2. Is the study current?

Yes

3. Is it applicable to the clinical context? Nothing is found Librarian selects the best resource, or the best 2 or 3 resources, if the answers are conflicting

Librarian briefly summarizes the evidence into an answer Librarian sends clinician a message that an answer could not be found in 15–20 minutes

Step 4: summarize

Librarian qualifies and contextualizes the answer Librarian adds references and web links

Step 5: Send

Librarian closes the ticket and sends answer to Clinician

Answer received by clinician within 1 minute

Step 6: tag & document Librarian tags the question so that it can be found efficiently in the just in time database

Librarian documents his or her strategy. What was learned? What was done differently?

Figure 3 (Continued)

end of the project, librarian interviews were conducted. These interviews were semi-structured and in-depth. They were used to explore the librarians’ sense of satisfaction with JIT and their thoughts about the service. A semi-structured interviewing guide was developed. It was used in a flexible manner in terms of question sequencing and general phrasing. A qualitative researcher, who was not part of the JIT project, conducted the interviews. Prior to the interview, the interviewer asked if audio-recording was acceptable, explaining that only she would have

access and listen to the original recording and that the purpose of audio-recording was to facilitate the writing of a more complete interview summary, which was used for analysis purposes. Clinicians answered an exit satisfaction survey that was delivered with mostly closed-ended Likert-scale items (1 = having positive impact; 5 = having no impact). The survey was designed and administered to all JIT clinicians at the end of the project. It was used to investigate the impacts to our participants in terms of service satisfaction.


17

18

Evidence-based service by librarians, Jessie McGowan et al. Table 1 Librarian checklist Check

JIT librarian checklist items

1.

Do you understand the question and terminology? • For difficult terminology, consult Taber’s encyclopaedia or eMedicine (which allows you to click on terms to look them up in Stedman’s dictionary). • For acronyms, try Harrison’s online. • If you only have the colloquial term for a symptom, disease, or procedure, try typing it into the Ovid interface and it will ‘link to a subject heading’ and give you the controlled term. Click the ‘i’ button for the scope note. Determine if the question has already been asked and answered. Consult the project database of answers to clinical questions. This database will contain the question summary and search details, including resources consulted, strategy, and search date; the information allows co-librarians to replicate the search, if necessary. Decide which resource would most efficiently answer the question Begin with pre-appraised literature (e.g. COCHRANE Library, TRIP, ACP Journal Club, Clinical Evidence), an evidence-based text (e.g. Harrison’s online) and lastly try MEDLINE. If after a few minutes, you do not find your answer, consider trying a different resource. Use the most specific keywords to describe the concept. For example, instead of prostate cancer tests, use PSA test or prostate-specific antigen. Consult an online textbook, if necessary. Consult MEDLINE, if necessary. In a MEDLINE search, combine a few specific terms (both MeSH and natural language). Use the study design filters in OVID MEDLINE, where possible.

2.

3.

4. 5.

Table 2 Top 10 resources used Times used percentage % (n = 1889)

Resource name 1. eMedicine 2. PUBMED 3. National Guideline Clearinghouse 4. BMJ CLINICAL EVIDENCE 5. Prodigy 6. The COCHRANE Collaboration’s Database of Systematic Reviews 7. Canadian Immunization Guide 8. NATURAL STANDARD DATABASE 9. Primary Care Question Answering Service (UK National Library of Health) 10. MEDLINE (OVID interface)

20.1% (380) 14.2% (268) 8.8% (166) 4.2% (80) 3.9% (73) 3.0% (57) 3.0% (57) 2.6% (50) 1.6% (30) 1.5% (29)

Results and lessons learned All librarians participated in the interviews. They indicated that the training and preparation they received prior to answering questions was adequate. All spoke very highly of their colleagues and work atmosphere, including the investigators and project staff. Overall, JIT fostered a highly

collaborative and egalitarian space, where the support of colleagues exceeded expectations. The librarians noted that they felt the position also carried with it a higher degree of responsibility than is normal for medical librarians. They also noted less explicit expectations including establishing rapport with clinicians, coaching and troubleshooting the use of the hand-helds, and teaching participants to frame questions in an evidence-based manner using the PICO format. Eighty-three per cent of clinicians responded to the exit satisfaction survey. The majority of these clinicians (86%) indicated that the service had a positive impact on care to patients, and that they would use a similar service to JIT, if it were available. Most clinicians preferred to use a Web interface or a hand-held device to access the service. A content analysis of open-ended questions showed that clinicians related improved decision making to the currency and quickness of information and that the service saved them time. Several clinicians spoke of improved confidence in their clinical actions because of JIT and some indicated that the JIT response confirmed their clinical plan. Many clinicians expressed appreciation for the JIT ser-



vice, and were supportive of such a service continuing. The following outlines the key lessons learned from the development and operation of the JIT service based on the interviews and surveys from the JIT librarians and clinicians. The importance of a complete and ‘in-scope’ question Clinicians have different ways to pose questions that are not always answerable in a time-sensitive manner. We wanted to assess a new service that was time sensitive and we did not want to spend time responding to questions that were not within the scope of the project. We had hoped that training clinicians to ask clinical questions would help teach them what was an in-scope question. During the sessions, examples of ‘in-scope’ and ‘outof-scope’ questions were presented. A total of five questions (0.4%; 5 ⁄ 1889) were determined to be out-of-scope during the trial phase of the service. Use of hand-held devices The integration of using a hand-held device during office hours by clinicians was a challenge and a concern some expressed upon entering the service. Librarians conducted a 45–60 min training session that focused on the use of the hand-held so that it was seen as a positive device. During this session, clinicians were encouraged to submit a question to demonstrate the ease of use. Subsequently, only 5.9% of clinicians expressed dissatisfaction with the hand-held. Specialised skills training for librarians Challenging situations occurred for the librarians. These included having multiple questions arriving at once, new or unfamiliar medical jargon and questions that could not be efficiently or effectively answered. These challenges fostered the creation of tools such as a resource list of highquality, reliable information sources with their topic areas, strengths and weaknesses given. Other resources were produced, including a document linking common ailments to appropriate resources

for dealing with the condition, and a list of definitions for medical terminology to streamline the question–answer process. Database of saved answers Several months into the service, we developed a database of saved answers. Although we had access to the contact centre database, that database was designed primarily for research purposes. A new database was created to store in a searchable format a critical mass of questions and answers that librarians accessed before searching other resources. This database provided additional information and was helpful in shortening the librarian response time and providing consistency and quality assurance of answers. We would recommend building this database at the beginning of a similar service. Just-in-time information as a knowledge translation tool The Canadian Institutes of Health Research (CIHR) define knowledge translation (KT) as ‘synthesis, dissemination, exchange and ethically sound application of knowledge to improve the health of Canadians, provide more effective health services and products, and strengthen the health care system.’20 Our service did this by providing clinicians with evidence-based information that they could apply to their patients. We accomplished this by ensuring that answers were based on the full body of research literature rather than the result of a single study, and librarians made use of synthesised research findings from systematic reviews, guidelines and health technology assessments. By synthesising research to create context-specific answers, KT is better positioned within a patient– physician clinical encounter, thus making it more personal and readily applied in practice. This is in keeping with previous research that physicians prefer summaries of evidence rather than producing them themselves.21 Discussion and next steps Overall, users were satisfied with the JIT service and clinicians who responded to the exit satisfaction survey rated their level of satisfaction with the service


19

20


as having a positive impact on care to patients, and that they would use a similar service. Some of the questions, however, were complex or referred to rare conditions or a patient-specific set of conditions where it was not possible to provide a targeted answer. A future possibility is to evolve a hybrid service to allow more complex questions but allow librarians a longer time frame to formulate a response. Other feedback found that clinicians did not always require an answer in 20 min or less for their clinical questions. This suggests that a service could have different levels of turn-around times available depending on the urgency of the request. We recommend that any Q&A service developed in the future take into consideration not only the clinicians’ needs, but also communicate what is achievable as it is important to manage the expectations of clinicians in order to maintain their satisfaction with the service. In Canada, primary care clinicians can find library services either in hospital libraries, academic libraries, the library of the College of Family Physicians of Canada or find information on their own. These libraries provide a varying degree of services to family physicians or primary care professionals. The JIT service could be integrated into any of these libraries. The service could be targeted at all primary care professionals or just specifically to family physicians. As part of the primary care team, the information needs of nurses and allied health professionals were sometimes found to be different than those of general practitioners during our study. In the future, a Q&A service should take into consideration preventive health, lifestyle management and patient education questions by including resources that are optimised for answering these queries and training librarians in finding high-quality information targeted at these information needs. Conclusions The JIT project has proved that it is possible to disseminate evidence-based answers to primary care clinical questions in 15 min or less through the development and operation of librarian consultation service. The project overcame the barriers of geographical distance (from the information source), hesitancy to use the hand-held device, unfamiliarity with asking PICO questions and lack

of initial specific knowledge or skills in the librarians. This project was very successful and positively received by both clinicians and librarians. Both librarians and clinicians would like to be part of a future service if it were available. We feel that our process is manageable and that health librarians can be trained to provide a similar service. Conflict of interests The views expressed in this report are the views of the authors and do not necessarily reflect those of the Ontario Ministry of Health and Long Term Care. Source of funding Support for this research was from the Primary Health Care Transition Fund, provided by the Ontario Ministry of Health and Long Term Care, Toronto, Ontario, Canada. Acknowledgements We thank Craig Campbell, who was a co-investigator on the project, and Peter Tugwell and Christine Urquhart for their support and supervision of J. McGowan’s PhD thesis at the Department of Information Studies, University of Wales, Aberystwyth. References 1 Health Canada. Primary Health Care Transition Fund [cited 2 April 2008], 2008. Accessible at: http://www.hc-sc.gc.ca/ hcs-sss/prim/phctf-fassp/index_e.html. 2 Katz, W. A. Introduction to Reference Work. Volume 1: Basic Information Sources. New York: McGraw-Hill, 1997. 3 McGowan, J., Hogg, W., Campbell, C. & Rowan, M. Justin-Time information improved decision-making in primary care: a randomized controlled trial. PLoS ONE 2008, 3, e3785. 4 Del Mar, C. B., Silagy, C. A., Glasziou, P. P., Weller, D., Spinks, A. B., Bernath, V., Anderson, J. N., Hilton, D. J. & Sanders, S. L. Feasibility of an evidence-based literature search service for general practitioners. Medical Journal of Australia 2001, 175, 134–137. 5 Brassey, J., Elwyn, G., Price, C. & Kinnersley, P. Just in time information for clinicians: a questionnaire evaluation of the ATTRACT project. British Medical Journal 2001, 322, 529–530. 6 Swinglehurst, D. A., Pierce, M. & Fuller, J. C. A clinical informaticist to support primary care decision making. Quality Health Care 2001, 10, 245–249.


Evidence-based service by librarians, Jessie McGowan et al. 7 Martin, P. & Kauser, A. An informaticist working in primary care: a descriptive study. Health Information Journal 2001, 7, 66–70. 8 Greenhalgh, T., Hughes, J., Humphrey, C., Rogers, S., Swinglehurst, D. & Martin, P. A comparative case study of two models of a clinical informaticist service. British Medical Journal 2002, 324, 524–529. 9 Verhoeven, A. A. & Schuling, J. Effect of an evidencebased answering service on GPs and their patients: a pilot study. Health Information and Library Journal 2004, 2, 27– 35. Available: PM:15317573. 10 Harrison, K. Clinical Question & Answer Service Evaluation. Rotherham, UK: Rotherham Primary Care Trust, 2006. 11 Hayward, J. A., Wearne, S. M., Middleton, P. F., Silagy, C. A., Weller, D. P. & Doust, J. A. Providing evidence-based answers to clinical questions. A pilot information service for general practitioners. Medical Journal of Australia 1999, 171, 547–550. Available: PM:10816708. 12 Markgren, S., Ascher, M. T., Crow, S. J., Lougee-Heimer, H., Markgren, S., Ascher, M. T., Crow, S. J. & LougeeHeimer, H. Asked and answered-online: how two medical libraries are using OCLC’s QuestionPoint to answer reference questions. Medical Reference Services Quarterly 2004, 23, 13–28. 13 Rockliff, S., Peterson, M., Martin, K. & Curtis, D. Chasing the sun: a virtual reference service between SAHSLC (SA) and SWICE (UK). Health Information and Library Journal 2005, 22, 117–123. Available: PM:15910582. 14 Koole, G. & Mandelbaum, A. Queueing models of call centers: an introduction, 2002 [cited 2008 December 12].

15

16

17

18 19

20

21

Accessible at: http://www.springerlink.com/content/ m5170t745702125j/fulltext.pdf. Carroll, A. E. & Christakis, D. A. Pediatricians and personal digital assistants: what type are they using? AMIA Annual Symposium Proceedings 2003, 2003, 130–134. Baumgart, D. C. Personal digital assistants in health care: experienced clinicians in the palm of your hand? Lancet 2005, 366, 1210–1222. Richardson, W. S., Wilson, M. C., Nishikawa, J. & Hayward, R. S. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club 1995, 123, A12–A13. Guyatt, G. H. & Rennie, D. Users’ guides to the medical literature. JAMA 1993, 270, 2096–2097. Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W. & Haynes, R. B. Evidence-based Medicine: How to Practice and Teach EBM, 2nd edn. London: Churchill-Livingstone, 2007. CIHR. Canadian Institutes of Health Research [homepage on the Internet]. About Knowledge Translation: The KT Portfolio at CIHR, 2000 [updated 22 October 2008; cited 12 December 2008]. Accessible at: http://www cihr-irsc gc ca/e/ 29418 html. McColl, A., Smith, H., White, P. & Field, J. General practitioner’s perceptions of the route to evidence based medicine: a questionnaire survey. British Medical Journal 1998, 316, 361–365. Available: PM:9487174.

Received 11 February 2009; Accepted 23 July 2009


21