Transistor Amplifiers

2 downloads 0 Views 29MB Size Report
Feb 1, 2014 - user. We will present how we are moving RevMan online to a ... Background: This group is bidding for EU Horizon 2020 support. (PHC 26 2014) ... which would be supported by a manual quality assurance, based on all extracted data ...... each study as an attempt to mimic a hypothetical RCT comparing the.
Abstracts of the 22nd Cochrane Colloquium Supplement 2014

Oral sessions Session 01: #CochraneTech and review production Session 02: Summary of findings tables and the quality of evidence Session 03: Review methods: missing data Session 04: Priority setting approaches Session 05: Tools for assessing bias Session 06: Citation screening and study selection (1) Session 07: Impact of Cochrane Reviews Session 08: Knowledge translation for policy makers Session 09: Review methods: outcomes Session 10: Statistical methods Session 11: Citation screening and study selection (2) Session 12: Complex reviews Session 13: Knowledge translation and communicating evidence Session 14: Review methods: publication bias Session 15: Reporting of trials and research (1) Session 16: Updating reviews Session 17: Issues around implementing evidence Session 18: GRADE and guidelines Session 19: Review methods: miscellaneous Session 20: Learning and capacity development Session 21: Reporting of trials and research (2) Session 22: Review production and dissemination

1 2 4 6 8 9 11 12 14 15 17 18 20 21 23 25 25 27 29 30 32 33

Poster Abstracts Session 1 Session 2 Session 3 Session 4 Session 5 Session 6

35 49 63 78 92 104

Workshops Workshops

122

Abstracts available online at www.cochrane.org

Abstracts of the 22nd Cochrane Colloquium Oral Presentations

Personally tailored, multilingual, up-to-date evidence-based summaries of effects of care, when and where you want it: an EU bid

Oral session 1 #CochraneTech and review production

Purgato M1 , Bagley S2 , Barbui C1 , Brailsford D2 , Furtado V2 , Leucht S3 , Salanti G4 , Shahar Y5 , Shuttleworth G2 , Soares-Weiser K6 , Tsafnat G7 , ¨ aki ¨ M8 , van Valkenhoef G9 , Xia J10 , Adams C2 Valim 1 University of Verona, Italy; 2 University of Nottingham, UK; 3 Technische ¨ Munchen, ¨ Universitat Germany; 4 University of Ioannina, Greece; 5 Ben Gurion University of the Negev, Israel; 6 Enhance Reviews, UK; 7 University of New South Wales, Australia; 8 University of Turku and Turku University Hospital, Finland; 9 University Medical Center Groningen, The Netherlands; 10 Systematic Review Solutions, UK

The future of review production for Cochrane Mavergames C1 , Thomas J2 and Knowledge Management Department, Germany; 2 Informatics and Knowledge Management Department, Denmark

1 Informatics

Background: Cochrane’s technology and processes for review production and evidence synthesis have served us well. RevMan, Archie and, more recently, the Cochrane Register of Studies (CRS) have provided a solid backbone to assist in moving Cochrane toward synthesizing and managing all available evidence. But are our tools and processes ready for the future? Do they work effectively and efficiently with each other? Are they optimized for the browser-based, mobile-friendly demands of today’s users? Objectives: This oral presentation will outline Cochrane’s Informatics and Knowledge Management Department’s plans to move technology forward in a more integrated, online way whilst encouraging the curating and analysis of knowledge for easier dissemination and re-use by the user. We will present how we are moving RevMan online to a browser-based environment, plans around implementing a Cochrane Author Support Tool, the implications for CRS and study screening, how the emerging Cochrane community and knowledge base site is taking shape, in addition to showing how the role of application programming interfaces (APIs) and linked data are helping to connect these systems to create a more coherent and effective user experience for all. In addition we will summarize in brief the outcomes of the #CochraneTech Symposium and the implications for future development. Methods: This presentation will take the form of a review of progress to-date on various technology and knowledge management projects within Cochrane as well as future strategy plans in these areas. Conclusions: We plan to show that Cochrane is working to meet the user’s demands through harnessing the power of technology and will also encourage feedback from attendees to get involved in the #CochraneTech conversation by sharing their innovative ideas and responding to the changes.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: This group is bidding for EU Horizon 2020 support (PHC 26 2014) using schizophrenia as an exemplar for all health care. Schizophrenia is a serious mental illness, the cost of which to health services alone is more than all of either cancer or coronary care. Schizophrenia affects 1 in 25 households and has a profound influence on a person’s ability to think, perceive, feel and move. An enormous effort is invested in producing and maintaining a syntheses of the effects of treatments in the light of valid criticism or new evidence. These highly technical summaries are, however, slow to produce, difficult to personalize and problematic to maintain. In addition, they are often not accessible or acceptable to most clinicians or patients. Objectives: To gather a team to create a multilingual, semi-automated system to produce understandable syntheses of best current evidence on interventions for schizophrenia. Methods: Since many people (in and out of Cochrane) are working in parallel around the European Union (EU) and elsewhere, gathering a European group together was not difficult. The universal ideas of evolving an automated process which would be supported by a manual quality assurance, based on all extracted data from all the relevant trials (20 000 in schizophrenia), as well as data on economic outcomes and adverse effects were common to many. This team hopes to produce a EU wiki-type repository, acting as a data source for all subsequent processing, auto-data extraction tools for PDFs of trials, and a tested, acceptable, user-interface which can accept personal queries in any language, and summarize and synthesize relevant data to produce reports ‘live’ in any language. It would be free to everyone. Results: We do not know now if our bid will be successful. We do know, however, that the vision was not hard to sell to a range of researchers, programmers and small businesses. Conclusions: Production of something nimble, attractive and useful is now possible. If we do not do it, someone else will.

1

Cochrane Database Syst Rev Suppl 1–150 (2014)

2 Piloting living systematic reviews in traumatic brain injury Synnot A1 , Maas A2 , Menon D3 , Clavisi O1 , Gruen R1 1 National Trauma Research Institute, Australia; 2 University of Antwerp, Belgium; 3 University of Cambridge, UK

Background: Living systematic reviews - online summaries of healthcare research that are updated as new research becomes available - are an attractive but largely untested answer to bridging the trade-off between systematic review currency and quality. Within the context of an international research project in traumatic brain injury (TBI), we have been trialling novel review methods, with a view to creating a suite of living systematic reviews in TBI. Objectives: To develop and sustain an international TBI knowledge community that maintains up-to-date, high quality living systematic reviews of the current state of knowledge in the most important questions in TBI. Methods: Teams of novice reviewers, supported by expert advisory panels and review methodologists, will be responsible for the creation of a suite of reviews on prioritized TBI topics. Online review workflow tools will facilitate the delegation of micro tasks (e.g. small amounts of screening, data extraction and critical appraisal) to dispersed groups of author teams as new studies are identified through frequent search updates. We will experiment with novel publication methods across interactive websites and traditional journals to allow the continual integration of new studies with the original review findings. Novel participation methods may be explored over time, as the project facilitates the use of large and evolving authorship teams, including expert crowdsourcing. Results: We have convened a ‘knowledge commons’ to oversee the project and commenced author training. We will report on the success of the remainder of the project against planned objectives. Conclusions: Truly living systematic reviews offer exciting possibilities for real-time knowledge translation products that build on systematic review evidence. Future project stages include integration of review findings with ‘living’ clinical practice guidance that feed into ‘living’ clinical decision support systems, involving a world-wide expert community, and bridging current temporal disconnects between evidence, guidelines and practice.

linkedPICO project: how to make different tools and platforms in the medical knowledge ecosystem talk past their silos Brandt L1 , Elliot J2 , Kunnamo I3 , Rada G4 1 MAGIC-Making GRADE the Irresistible Choice, Norway; 2 Covidence and Australasian Cochrane Centre, Monash University, Australia; 3 EBMeDS, Finland; 4 Epistemonikos, Chile

Background: PICO is an acronym for the framework of structured clinical questions consisting of population, intervention, comparator and outcome. Although variations, all or parts of this framework can generally be applied to most data in the medical knowledge ecosystem. An increasing number of collaborators share a common interest in harmonizing data models and representation of data in single studies, systematic reviews, guidelines and decision support systems in order to improve patient care, while reducing duplication of effort. The linkedPICO project has been established to facilitate Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

this harmonization, to help knowledge break free of its silos and join knowledge management tools with the linked world of the semantic web. Objectives: With the linkedPICO project we aim to leverage modern technology to achieve easier export and import of data between knowledge management tools dealing with single studies, systematic reviews, guidelines and decision support. We further aim to establish better flow and accessibility of information through identifying and standardizing core elements centered around structured PICO questions in these tools. We have set up actual data transfers and integrations between various tools, to showcase how linking PICO data works in real life.

Oral session 2 Summary of findings tables and the quality of evidence Enhancing the acceptance and implementation of summary of findings tables in Cochrane Reviews: user testing study Carrasco-Labra A1 , Brignardello-Petersen R2 , Santesso N1 , Mustafa R3 , ¨ Schunemann H1 1 McMaster University, Canada; 2 University of Toronto, Canada; 3 University of Missouri-Kansas City, USA

Background: Cochrane has adopted the principles of the GRADE approach for evaluating the quality of evidence for outcomes reported in systematic reviews. Our aim was to determine the effect of alternative formats and columns of summary of findings (SoF) tables on stakeholder’s understanding and acceptability of the presentation of review results of continuous and dichotomous outcomes. Methods: We presented alternative and current SoF table formats and conducted direct comparative tests. Two researchers conducted semi-structured interviews with end users (clinicians, guideline developers, researchers and policy makers) using standardized scripts. The iterative process included three main rounds: (i) GRADE working group meetings (two sessions; 10–20 participants each); (ii) one-on-one, 30-min interviews (10 participants); (iii) international conferences and meetings (10 participants). We asked 7-point Likert-type and open-ended questions. The information collected was tabulated and coded focusing mainly on limitations and barriers against understanding, usability and acceptability of the formats. Results: All participants agreed that SoF tables are helpful to present systematic review results. There was preference for simplification of tables (e.g. reducing the number of columns, minimising comments and reducing text). For dichotomous outcomes, participants considered that the number needed to treat for benefit (NNTB) and harm (NNTH) and the risk difference expressed as percentage (RD%) are useful to display review results (Likert 1–7 = 5.3; SD = 1.7). For continuous outcomes, the minimal important difference units (MIDU) of improvement and the proportion of participants showing an improvement (PPSI) showed to be easier to understand (Likert 1–7 = 6.1; SD = 2.1). The ‘what happens’ column was perceived as very valuable by most participants. Conclusions: Participants prefer simple and less crowded tables. NNTs and RD%, and MIDU and PPSI seem to facilitate understanding of dichotomous and continuous outcomes results, respectively. Cochrane Database Syst Rev Suppl 1–150 (2014)

3 Enhancing the acceptance and implementation of summary of findings tables in Cochrane Reviews: two non-inferiority randomized controlled trials Carrasco-Labra A1 , Brignardello-Petersen R2 , Carrasco-Labra NS1 , Mustafa ¨ H1 R3 , Etxeandia-Ikobaltzeta I4 , Schunemann 1 McMaster University, Canada; 2 University of Toronto, Canada; 3 University of Missouri-Kansas City, USA; 4 OSTEBA, Spain

Background: Although systematic reviews represent important tools for knowledge translation, users struggle with understanding and interpreting their results. GRADE summary of findings (SoF) tables have been developed to display results of reviews in a concise and transparent manner. However, it has been suggested that additional methods to present risks and display review results in the SoF tables are needed. Methods/design: We conducted two non-inferiority parallel, two-armed randomized controlled trials to determine whether alternative formats to present risks and display SoF tables were not inferior compared to the current standard formats (Table 1). We measured participant understanding, accessibility of the information, satisfaction and preference for either formats. We invited systematic review users to participate (i.e. clinicians, guideline developers and researchers). For the outcome understanding, the non-inferiority boundary was a 10% difference in the proportion of participants answering a given question correctly. Results: Participants allocated to the alternative formats had a higher understanding than participants allocated to the current formats. The presentation of risk differences (95% CI −0.66 to −0.36) and the ‘what happens’ column (95% CI −0.66 to 0.04) showed to be non-inferior regarding understanding compared to the current formats tested. Alternative items also showed to be more accessible overall [Mean Difference (MD) = 0.3; SE = 0.11, P value = 0.001] and assisting decision making (MD = 0.5; SE = 0.18, P value = 0.011). Participants from both groups strongly preferred the SoF table showing alternative formats. Conclusions: These trials provide systematic review authors and users with a series of alternative items that can be used along with the current formats of SoF tables. Some of these items have proven to be at least as effective as the current formats, with higher accessibility of information and preference from systematic review users.

An approach to evaluate the quality of evidence from a network meta-analysis Salanti G1 , Del Giovane C2 , Chaimani A1 , Caldwell D3 , Higgins JP3 1 University of Ioannina, Greece; 2 University of Modena and Reggio Emilia, Italy; 3 University of Bristol, UK

Background: Systematic reviews (SRs) that collate data about the relative effects of multiple interventions via network meta-analysis (NMA) are highly informative for decision-making purposes. A NMA provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. Objectives: To propose an approach based on the methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses to determining confidence in the output of a NMA. Five domains are judged: study limitations, indirectness, inconsistency, imprecision and publication bias. Methods: The suggested framework for evaluating a NMA acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the NMA estimates of effect size; (iii) the importance of the transitivity assumption to the validity of NMA; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We applied the strategy to a SR comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. Results: The suggestions proposed were workable in the empirical example. Out of seven effect estimates, two had moderate confidence, four low and one very low confidence mainly due to study limitations, inconsistency and imprecision. Our confidence in the ranking of the included treatments was low. Conclusions: The proposed framework can be used to determine confidence in the results from a NMA. The assumption of transitivity which is key to a NMA should be considered in the indirectness domain and its assessment is critical. In the inconsistency component, both between-study variance within a comparison and differences between direct and indirect evidence in the network should be evaluated. The contributions of each piece of direct evidence can make the judgments less subjective.

Rating the quality of the evidence for the studies that evaluate the importance of the outcomes of interest Alonso-Coello P1 , Zhang Y2 , Selva A1 , Sanabria AJ1 , Rigau D1 , Sola` I1 , ¨ H2 Guyatt G2 , Schunemann 1 Iberoamerican Cochrane Center, Spain; 2 McMaster University, Canada

Background: When developing systematic reviews (SR) and clinical practice guidelines (CG), consideration of patients’ values and preferences (VPs) is an important issue. GRADE (Grading of Recommendations Assessment, Development and Evaluation) considers the quality

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

4 of the evidence supporting estimates of VPs and their variability when moving from evidence to recommendations. Nevertheless, detailed guidance about how to evaluate quality of evidence of studies on VPs is unavailable. Objectives: To develop a framework to rate the quality of the evidence about VPs information. Methods: Iterative process including: brainstorming of international groups of methodologists and feedback from the GRADE Working Group; systematic review of methodological guidelines handbooks; and review of the methodological literature in the field of guideline development and health economics. Results: We have developed an initial proposal to rate the quality of evidence from primary studies addressing patients’ typical VPs. For the risk of bias we propose to consider several aspects common across study designs (e.g. selection bias) and some specific to the type of methodology used (e.g. for standard gamble, qualitative research). The framework considers other factors including consistency, precision, directness and publication bias. Starting from high quality, limitations in these aspects lowers confidence in the VPs results. We will present the full framework with examples and a presentation of the results in a summary of findings table. Conclusions: There is an urgent need for a framework to evaluate the confidence we can place in bodies of evidence addressing typical VPs. Our framework could help the guideline community to bring more rigour into the consideration of VPs when developing recommendations.

had statistically significant results in the complete case analysis. The table lists for each of these outcomes the effect of each assumption on its statistical significance (Table). The impact of the common assumptions varied significantly with no change for two of them (best case scenario and ‘none had the event’) and major change for the other two (‘all had the event’ and, particularly, worst case scenario). Under the plausible assumptions (based on the review and lost to follow up/follow up: RILTFU/FU), an increasing number of outcomes, up to five, lost statistical significance, with one changing direction. Conclusions: The proposed assumptions may help in assessing risk of bias associated with MPD. The increasingly stringent plausible assumptions may be more helpful with assessing risk of bias associated with MPD.

Oral session 3 Review methods: missing data Impact of missing outcome data for participants in trials included in five Cochrane Reviews: an imputation study Kahale L1 , Ramly E1 , Barba M2 , Sperati F2 , Terrenato I2 , Ballout R1 , Guyatt G3 , Kahale L4 , Schunemann H3 1 American University of Beirut, Lebanon; 2 National Cancer Institute Regina Elena, Italy; 3 McMaster University, Canada; 4 Cochrane Gynaecological and Orphan Cancer Group, Lebanon

Background: We recently proposed a number of assumptions for handling missing participant data (MPD), and assessing the associated risk of bias. Objectives: To assess the impact of applying the proposed assumptions about the outcomes of participants with missing data on the statistically significant effect estimates in five Cochrane Reviews. Methods: We conducted this study during the process of updating a series of five Cochrane Reviews on anti-coagulation in patients with cancer. We considered patients described as ‘withdrew consent’, ‘lost to follow-up’ or ‘outcome not assessable’ as having MPD. We focused on outcomes for which the primary analysis, a complete case analysis, found statistically significant results. We applied nine assumptions about the outcomes of participants with missing data on the effect estimates. Four of those assumptions are commonly used (e.g. best and worst case scenario). The remaining five and increasingly stringent assumptions are considered more plausible as they are based on incidences observed among followed-up participants. We assessed how many outcomes lost statistical significance or changed direction with each of the assumptions. Results: We included 12 outcomes that Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Interpreting trial results following use of different intention-to-treat approaches for preventing attrition bias: a meta-epidemiological study Døssing A1 , Tarp S1 , Furst D2 , Gluud C3 , Beyene J4 , Brandt Hansen B1 , Bliddal H1 , Christensen R1 1 The Parker Institute, Frederiksberg and Bispebjerg Hospital, Copenhagen, Denmark; 2 David Geffen School of Medicine, University of California, Los Angeles, USA; 3 Copenhagen Trial Unit, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark; 4 Department of Clinical Epidemiology and Biostatistics, McMaster University, Canada

Background: In randomized clinical trials (RCTs) exclusion of participants post-randomization from the final analysis population frequently happens. This leads to deviations from the intention-to-treat (ITT) principle, potentially leading to attrition bias. Modified ITT (mITT) is commonly used as an alternative to ITT; mITT is vaguely defined, but most often excludes patients who failed to receive treatment or patients without post-baseline assessment. Objective: To investigate potential bias of performing mITT compared to ITT. Methods: A systematic search in MEDLINE, EMBASE, and CENTRAL was performed. Inclusion criteria: blinded RCTs investigating approved biological agents for rheumatoid arthritis (RA); using the American College of Rheumatology 20 response (ACR20) as an outcome. Exclusion criteria: non-inferiority trials, as per protocol is the preferred analysis method. A meta-epidemiological random-effects analysis was used to calculate the odds ratio (OR) of analysis population (ITT and mITT). Statistical analyses Cochrane Database Syst Rev Suppl 1–150 (2014)

5 were based on the GLIMMIX procedure in SAS. Results: Among 5237 references, 72 RCTs were included and analysed (incl. 23 842 patients). Figure 1 illustrates the OR for each analysis population reflecting the pooled estimate of treatment effect; all in favour of the intervention. There was no statistically significant difference in treatment effect when applying mITT compared to ITT. Figure 2 illustrates the treatment effect corresponding to analysis population, when mITT is subdivided based on the number of modifications. The treatment effect increases gradually from ITT(0) to mITT(1) to mITT(2) and decreases again at mITT(3). mITT trials did not differ statistical significant from ITT trials, regardless of the number of modifications applied (4). Conclusion: Both ITT and mITT analyses can be applied; however, more than one post-randomization modification may threaten a conservative estimate. This work was supported by grants from the Oak and the Michaelsen foundation.

whether one ‘ought’ to conduct a meta-analysis; (ii) examination of aggregate treatment effects, meta-analysts managed a ‘part-whole’ tension and shifted between focusing on individual studies and the entire dataset, with some displaying a moral commitment to maintain the whole dataset integrity; (iii) translation of interpretation of data into action, participants deliberated about making a treatment decision, including moral concerns about financial resources invested, invoking ‘do no harm’, and a sympathy towards the patient population. These concerns sometimes led to a refusal to assign a point estimate, make a treatment decision, or withdraw from the meta-analytical process. Conclusion: This research highlights how data aggregation involves important moral considerations that are commonly concealed in ‘objectivist’ characterizations of meta-analysis. Understanding these processes should improve our understanding why different conclusions are drawn by different authors, and improve transparency for consumers of meta-analyses.

Prediction study risk of bias assessment tool (PROBAST) Wolff R1 , Whiting P1 , Mallett S2 , Riley R3 , Westwood M1 , Kleijnen J1 , Moons K4 1 Kleijnen Systematic Reviews Ltd, UK; 2 University of Oxford, UK; 3 University of Birmingham, UK; 4 University of Utrecht, The Netherlands

The moral dimensions of meta-analysis: a grounded theory of the meta-analytic process Macdonald ME, Carnevale FA, Chan LS, Steele RJ, Shrier I McGill University, Canada

Background: Discordant conclusions between meta-analyses occur even when meta-analysts are presented identical data, suggesting subjectivity is inherent in the ‘objective process’. Objective: To better understand the subjectivity in the meta-analytic interpretation process. Methods: We recruited 14 authors of meta-analyses and systematic reviews using theoretical sampling based on gender, training and experience. We asked participants about their general meta-analytical approach, and then provided simulated data sets (based on published meta-analyses) for treatments on four conditions. They were then asked to provide treatment effect estimates and confidence intervals after 3, 5, 10, and 20 studies for each condition. We used grounded theory methodology and semi-structured interviews to render explicit the analytical processes employed. Results: Our results suggest moral considerations shaped participants thinking at each stage of the interpretive process. (i) operationalization of the ‘should’, meta-analysts scrutinized the data pool. Many participants expressed strong moral reservations about certain data and questioned Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: Quality assessment of included studies is a crucial step in any systematic review. The review and synthesis of prediction modelling studies are a relatively new and evolving area and a tool facilitating quality assessment for prognostic and diagnostic prediction modelling studies is needed. Objectives: To develop PROBAST, a tool for assessing the risk of bias and applicability of prediction modelling studies. Methods: A Delphi process, involving 42 experts in the field of prediction research, was used until agreement on the content of the final tool. Existing initiatives in the field of prediction research such as the REMARK (Reporting Recommendations for Tumor Marker Prognostic Studies) guidelines and the TRIPOD prediction model reporting guidelines formed part of the evidence base for the tool development. The scope of PROBAST was determined with consideration of existing tools, such as QUIPS and QUADAS. Results: After six rounds of the Delphi procedure, a final tool was developed which utilizes a domain-based structure supported by signalling questions similar to QUADAS-2, which assesses risk of bias and applicability of diagnostic accuracy studies. PROBAST assesses the risk of bias and applicability of prediction modelling studies. It consists of five domains (participant selection, outcome, predictors, sample size and flow, and analysis) and 27 signalling questions grouped within these domains. Risk of bias addresses the extent to which reported estimates of the predictive performance/accuracy [e.g. discrimination, calibration and (re)classification estimates] of the prediction model are potentially biased. Applicability refers to the extent to which the reported prediction model and the population used to measure model performance matches the review question and intended use of the model. Conclusions: PROBAST can be used for the quality assessment of prediction modelling studies. The presentation will give an overview of the process, the current version of the tool (including the addressed domains and signalling questions) as well as an insight into underlying discussions.

Cochrane Database Syst Rev Suppl 1–150 (2014)

6

Oral session 4 Priority setting approaches Global burden of skin disease in 2010 and systematic reviews and protocols in Cochrane Database of Systematic Reviews Karimkhani C1 , Boyers L2 , Prescott L3 , Welch V4 , Dellavalle F5 , Nasser M6 , Zaveri A7 , Hay R8 , Vos T9 , Murray C9 , Margolis D10 , Hilton J11 , MacLehose H11 , Williams H3 , Dellavalle R12 1

Columbia University College of Physicians and Surgeons, USA; University School of Medicine, USA; 3 Centre of EvidenceBased Dermatology, University of Nottingham, UK; 4 Bruyere Research Institute, University of Ottawa, Canada; 5 Cochrane Skin Group, Nottingham University Hospitals NHS Trust, UK; 6 Peninsula Dental School, Plymouth ¨ Informatik, Universitat ¨ Leipzig, Germany; University, UK; 7 Institut fur 8 International Foundation for Dermatology, UK; 9 Institute for Health Metrics and Evaluation, University of Washington, USA; 10 Perelman School of Medicine, University of Pennsylvania, USA; 11 Cochrane Editorial Unit, UK; 12 Colorado School of Public Health, US Department of Veterans Affairs, USA 2 Georgetown

Background: Disease burden should help guide research prioritization. The Global Burden of Disease (GBD) Study 2010 compiled data from 1990 to 2010 on 291 diseases and injuries, 1160 disease and injury sequelae, and 67 risk factors in 187 countries. The Cochrane Database of Systematic Reviews (CDSR) is the leading resource for systematic reviews in healthcare, with peer-reviewed systematic reviews that are published by Cochrane Review Groups. Objectives: To determine whether systematic review and protocol topics in CDSR reflect disease burden, measured by disability-adjusted life years (DALYs) from the GBD 2010 project. This is one of a series of projects mapping GBD 2010 medical field disease burden to corresponding systematic reviews in CDSR. Methods: Two investigators independently assessed 15 skin conditions in CDSR for systematic review and protocol representation according to subject content. The 15 skin diseases were matched to their respective DALYs from GBD 2010. An official publication report by the Cochrane Skin Group was also obtained to ensure that no titles were missed. Results: All 15 skin conditions were represented by at least one systematic review in CDSR. Sixty nine percent of systematic reviews and 67% of protocols by the Skin Group covered the 15 skin conditions. Comparing the number of reviews/protocols and disability, dermatitis, melanoma, non-melanoma skin cancer, viral skin diseases, and fungal skin diseases were well-matched. Decubitus ulcer, psoriasis, and leprosy demonstrated review/protocol over-representation when matched with corresponding DALYs. In comparison, acne vulgaris, bacterial skin diseases, urticaria, pruritus, scabies, cellulitis, and alopecia areata were under-represented in CDSR when matched with corresponding DALYs. Conclusions: Degree of representation in CDSR is partly correlated with DALY metrics. The number of published reviews/protocols was well-matched with disability metrics for 5 of 15 studied skin diseases, while 3 skin diseases were over-represented and 7 under-represented. Our results provide good quality and transparent data to inform future prioritization decisions.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Systematic reviews addressing identified policy priorities in Eastern Mediterranean countries: a situational analysis El-Jardali F1 , Akl E1 , Bou Karroum L1 , Kdouh O1 , Akik C2 , Fadlallah R1 , Hammoud R1 1 American University of Beirut, Lebanon; 2 London School of Hygiene and Tropical Medicine, UK

Background: The production and use of systematic reviews (SRs) are limited in the Eastern Mediterranean Region (EMR). The extent to which published SRs address policy priorities in the region is still unknown. Objectives: This situational analysis is the first of its kind to assess the extent to which published SRs address policy priorities identified by policymakers and stakeholders in the EMR. It also provides an overview of the state of SR production in the region and identifies knowledge gaps. Methods: We conducted a systematic search of Health Systems Evidence database to identify published SRs on policy-relevant priorities pertaining to the following themes: human resources for health; health financing; role of the non-state sector and access to medicine. Priorities were identified from two priority-setting exercises conducted in the region. We describe the distribution of these SRs across themes, sub-themes, authors’ affiliations, and countries where the included primary studies were conducted. Results: Out of 1045 SRs identified in Health Systems Evidence on the selected themes, 200 SRs (19%) addressed the priorities from the EMR. The theme with the largest number of SRs was human resources for health (n = 115) followed by health financing (n = 33), access to medicine (n = 27), and role of the non-state sector (n = 25). Authors based in the region produced only three SRs addressing regional priorities (1.5%). Furthermore, no SRs focused on the EMR. Seventeen SRs (8.5%) included primary studies conducted in the region. Conclusions: There are still gaps in the production and alignment of SR production with policymakers’ and stakeholders’ priorities in the EMR. The findings highlight the need for individual and institutional capacity development in SR production and communication. They can also inform the agendas of researchers, research institutions and funding agencies of priority areas that require SRs, thus pushing towards evidence-informed policy making.

Cochrane Database Syst Rev Suppl 1–150 (2014)

7 Background: Preterm birth is the most important determinant of adverse infant outcomes including survival, quality of life, psychosocial impact on the family, and healthcare costs. Research agendas in this area have been determined primarily by researchers. Objectives: To identify and prioritize future research areas in preterm birth that are most important to service users and practitioners. Methods: A priority setting partnership was established with: organisations representing families in the British Isles with experience of preterm birth, obstetricians, neonatologists, midwives, and neonatal nurses. Research uncertainties were gathered from surveys of service users and healthcare practitioners, and analyses of systematic reviews and clinical guidance. Priorities were set through on-line voting and paper surveys, and further distilled in a facilitated workshop, as advocated by the James Lind Alliance. Results: Eliciting research suggestions from all types of stakeholders required extensive networking. Five hundred and ninety three uncertainties were submitted by 386 respondents (58% service users, 30% healthcare professionals and 12% from those in both roles); 52 were identified from literature reviews. After merging similar questions, 104 were distributed for voting. The 30 most popular items were discussed at a workshop with service users and professionals, and the top 15 questions put in priority order. These include prevention and prediction of preterm birth, neonatal infection, lung damage, necrotising enterocolitis, pre-eclampsia, preterm premature rupture of the membranes, optimal neonatal feeding strategy, pain perception and management, a care package at neonatal discharge, emotional and practical support, attachment and bonding, and the best time for cord clamping. Conclusions: Fifteen top priorities provide guidance for researchers and funding bodies to ensure that future research addresses questions that are important to service users and clinicians. Challenges for the priority setting partnership included maximizing participation amongst people most affected by preterm birth; the breadth of the topic; and securing input from an appropriate range of clinicians.

Update of the Cochrane Review of interventions for preventing obesity in children: priority setting Waters E, Steele E, Nwagbara B, Pettman T Cochrane Public Health Group, Australia

Setting research priorities with multiple professions and service users Uhm S1 , Alderdice F2 , Brady I3 , Chambers B4 , Chivers Z5 , Crowe S6 , David A7 , Deshpande S8 , Gale C9 , Gyte G10 , Oliver S11 , James P7 , Duley L12 , McNeill J2 , Shennan A13 , Turner M14 , Oliver S11 1 Institute of Education, London and Kookmin University, UK; 2 TinyLife, Belfast, UK; 3 Irish Premature Babies, Ireland; 4 Unknown, Unknown; 5 Bliss, London, UK; 6 James Lind Alliance, UK; 7 UCL Institute for Women’s Health, London, UK; 8 British and Irish Paediatric Pathology Association, UK; 9 Imperial College, London, UK; 10 National Childbirth Trust, UK; 11 EPPI-Centre, Institute of Education, UK; 12 Nottingham Clinical Trials Unit, University of Nottingham, UK; 13 Kings College London, London, UK; 14 Liverpool Women’s NHS Foundation Trust, Liverpool, UK Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: The Cochrane Review ‘Interventions for preventing obesity in children’ (Waters 2011) synthesises evidence of the effectiveness of childhood obesity prevention activities. A significant amount of relevant primary research has been conducted since the latest review update, prompting planning for the next update. To ensure the updated review meets the needs of key policy stakeholders, a structured program of stakeholder engagement will guide development of the systematic review protocol. This paper describes the first stage of this program. Objectives: The aim of this stage was to determine research gaps in the area of effectiveness of interventions for preventing obesity in children, as identified by policy-makers in developed and developing contexts. Methods: A structured website search was conducted to find appropriate policy documents. Research priorities were extracted from each document and categorised as socio-ecological, settings-based, contextual, equity-based, implementation factors, methodological factors or issues concerning outcome measures. A web-based survey of international obesity experts was then undertaken to validate Cochrane Database Syst Rev Suppl 1–150 (2014)

8 the list of research priorities. Results: The search identified nine documents from which 30 research gaps were identified. The gaps were evenly distributed amongst the categories of the research priority framework. Results of the web-based survey will be available prior to the Cochrane Colloquium. Conclusions: This presentation will present the background, aims, methods and results of this exercise, and discuss how results will guide the development of the update of the Cochrane Review ‘Interventions for preventing obesity in children’.

Oral session 5 Tools for assessing bias Introducing LATITUDES: a library of assessment tools and instruments used to assess data validity in evidence syntheses Whiting P1 , Wolff R1 , Savovic J2 , Simera I3 , Mallett S3 1 Kleijnen Systematic Reviews, UK; 2 University of Bristol, UK; 3 University of Oxford, UK

Background: Evidence syntheses provide the most reliable form of evidence for decision makers. A formal assessment of the validity of included studies is an essential component of any systematic review. This requires a formal risk of bias or quality assessment tool. Many tools are available, usually designed specifically for different study designs and/or topic areas. Objectives: To introduce the LATITUDES Network (www.latitudes-network.org), a library of assessment tools and instruments used to assess data validity in evidence syntheses. Methods: We are establishing LATITUDES to increase the robustness of evidence syntheses by improving the process of validity (risk of bias or quality) assessment. Our goals are to increase the use of key tools, help people use tools more effectively, improve incorporation of the validity assessment into the review and disseminate best practice in validity assessment. Results: LATITUDES (www.latitudes-network.org) aims to improve the overall quality of research syntheses by improving the process of validity assessment. We will provide a complementary initiative inspired by the EQUATOR Network (www.equator-network.org), an influential international resource to improve the reporting and transparency of health research studies. The LATITUDES network will index validity assessment tools developed for healthcare studies in an online library. Key risk of bias tools will be highlighted to help reviewers identify the optimum tool for their review. We will provide toolkits to aid the use of validity assessment tools. Toolkits will aim to enable better and effective use of existing tools and to improve the incorporation of the validity assessment into the review. Project outputs and communication will focus around a freely available online platform featuring the library, key tools, toolkits, online training materials, and links to courses and events. Conclusions: LATITUDES aims to be the primary resource, networking reviewers to key information when planning and conducting validity assessment for their review.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Methodological considerations for assessing risk of bias in non-randomized studies ´ M2 , Savovi´c J1 , Turner L3 , Higgins JP1 for the Sterne JA1 , Reeves B1 , Hernan Development group for the Cochrane risk of bias tool for non-randomized studies4 1 University of Bristol, UK; 2 Harvard School of Public Health, USA; 3 Ottawa Hospital Research Institute, Canada; 4 Various, International

Background: Cochrane intervention reviews rely primarily on evidence from randomized trials (RCTs), but this is not always available. Some clinical questions such as those about rare, long term or unexpected harms, are often addressed incompletely by RCTs. There is no widely accepted tool for assessing risk of bias in non-randomized studies (NRS) of interventions. Objectives: Describe key methodological concepts that underpin development of the new Cochrane tool for assessing risk of bias in NRS. Methods: We developed the tool through a wide consultation with methodologists, authors, Cochrane Review Groups and other stakeholders. Methodological considerations were addressed primarily through discussions within domain-focused working groups and a face-to-face meeting of all contributors. Results: Evaluations of risk of bias in the results of NRS are facilitated by considering each study as an attempt to mimic a hypothetical RCT comparing the same interventions as the NRS (the ‘target trial’) and by specifying the intervention effect of interest (either assignment to the interventions at baseline, or receiving the interventions as specified in the protocol). Bias domains can be divided into ‘pre-intervention’ domains (confounding and selection bias), an ‘at intervention’ domain (bias in measuring interventions), and ‘post-intervention’ domains (bias due to departures from intended interventions, missing data, measurement of outcomes and selection of the reported result). Assessment of pre-intervention bias domains is mainly distinct from assessments in RCTs (these focus on randomization procedures), but there is substantial overlap in assessment of post-intervention domains in RCTs and NRS. Studies that split follow-up according to intervention received require special consideration because confounding can occur at baseline and at the time of intervention change. Conclusions: Evaluating risk of bias in a systematic review of NRS requires both methodological and content expertise. A structured approach to assessing risk of bias in different domains should provide a systematic way to organize and present the available evidence relating to risk of bias.

A new tool to assess risk of bias in non-randomized studies Savovi´c J1 , Higgins JP1 , Reeves B1 , Turner L2 , Sterne JA1 for the Development group for the Cochrane risk of bias tool for non-randomized studies3 1 University of Bristol, UK; 2 Ottawa Hospital Research Institute, Canada; 3 Various, International

Background: Cochrane intervention reviews rely primarily on evidence from randomized trials (RCTs), but this is not always available. Some clinical questions, such as those about rare, long term or unexpected harms, are often addressed incompletely by RCTs. There is currently no widely accepted tool for assessing risk of bias in non-randomized studies (NRS) of interventions. Objectives: To present the new Cochrane tool for assessing the risk of bias in NRS. Methods: Cochrane Database Syst Rev Suppl 1–150 (2014)

9 We developed the tool through consultations with methodologists, authors, Cochrane Review Groups and other stakeholders. We used a combination of interactive and iterative methods including: a scope meeting with key stakeholders; survey of review groups’ needs; recruitment of experts; small domain-focused working groups; and a meeting with all contributors. The draft tool was revised following piloting. Results: Protocol considerations include specification of a target trial, the effect of interest, and likely confounding domains and co-interventions. The tool consists of seven bias domains: confounding; selection of participants into the study; measurement of interventions; departures from intended interventions; missing data; measurement of outcomes; and selection of the reported result. Each domain includes 3–5 signalling questions that focus on key aspects of study conduct. Answers to these questions inform a domain-level judgment on risk of bias that has five levels: low risk, when a study is comparable to a well-performed RCT; moderate risk, consistent with a sound NRS but not comparable to an RCT; Serious risk, when a study has some important problems; Critical risk, when a study is too problematic to provide useful evidence; and No information. Criteria for reaching a judgment about the overall risk of bias are specified. Conclusions: This is the first domain-based risk of bias tool for NRS of interventions, developed with wide international participation of methodological experts and stakeholders. It provides a structured approach to organizing and presenting available evidence relating to risk of bias and will be disseminated across the Collaboration after the Hyderabad Colloquium.

ROBIS: a new tool to assess the risk of bias in a systematic review Whiting P1 , Savovic J2 , Higgins J2 , Shea B3 , Reeves B2 , Caldwell D2 , Lasserson T4 , Davies P2 , Kleijnen J5 , Tovey D4 , Wells G6 , Churchill R2 1 Kleijnen Systematic Reviews, UK; 2 University of Bristol, UK; 3 Univerisity of Ottawa, Canada; 4 Cochrane Editorial Unit, UK; 5 University of Maastricht, The Netherlands; 6 University of Ottawa, UK

Background: Systematic reviews are now generally consulted as the primary source of evidence for the effects of an intervention. As with any study, systematic flaws or limitations in the design or conduct of the review have the potential to bias results. Several tools are available for the critical appraisal and quality assessment of a systematic review. However, none aims specifically to assess the risk of bias introduced by the review conduct or its interpretation of findings. Objectives: To describe the new ‘ROBIS’ tool, developed using rigorous methodology, to assess the risk of bias in a systematic review. Methods: We used a five phased approach to develop ‘ROBIS’: definition of the scope of the tool, development of the evidence base concerning biases in systematic reviews, face to face meeting, the Delphi prediction method to develop a first version of the tool, and refinement through piloting. We established a broad team including methodological experts in systematic reviews and review authors. Results: We agreed that ‘ROBIS’ should consider the relevance of a review to the research question at hand (where appropriate) and assess the risk of bias in the review. Discussions at the face-to-face meeting informed decisions and the subsequent Delphi procedure led to a tool consisting of five key domains: (i) study eligibility criteria, (ii) identification and selection of studies, (iii) data collection and study appraisal, (iv) synthesis and findings, and (v) interpretation. A series of questions within each domain elicits information about possible limitations of the systematic Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

review, leading to a judgement about the risk of bias within that domain (low, high or unclear risk of bias). Piloting of ‘ROBIS’ is ongoing at the time of writing. Conclusions: The development of ‘ROBIS’ was evidence based and included detailed piloting and refinement. We believe that ‘ROBIS’ will allow objective rating of the risk of bias and relevance of systematic reviews.

Oral session 6 Citation screening and study selection (1) Rayyan: a systematic reviews web app for exploring and filtering searches for eligible studies for Cochrane Reviews Elmagarmid A1 , Fedorowicz Z2 , Hammady H1 , Ilyas I3 , Khabsa M4 , Ouzzani M1 1 Qatar Computing Research Institute, Qatar; 2 Bahrain Cochrane Branch, Bahrain; 3 University of Waterloo, Canada; 4 Pennsylvania State University, USA

Background:Preliminary filtering of searches is one of the most time consuming aspects of systematic reviewing. Cochrane Review authors use a variety of approaches (manual and electronic) but no single method satisfactorily fulfills the principal requirements of speed with accuracy. Objectives: Pilot testing of Rayyan (rayyan.qcri.org) focused on usability and assessment of how accurately the tool performed against manual methods, and evaluation of the added benefit of the prediction feature. Methods: Searches from two published Cochrane Reviews (1030 and 273 records, respectively) were used to test the app (December 2013 to March 2014). Included studies had been previously selected manually for the reviews; original searches were uploaded into Rayyan. Results: One recently updated Cochrane Review (273 records) was used as a taster, allowing a quick overview of the look and feel of Rayyan and for early feedback on usability to be addressed by the developers. The second Cochrane Review (1030 records) required several iterations to identify the 11 trials that had previously been selected manually. The suggestions and hints, based on the prediction rules, appeared as the testing progressed beyond five included studies. The selection process was responsive and effective; the options undecided/included/excluded/suggested were clearly displayed. Search functions included limiters relevance/title/date. Innovative features include: word clouds as graphical indicators of key terms; translation option linked to Google Translate to enable a quick translation or forwarding to a translator; similarity-based exploration of studies; labelling of studies, including reasons for exclusion. Key functionality includes the unambiguous way in which studies could be viewed in context together with the completed selections, and how undecided studies could be fed back into the system and were then highlighted as hint. Conclusion: Rayyan (beta-testing phase) is responsive, and largely intuitive to use, with a significant potential to lighten the load of review authors by speeding up this tedious part of the process.

Cochrane Database Syst Rev Suppl 1–150 (2014)

10 Using text mining technologies can reduce screening workload in systematic reviews in practice as well as in theory Thomas J1 , Ananiadou S2 , Brunton J1 , McNaught J2 , Miwa M2 , O’Mara-Eves A1 1 EPPI-Centre, Institute of Education, London, UK; 2 University of Manchester, UK

Background: The task of identifying relevant studies for systematic reviews in an unbiased way is increasingly time consuming. Text mining may be able to assist in the screening process by prioritizing the list of items for manual screening so that the studies at the top of the list are those that are most likely to be relevant (‘screening prioritization’). This can assist in the review by enabling reviewers to move to full-text screening (and later stages) earlier than they might otherwise have done; they may also not need to look at every retrieved reference. Objectives: To evaluate the performance of text mining methods to reduce screening workload by assessing their performance in (i) completed reviews and (ii) on-going reviews. Also, to evaluate implementation issues and user experiences through a process of evaluation. Methods: Simulation studies were conducted on eight completed systematic reviews in public health and clinical areas. Performance was assessed using accuracy, precision, recall/sensitivity, F-measures, and Area Under a (ROC) Curve and burden. Additionally, screening prioritization was used in two large systematic reviews during the actual process of screening. A process evaluation assessed the implementation of the methods and user experiences of the system. Retrospective simulation studies of these datasets were also conducted. Results: Text mining methods successfully ‘pull’ the relevant studies towards the beginning of the screening process; and this can apply in complex areas such as public health (though more complex tools are required) as well as the more clinical topics. The process evaluation shows that the performance of these tools in ‘live’ reviews may not reach the levels achieved in simulation studies for operational reasons. Conclusions: Simulation studies show that text mining technologies are able to ‘prioritize’ relevant items for early screening in reviews. The use of these technologies in ‘live’ reviews requires further work, since the practicalities of undertaking screening in large reviews with potentially many reviewers may prevent optimal performance.

Automatic information retrieval: citation tracking, deduplication and full-text fetching Tsafnat G, Choong M Centre for Health Informatics, University of New South Wales, Australia

Background: In evidence-based medicine, systematic reviews are important devices for medical practitioners in making clinical decisions. A systematic review is a summary of evidence on a clearly formulated question based mostly on randomized controlled trials. Development of a systematic review is a slow and complex process and can take between 1 and 2 years to complete. Updates are costly in terms of time and money, and can take as much time and effort as the original systematic review. Objectives: To support new evidence from the conducting and updating of systematic reviews using a software tool that automatically extracts citation information, retrieves citation data and full text of scientific papers. Methods: We developed gold standards by manually extracting citations from systematic reviews and Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

using search results provided by Cochrane authors. We measured our tool’s capacity to extract citations from PDFs, retrieve full text or abstracts, duplicate citations and extract citations from the full text. Results: We have developed ESuRFr, a tool that automatically extracts citation information, retrieves citation data and retrieves the full text of scientific papers. ESuRFr extracted 1020 (96.5%) citations from

20 review papers. ESuRFr achieved F1-score of 0.912 for citation retrieved and 0.797 for abstract/full text fetched of the available citations. ESuRFr identified 100% (n = 17) duplicates but identified two citations as duplicates erroneously. ESuRFr automatically retrieved 86% (n = 12) of 17 papers included in a systematic review update by tracking two iterations of citations from the original review. Conclusions: ESuRFr automates new evidence, duplication of records from different databases and automatic citation tracking to identify relevant literature for systematic reviews and systematic review updates.

LESS (Logigramme for Efficient and Systematic Search): the logical tool for the development and management of standardised search strategy for electronic databases Salunke S1 , Brandys B2 , Jain S3 , Upadhyay P3 , Tuleu C1 1 University College London, UK; 2 National Institutes of Health (NIH) Library, USA; 3 Synfo Data Services, India

Background: Developing search strategies to retrieve research evidence from databases is challenging and requires thorough knowledge of database features. The development of a search strategy is likely to require several iterations and several approaches (e.g. free text, index terms, etc.). This exercise can be very daunting and time consuming if not performed in a structured or systematic manner. This study describes a novel technique called LESS (Logigramme for Efficient and Systematic Search) for the development of a standardized search strategy. Objectives: The aim of the study was to build, test and validate LESS for the development of a systematic and standardized search strategy for collecting relevant information about safety and toxicity information. Methods: An in-depth search was conducted as per LESS in 10 different databases for one model chemical, propylene glycol (PG). The relevant records found for PG’s safety and toxicity in each database acted as the gold standard for that particular database

n w

a r d

ith

W

Cochrane Database Syst Rev Suppl 1–150 (2014)

11 against which the sensitivity and precision of different combinations of the search approach was tested. The search strategy developed was further validated by performing search with same strategy but another excipient. Results: A successful LESS was built up, considering the bibliographic database’s strength and avoiding its shortcoming. The LESS approach uses the indexing and free text terms for developing a search strategy in a systematic way. It also implements a logical flow chart that can help generate a search string for re-use by alternating only the main concept. This method documents each step of the search process and can be especially useful for comprehensive searching of publications on large sets of references on similar topics. Conclusions: The LESS adopted for literature search provides an in-depth retrieval of the safety and toxicity information and a logical way of reducing the number of irrelevant records to be assessed for relevance. It provides a new transparent and reproducible technique for researchers, information specialists and searchers new to conducting literature searches.

n w

a r d

ith

W

outperform other systematic reviews (SRs); and if they include all the relevant evidence. Methods: In the context of clinical meetings of one public and one private hospital in Chile, we selected one condition per month to be discussed. A team generated a list of clinical questions or hot topics, and the staff selected 2–5 questions that were worth reviewing during the meetings. We prepared matrices of evidence in Epistemonikos (a table comparing all of the reviews and all of the primary studies for a question). Epistemonikos searches for SRs across 20 databases. We measured the number of questions answered by any SR; number of questions answered by a CR; number of times a CR was the newer/more complete; and if the CR included all the relevant trials identified in the matrix. Results: Thirty-two clinical questions were reviewed during an 8-month period. The average of SR per question was 4.4 (0–19). For 25/32 (78%) we identified at least one SR, but only 12/32 (38%) were answered by a CR (Figure). From 12 questions for which a CR was available, in 9/12 (75%) Cochrane was the more complete, and in 8/12 (66%) it was the newest. We checked the CR against all the relevant primary studies identified in the matrix, and went to the full text to evaluate if the authors had excluded or missed

the studies that other reviews had included. Only five CRs had found all the relevant evidence. Conclusions: A substantial proportion of questions can be answered with SRs. However, non-Cochrane reviews (NCRs) had much better coverage than CRs. When there were both CRs and NCRs, CRs were more complete and newer than NCRs, but there is still room for improvement. Only one in every six questions was answered by a Cochrane Review including all of the relevant evidence. Our results emphasize the need for producing reviews covering the more relevant topics, and the challenges for Cochrane to be the more reliable source of evidence.

Oral session 7 Impact of Cochrane Reviews Do we have Cochrane Reviews for the more relevant questions? Are they the best choice? Analysis of clinical questions in internal medicine ˜ J, Turrillas P, Espinosa P, Monsalve X, Aizman A Rada G, Pena ´ Internal Medicine Department, Pontificia Universidad Catolica de Chile, Chile

Background: Relying on Cochrane Reviews (CR) is recognized as an efficient way of making evidence-based health decisions. Objectives: To explore how often CRs are available for clinical questions; if they Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Evaluating the impact of Cochrane Reviews Trivedi D1 , Bunn F1 , Alderson P2 , Iliffe S3 1 Cochrane Injuries Group, University of Hertfordshire, UK; 2 Centre for Clinical Practice, National Institute for Health and Care Excellence, UK; 3 Primary Care for Older People, University College London, UK

Background: The need to develop an evidence-based approach to decision making has been recognised by clinicians, healthcare managers and policy makers. The National Institute for Health Research (NIHR) funds Cochrane Review Groups (CRGs) in the UK, but there is less evidence about the impact of Cochrane Reviews on policy, practice and research. Objectives: This study, funded by the NIHR, aimed to (i) evaluate the impact of Cochrane Reviews from 20 NIHR-funded CRGs from 2007 to 2011 on clinical practice, use in Cochrane Database Syst Rev Suppl 1–150 (2014)

12 clinical guidance and influence on future research; and (ii) to identify barriers and facilitators to the use of reviews in developing guidelines. Methods: We used bibliometrics and documentary review to assess CRG impact, with an in-depth evaluation of a sample of 60 Cochrane Reviews. The analysis was based on a framework of knowledge production, research targeting, informing policy development, and impact on practice/services. We employed mixed methods, including questionnaires to CRGs and interviews with guideline developers. Results: Overall 483 systematic reviews were cited in 247 sets of guidance. We identified 40 examples of reviews influencing primary research, with many contributing to new knowledge. Examples of health benefits included improved patient safety or the identification of new effective treatments. Potential economic benefits included cost savings, improved quality of care, decreased use of unwarranted practices and improvements in patient/carer experiences. Thematic analysis from interviews with guideline developers identified the need for up-to-date evidence, better fit with the guideline scope, positive collaboration and timely communication between guideline developers and CRGs. Conclusions: Whilst there were clear impacts on research targeting and healthcare policy, appropriate methods need to be developed for measuring impact on clinical practice and service organisation and delivery, for which there was less evidence. Guideline developers, CRGs and review authors should aim to improve collaboration and communication to ensure that guidance development is useful for decision makers.

Cochrane Reviews: the linchpins of knowledge translation and impact Middleton P1 , Green S2 , Crowther C1 1 Cochrane Pregnancy and Childbirth Group, Australia; Cochrane Centre, Australia

2 Australasian

Background: As systematic reviews represent the synthesis of a body of evidence, it stands to reason that they should often be the basic unit of knowledge translation. Objectives: To measure the use of Cochrane Reviews (CRs) in several different knowledge transfer systems including clinical practice guidelines (CPGs), cites within The Cochrane Library and in SCOPUS; ‘actionable’ Cochrane Clinical Answers (CCA and altmetric) and to assess if there were relationships between systems and other factors. Methods: We used RevMan and Excel to manage our database of 790 perinatal or part-perinatal CRs (pCRs) from 25 Cochrane Review Groups. We manually searched several hundred open access perinatal evidence-based guidelines for cites of these pCRs. We used Cochrane Library citations and altmetric tabs within pCRs; accessed a spreadsheet of all CCAs and searched SCOPUS for cites of CRs. Results: Over a third of pCRs (284/790; 36%) were cited in at least one CPG (range 1–8), with differences between review focus (less than 10% of neonatal CRs cited in CPGs). CPG citations are rarely captured (in only 3/284 CR cite tabs). pCRs selected for CCAs were just as likely to be cited in a CPG as not; and SCOPUS citation rates showed no clear relationships with any other factor. Four pCRs had exceptionally high altmetric scores (exercise for depression, home births, midwife-led models and cord clamping in term babies) and a further 13 were in the top 5%, but 136 pCRs had a score of 0 (mostly older CRs). Conclusions: Cochrane Reviews make a substantial, but underestimated contribution to knowledge translation and impact. As there are presently no systems to capture the substantial use of CRs in CPGs, linkage systems are necessary not only between trials Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

and CRs but also between CRs and CPGs. Usage patterns are very different between review focus (e.g. neonatal/antenatal) and between bibliometric and social networking systems, indicating wide and diverse use of Cochrane Reviews and the concomitant need for sophisticated analysis and interpretation.

‘Buying’ into Cochrane: how do fields measure success? Lockwood C, Lisy K, Stern C Cochrane Nursing Care Field, Australia

Background: The core functions of a Cochrane Field focus on supporting the conduct, dissemination and utilization of relevant systematic reviews to professions aligned with the Field. These aims rely on the Field having strong relationships within Cochrane, particularly with Cochrane Review Groups and other Fields, as well as external stakeholders. How Fields interpret and apply these core functions may vary, but each requires an engaged membership and active participation by leadership groups and teams who coordinate activity. The Nursing Care Field is structured to provide an avenue for professional engagement that brings together members of the wider profession in a partnership-based model. Objectives: To provide an account of the Nursing Care Field’s experiences with metrics in stakeholder engagement across Cochrane as well as external constituents. Methods and results: The Cochrane Nursing Care Field was established in 2009, has a very large group of stakeholders and a rapidly growing membership. The Field utilizes a number of strategies to engage with constituents within the Cochrane Collaboration and specific approaches to wider engagement with relevant professions. These have met with varying success, but overall, have had a positive impact on Field membership growth and Field outputs. A summary of lessons learned to date will be presented that relate to building and maintaining relationships, activities and outputs that meet the strategic goals set by Cochrane for Fields. The challenges involved in measuring this type of output will also be discussed. Conclusions: The role of Fields can be complex and challenging. Many people, both within and outside of the Collaboration are unaware of the work Fields undertake, how it is accomplished and how we can demonstrate success. The experiences of the Nursing Care Field may help shed light on this work and provide some guidance for other Cochrane entities.

Oral session 8 Knowledge translation for policy makers Policy ‘buddies’: policymaker and researcher engagement for evidence-informed policy Young T1 , Ongolo-Zogo P2 , Naude C3 , Zani B4 , Kredo T4 , Wiysonge C3 , Dudley L5 , Garner P6 1 Stellenbosch University and South African Cochrane Centre, South Africa; 2 Centre for the Development of Best Practices in Health, Cameroon; 3 Centre for Evidence-based Health Care, Stellenbosch University, South Africa; 4 South African Cochrane Centre, South Africa; 5 Stellenbosch University, South Africa; 6 Liverpool School of Tropical Medicine, UK

Background: Robust evidence shapes effective and efficient health services. Policy- makers need to understand what research can offer, Cochrane Database Syst Rev Suppl 1–150 (2014)

13 and researchers need to understand policy-makers’ needs. ‘Buddies’ aims to help policy-makers to demand and use systematic reviews, and researchers to respond to their requirements. Methods: Conducted in South Africa and Cameroon at provincial and regional government using five phases: (i) a situational analysis to understand policy-makers’ requirements and what influences their use of evidence; (ii) workshops to help policymakers identify research questions and draw on systematic reviews to inform decisions; (iii) a ‘buddy’ model to create links between policy-makers and researchers around specific priority policy questions identified during the earlier phases of the project; (iv) provision of resources to guide policy-makers and links to relevant researchers; and (v) evaluation of the model to identify lessons learnt. Results: Good relationships between policy-makers and researchers are essential to aid evidence informed decision making. Researchers need to understand how policy-makers operate, what their priorities are and where evidence fits into decision-making to tailor strategies to improve the use of evidence in policy-making. Preand post-workshop assessments demonstrated an improvement in the capacity of policy-makers to formulate questions, find and understand evidence from systematic reviews and interpret it using tools such as GRADE (Grading of Recommendations Assessment, Development and Evaluation). Results of the ‘buddy’ model and evaluation, currently in progress, will be reported at the colloquium. Discussion and conclusions: Building capacity and effective engagement between researchers and health service decision makers can increase demand for evidence and help researchers prioritize topics for reviews. Dialogue and personal contact, central to the ‘Buddies project’, helps facilitate this partnership. Such partnership provides a knowledge sharing platform that could strengthen the health system through enhancing evidence informed decision making.

Seizing opportunities: engaging policy makers in evidence informed discussions, planning and policy making Aryal K1 , Rana S1 , Mehata S2 , Dahal G3 , Welch V3 , Lohani GR4 1 Nepal Health Research Council, Nepal; 2 Ministry of Health and Population, Nepal Health Sector Support Programme, Nepal; 3 Centre for Global Health, Institute of Population Health, University of Ottawa, Canada; 4 Ministry of Health and Population, Nepal

Background: Effective engagement, trust and shared interest with policy makers are the core requirements of evidence informed policy making. However, poor engagement with policy makers and a lack of shared priorities serve as significant barriers to bringing evidence from systematic reviews into health policy development. Objectives: This presentation aims to share experiences and analyze strategies to engage policy makers for the uptake of evidence in health policy. These activities were conducted in preparation for and following on from the momentum created by an exploratory meeting in February 2014 to develop a Nepal branch of the South Asian Cochrane Centre. Methods: The presentation will describe how the general framework of four essential knowledge exchange strategies was used to facilitate ‘user pull’ and engage policy makers in evidence informed policy making in Nepal: (i) building trust and visibility; (ii) informing and training policy makers; (iii) ensuring one-on-one interactions; (iv) formalizing partnerships. Results: A team to support current policy making efforts with evidence based methods was established, with Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

developed links to obtain Cochrane-specific training and share robust priority setting methods and policy brief formats for policy makers to incorporate evidence and context. Conclusions: These results provide an adaptable method to ensure stakeholder support for establishing Cochrane activities in a low or middle-income country, through proactive engagement throughout the process.

Linking diagnostic test accuracy reviews to global priorities: case study of TB Steingart K1 , Richardson M2 , Stephani AM2 , Garner P2 1 Cochrane Infectious Diseases Group, UK; 2 Liverpool School of Tropical Medicine, UK

Background: The Cochrane Collaboration is finding effective mechanisms for aligning intervention reviews and global guidelines processes, but experience with diagnostic test accuracies (DTA) reviews appears limited. Here we report on how an effective linkage between a Cochrane Review Group and the World Health Organization’s (WHO) global tuberculosis (TB) programme. Approach: In early 2011, WHO ® issued policy recommendations on Xpert MTB/RIF (Xpert), a new test that can simultaneously detect TB and TB drug resistance in around 2 hr. In 2013, given additional data on Xpert, WHO convened a process to update recommendations, including commissioning of DTA reviews. The Cochrane Editor is a member of ‘STOP TB Partnership’s New Diagnostic Working Group’ and participates in consideration of new TB tests. The Cochrane Editor worked with (i) researchers in the field and Cochrane Infectious Diseases Group (CIDG) to prioritize topics; (ii) WHO to formulate PICO questions; (iii) biostatisticians to perform data analysis; and (iv) the Cochrane Managing Editor to ensure rapid production of the Xpert review1. Based on this approach, two new DTA reviews are underway. Results: The updated Cochrane Review on Xpert included 27 studies (integrating nine new studies) and was presented at the WHO Expert Group meeting in May 2013. The review found: in adults thought to have TB, with or without HIV infection, Xpert is accurate for detection of TB and rifampicin resistance and can allow for rapid initiation of treatment for multi-drug-resistant TB. Updated WHO recommendations on Xpert were issued October 2013; the updated Xpert review was published open access January 2014. For the second review (MTBDRsl), we are completing data analysis; for the third (urine lipoarabinomannan), we are writing the protocol. Conclusion: Despite complexity of DTA reviews, it is possible with teamwork and leadership to ensure reviews are prioritized and completed within WHO timelines.

Social values underlying priority setting Gough D1 , Kenny C2 1 EPPI-Centre, Institute of Education, London, UK; 2 Parliamentary Office for

Science and Technology, UK

Background: Many health and social care systems create guidance to inform allocation of resources and thus services. Such guidance depends on decisions about opportunity costs and thus is based on social values as to the relative importance of different priorities. Research in this area has shown international variation in these values underlying differences in priority setting systems (Littlejohns 2012). The National Institute for Health and Care Excellence (NICE) in England has a policy of Social Value Judgements (2008) that informs the use Cochrane Database Syst Rev Suppl 1–150 (2014)

14 of research evidence in the production of health systems guidance for the National Health Service. NICE is currently revising its social values policy and this paper reports on a review of the literature on social values commissioned to inform that revision. Aim: To identify social values and related debates relevant to priority setting by health and social care systems. Method: As the purpose was to identify issues, a configuring rather than aggregative review methodology was adopted (Gough 2012). The search strategy included a priori inclusion criteria and exhaustive and iterative search components. Nearly 2000 papers were identified, of which over 800 were considered highly relevant. The issues were configured using an adapted version of Clarke and Weale’s (2012) typology for social values consisting of: Utility and efficiency, Justice and equity, Autonomy, Solidarity, Participation, Sustainability, Transparency and accountability, and Rigour in methods of guidance development. Results and conclusion: The review identified 84 main issues under the eight social value categories and proposed 20 main issues that NICE might wish to consider under 10 main themes. These 10 themes and their relevance to the development of priority setting in health and social care systems will be presented and discussed.

Oral session 9 Review methods: outcomes Completeness of outcomes description reported in low-back pain rehabilitation interventions: a survey of trials included in Cochrane Reviews Gianola S1 , Frigerio P2 , Agostini M3 , Bolotta R4 , Castellini G5 , Corbetta D6 , Gasparini M1 , Gozzer P7 , Guariento E1 , Li LC8 , Pecoraro V1 , Sirtori V6 , Turolla A3 , Moja L5 1 IRCCS Galeazzi Orthopaedic Institute, Milan, Italy; 2 Niguarda Ca’ Granda Hospital, Milan, Italy; 3 IRCCS Fondazione Ospedale San Camillo, Venezia, Italy; 4 Servizio di Fisioterapia Inail, Milano, Italy; 5 University of Milan, Italy; 6 Fondazione Centro San Raffaele del Monte Tabor, Milan, Italy; 7 APSS Tn, Villa Igea, Trento, Italy; 8 University of British Columbia, Vancouver, Canada

Background: Selection of appropriate outcome measures is crucial in clinical trials in order to minimize bias and allow for precise comparisons of effects between interventions. Objectives: We aimed to assess the frequency and completeness of outcome measures in randomized controlled trials (RCTs) included in Cochrane systematic reviews (SRs), focusing on rehabilitation interventions for mechanical low-back pain. Methods: We performed a cross-sectional study of all RCTs included in all Cochrane SRs (full-text) published in the Cochrane Database of Systematic Reviews in February 2013 that focused on evaluations of the efficacy and safety of rehabilitation interventions for mechanical low-back pain. Two authors independently evaluated the type and frequency of each outcome measure reported in the full-text of RCTs, the methods used to measure outcomes, and the proportion of outcomes fully replicable based on the reported information. Results: Our literature search identified 11 Cochrane SRs, including 185 RCTs. Across all RCTs, 36 different outcomes were investigated. Those most commonly reported were pain [n = 164 RCTs (89%)] and disability [n = 119 RCTs (62%)], which were measured by 66 and 44 measurement tools, respectively. Other frequently reported outcomes were range of motion [n = 70 RCTs (38%)] and quality of life [n = 45 Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

RCTs (24%)]. The procedure of blinding assessment was reported in 52% of the RCTs for pain (n = 85) and 44% of RCTs for disability (n = 52). Pain, disability, range of motion, and quality of life outcomes were reported as fully replicable in 10% (n = 17), 10% (n = 12), 6% (n = 4), and 7% (n = 3) of the RCTs, respectively. Conclusions: A large number of outcome measures and a myriad of measurement instruments were used across all RCTs. The reporting was largely incomplete, suggesting better opportunities for the standardization of approaches and reporting.

Survey of outcomes in Cochrane Reviews: support for ‘core outcome set’ (COS) development Smith V1 , Clarke M2 , Williamson P3 , Gargon E3 1 Trinity College Dublin, Ireland; 2 Queens University Belfast, Northern Ireland; 3 University of Liverpool, UK

Background: Many systematic reviewers encounter difficulties when synthesising evidence from individual studies because of heterogeneity in the outcomes measured in those studies. Developing and applying agreed standardized sets of outcomes, known as ‘core outcome sets’ (COS), would help overcome this through more consistency in the measurement and reporting of outcomes in trials. COS might also make reviews more useful to decision makers. The Core Outcome Measures in Effectiveness Trials initiative (COMET) has been helping since 2010, by facilitating the development and use of COS and is developing a strategy to assist Cochrane Review Groups (CRGs) wishing to make more use of COS. Objectives: To describe the variety of outcomes used in Cochrane Reviews, which were published in full for the first time in 2007 or 2011. Methods: We did a descriptive survey of new Cochrane Reviews from 2007 and 2011, by extracting information on the outcomes specified in the Methods section and reported in the Results section of each review. Numbers, types and frequencies of similar outcomes were identified across reviews. Results: We included 788 reviews, specifying 6127 outcomes. After excluding outcomes that had been specified in the 86 reviews that did not include any studies, we found that 1996 (37%) outcomes were not reported. The median number of specified outcomes per review was 7. Of the 2011 reviews that included studies, 361 (31%) published a ‘Summary of Findings’ table (SoF). Of these SoFs, 27% included the recommended maximum of seven outcomes. Fifteen broad outcome categories were used to manage the outcome data during data extraction and analyses. There was consistency in use of these categories across CRGs, but inconsistency in outcomes and outcome measurement within the categories. Conclusions: Further work on COS will make it easier for the results of studies to be compared, contrasted and combined, as appropriate. This will reduce waste in research and produce more informed decisions about care.

Progressing towards core outcomes in gestational diabetes clinical trials and Cochrane Reviews Bain E, Crowther C, Middleton P Robinson Research Institute, University of Adelaide, Australia

Background: As defined by the COMET initiative, core outcome sets (COS) are an agreed minimum set of outcomes to be measured and Cochrane Database Syst Rev Suppl 1–150 (2014)

15 reported in trials in a specific area, making it easier for results to be combined and compared. In Australia, the WOMBAT Collaboration, which supports high-quality perinatal trials, recognises the need for standard outcomes in research, including for gestational diabetes (GDM), a common pregnancy complication associated with significant adverse outcomes for women and babies. Objectives: To assess progress made towards COS in GDM Cochrane Reviews and clinical trials. Methods: We identified all Cochrane Reviews/Protocols of interventions for the prevention, detection, management or follow-up of GDM. We reviewed choices of review outcomes, reporting of outcome data by included trials, and compared review outcomes pre and post development of standard outcomes by the WOMBAT Collaboration (developed in 2009 through extraction and group harmonisation of outcomes from selected clinical trials and reviews). Results: Following the WOMBAT Collaboration’s release of a GDM COS, there has been increased consistency in pre-specified primary and secondary review outcomes (in 11 Cochrane Reviews/Protocols published 2010–2014 versus 6 published 2001–2009). The total number of pre-specified review outcomes has increased over time [mean: 52 (2010–2014) vs 29 (2001–2009)], signifying heightened recognition of important long-term maternal and child health outcomes. While on average over 80% of pre-specified review outcomes in recent reviews still have no reported outcome data, this in part reflects the age of included trials (published from 1985). Conclusions: While some inconsistency in outcome choices between GDM trials and reviews persists, important progress towards core outcomes in this area has been made. This will increase value and reduce waste in future research, such as in a planned overview of Cochrane Reviews for GDM prevention. Selective reporting of outcomes in trial publications represents an ongoing challenge. Approaches to ensure that COS are further developed, adhered to and endorsed are required.

Do Cochrane Reviews measure enough of what patients want? A collaborative study of Cochrane Reviews on HIV/AIDS Saldanha I1 , Dickersin K1 , Ugarte-Gil C2 , Li T1 , Rutherford G3 , Volmink J4 1 Cochrane Eyes and Vision Group and US Cochrane Center, Johns Hopkins Bloomberg School of Public Health, USA; 2 Johns Hopkins Bloomberg School of Public Health, USA; 3 Cochrane HIV/AIDS Group, University of California San Francisco, USA; 4 Cochrane HIV/AIDS Group and South African Cochrane Centre, Stellenbosch University, South Africa

Background: To minimize bias during systematic reviews (SRs), outcomes should be completely pre-specified in the protocol, including: domain (title), specific measurement (instrument), specific metric (e.g. change from baseline), method of aggregation (e.g. mean), and time-points. Also, SRs must include patient-important outcomes (PIOs), as opposed to interim outcomes only. Objectives: To evaluate the number of outcomes per SR, extent of completeness of specification of outcomes and inclusion of PIOs in SRs on HIV/AIDS. Methods: The Cochrane Eyes & Vision Group, US Cochrane Center, South African Cochrane Centre, and Cochrane HIV/AIDS Group collaborated to assess all SRs published by the Cochrane HIV/AIDS Group by June 2013. For ongoing SRs, we included the published protocol. We extracted from the Methods section of each SR each outcome domain, the other four elements for that domain and whether that domain was a PIO. Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Results: We identified 140 SRs with a median of seven outcome domains each (range 1–30). Overall, there were 294 unique outcome domains, included a total of 1140 times (‘instances’). Outcome domains were each included median = 2 (range 1–68) SRs. We developed a comprehensive classification system for outcome domains, with 14 major and 32 sub-categories. The largest major categories were clinical/biological (160/294 domains, 54%) and behavioral (51/294 domains; 17%) (Table). Overall, median = 1 (range 1–4) element was specified per outcome. Domain, specific measurement, specific metric, method of aggregation, and time-points were specified for 100%, 9%, 16%, 12%, and 2.5% of instances respectively. We classified 747/1140 instances (66%) as PIOs. Most SRs (136/140; 97%) included at least one PIO (median = 5; range 0–15 PIOs per SR). Conclusions: Cochrane SRs on HIV/AIDS include a large number of outcome domains and outcome pre-specification is incomplete. Almost two-thirds of outcomes are PIOs; the abundance of outcomes could limit SR utility for decision-makers. The classification system we developed could provide a useful framework for developing core outcome sets for primary and synthesis research in HIV/AIDS.

Oral session 10 Statistical methods Exploring treatment by covariate interactions in Cochrane Reviews: recent practice Donegan S1 , Williams L1 , Dias S2 , Welton N2 , Tudur-Smith C1 1 University of Liverpool, UK; 2 University of Bristol, UK

Background: Treatment by covariate interactions can be explored in reviews using interaction analyses (i.e. subgroup analysis or meta-regression techniques). Such analyses provide information regarding how the covariate modifies the treatment effect. Guidance exists regarding how to apply such analyses but there is limited research detailing how review authors explore interactions. Objectives: To review Cochrane Reviews (CRs) to establish how well interaction analyses are designed, applied, interpreted and reported. Methods: For each Cochrane Review Group, we included the most recently published review for which the protocol was accessible. We excluded review updates, diagnostic test accuracy reviews, withdrawn reviews and overviews of reviews. The Cochrane Database of Systematic Reviews was searched on 19 August 2013. We devised criteria using literature to assess how well interaction analyses were designed, applied, interpreted and reported. Data were extracted and summarized regarding review and covariate characteristics and analyses. Results: Cochrane Database Syst Rev Suppl 1–150 (2014)

16 No review reported how or why each covariate was chosen. Forty two percent of reviews reported each covariate in the protocol. No review labelled post hoc covariates as such. Twenty one percent of reviews mentioned five covariates or less. Two percent of reviews sought missing covariate data from study authors. Fifteen percent of reviews that did aggregate data analyses included at least 10 trials. Two percent of reviews planned to use a method to detect interactions (e.g. interaction test) but 3% of reviews reported whether an interaction was detected. No review discussed the importance, or plausibility, of the results, or the possibility of confounding. Thirty nine percent of reviews reported the number of trials with each covariate value. Fifty eight percent of reviews reported results from the interaction analysis. Eight percent of reviews reported results from methods to detect interactions. Conclusions: This review highlights flaws of interaction analyses in recent Cochrane Reviews and suggested areas where such analyses can be improved. We recommend that authors use our devised criteria to more carefully consider the design, application, interpretation, and reporting of analyses.

Reliability of estimating odds ratios of response and the corresponding standard errors from continuous rating scale scores for meta-analysis: a case study in trials on depression treatment Meister R, von Wolff A, Kriston L University Medical Center Hamburg-Eppendorf, Germany

Background: While conducting a meta-analysis researchers are occasionally faced with the problem that the relevant endpoint is reported in some studies as a continuous endpoint (e.g. group specific mean on a rating scale) and in other studies as a dichotomous endpoint (e.g. group specific number of responders), which makes the synthesis of primary trials difficult. Hence, effect sizes derived from dichotomous endpoints (e.g. odds ratios) are estimated out of effect sizes derived from continuous endpoints (e.g. standardized mean differences). Yet the reliability of currently available methods is insufficiently evaluated. Objectives: The objective is to empirically evaluate the performance of five methods to estimate both odds ratios and the corresponding standard errors. Methods: A database of randomized controlled trials searched and selected for earlier meta-analyses on the effectiveness of psycho therapeutic, pharmacological, or combined treatments for chronic depression was used. Only those trials of the database were considered that reported both continuous and dichotomous endpoints. For all included trials odds ratios and the corresponding standard errors were calculated and additionally estimated using one of the five methods by Hasselblad & Hedges (H&H), Cox & Snell (C&S), Furukawa (F), Suissa (S) and Kraemer & Kupfer (K&K). Observed and estimated values were compared on a trial level and on the level of meta-analyses using relative odds ratios and percentage deviations. Results: A total of 26 trials were included. On a trial level four out of five methods performed well in estimating odds ratios out of continuous endpoints (H&H, C&S, F, S). Regarding standard errors three out of four methods considerably underestimated the standard errors (H&H, C&S, F), whereas one method (S) reliably estimated the standard errors. On the level of meta-analyses all overall effect sizes were comparable with overlapping confidence intervals. Again, the method Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

by Suissa performed best. Conclusions: Odds ratios of response and the corresponding standard errors can reliably be estimated out of continuous endpoints with the method by Suissa performing best.

Multiple imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE Debray T1 , Jolani S2 , Koffijberg H1 , van Buuren S3 , Moons K1 1 University Medical Center Utrecht, The Netherlands; 2 Utrecht University, The Netherlands; 3 TNO Quality of Life, The Netherlands

Background: Individual participant data meta-analyses (IPD-MA) are increasingly used for developing and validating multivariable (diagnostic or prognostic) risk prediction models. Unfortunately, some predictors or even outcomes may not have been measured in each study and are thus systematically missing in the IPD-MA. As a consequence, it is no longer possible to evaluate between-study heterogeneity and to estimate study-specific predictor effects, which severely hampers the development and/or validation of novel prediction models. Methods: Here we describe a novel approach for imputing systematically missing data and adopt a generalized linear mixed model to allow for between-study heterogeneity. This approach can be viewed as an extension of Resche-Rigon’s method (Stat Med 2012), but relaxes assumptions regarding variance components and allows imputation of linear (e.g. continuous) and non-linear (e.g. categorical) predictors. Results: We illustrate our approach in a case study with the IPD from 13 studies for predicting the presence of deep venous thrombosis. We compare the results after applying various imputation methods, and make recommendations about their implementation. Conclusions: Our approach improves the estimation of predictor effects and between-study heterogeneity, thereby facilitating the development and validation of novel prediction models from an IPD-MA.

Testing for intervention effects in the random-effects model: a comparison of different statistical methods Beyene J McMaster University, Canada

Background: A random-effects model is typically used to dealing with unexplained heterogeneity in meta-analysis. However, when the number of studies available for analysis is small, accuracy of asymptotic results may be compromised. Objectives: To compare alternative statistical methods for testing intervention effect with random-effects model. Methods: Using data extracted from The Cochrane Library, empirically we compared statistical significance levels for testing intervention effect based on: (i) currently used methods that assume large sample; (ii) second-order likelihood method based on Skovgaard’s statistic; (iii) bootstrap-based approach. We focused on continuous outcomes and investigated three effect measures [mean difference (MD), standardized mean difference (SMD), ratio of means (RoM)], and three methods of estimation of the heterogeneity parameter [DerSimonian-Laird (DL), maximum likelihood (ML), and restricted maximum likelihood (REML)]. Results: After excluding meta-analyses for which the heterogeneity parameter is significantly different from 0, Cochrane Database Syst Rev Suppl 1–150 (2014)

17 66 meta-analyses were used with MD and 106 meta-analyses were analyzed with SMD and RoM. For a large majority of the analyses, P values were smaller for the traditional methods. With MD as effect measure and heterogeneity estimated by ML, we observed a significant discrepancy in P values between Skovgaard’s and the traditional Z test (mean difference = 0.05, SD of difference = 0.09). Overall, similar patterns emerged when we used the other effect measures (SMD, RoM) as well as alternative methods of estimating the heterogeneity parameter (DL, REML). The bootstrap-based significance levels were generally closer to the higher-order methods rather than the Z test. Conclusions: Inferential methods in random-effects model need to be carefully assessed when the number of studies is small. Alternative statistical procedures are available and easy to compute.

Oral session 11 Citation screening and study selection (2) Facilitating research on research using SWAR (studies within a review) and SWAT (studies within a trial) Clarke M1 , Devane D2 , Lang T3 , Maguire L4 , Savage G5 , Shorter G6 , Smith V7 1 All-Ireland Hub for Trials Methodology Research, UK; 2 School of Nursing and Midwifery, NUI Galway, Ireland; 3 Global Health Network, Oxford, UK; 4 All-Ireland Hub for Trials Methodology Research, Queen’s University Belfast, UK; 5 Centre for Public Health, Queen’s University Belfast, UK; 6 All-Ireland Hub for Trials Methodology Research, University of Ulster, UK; 7 School of Nursing and Midwifery, Trinity College Dublin, Ireland

Background: Cochrane Reviews, other systematic reviews, randomized trials, and other prospective studies of health and social care seek to resolve uncertainties about the effects of different interventions, actions, and strategies. However, despite hundreds of thousands of reviews and trials, there are probably only a few hundred studies assessing the effects of different methods for doing this research. This is well illustrated by the findings of Cochrane Methodology Reviews. Many uncertainties remain about the most appropriate methods for ensuring that estimates of the effects of health and social care interventions are reliable and robust. The SWAR (studies within a review) and SWAT (studies within a trial) programme is seeking to help fill this evidence gap. It was established by the All-Ireland Hub for Trials Methodology Research in collaboration with the Medical Research Council’s Network of Hubs in the UK and the Global Health Network, to facilitate this research into research. Simple, one and two page protocols are being prepared and made available to support the embedding of these studies in trials and reviews, encourage modifications to them and stimulate interest and additional ideas. Objectives of this presentation: (i) To describe how the SWAR and SWAT programme will help resolve uncertainties about the effects of different methods for all aspects of reviews and trials. (ii) To enthuse reviewers, researchers and others about their ability to conduct research within their own research. Content of this presentation: The concept of SWAR and SWAT will be presented. This includes the core outline: background, intervention, comparator, allocation, primary Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

outcomes, secondary outcomes, analysis, possible problems, likely costs, publications, and version information. The SWAR and SWAT website will be available to view, and examples of completed SWAR and SWAT will be shown. The presentation should stimulate ideas for future SWAR and SWAT, and encourage researchers to see how they might maximize the impact of embedding research into research, leading to improvements in the design of future trials and reviews.

The EMBASE project: an analysis of highlighted words and phrases Noel-Storr A1 , Dooley G2 , Glanville J3 1 Cochrane Dementia and Cognitive Improvement Group, Oxford University, UK; 2 Metaxis Ltd, UK; 3 York Health Economics Consortium, UK

Background: The EMBASE project utilizes a crowd model to help screen search results for relevance within the Cochrane Central Register of Controlled Trials (CENTRAL). To help screeners, approximately 80 words or phrases were selected to be highlighted, some in yellow and some in red. The yellow highlights are considered more likely to appear in records eligible for CENTRAL. The red highlights are ‘warning’ highlights considered likely to appear in records to be rejected. Objectives: We sought to assess the usefulness of these highlighted words. Do they speed up the screening? What proportion of records screened in the first 6 months of the project: (i) contained only red highlights and were rejected?; (ii) contained only yellow highlights and were accepted?; (iii) does the location within the citation of the highlighted word indicate that a record is very likely a reject or not?; and (iv) are certain highlights more effective than others? Methods: Our analyses were based on all records screened by the crowd from January to July 2014. A sample of EMBASE screeners were asked to complete a brief questionnaire regarding the usefulness of highlights. Another random sample was selected to screen a batch of records without the highlights activated. Their screening speed and accuracy was compared to those who screened with the highlights. Results: Complete analyses will be presented. Interim results indicate some key findings: screeners want and use the highlights a great deal; almost all accepted records contained yellow highlights; all records with red highlights in the title of the citation were rejects; records with yellow highlights found only in the final half of the abstract were almost always rejects. Conclusions: Highlighting words and phrases within search results plays a significant role in speeding up the assessment of citations. Screeners are overwhelmingly in favour of highlights, both yellow and red. Highlighted terms are designed to help guide the screener; they are not designed to negate the need for human intervention.

Cochrane Database Syst Rev Suppl 1–150 (2014)

18 Citation screening in systematic reviews: two approaches, two authors and time taken [SWAR-1 (Study Within A Review 1)] Devane D1 , Clarke M2 , McCarthy B3 , Casey D3 1 National University of Ireland Galway/West Northwest Hospitals Group, Ireland; 2 Queen’s University Belfast, Northern Ireland; 3 National University of Ireland Galway, Ireland

Background: Conducting a high quality systematic review can be time consuming and costly, with conservative estimates of more than 1000 person-hours for an average review. One important, but time consuming, step is the selection of studies for inclusion in the review. This is done by firstly applying the study eligibility criteria to citations retrieved from the searches. However, there is little evidence to guide review teams in choosing the most effective method for screening citations. Objectives: To evaluate the effects of two different methods of screening citations for inclusion of papers in a systematic review. Methods: A two-group randomized trial. A database of 1072 citations was divided randomly into two groups i.e. (1) a single or a (2) two-stage method of citation screening. In the single-stage method, the title and the abstract were made available simultaneously, while in the two-stage approach the citation was screened by title only (stage 1) and then by screening the records judged to be potentially eligible using the title and abstract (stage 2). Two authors screened each citation independently. Results: The average time taken, across both reviewers to screen 100 citations was 120 min using the one-stage process compared with 170 min using the two-stage process. There was a significantly higher rate of rejection associated with the one-stage process (76.9%) compared with the two-stage process (66.2%) (RR 1.16, 95% CI 1.08–1.25). Inter-observer levels of agreement between both reviewers were moderate across all stages (kappa 0.41–0.57). Conclusions: Screening using a method in which titles and abstracts are presented simultaneously is less time consuming than a two-stage process of presenting the title followed by the abstract.

evaluation of the different synthesis and presentation methods used in complex reviews. Objectives: (i) To describe and estimate the prevalence of different synthesis and presentation methods used in a sample of complex systematic reviews. (ii) To describe the advantages and disadvantages of each of the identified synthesis and presentation methods. Methods: Systematic reviews published between 2008 and 2012 were identified from the Health Systems Evidence Database. A sample of 50 Cochrane Reviews was randomly selected. Data extracted included: diversity of interventions, settings, conditions, outcomes, and study designs; use of outcome categories; synthesis and presentation methods; and rationale for the choice of methods. Results were summarized using descriptive statistics. Results: Outcome categories (domains) were used in 78% of reviews to summarize the results of included studies, but were less commonly used to aid synthesis (30% of reviews). Synthesis was undertaken in 60% of reviews, most commonly meta-analysis (Fig. 1). Forest plots were used in 60% of reviews but rarely in reviews with no syntheses (5%) compared with reviews with synthesis (97%); risk difference −92% (95% CI −100 to −80%). Only 30% of reviews pre-specified selection methods for dealing with multiplicity of outcomes. Synthesis was more common in these reviews (87%) compared with reviews where no selection method was provided (49%); risk difference 38% (95% CI 14–62%). Conclusions: Outcome categorization and selection methods can facilitate presentation and synthesis in complex reviews, enabling best use of available research.

Oral session 12 Complex reviews From summary to synthesis: a review of statistical synthesis and presentation methods used in complex reviews J1 ,

S2 ,

M2 ,

M3 ,

S2 ,

M3

McKenzie Brennan Page Chau Kramer Bosch 1 Statistical Methods Group, Australasian Cochrane Centre, Australia; 2 Australasian Cochrane Centre, Australia; 3 Central Clinical School, Monash University, Australia

Background: Systematic reviews evaluating public health and health services interventions involve additional complexity compared with clinical reviews. This complexity arises from multi-faceted interventions, evaluated in varied settings, conditions, outcomes, and study designs. Application of meta-analytical methods in these reviews can be challenging, and the use of ad hoc approaches (e.g. counting the number of studies with statistically significant results) may under-utilize or misrepresent available research. To date, there has been no Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Can qualitative comparative analysis help us to identify important components of complex interventions? Thomas J1 , O’Mara-Eves A2 , Brunton G2 1 CQIMG, EPPI-Centre, Institute of Education, London, UK; 2 EPPI-Centre, Institute of Education, London, UK

Background: Systematic reviews addressing policy and practice questions concerning complex interventions frequently need to both assess the efficacy of a given intervention and to identify which intervention and its components might be relevant in particular circumstances. Here, intervention replication is rare, and commonly used synthesis methods are less useful when analyses focuses on the Cochrane Database Syst Rev Suppl 1–150 (2014)

19 identification of those components of an intervention which are critical to its success. Methods: Through a worked example, we examine the potential of Qualitative Comparative Analysis (QCA) to assist with complex syntheses. Originating from political science and historical sociology, QCA can identify a number of configurations of various participant, intervention, and contextual characteristics which are (or are not) present when the intervention has been successful (or not) in obtaining the outcome. Analysing studies in these terms facilitates the identification of necessary and sufficient conditions to obtain the outcome. Since QCA is predicated on the assumption that multiple pathways might lead to the same outcome, and does not assume a linear additive model in changes to a particular condition, it appears less sensitive to some limitations of statistical meta-analytic methods. Results: The worked example shows how the QCA reveals our initial theories of change to be unable to distinguish between ‘effective’ and ‘highly effective’ interventions. Through the iterative QCA process, other intervention characteristics are identified which better explain the observed results. Conclusions: QCA is a promising alternative (or adjunct) synthesis method, particularly in comparison to the standard fall-back of a ‘narrative synthesis’ when quantitative synthesis is impossible, and might be considered when reviews are broad and heterogeneity is significant. There are currently very few examples of its use with systematic review data, and further methodological work is needed to establish optimal conditions for its use and to document process, practice, and reporting standards.

Characteristics of realist synthesis reviews: a systematic mapping review of realist reviews Berg RC1 , Nanavati J2 Norwegian Knowledge Centre for the Health Services, Norway; 2 Loyola/ Notre Dame Library, USA 1

Background: Realist synthesis reviews are a new type of literature review that aims to provide an explanatory analysis of how and why complex social interventions work, or don’t work, in particular contexts or settings. A realist review is explicitly concerned with the context-mechanisms-outcomes configurations that underlie interventions. Objectives: To describe, with regard to realist reviews, how much has been done, in what areas, what are the methodological characteristics and what are the opportunities and challenges for its methodological standardization and wider use. Methods: A systematic mapping review approach, which captures and describes the literature in one specific field of study, is used. Systematic searches are conducted in electronic databases (The Cochrane Library, MEDLINE, EMBASE, CINAHL, ERIC, PsycINFO), relevant websites and listservs are examined and experts contacted. Study selection is performed by two reviewers independently and data extraction is performed by one reviewer and checked by a second. For each included review, details of the methodological process are extracted, and descriptive analyses are performed. Results: Electronic searches identified 154 unique references and other sources 79 relevant references published from 2004 to 2014. Half (52%) were promoted to full text screening. The final study selection process is near completion and we anticipate about 75 included papers, approximately two-thirds of which were published in the last 5 years. In 10 years, there has been an exponential growth of published realist reviews from a broad spectrum of health and related research areas. The included reviews vary greatly in terms of Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

purpose, methodological rigor, and quality of reporting. Many consult stakeholders/experts as part of the review process. Quality assessment of included studies is infrequently performed. Conclusions: Realist synthesis review is a new review approach of increasing focus that can be useful for taking stock of complex social programs. Due to great variability in its conduct, there may be a need for methodological standardization to ensure the utility and strength of its evidence.

Assessing how much confidence to place in findings from qualitative evidence syntheses: a new version of the CERQual tool Munthe-Kaas H1 , Glenton C2 , Lewin S3 , Carlsen B4 , Colvin C5 , Noyes J6 , Rashidian A7 , Booth A8 , Garside R9 , TCWG10 1 Norwegian Knowledge Centre for the Health Services, Norway; 2 Norwegian branch, Nordic Cochrane Centre, Norwegian Knowledge Centre for the Health Services, Norway; 3 Global Health Unit, NOKC and Health Systems Research Unit, Medical Research Council of South Africa, Norway; 4 Uni Rokkansenteret, Norway; 5 School of Public Health and Family Medicine, University of Cape Town, South Africa; 6 School of Healthcare Sciences, Bangor University, UK; 7 National Institute of Health Research, Iran; 8 School of Health and Related Research, University of Sheffield, UK; 9 European Centre for Environment and Human Health, University of Exeter Medical School, UK; 10 Various, International

Background: Qualitative evidence syntheses are increasingly used to bring together findings from qualitative studies. However, it is difficult to use these findings to inform decisions because the methods to assess how much confidence to place in these synthesis findings are poorly developed. Objectives: To describe a new version of a tool for assessing how much confidence to place in the evidence from qualitative research reviews. Methods: The Confidence of the Evidence from Reviews of Qualitative research (CERQual) tool was developed through the review of existing tools; working group discussions; and piloting of the tool on several qualitative evidence syntheses. Results: CERQual bases the assessments of confidence on four components: (i) the methodological quality of the individual studies contributing to a review finding, was assessed by using a quality-assessment tool for qualitative studies (ii) the coherence of each review finding, assessed by looking at the extent to which the finding is based on data that is similar across multiple individual studies and/or incorporates (plausible) explanations for any variations across individual studies (iii) the relevance of a review finding, assessed by determining to what extent the evidence supporting a review finding is applicable to the context specified in the review question (iv) the sufficiency of data supporting a review finding, assessed by an overall determination of the degree of richness and/or scope of the evidence and quantity of data supporting a review finding. After assessing each component, an overall judgement of the confidence in each review finding is made. The confidence can be judged as high, moderate, low, or very low. This assessment should be described and justified in a transparent manner, preferably in a summary of qualitative findings table that includes narrative statements. Conclusions: CERQual provides a transparent method for assessing the confidence of evidence from reviews of qualitative research. Like the GRADE approach, it may facilitate the use of these findings alongside reviews of effects and in guideline development processes.

Cochrane Database Syst Rev Suppl 1–150 (2014)

20

Oral session 13 Knowledge translation and communicating evidence Do Cochrane summaries help student midwives understand the findings of Cochrane reviews: the BRIEF randomised trial Alderdice F1 , Lasserson T2 , McNeill J1 , Beller E3 , Carroll M4 , Hundley V5 , Sunderland J6 , Devane D7 , Noyes J8 , Keyes S9 , Norris S10 , Wyn-Davies J11 , Clarke M1 1 Queens University Belfast, UK; 2 Cochrane Editorial Unit, UK; 3 Bond University Queensland, Australia; 4 Trinity College Dublin, Ireland; 5 Bournemouth University, UK; 6 City University London, UK; 7 National University of Ireland, Galway, Ireland; 8 Bangor University, UK; 9 Edinburgh Napier, UK; 10 Swansea University, UK; 11 University of South Wales, UK

Background: Abstracts and plain language summaries (PLS) are often the first, and sometimes the only, point of contact for systematic reviews. It is important to identify how they are used and to know the impact of different elements, including the authors’ conclusions. Objectives: To assess whether (i) the abstract or the PLS of a Cochrane Review is a better aid for midwifery students in assessing the evidence; (ii) inclusion of authors’ conclusions helps them; (iii) there is an interaction between the type of summary and the presence or absence of the conclusions. Methods: Eight hundred and thirteen midwifery students from nine universities in the UK and Ireland were recruited to this 2 × 2 factorial trial (abstract vs PLS, conclusions vs no conclusions). They were randomly allocated to one of the four groups and asked to recall knowledge after reading one of four types of summaries of two Cochrane Reviews, one with clear findings and one with ambiguous findings (as assessed by an expert panel). The primary outcome was the proportion who gave the response the panel judged to be correct. Results: There was no statistically significant difference in correct response between Abstract and PLS groups in the clear findings example (Abstract: 59.6%; PLS: 64.2%; Risk difference 4.6%; CI −0.2 to 11.3) or the ambiguous findings example (42.7%; 39.3%; −3.4%; −10.1 to 3.4). There was no significant difference between conclusions and no conclusions groups in the example with clear findings (conclusions: 63.3%; no conclusions: 60.5%; 2.8%; −3.9 to 9.5) but there was a significant difference in the example with ambiguous findings (44.7%; 37.3%; 7.3%; 0.6–14.1 p = 0.03). PLS without conclusions in the ambiguous findings example had the lowest proportion of correct responses (32.5%). Students given a PLS were more likely to report wanting to read the full review than those given the abstract. Conclusions: Abstracts with and without conclusions generated similar student responses. PLS with conclusions gave similar results to abstracts with and without conclusions but removing the conclusion from a PLS with ambiguous findings led to more problems with interpretation.

SWAR 2: How much do you need? Can readers get the key messages from summaries of Cochrane Reviews, without reading the full review? Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Maguire L, Clarke M Queen’s University Belfast, UK

Background: Cochrane Reviews (CR) provide an important source of evidence. Many readers will only access the various summaries available for CRs rather than the full review, so it is essential that these are clear and understandable. Objectives: We explored whether readers can understand key messages without having to read the full review, and if there were differences in understanding between various types of summary, including an audio podcast. Methods: The CR we focused on examines the health impacts of the use of electric fans during heat waves. Thirty-six members of staff at various universities in Ireland participated. Participants were asked if they knew of the review and to select from a list of options the one that matched their expectation of the effect on mortality of using electric fans during heat waves. They were then randomly assigned one of four summaries of the review (i.e. abstract, plain language summary, podcast or podcast transcript). They were asked to spend no more than 15 min reading or listening to the summary and then to answer again the question about effect of electric fans on mortality and to indicate whether or not they would now want to read the full Cochrane Review. Results: After considering their summary, just over half (53%) the participants identified what the authors of the review regard as its key message on mortality, which is that the research evidence is mixed. The figures were 33% for the abstract group, 50% for both the plain language summary and the transcript groups and 78% for the podcast group. Only 13 (37%) of the respondents said that they would want to read the full review after having considered their summary. Conclusions: The differences between the groups were not statistically significant but suggest that the audio summary might improve knowledge transfer compared to written summaries. This finding should be repeated using a larger sample size and with other reviews. This study is part of the SWAR (study within a review) programme which was developed by the All-Ireland Hub for Trials Methodology Research and the Medical Research Council Hub Network.

Online filter bubbles and confirmation bias in health care: narrative of a vaccine skeptic ´ Gaxiola G1 , Badenoch D2 Perez 1 HPS, Cochrane Mexico, ´ Mexico; 2 Minervation Ltd., UK

Background: Websites such as Facebook and Google use information about our previous behaviour to prioritize and display information for us. This phenomenon is sometimes called a ‘filter bubble’ because it means we are less likely to find information that is novel or challenging. Filter bubbles may reinforce anti-science health beliefs and make it harder to disseminate evidence-based information to the people that need it most. Objectives: To explore the effect of filter bubbles in Facebook in the context of a controversial health belief and to consider the implications for evidence-based health information. Methods: A Facebook account was created and a search for ‘vaccine harms’ was performed. Groups and pages suggested by Facebook were browsed and the first results were ‘liked’, as were the suggestions. After the timeline was populated, a relevant article was ‘liked’ and suggestions were classified into pro-vaccines or anti-vaccines. Screen capture software was used to record the experience. The process was repeated with different accounts to enable a comparison between different ‘experiences’ of the same website according to: pro-versus anti-vaccine search terms; UK versus American user; English language Cochrane Database Syst Rev Suppl 1–150 (2014)

21 versus Spanish language user (location controlled). Results: The search for ‘vaccine harms’ was performed in January 2014. Within three clicks of the search results we identified three anti-vaccine pages with over 125 000 ‘likes’. Very quickly, our timeline was populated with anti-vaccine information. After ‘liking’ an article, four more anti-vaccination suggestions appeared, all with over 10 000 ‘likes’. Our comparisons revealed a different Facebook experience according to opinion (pro- or anti-vaccine), location and language. Conclusions: Personalization of online searching makes it more likely that people will find information that they already agree with and less likely that they will find information that challenges their views. As personalization becomes the norm in online services, research is needed to assess the impact on public health and the prospects for interventions to counter their impact.

Implementing multilayered presentation formats in a new generation of trustworthy clinical practice guidelines 1

1

1

2

3

Brandt L , Kristiansen A , Vandvik PO , Alonso P , Guyatt G 1 MAGIC-Making GRADE the Irresistible Choice, Norway; 2 Centro Cochrane Iberoamericano, Barcelona, Spain; 3 McMaster University, Canada

Background: Clinical practice guidelines face challenges with limited availability and usefulness for clinicians at the point of care, often being published as lengthy and cumbersome PDF documents with evidence hidden away in appendixes. Increasing accessibility by structuring content and making guidelines available in optimized presentation formats on a multitude of platforms can facilitate successful dissemination. Methods: The MAGIC research and innovation program (www.magicproject.org) has developed a platform (www.magicapp.org) to author, publish and dynamically update trustworthy guidelines developed with the GRADE system. The MAGICapp was large scale tested in an adaptation of the nineth iteration of the American College of Chest Physicians Antithrombotic Therapy and Prevention of Thrombosis, performed by the Norwegian Society of Thrombosis and Hemostasis. We used the MAGICapp to create recommendations in a novel multilayered presentation format, developed and evaluated through an iterative process of brainstorming, sketching, stakeholder feedback and user testing, in collaboration with the DECIDE project (www.decide-consortium.eu). Results: We published the Norwegian guideline with 249 recommendations for antithrombotic therapy in November 2013. The guideline is freely available to inhabitants of Norway, accessible online, on smartphones and tablets, with planned integration in the electronic medical records. Clinicians are presented actionable recommendations first, with the option to delve into deeper layers of information: the key information (balance between the desirable and undesirable consequences of an intervention, confidence in the effect estimates, typical values and preferences and costs), rationales describing the reasoning behind the recommendation, practical information (e.g. risk scores, contraindications), evidence profiles with effect estimates and links to references. Conclusion: The Norwegian adaptation endeavour demonstrated the feasibility of authoring an entire guideline in the multilayered format which makes the evidence more readily available. We will provide an online demonstration.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Oral session 14 Review methods: publication bias A systematic review on the impact of studies that are not fully published on results of meta-analyses ¨ Schmucker C1 , Schell L1 , Blumle A1 , Briel M2 , Schwarzer G3 , Basler D4 , Von Elm E5 , Meerpohl J1 1 German Cochrane Center, University Medical Center Freiburg, Germany; 2 Institute for Clinical Epidemiology and Biostatistics, University Hospital Basel, Switzerland; 3 Institute of Medical Biometry and Medical Informatics, University Medical Center Freiburg, Germany; 4 Department of Neonatology, University Hospital Zuerich, Switzerland; 5 Cochrane Switzerland, University Hospital Lausanne, Switzerland

Background: As part of a systematic review a meta-analysis aims to provide an unbiased summary of data from the literature. However, potentially important studies could be missing because of selective publication and inadequate dissemination of results. If results of missing studies differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention’s effects. Objectives: As part of the OPEN project (www.open-project.eu) we conducted a systematic review of methodological research projects (MRPs) that assessed whether inclusion of studies that are not published or published in the grey literature impacts on pooled effect estimates in meta-analyses (quantitative measure) and leads to different conclusions in meta-analyses (qualitative measure). Methods: Four bibliographic databases were searched with no limit to publication year or language. MRPs were considered eligible for inclusion if they reviewed a cohort of meta-analyses which (i) compared pooled effect estimates of meta-analyses of health care interventions according to publication status or (ii) examined whether the inclusion of unpublished/grey studies impacts the overall finding of a meta-analysis. Results: Seven MRPs including 1866 studies (from 1348 published studies) that compared pooled treatment effect estimates between published and unpublished/grey studies were identified. One MRP showed that published studies had larger pooled treatment effects in favour of the intervention than unpublished studies (Ratio of ORs 1.15, 95% CI 1.04–1.28). In the remaining six MRPs pooled effect estimates and overall findings were not changed by the inclusion of unpublished/grey studies. Conclusion: There is only limited evidence from MRPs for an impact of studies that are not fully published on results of meta-analyses or systematic reviews. However, in several individual examples (e.g., a meta-analysis of reboxetine by Eyding et al 2010 published in BMJ) it could be shown that the inclusion of unpublished studies led to findings that differed from those based on published data.

Cochrane Database Syst Rev Suppl 1–150 (2014)

22 An empirical investigation of the potential impact of selective inclusion of results in systematic reviews of interventions Page MJ1 , McKenzie JE1 , Chau M2 , Green SE1 , Forbes A3 1 Australasian Cochrane Centre, Monash University, Australia; 2 National Trauma Research Institute, Central Clinical School, Monash University, Australia; 3 School of Public Health and Preventive Medicine, Monash University, Australia

Background: For a particular meta-analysis, there may be multiple effect estimates available to select from a trial report, resulting, for example, from multiple scales, time points, or analyses. Multiplicity can potentially lead to data driven decisions regarding which data to include in a meta-analysis, which we refer to as ‘selective inclusion of results’. Selective inclusion of results may bias meta-analytic effect estimates. Objective: To investigate whether there is evidence of selective inclusion of results in systematic reviews, and what impact it has on meta-analytic effects. Methods: Systematic reviews published between January 2010 and 2012 were randomly sampled. Trial effect estimates that were available for inclusion in the first continuous outcome meta-analysis in each review were extracted. For each trial, extracted effect estimates were ranked according to magnitude, and the selected effect estimate’s location between the lowest and highest rank was calculated. A Potential Bias Index (PBI) was developed to investigate whether there was evidence of selective inclusion. The PBI quantifies the average ranking location of the trial effect estimates selected for inclusion in a meta-analysis (where PBI = 1 when the highest ranked effect estimate in each trial is always selected, PBI = 0 when the lowest ranked effect estimate is always selected, and PBI = 0.5 under a process consistent with random selection). Results: Thirty-one reviews including 228 trials were evaluated. Half the trials had multiple effect estimates that were available for inclusion in a particular meta-analysis. The PBI was 0.58 (95% confidence interval 0.51–0.65; two-tailed p-value of 0.049), providing some evidence that on average, larger effect estimates were included than we would expect through a process consistent with random selection. Analysis of the potential impact of selective inclusion of results on meta-analytic effect estimates is ongoing and will be presented at the Colloquium. Conclusion: This is the first study to explore selective inclusion of results in systematic reviews. The findings may inform guidance for dealing with multiplicity of data in trial reports.

Massive production of meta-analyses of antidepressants with industry authorship or industry-related conflicts Ebrahim S1 , Bance S2 , Athale A3 , Malachowski C2 , Ioannidis J4 1 Stanford University/McMaster University, USA; 2 University of Toronto, Canada; 3 McMaster University, Canada; 4 Stanford University, USA

Background: As meta-analyses have become highly influential for guiding clinical practice, they may be used as marketing tools by the industry. Objectives: To identify how many recent meta-analyses of antidepressant trials in depression are authored or sponsored by the Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

industry or have authors with conflicts of interest; and evaluate whether this may affect the overall interpretation of their results. Methods: We searched MEDLINE to identify all published meta-analyses of randomized controlled trials evaluating antidepressants in patients with depressive conditions from 1 January 2007 to 5 March 2014. We extracted data pertaining to author’s affiliations, conflicts of interests and whether the abstract of the meta-analysis included a negative concluding statement as to whether the antidepressant(s) were effective or safe. Results: Of the 1111 citations, we retrieved 262 in full text, and eventually identified 187 meta-analyses that were eligible. One hundred and thirty four (72%) meta-analyses had authors who were industry employees and/or had conflicts of interest for a for-profit organization. One hundred and twenty nine meta-analyses (69%) did not include any caveats or negative statements in the concluding statement of the abstract. Meta-analyses having at least one author employed by a for-profit organization or having financial conflicts of interest with the manufacturer of the assessed drug were significantly associated with fewer negative statements about the assessed antidepressant in the abstract’s concluding statement (p < 0.001) (Table 1). Conclusions: There is a massive production of meta-analyses of antidepressants in depression performed by the industry or by authors with financial conflicts of interest with industry and these meta-analyses almost never include any negative statements about antidepressants.

Publication bias tests for survival data: a simulation study Debray T1 , Moons K1 , Koffijberg H1 , Riley R2 1 University Medical Center Utrecht, The Netherlands; Birmingham, UK

2

University of

Background: The presence of publication bias is often verified in meta-analyses by visual inspection of the funnel plot. Statistical tests may estimate the association between the reported effect size and their standard error (Egger’s test), total sample size (Macaskill’s test) or inverse of the total sample size (Peter’s test). Although these tests have been evaluated for pooling odds ratios, their application may be less appropriate for survival data where censoring influences statistical significance (and thus selective reporting) of the hazard ratio. Methods: We propose and evaluate two new publication bias tests for survival data that are based on the total number of events (Test-1) and the total survival time (Test-2). We compare their performance to existing tests in an extensive simulation study where more than 20 000 000 meta-analyses were generated. Here, we varied the true hazard ratio (HR = 0.20–1.00), the number of available studies (N = 10–100), the censoring proportion (cp = 0.10–0.50) and the scale of the hazard distribution. Furthermore, we used a set of predefined reflecting meta-analyses of randomized clinical trials in the medical literature. Results and conclusions: When treatment groups are balanced, simulation results demonstrate that type-1 errors Cochrane Database Syst Rev Suppl 1–150 (2014)

23 are problematic for Egger’s test (averaging from 0.110 for N = 10 to 0.195 for N = 100), but consistently good (around 0.10) for Peter’s test and Test-1. The power of all tests was low; for example Test-1 yielded power from 0.112 (for N = 10) to 0.208 (for N = 100). Finally, we compare and discuss the performance of Peter’s test and Test-1 in imbalanced treatment groups, and make recommendations for practice.

disease examined by each RCT was classified using the GBD criteria. Multivariable regression was used to examine the relationship between number of RCTs and total DALYs (Disability Adjusted Life Years), proportion of DALYs in middle-income (pDALYs-MICs) and proportion of DALYs in low-income countries (pDALYs-LICs). Results: Four thousand hundred and ninety abstracts were reviewed and 1370 primary RCTs were identified, of which 1112 could be classified using the GBD taxonomy. Eight hundred and ninety two RCTs (82%) examined non-communicable diseases whereas 69 (6.2%), 14 (1.3%), 23 (2.1%), 23 (2.1%) and 91 (8.2%) examined communicable, maternal, neonatal and nutritional diseases and injuries, respectively. Total DALYs was poorly associated with number of RCTs in univariable analysis (R2 = 0.25), although it was a significant predictor in the univariable and multivariable models (p < 0.01). pDALYs-LICs was

Oral session 15 Reporting of trials and research (1)

inversely associated with number of RCTs (p < 0.01) and pDALYs-MICs was not a significant predictor. Relative to total DALYs, neonatal sepsis was the most underrepresented disease, while female infertility was the most overrepresented (Table 1). Conclusions: Overall, there was a poor association between global burden of disease and number of published RCTs. Additionally, diseases with a high proportion of DALYs in LICs were associated with significantly fewer RCTs.

Relationship between global burden of disease and quantity of published randomized controlled trials Emdin C1 , Odutayo A2 , Hsiao A3 , Rahimi K1 , Hopewell S2 , Altman D2 1 George Institute for Global Health, UK; 2 Centre for Statistics Medicine, UK; 3 Department of Economics, University of Oxford, UK

Background: The output of randomized controlled trials (RCTs) is an important indicator of research priorities. Whether an association exists between the number of published RCTs and the global burden of disease has yet to be examined systematically. Objective: To assess the association between the quantity of RCTs and the global burden of disease, as determined in the 2010 Global Burden of Disease (GBD) study (Lancet 2012;380:2197–223). Methods: We identified and reviewed all primary reports of RCTs published in December 2012 and indexed in PubMed by 17 November 2013. Review of abstracts, review of full text papers and extraction of trial characteristics was performed in duplicate. No language restrictions were applied. The Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

The use of systematic reviews in the design of randomized controlled trials of acupuncture: a systematic review Kim KH1 , Lee S2 , Choi J3 , Lee JA3 , Lee MS3 1 Pusan National University, South Korea; 2 Kyung Hee University, South Korea; 3 Korea Institute of Oriental Medicine, South Korea

Background: One of the major purposes of systematic review is to guide future randomized studies to resolve current conflicts or address uncertainty in clinical practice. Concerns have been raised regarding waste in research due to inadequate use of existing evidence for planning or designing future research. However, whether and how systematic reviews were used in the design of randomized trials remains uncertain. Objectives: To investigate how systematic reviews have Cochrane Database Syst Rev Suppl 1–150 (2014)

24 been used in the planning and design of randomized acupuncture trials. Methods: Study protocols for randomized controlled acupuncture trials with trial registration numbers published from January 2008 to April 2013 were searched through four English databases (EMBASE, MEDLINE, CENTRAL and AMED). A full list of reference titles in the included study protocols was screened to identify cited systematic reviews with or without meta-analyses. Results: Thirty-eight study protocols that referenced a total of 108 systematic reviews 118 times were eligible. Most protocols (n = 30; 79%) were for randomized parallel trials. Thirty-seven protocols (97%) cited at least one review (median value of 3). Cochrane reviews were referenced in 16 protocols (42%). Reviews were predominantly used for identifying current evidence levels of diseases/conditions or available treatments in 36 of 37 review-cited protocols (97%). Other areas of use included justification of study interventions and comparisons (n = 12; 32%), estimation of the difference to detect or margin of equivalence (n = 2; 5%), formulation of acupuncture treatment details (n = 2; 5%), estimation of the control group event rate (n = 3; 8%), selection or definition of outcome (n = 6; 16%), collection and analyses of adverse events (n = 3; 8%), duration of follow-up (n = 1; 3%) and other areas that were not directly related to the design of the respective protocol (n = 9; 24%). Conclusions: Most protocols used systematic reviews for the purpose of background description and only a minor portion referenced systematic reviews to inform the design. Adequate use of systematic reviews in the design of future randomized acupuncture trials is needed to reduce waste in research.

Quality of descriptions of rehabilitation interventions in low back pain: a review of published randomized controlled trials Gianola S1 , Castellini G2 , Agostini M3 , Bolotta R4 , Corbetta D5 , Frigerio P6 , Gasparini M7 , Gozzer P8 , Guariento E9 , Li LC10 , Pecoraro V9 , Sirtori V5 , Turolla A3 , Moja L11 1 University of Milano-Bicocca, Monza, Italy; 2 University of Milano, Italy; 3 IRCCS Fondazione Ospedale San Camillo, Venezia, Italy; 4 Servizio di Fisioterapia Inail, Milano, Italy; 5 Fondazione Centro San Raffaele del Monte Tabor, Milano, Italy; 6 Niguarda Ca’ Granda Hospital, Milano, Italy; 7 Asl Biella, Italy; 8 APSS Tn, Villa Igea, Trento, Italy; 9 IRCCS Galeazzi Orthopaedic Institute, Italy; 10 University of British Columbia, Vancouver, Canada; 11 University of Milan, Italy

Background: Published reports of randomized controlled trials (RCTs) often focus on the results, thereby failing to adequately describe the characteristics of the interventions studied. The findings of these reports are of little use as they cannot be replicated in real-life settings. Objectives: We aimed to assess the quality of descriptions provided for low back pain rehabilitation interventions in RCTs. Methods: We systematically searched for all RCTs considered eligible in Cochrane Systematic Rviews (SRs) published (full-text) on The Cochrane Database of Systematic Reviews in February 2013, focusing on the evaluation of rehabilitation interventions for mechanical low back pain. Two authors independently evaluated through an ad hoc checklist the quality of descriptions of rehabilitation interventions. The primary outcome was the proportion of items considered to be able to be fully replicated based on the reported information. Results: Across 11 SRs, 185 RCTs were included [median (IRQ) of publication year 1998 Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

(1990–2004)]. Less than one fifth (170.8%) of the RCTs were able to be replicated. Most RCTs (490.7%) did not report key information, compromising the trials’ ability to be replicated; another one fifth (170.3%) of RCTs gave insufficient descriptions of interventions. Who provides and who receives the intervention were the most commonly reported items in the description of interventions (91% and 81%, respectively) while materials and procedure were less reported (48% and 43%, respectively). Over time, the reporting of intervention descriptions improved. Conclusions: Despite the remarkable amount of effort spent on conducting RCTs to compare the effectiveness of rehabilitation approaches for low back pain, a large proportion of trials failed to report key information to allow for the repetition of the intervention. This is likely to hinder practical application of the research findings.

Reporting of randomized clinical trials discontinued due to poor recruitment: a literature review Olu K1 , Briel M1 , Kasenda B2 , von Elm E3 , Schandelmaier S1 1 Basel Institute for Clinical Epidemiology and Biostatistics, Switzerland; 2 University Hospital of Basel, Switzerland; 3 Institute of Social and Preventive Medicine, University Hospital of Lausanne, Switzerland

Background: Poor recruitment is the most frequent reasons given for discontinuation of randomized controlled trials (RCTs). Ideally, such RCTs should disseminate both the lessons learned from recruitment failure, which might inform the planning of future trials, and the results in sufficient detail for inclusion in meta-analyses. To date, a comprehensive empirical collection of reasons for insufficient recruitment remains missing. Objectives: To examine the extent to which published RCTs that were discontinued due to poor recruitment reported on the recruitment process, lessons learned and trial results; and to gather reported reasons for poor recruitment. Methods: We identified eligible publications using (i) a previous collection of discontinued RCTs (published 2002–2013, Kasenda et al. JAMA 2014), and (ii) a search of MEDLINE and EMBASE (2010–2013) using the recently introduced medical subject heading ”Discontinued Trial” and synonyms for discontinuation and poor recruitment. We extracted RCT characteristics and the reporting of recruitment failure. We created a new classification system of reasons for poor recruitment. Results: We included 119 publications of RCTs discontinued due to poor recruitment. The median proportion of recruited participants from the planned sample size was 27% (interquartile range, 15–44%); 108 RCTs (91%) reported results in sufficient detail for inclusion in meta-analyses; 36 (30%) described the recruitment process, and 20 (17%) explicitly reported lessons learned. Overall, we identified 40 different reasons for insufficient recruitment; most frequent were ‘overestimated number of eligible participants’ and ‘strong opinions among recruiters about study interventions’. Twenty-nine (73%) reasons for poor recruitment could have been addressed at the planning stage of the RCT. Conclusions: Most reports of RCTs discontinued for poor recruitment presented results in sufficient detail for inclusion in meta-analyses. Only a few stated lessons learned and details about participant recruitment. Most reasons for poor recruitment could have been addressed at the planning stage of the RCT.

Cochrane Database Syst Rev Suppl 1–150 (2014)

25

Oral session 16 Updating reviews A new approach to update Cochrane Reviews focusing on the ‘Summary of findings’ (SoFs) Soares-Weiser K1 , Bergman H1 , Maayan N1 , Marshall R2 1 Enhance Reviews, UK; 2 Cochrane Editorial Unit, UK

Background: The Enhance Reviews (ER) team has been involved in updating reviews for Cochrane Groups over the past 5 years. Some of the challenges are: (i) changes in methods; (ii) the need to cross-check data extraction and risk of bias for studies previously included in a review; (iii) the lengthy peer-review process; and (iv) authors’ unwillingness to get involved in the update. As a result the process is often very time-consuming, and it is difficult to estimate the time needed to complete an update. This increases the Review Groups’ burden and delays publication of updates. As a pilot project, ER are working together with the Cochrane Editorial Unit to test a faster and more focused way of producing updates. Whilst these ‘focused updates’ are simple and short, they maintain a very high methodological standard, focusing on the ‘Summary of findings’ (SoFs). Objectives: To compare time to complete a regular update to a focused update and to discuss acceptability and usefulness of the focused update Description: To perform the pilot, one of the authors (RM) chose three recently completed reviews. The ER team received the out-of-date versions of the reviews and: (i) selected the seven most important outcomes for policy-makers; (ii) performed searches and screening of the retrieved references; (iii) performed data extraction and risk of bias for new studies (iv) cross-checked data extracted from the already included studies; (v) performed meta-analyses as appropriate; (vi) created SoF tables; and (vii) updated abstract, plain language summary (PLS), implications for practice and research. For each review a short report (two-sides of A4) was created in a standardized template. This report contains the main conclusions of the review, followed by the SoFs, updated abstract and PLS, and is linked to complete table of characteristics, risk of bias assessments, and updated meta-analyses. All stages of the updating process were fully documented to allow comparison with the full review update. An appraisal form was used to estimate the actual time taken to complete each stage of the review, noting the reasons why a task took more or less time than anticipated.

Updating for quality: reflections on our editorial policy Garner P, Stephani AM, Sinclair D Liverpool School of Tropical Medicine, UK

Background: Our review group only update topics where the question is still current and with persistent equipoise, which allows the group to focus on improving the quality of reviews in updates, by using a logic framework where possible, and almost always including a ‘Summary of findings’ table. We reflect on the last 12 months of implementing this policy. Objectives: To document the outcome of editors applying the Classification Framework to identify the subset of reviews needed for update; encouraging authors to use a logic framework; and insisting that authors complete a ‘Summary of findings’ table. Methods: The Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

editorial team developed a ‘classification framework’ and two editors agreed which reviews could be excluded from any consideration for updating. We worked with authors in developing logic framework, and provided advice and support to authors in revising their reviews in the light of the proposed ‘Summary of findings’ table. Result: About one third of our reviews were successfully ‘parked’: that is classified as a historical question, or a current question where no update was intended. Logic frameworks were used in a variety of ways, including displaying the outcomes, or explaining the theory behind the intervention and the outcomes. It became clear that editorial staff have to be pro-active in assisting authors develop ‘Summary of findings’ tables, and frequently this leads to dramatic changes in the way the results are presented, and has sometimes changed the conclusions of the review. We will present these findings in a more structured, analytical way at the Colloquium. Conclusion: Cochrane Reviews need to continue to lead the market. Introduction of the classification framework across the Collaboration in late 2014 will simplify updating and improve the product for the reader. Summary of findings can dramatically improve quality, but authors need the guidance of senior editors.

Five evidence ‘game changers’ in 2013 and what this means for Cochrane Reviews Alper B1 , Fedorowicz Z2 1 EBSCO Information Services, USA; 2 Bahrain Cochrane Branch, Bahrain

Background: Reviewing the best evidence is especially challenging when the research is unpublished, when the published evidence is not reliable (for reasons not apparent through routine critical appraisal methods), and when the research framework differs from common clinical practice. Understanding relevant examples can provide insight for the planning and updating of Cochrane Reviews. Discussion: In 2013 there were five high-impact changes in systematically synthesized evidence for guiding clinical practice: sodium restriction for heart failure (January 2013), gliptins for diabetes (February 2013), iImiquimod for molluscum contagiosum (July 2013), perioperative beta blockers for noncardiac surgery (July 2013), threshold for recommending statins for primary prevention (November 2013). We will share the ‘behind the scenes’ efforts to better understand and convey these changes, and discuss what can be learned from these experiences. These will be presented in terms of: what we thought we knew, what we found out, and the challenges faced. These five experiences represent unique cautions for Cochrane reviewers and we will provide five corresponding strategies that can be used to enhance the validity, relevance, and currency of Cochrane reviews.

Oral session 17 Issues around implementing evidence The importance of process evaluation: a tale of two reviews Kristjansson E1 , Welch V1 , Liberato S2 , Francis D3 1 University of Ottawa, Canada; 2 Charles Darwin University, Australia; 3 University of West Indies, Jamaica

Cochrane Database Syst Rev Suppl 1–150 (2014)

26 Background: We have completed two complex systematic reviews on food supplementation: one for young children and one for school children, which was recently updated. In addition to rigorous Cochrane methodology, we used process evaluation to understand which implementation factors were important for the outcomes. Objectives: To present results of the reviews and process evaluations, and from this, to identify policy and program implications for those who develop and implement supplemental food programs for children. Methods: We carefully followed Cochrane procedures, developing a logic model, standardized searching and involving at least two authors in reviewing searches, inclusion/exclusion and conducting data extraction, analyses and bias assessment. We also conducted relevant subgroup analyses in order to assess key implementation and participant factors. Results: Thirty studies were in the preschool review, 27 from low and middle income (LMIC). Eighteen studies were originally in the school meals review with eight more in the update; 17 from LMIC. We will focus on findings from LMIC. Both reviews showed small effects on weight (preschool: 0.12–0.25 kg/year, school: 0.25–0.75 kg/year) and height (preschool: 0.48–0.67 cm/year, school 0.25–1.47 cm/year). The preschool review showed significant, moderate effectiveness for psychomotor development in four out of five studies and both reviews demonstrated significant effects on mental development; school meals showed consistent effectiveness for attendance and performance in math. Factors that impacted on the effectiveness included the child’s nutritional need, broken supply chains, redistribution of the supplement within the family, amount of energy given in the supplements, parental or school capacity to give food, location and level of staff supervision of the supplementation. Policy/progam implications: Food supplementation programs for young children have unexpectedly small benefits for growth; the effects for psychomotor development are somewhat larger in the School feeding programs and have small benefits for growth, and also for attendance and academic performance.

Intervention component analysis (ICA): enhancing the‘implementability’ of review findings on complex interventions Sutcliffe K, Thomas J EPPI-Centre, Institute of Education, London, UK

Background: Decision-makers often find it difficult to draw clear messages for implementation from systematic reviews due to inadequate detail in their descriptions of complex interventions. Detailed scrutiny of intervention components allows reviewers to consider whether certain features are critical for success, thus uncovering important messages for implementation. Objectives: To support the implementation of evidence from a systematic review on the efficacy of Paediatric Electronic Prescribing (PEP) interventions. Methods: To complement a synthesis of statistical evidence establishing the efficacy of PEP, we sought to examine what a successful PEP intervention ‘looks like’. We drew on the TIDieR (Template for Intervention Description and Replication) framework to examine: (i) which features were present in effective PEP systems; (ii) evidence on the strengths and weaknesses of individual intervention components; (iii) evidence on developing and implementing successful interventions. Results: Inconsistency in the level of detail provided, and in the terminology and definitions used, restricted our ability to provide a comprehensive and accurate picture of PEP interventions. However, Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

we were able to identify critical features of PEP interventions. For example, we identified that PEP interventions which were not tailored for use in paediatrics were less successful, and in some cases harmful. Notwithstanding its limitations, thematic analysis of qualitative data, such as authors’ observations of the impact of interventions on working practices, and authors’ hypothetical conclusions regarding associations between intervention features and the success (or otherwise) of PEP interventions, enabled us to identify the key strengths and weakness of different PEP components and the barriers to and facilitators of PEP development and implementation. Conclusions: Even in the absence of consistently well reported information on the content of PEP interventions, innovative approaches enabled us to identify critical components of PEP, the strengths and weaknesses of individual components, and useful evidence on how to develop and implement successful PEP system.

Using evidence syntheses: views and experiences of public health decision makers and review authors in the UK Francis D1 , Turley R2 , Thomson H3 , Weightman A4 , Moore L5 , Waters E1 , Doyle J1 , Anderson R6 , Shemlit I7 1 Cochrane Public Health Group, Australia; 2 Cardiff University/Cochrane Public Health Group, UK; 3 Glasgow University/Cochrane Public Health Group, UK; 4 Cardiff University, UK; 5 Glasgow University, UK; 6 University of Exeter/Cochrane Public Health Group, UK; 7 University of Cambridge, UK

Background: Review questions initiated by policy makers and evidence users in public health and related fields are often broad and incorporate high levels of complexity. Preparing rigorous systematic reviews to address these policy relevant questions can present many methodological challenges. A qualitative study was conducted in order to advertise Cochrane Public Health Group activities. The project sought to elicit perceptions and experiences of public health decision makers and review authors with respect to the use and production of systematic reviews. This presentation will focus on findings relating to the understanding and utility of systematic reviews in public health and how this might be improved. Methods: Telephone interviews were conducted with 14 policy makers and 13 systematic review authors from the UK (UK). Purposive sampling was used to select users and funders of public health research and systematic reviewers with varying degrees of experience. Interview data were coded using framework analysis. Results: Difficulties with applying and using systematic reviews to inform policy reiterated well established challenges to bridging the research-policy gap. Yet policy makers reported that systematic reviews were an integral and valuable component for informing decision making. Participants believed that the utility of reviews could be improved by addressing questions beyond effectiveness, incorporating diverse evidence types and providing brief, accessible, and timely summaries addressing specific policies and contexts. Greater collaboration between policy makers and reviewers and a greater focus on communication could help overcome some barriers. Review authors spoke of the varying challenges in preparing a complex review which is rigorous, relevant, timely and affordable. Conclusions: These findings reflect growing recognition, in and beyond the Cochrane Collaboration that systematic review authors need to be increasingly mindful and considerate of the needs of end users in order to meet challenging and complex requirements. Cochrane Database Syst Rev Suppl 1–150 (2014)

27 The Cochrane Review moving beyond RCTs: philosophical notions of causality and its implications Chandler J The Cochrane Collaboration, UK

Background: Cochrane’s rigorous and well-established methodology, and reproducible methods to conduct meta-analyses of randomized controlled trials, although not exclusively, is under pressure. A more flexible systematic review structure that addresses broader questions other than intervention effectiveness and incorporates non-randomized designs and qualitative research, for example, are pushing forward. Cochrane already includes diagnostic test accuracy reviews and is evaluating extending this to prognostic reviews. Other systematic approaches seek to come under the Cochrane umbrella such as rapid reviews and mixed methods synthesis. This increasing methodological diversification in Cochrane brings challenges to the underpinning premise and assumptions of what constitutes ‘evidence’ in the evidence based medicine (EBM) hierarchy and the systematic synthesis of RCTs; which allows conclusions to be drawn about causality even if the conclusion does not transfer the application of the findings to real world situations. Discussion: This oral presentation will briefly outline some of the recent philosophical developments in causation. The purpose is to consider the implications these shifts in thinking have on notions of causality within the EBM paradigm as espoused by Cochrane. Causation is, often implicit through methodology and methods and thereby for the interpretation of results. Assumptions of causality need to be more explicit as experimental, observational and qualitative methods are increasing used in studies and systematic reviews. Current debates on notions of causality in philosophy have become increasingly active in the last two decades, addressing qualitative reasoning, mechanistic reasoning, thick causality (causal pluralism) and mathematical developments using causal diagrams. In focusing on the need for applicable evidence that has utility for users of Cochrane Reviews, that is the metaphysics of causation should be tied to methods on the one hand but must be equally tied to use in the target situation. These debates propose a broader approach to evidence and causal assumptions, and will be presented for discussion.

to date has assessed the association of these factors with the strength of recommendations. Objective: Empirical evaluation of the impact of these factors on the strength of recommendations in the setting of development of the guidelines for the use of platelets by the AABB (formerly, the American Association of Blood Banks). Methods: AABB convened a 17 member, multidisciplinary panel to develop guidelines for the use of platelets for 11 different clinical indications. One week after the meeting, panel members used a web-based survey tool to document their judgments of the strength of recommendation and all four GRADE factors. We used both a multilevel mixed effect logistic regression model and a simpler regression model to assess the relationship between the strength of recommendation and the GRADE factors. The results were similar, and we here present findings from the simpler model. We used Cronbach alpha to assess the consistency in rating of GRADE factors. Results: Cronbach alpha ranged from poor for judgment of the balance of benefits and harms (0.55) to good for judgment on the patients’ values and preferences (0.73). The quality of evidence was the only statistically significant (OR = 4.5; p < 0.001) GRADE factor associated with the strength of recommendations: the higher the quality of evidence, the higher the probability of a strong recommendation. The predictive model (Fig. 1) suggested an approximately 90% probability of a strong recommendation when confidence in the effects of intervention was high, and 10% when the quality of evidence was very low. Conclusion: In this first comprehensive empirical study, we show that a guideline panel’s confidence in estimates of effect is the major determinant of making a strong versus weak recommendation. This confirms the fundamental EBM view that when the quality of evidence is high, evidence indeed can serve as a neutral arbiter among competing views.

Oral session 18 GRADE and guidelines The quality of evidence is a key determinant of strength of recommendations Djulbegovic B1 , Kaufman R2 , Tobian A3 , Kumar A4 , Mhaskar R4 , Guyatt G5 1 Univeristy of South Florida, USA; 2 Brigham and Women’s Hospital, Boston, USA; 3 Johns Hopkins University, Baltimore, USA; 4 University of South Florida, USA; 5 McMaster University, Canada

Background: According to the GRADE system, guidelines panels consider four factors when making recommendations: quality of evidence, balance between the intervention’s benefits and harms, resource use and patient’s values and preferences. However, no study Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Use of information from summary of findings tables in the clinical practice guidelines development: a pilot study in Colombia ´ Rojas MX1 , Rodriguez A1 , Gomez C1 , Santesso N2 1 Pontificia Universidad Javeriana, Colombia; 2 McMaster University, Canada

Background: The goal of the inclusion of GRADE (Grading of Recommendations Assessment, Development and Evaluation) summary of findings (SoF) tables in Cochrane Reviews (CRs) is to provide a Cochrane Database Syst Rev Suppl 1–150 (2014)

28 concise and transparent summary of the key findings of the review. The SoF tables in CRs differ from GRADE evidence profiles (GP) in their detail and quality; while the GP present it in detail, the SoF table only provides the key information needed for decision-making. How useful the guideline developing groups (GDG) find the SoF tables as they are currently presented in the CRs remains uncertain. This study aimed to describe the experience of using the SoF tables from CRs by the Colombian GDGs that have adopted the use of GRADE as part of the guideline development. Objectives: To assess the understanding of SoF tables and GP information by GDGs. To describe the use that GDGs are doing with SoF tables and GP information in the formulation of recommendations. To identify: (i) how much technical teams and GDGs use the SoF tables of Cochrane reviews in the preparation of information for panel members; (ii) the reasons for not using SoF tables of CRs or for developing new GRADE profiles based in the same CRs. Methods: We conducted a semi-structured interview with individuals and groups that have applied the GRADE approach in the context of guidelines development. From the national register of GDGs, we identified and invited the eligible participants to take part in the study. Personal interviews were conducted relating to specific CRs or guidelines to assess the understanding the SoF tables and the application of GP results in the formulation of their recommendations. All interviews responses were recorded and written interview summaries were validated with the participants. Results: As part of a pilot study we have performed 12 interviews to technical team members and 8 to members of GDG. The responses obtained from implementing the above mentioned methods will be presented. Conclusions: Results provide useful information for developing future larger survey in GDGs from other organizations and generating actions for improving the CRs.

Exploring why WHO guideline panels are frequently making strong recommendations based on low study quality (high uncertainty) Alexander P1 , Norris S2 , Bero L3 , Montori VM4 , Brito JP4 , Stoltzfus R5 , Djulbegovic B6 , Neumann I1 , Gionfriddo M4 , Guyatt G1 1 McMaster University, Canada; 2 Guidelines Review Committee Secretariat, WHO, Switzerland; 3 University of California San Francisco, USA; 4 Mayo Clinic, USA; 5 Cornell University, USA; 6 University of South Florida, USA

Background: The GRADE approach to guideline development entails grading the guideline recommendations as strong or weak (conditional) and rating confidence in effect estimates of intervention (quality of evidence) as high, moderate, low and very low (l/vl). GRADE suggests guideline panels seldom make strong recommendations based on l/vl confidence. Between 2007 and 2013 World Health Organization (WHO) guideline developers using GRADE had 60% of their recommendations as strong, and 60% of these based on l/vl evidence. Using a GRADE-developed taxonomy, we classify these recommendations as either optimal or a number of categories of sub-optimal. Objectives: We are now in the process of interviewing WHO guideline panel chairs to explore the thought processes of their panels in making strong recommendations based on l/vl confidence. Methods: We will use examples of strong recommendations that we have classified as best practice (and thus inappropriate for grading); incorrect classification of confidence as l/vl when moderate is warranted; and recommendations better classified as conditional on the basis of uncertainty in effects. Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

One hour duration, one-on-one, semi-structured interview format (15 questions) will seek guideline panelists’ thinking and considerations they used to arrive at these strong recommendations. Results: Full results will be available by September 2014. Conclusions: A clear understanding of when and why strong recommendations are being formulated at WHO when the uncertainty is high is an important methodological issue that has implications not just for WHO, but for a wide variety of guideline development groups internationally. It may be that guideline developers perceive weak or conditional recommendations as unhelpful, considerations such as equity and local values may drive their decisions, or they may be inappropriately neglecting the uncertainty and making recommendations in accordance with their personal views and current policies and practice. The findings may warrant modifications of the GRADE approach, increased education of guideline panelists, increased attention to conflict of interest, or some combination.

Understanding the determinants WHO guideline panels consider in making strong recommendations in the face of low or very low confidence (study quality) in effect estimates Alexander P1 , Norris S2 , Bero L3 , Brito JP4 , Neumann I1 , Djulbegovic B5 , Montori VM4 , Rave S6 , Gionfriddo M4 , Guyatt G1 1 McMaster University, Canada; 2 Guidelines Review Committee Secretariat, WHO, Switzerland; 3 University of California San Francisco, USA; 4 Mayo Clinic, USA; 5 University of South Florida, USA; 6 Health Canada, Toronto, Canada

Background: In 2007, the World Health Organization (WHO) adopted the GRADE approach to guideline development which characterizes evidence confidence in estimates from high to very low, and recommendations as strong or weak. Because low or very low confidence in estimates (l/vl) is associated with recommendations sensitive to values and preferences, GRADE suggests that guideline panels make strong recommendations based on l/vl sparingly. We have previously developed a taxonomy of strong recommendations based on l/vl comprising: (i) five identified paradigmatic situations in which strong recommendations are warranted despite l/vl; (ii) best practice recommendations; (iii) misclassification of confidence in estimates; (iv) unwarranted strong recommendations. Objectives: To characterize WHO strong recommendations based on l/vl confidence in estimates. Methods: We previously reported on 456 WHO GRADEd recommendations published 2012 or earlier, of which 63% were strong and, of these, 55% based on l/vl confidence. We are in the process of characterizing these recommendations according to our taxonomy. We have re-examined WHO guidelines for 2013 and found that 70% of recommendations were strong and of these, 76% based on low or very low confidence in effect estimates. Results: Thus far we have noted that all four categories are well represented. Full results will be available by September 2014. Conclusions: Our examination of the guideline documents may reveal that: (i) WHO recommendations are based on reasonable judgements and consistent with GRADE guidance; (ii) recommendations may not have been formulated with correct applications of the GRADE approach and panellists judgements were incorrect; or (iii) even if the determination is accurate, the reasoning of the panel may highlight limitations in GRADE guidance. Understanding Cochrane Database Syst Rev Suppl 1–150 (2014)

29 when and why strong recommendations are being formulated at WHO with l/vl is an important methodological issue that has implications not just for WHO, but for a wide variety of guideline development groups internationally. This project will help identify potential educational needs of panellists regarding optimal use of GRADE.

Oral session 19 Review methods: miscellaneous Multiple systematic reviews published on the same topic area: an analysis of systematic reviews that overlap in content Foisy M1 , Dryden DM1 , Fernandes RM2 , Hartling L1 1 Alberta Research Centre for Health Evidence, University of Alberta, Canada; 2 Clinical Pharmacology Unit, Instituto de Medicina Molecular, Portugal

Background: The Cochrane Collaboration attempts to avoid duplication of effort by publishing only one systematic review (SR) per topic of interest. However, there are often many non-Cochrane reviews published on the same topic. Objectives: To quantify and describe the overall body of SR evidence published in different clinical areas, including an analysis of Cochrane and non-Cochrane SRs that overlap in content. Methods: We chose five topics (eczema prevention, bronchiolitis, alcohol withdrawal, acute severe asthma, procedural sedation) and searched for all Cochrane and non-Cochrane SRs that met pre-defined inclusion criteria. We extracted data on the scope, methods, included studies, results, conclusions, year of publication, and quality of each relevant SR. Results for eczema prevention are presented here. Complete data for other topics will be presented at the Colloquium. Results: There were 30 SRs published on eczema prevention. Six Cochrane SRs included a total of 65 trials and 24 541 subjects; 24 non-Cochrane SRs included 806 studies and 945 234 subjects. Fifteen interventions were examined across the 30 SRs, and five interventions were examined in four or more SRs (including a Cochrane SR; see Table 1). In each cluster the Cochrane SR was never the most recent but was often the highest quality SR (as assessed by AMSTAR). It was not uncommon for overlapping SRs to have discordant results and conclusions. Conclusions: For each SR, it is important to examine the characteristics (population, intervention, comparator, outcomes, study designs, setting), quality (using AMSTAR), and included studied to determine the amount and nature of the overlap across SRs. Some SRs that included a large proportion of overlapping studies came to entirely different conclusions, while other SRs that included very different studies came to similar conclusions. The large number of SRs published in a single topic area indicates duplication of effort and warrants further examination. These findings also have implications for inclusion of non-Cochrane SRs in overviews of reviews. Restricting overviews to Cochrane SRs may not provide a comprehensive synthesis of all treatment options.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Conducting overviews of reviews: lessons learned since 2006 Foisy M1 , Thomson D2 , Dryden DM1 , Fernandes RM3 , Hartling L1 1 Alberta Research Centre for Health Evidence, University of Alberta, Canada; 2 Cochrane Child Health Field, University of Alberta, Canada; 3 Clinical Pharmacology Unit, Instituto de Medicina Molecular, Portugal

Background: Overviews of reviews, previously called ”umbrella reviews”, are now indexed in the Cochrane Database of Systematic Reviews. Overviews compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. The Child Health Field has produced and published overviews regularly since 2006. Objectives: To summarize and describe our experience conducting overviews, and to provide guidance on the steps involved. Methods: Descriptive summary of our experience conducting over 30 overviews, including recommendations and practical considerations for issues commonly encountered. Results: Overviews are broader in scope than any individual SR, but use methods similar to SRs. Searching, screening, and inclusion are typically more straightforward at the overview level, but procedures differ based on whether non-Cochrane SRs are included alongside Cochrane SRs. There are often a number of relevant non-Cochrane SRs that may overlap in content with relevant Cochrane SRs. In these cases, extracting information from each SR about the scope (population, intervention, comparison, outcomes, study designs, setting), quality (using AMSTAR), year of publication, and included studies can make it easier to determine which SRs would be most appropriate for inclusion in the overview. Steps must be taken to describe the characteristics and methodological quality of the included SRs, as well as the studies within those SRs. Extraction and re-analysis of study data from SRs should be driven by a well-defined clinical question, and not by the individual SR data. We encountered challenges when including non-Cochrane SR in overviews and when grading the quality of evidence. Conclusions: Overviews bring together the highest quality evidence into a ”friendly front end” for decision-makers. However, including non-Cochrane SRs in overviews is often challenging, and many issues stem from reliance on methods and decisions made by authors at the SR level. We are currently conducting research on overview methodology and will use this research to develop evidence-based guidance for the conduct and reporting of overviews.

Cochrane Database Syst Rev Suppl 1–150 (2014)

30 Risk of bias: are judgements consistent between reviews? Lensen S, Farquhar C, Jordon V University of Auckland, New Zealand

Background: In Cochrane Reviews (CR) studies at a high or unclear risk of bias (RoB) are often down-graded or removed in sensitivity analyses, impacting on review conclusions. It is not clear whether review authors are consistent in their approach to these assessments. Objectives: To assess the consistency in RoB judgements across CR in the field of subfertility. Methods: Eighty four reviews were identified which comprised 981 included studies. There were 46 studies that appeared in at least two reviews (duplicates). RoB judgements (low-risk/yes, unclear, high-risk/no) and supporting text were extracted where possible for each of the following domains: sequence generation, allocation concealment, blinding, incomplete outcome data and selective reporting. Results: Of the 46 duplicates, only 21 had judgements for all five bias domains, resulting in varying denominators for each domain (Table 1). Overall, 46% of bias judgements differed between reviews. This inconsistency varied across domains; 29% of judgements on sequence generation differed compared to 65% of blinding assessments. Of interest is whether the inconsistencies change the judgement from low to unclear/high-risk (or vice-versa), and thus shift the study between subgroups in a sensitivity analyses. Overall, 35% were differences that would change the subgrouping (range 22–54% across domains). An explanation for differences may have been changing recommendations to authors over time. However, only 23% of differences were in reviews published using both Revman 4 and 5. Of all instances with different bias judgements, 27% supported their differing judgement with similar or identical reasoning; 73% of the time the support for the judgement was based on different information, potentially due to unmatched author correspondence, or a misunderstanding of either the bias domain or content from the papers. Conclusions: Here is a large amount of inconsistency in RoB judgements in subfertility reviews, the majority of which change the study from a low to unclear/high-risk of bias. Improved consistency across reviews could be achieved by updating and linking RoB judgements to studies within the CRS.

meta-analyses of such outcomes frequently require the re-analysis of individual participant data (IPD), an approach which is considered to be ‘gold standard’. However, procedures for obtaining IPD are time-consuming, resource intensive and often unsuccessful due to unavailability of data or concerns regarding collaboration and data sharing. If IPD cannot be obtained for a study, it may have to be excluded from meta-analysis, potentially introducing bias into the results of a review. Methods: Since 1997, the Cochrane Epilepsy Group has undertaken an IPD approach for seven pairwise Cochrane reviews and recently for an overview of reviews and network meta-analysis. For all studies identified as eligible for these reviews, an IPD request was made to the lead/corresponding author or the sponsoring organisation if a study was industry funded. Results: Within the original seven reviews published between 2000 and 2006, we successfully obtained 78% of IPD for 5363 participants from 30 studies. Following IPD requests for review updates and the overview of reviews from 2011 to 2014, to date, we have obtained IPD for only 299 participants from four studies out of 7811 eligible from 37 studies (4%). Requests also include nine ongoing industry based requests for data of 4819 participants, some which have been outstanding for over 2 years. The poor response rate has substantially delayed us from updating our reviews and the proportion of IPD available for inclusion in some reviews has dropped to below 50%. Review specific results and the potential impacts of missing IPD will be presented Conclusions: The Cochrane Epilepsy Group has previously had a high success rate in obtaining IPD for Cochrane Reviews however recently our success rate has declined. We are uncertain regarding the reason for this decline, how to improve our success rate and the impact on our current and future IPD reviews.

Oral session 20 Learning and capacity development Strengthening capacity for producing systematic reviews in low and middle income countries Bangpan M1 , Stansfield C1 , Dickson K1 , Vigurs C1 , Stewart R2 , Oliver S1 1 EPPI-Centre, Institute of Education, London, UK; 2 University of Johannesburg, South Africa

Data sharing: is it getting easier to access individual participant data? Experiences from the Cochrane Epilepsy Group Nolan S, Marson A, Tudur Smith C Cochrane Epilepsy Group, UK

Background: Inadequate and un-standardized reporting of time-to-event (censored) outcomes and statistical analyses in individual randomized controlled trials is well documented. Therefore, Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: Systematic reviews are increasingly recognized as important for policy decisions across sectors. However, global capacity is variable and reviews address fewer policy dilemmas than clinical practice questions. Methods: Information and reflections on producing systematic reviews in low and middle income countries (LMICs) were sought from review funders, review support organisations and review authors. Current capacity and capacity strengthening were considered for: individual authors, review teams, institutions that fund, support or conduct systematic reviews; networks and systems to support systematic reviewing internationally. Interviews with systematic reviewers and policy makers focused on the relevance of reviews. Results: The largest network of a skilled workforce and established centres is The Cochrane Collaboration. Smaller networks provide specialist skills in the production of reviews beyond health care and beyond questions about the effects of interventions. Participants can only make good use of training programmes that are closely aligned with reviews in progress. Developing individuals with key skills in organisations where systematic reviews are not seen as valuable Cochrane Database Syst Rev Suppl 1–150 (2014)

31 is challenging and can have negative consequences if those skills are used to boost careers by moving out of LMICs rather than conducting systematic reviews in and for LMICs. Systematic searching faces challenges in terms of: access to appropriate sources of research, functionality of bibliographic databases, and developing technical skills when frequently interrupted by poor internet connectivity. Capacity development needs a coherent system of supply and demand. Yet producers and users of systematic reviews often struggle to engage with each others’ worlds. Conclusions: Investment needs to consider institutions’ current capacity and readiness to change. People in LMICs are well placed to consider the feasibility of technical and social solutions for strengthening research capacity, with people in high income countries taking a supportive role. Capacity development for policy-relevant reviews requires culture change on both sides of the research-policy divide.

Primer in systematic reviews and research synthesis: helping people to find, appraise, interpret and use systematic reviews in Africa Young T1 , Garner P2 1 Stellenbosch University and South African Cochrane Centre, South Africa; 2 Liverpool School of Tropical Medicine, UK

Background and objectives: Cochrane centres and other Cochrane groups are continually running protocol or review workshops, however the main gap in many countries is actually to understand and read systematic reviews. We put together an intensive 4-day course based on existing Cochrane reviews and offered it to institutions in Africa, and reported on the experience. Implementation: Host organisations arranged all the course logistical details and skilled Cochrane trainers simply facilitated the course. Through seminars and group work participants are equipped with knowledge and skills to find, understand, appraise and use systematic reviews of effects of interventions using Cochrane reviews relevant to the host country or institution. An online learning management site contains all the material of the course, allows for engagement with participants before, during and after the course. Course evaluations inform the ongoing refinement of the offering. More than 70 participants have attended the course in Malawi, Namibia, South Africa and Tanzania, and for many it was the first time to read a systematic review. Participants enjoyed the participatory nature, use of relevant examples, blended teaching approach and teaching style, and called for expansion and roll-out of the workshop to reach a wider audience. Conclusions: Increasing people’s understanding and use of systematic reviews is essential in promoting evidence-informed practices and to identify relevant new research questions. Experienced Cochrane authors are well placed to do this. We propose that centres, branches and individual experienced authors seek opportunities to provide training in finding, critically appraising and interpreting systematic reviews, and that relevant material to low and middle income countries is stored centrally in an open access format.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

GAME-IT: explore peoples choices and facilitate teaching by turning evidence processing into a game Brandt L1 , Magdin V2 , Luders P2 , McCallum S2 , Kristiansen A1 , Vandvik PO1 1 MAGIC-Making GRADE the Irresistible Choice, Norway; 2 Game Lab, Gjøvik University College, Norway

Background: The process of turning evidence from systematic reviews into actionable recommendations when developing guidelines according to standards for trustworthiness can be difficult to understand and learn; both for participating clinical experts, physicians and lay people. We also know little about what individual clinicians, patients or healthy lay people would have recommended if they were provided with the same evidence as the experts. When guidelines and evidence data are stored as data elements the content can be used to make an interactive game that can be used to learn guideline methodology and harvest data about peoples choices. Objectives: To develop and test perceived usefulness of an ‘evidence processing game’ which can be used to: (i) practically learn the principles of applying evidence made according to standards for trustworthy guidelines. (ii) harvest data about users’ choices when they are presented with the original evidence. (iii) give feedback to systematic reviewers about people’s choices when give their guideline content. Methods: We used modern game technology to make an online recommendation making game based on structured guidelines published in the MAGIC (Making GRADE the Irresistible Choice) authoring and publication platform (www.magicapp.org). Testing will be done as observations using qualitative research methods. Results: We will display the game at the conference, along with the initial testing and feedback. Discussion: Does clinical experts’ reasoning match that of people playing the game? Can we make evidence process training fun? Implications for guideline developers/users: GAME-IT brings a new way to harvest information regarding values and preferences in decision making, and it can potentially make evidence processing training easier.

Engaging with younger generations through Students 4 Best Evidence: the follow-up Millward H, Burton M UK Cochrane Centre, UK

Background: Students 4 Best Evidence (S4BE) is an online community for students interested in evidence-based health care, with the purpose of helping students to learn more about evidence-based practice and the concepts involved. S4BE aims to involve students from school age to university through relevant, useful resources as well as being a space for them to communicate their knowledge and interact with fellow students. S4BE was introduced to Cochrane at the 2013 Colloquium in Quebec and formally launched in December 2013. S4BE is supported by Cochrane UK and partnered with 24 organizations, including Testing Treatments interactive English, IDEAL Collaboration, Sense About Science and Radboud University Medical Center in The Netherlands. Presentation objectives: We want to present a follow-up report on the community’s progress since the Cochrane Colloquium in 2013 and discuss how the community has grown and evolved with the help of the partners. We would also like to share ideas for the Cochrane Database Syst Rev Suppl 1–150 (2014)

32 future of S4BE and how we can increase student engagement not only with S4BE but also with evidence-based health care as a whole. Results: Following the formal launch of S4BE in December 2013 and the subsequent sign-up of partner organizations, we would like to see even more students engaging with each other through the website and social media platforms as well as with partner organizations and the projects that have evolved from the partnerships. Conclusions: For Students 4 Best Evidence to be a key website for all students globally interested in evidence-based health care.

Oral session 21 Reporting of trials and research (2)

Changing patterns of clinical trial activity in Australia: 2005–2013 Smith E1 , Ko H2 , Hunter K2 , Askie L2 1 University of New England Medical Program, Australia; 2 NHMRC Clinical Trials Centre, University of Sydney, Australia

Registration of randomized controlled trials published in PubMed indexed journals: a cross-sectional analysis 1

2

3

2

1

1

Odutayo A , Emdin C , Hsiao A , Rahimi K , Hopewell S , Altman D 1 Centre for Statistics in Medicine, University of Oxford, UK; 2 George Institute for Global Health, UK; 3 Department of Economics, University of Oxford, UK

Background: The registration of clinical trials is an ethical and scientific imperative. It assures accountability for the conduct of ethical research and reduces duplication and biased reporting. The World Health Organization and the International Committee of Medical Journal Editors (ICMJE) have introduced mandatory trial registration policies since 2005 but few countries have supported these efforts with similar legislation. To date, it is unclear how many unregistered randomized controlled trials (RCTs) are conducted and subsequently published by scientific journals. Objectives: To determine the frequency of reporting of RCT registration by country and by ICMJE member status, without restriction by journal, disease or specialty. Methods: We identified and reviewed all primary reports of RCTs published in December 2012 and indexed in PubMed by 17 November 2013. There were no language restrictions. Reviews of abstracts, full text and data extraction were performed in duplicate. Sixty two non-English articles were reviewed by the same person on two occasions, several weeks apart. Country specific data were analyzed for locations wherein >40 RCTs were conducted. Registration rates in ICMJE and non-ICMJE journals were compared by a chi-squared test. Results: Four thousand one hundred and ninety abstracts were reviewed, 1370 primary RCTs were identified and 560 (40.9%) RCTs were reported as registered in the final publication. Among countries with >40 RCTs (Fig. 1), trial registration was highest in the United States at 53.2% and lowest in China at 18.6%. ICMJE journals published 297 (21.7%) RCTs and the proportion of registered trials was significantly higher than in non-ICMJE journals [185 (62.3%) vs. 375 (35.0%), p < 0.001]. 153 613 participants (34.2%) were randomized for RCTs that were not reported as registered. Conclusions: Irrespective of medical specialty, journal or disease, reporting of RCT registration remains problematic. The majority of RCTs are published in non-ICMJE member journals, limiting the broad applicability of ICMJE policy. Country specific legislation and efforts by national research authorities are needed to achieve universal reporting of trial registration.

Background: In the past decade there has been a worldwide initiative to make all clinical trials public. With the establishment of clinical trials registries, such as the US based ClinicalTrials.gov in 2000 and the Australian New Zealand Clinical Trial Registry (ANZCTR) in 2005, researchers and consumers are now able to find which clinical trials are taking place across all areas of health. This not only protects against publication bias and selective reporting, but also provides a valuable resource to evaluate trends in clinical trial activity. Objectives: The aim of this study was to analyse the trends over time for clinical trial activity in Australia, including the number of trials across all disease areas, the types of interventions being evaluated, the size of clinical trials being conducted and the types of funding sources. Methods: Data was sourced from ClinicalTrials.gov and ANZCTR for all registered intervention trials that had at least one Australian recruitment site. These registries combined cover 98% of registered trials in Australian sites (Dear, MJA 2011). The search results were exported as XML files and then converted into MS Excel spreadsheets, where they were cleaned and coded for analysis. Results: Data were extracted for a total of 7891 registered trials. The total number of Australian trials registered annually on ANZCTR/ClinicalTrials.gov has steadily increased over the last 9 years, from 567 in 2005 to 1028 in 2013. There was a 30% decrease in median sample size from 150 in 2005 to 84 in 2013. The proportion of industry trials has decreased from a peak of 55% in 2007 to 39% of trials commencing in 2013. Studies evaluating behaviour and lifestyle interventions have increased sixfold from 26 in 2005 to 185 in 2013. The top condition category studied was cancer, and mental health has overtaken cardiovascular disease as the category with the second most numerous trials. Conclusions: A trend towards smaller sample sizes underscores the importance of meta-analyses to enable effects of interest to be detected. Other patterns in clinical trial activity may have important implications for systematic review planning and methodology.

Editorial policies and trial registration status for published trials in Indian medical journals Bhaumik S, Kirubakaran R, Matthew RJ, Tharyan P South Asian Cochrane Network & Centre, India

Background: Clinical trial registration enhances transparency, identifies reporting biases and increases public accessibility of trial Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

33 results. Trial registration has been mandatory since 2009 for trials seeking licensing of drugs in India. Editors of 11 Indian medical journals jointly published a statement that prospective trial registration would be mandatory for all trials submitted from January 2010 (Satyanarayana et al. Indian J Med Res. 2008;127:104-5). Objectives: To evaluate the impact of the editorial policy on mandatory registration on trials published in journals that were, and were not, signatory to this policy. Methods: In August 2013, we assessed the instructions to authors of 63 Indian journals for their editorial policies regarding prospective trial registration. We identified controlled clinical trials published in these journals from January 2010 to August 2013. We extracted and analysed data on the type of trial, year of publication, country of origin, registration details and mention of trial registration number in the published article. We also searched the clinical trials registers (CTR) for registration details of the identified trials. Results: Only 21/63 (32%) of journal policies mandated clinical trial registration as a prerequisite for publication. During the period under survey, 686 controlled clinical trials were published in 58 of the 63 journals. Only 63 (9%) were registered in a CTR. Of the 560 trials (82%) conducted in India, only 48 (9%) were registered in a CTR. Only 15 (12%) of the 126 trials conducted outside India were registered in a CTR. Of the 63 trials which were registered, the trial registration number was not mentioned in 22 trials (35%). Only 14 of the 115 (12%) trials published in 10 of the 11 journals that were signatory to the mandatory trial registration statement were registered. Conclusion: Prospective registration of clinical trials in Indian medical journals continues to be substandard. This raises doubts about the validity of the results of the trials that were not properly registered.

Transparent reporting of a multi-variable prediction model for individual prognosis or diagnosis: the TRIPOD statement Moons K1 , Altman D2 , Reitsma J1 , Collins G2 1 University Medical Center Utrecht, The Netherlands; 2 Centre for Statistics in Medicine, University of Oxford, UK

Background: Prediction models are developed to aid healthcare providers in estimating the probability that a specific outcome or disease is present (diagnostic models) or will occur in the future (prognostic models), to inform their decision-making. Clinical prediction models are abundant in medical literature. Some disease areas show an overwhelming number of competing prediction models (sometimes even in excess of 100) for the same outcome or target population. Only when full information on all aspects of a prediction model study are clearly reported can risk of bias and potential usefulness of the prediction model be adequately assessed. Many reviews have shown that the quality of published reports on the development, validation and updating of prediction models, is very poor. Objectives: The transparent reporting of a multi-variable prediction model for individual prognosis or diagnosis (TRIPOD) initiative therefore developed a set of consensus-based recommendations for the reporting of studies developing, validating or updating a prediction model, whether for diagnostic or prognostic purposes. Methods: The development was based on systematic reviews of the literature, web-based surveys and a 3-day expert meeting among methodologists, healthcare professionals and journal editors. Results: The TRIPOD checklist includes 22 Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

items deemed essential for transparent reporting of a prediction model study. The development and contents of the TRIPOD checklist will be presented and illustrated, along with empirical evidence and rationale for their inclusion. Conclusions: The TRIPOD statement intends to improve the transparency and completeness of reporting of studies that report the development, validation, or updating of a diagnostic or prognostic prediction model.

Oral session 22 Review production and dissemination Trial data available in FDA and EMA reports: a cross-sectional study Schroll J1 , Abdel-Sattar M2 , Bero L2 1 Nordic Cochrane Centre, Denmark; 2 University of California San Francisco, USA

Background: Drug approval reports have been made publicly available by both the European Medicines Agency (EMA) and the Food and Drug Administration (FDA). To what extent data are useful is uncertain. Objectives: The purpose of this study is to determine whether information publicly available from EMA and FDA is easily accessible, provides useful data on efficacy and safety, and whether the information from the two agencies differs. Methods: We conducted a cross-sectional study by sampling all new molecular drugs approved between January 2011 and December 2012 from the FDA and EMA websites and pairing identical drug reports from the two agencies. Two researchers independently extracted data about identifiers, efficacy and harms from each report. Results: We included 27 pairs of drug reports. Almost all were searchable but the table of contents in FDA reports often did not match the electronic documents’ page numbers. The reports contained information about trial methodology but did not include trial registry IDs or investigator names. All but one report contained sufficient information, including efficacy data, to be used in a meta-analysis. Comprehensive information about harms was reported for 93% of the FDA reports (25 of 27 reports) and 26% of the EMA reports (7 of 27 reports). Even though it was beyond our objective, we noticed that the approved indications differed substantially for 15 of the 27 drugs. In 14 of these cases, the EMA was more restrictive. Conclusions: Comprehensive data on efficacy and harms are made available by the two agencies. The FDA has more data on harms but the documents are harder to navigate. The FDA seems to approve wider indications for drugs than the EMA.

How can we accurately diagnose schizophrenia? Challenges in performing Cochrane diagnostic test accuracy reviews in schizophrenia Maayan N1 , Adams CE2 , Bergman H1 , Davenport C3 , Palaniyappan L4 , Soares-Weiser K1 1 Enhance Reviews, UK; 2 Cochrane Schizophrenia Group, UK; 3 University of Birmingham, UK; 4 University of Nottingham, UK

Background: Since the 1950’s researchers have been trying to standardize the way schizophrenia is diagnosed. We recently completed Cochrane Database Syst Rev Suppl 1–150 (2014)

34 six Cochrane Reviews assessing the accuracy of four checklists (First Rank Symptoms, OPCRIT+, K-SADS and CIDI), magnetic resonance imaging (MRI) and blood tests to diagnose schizophrenia. Objectives: To report the challenges we encountered while performing diagnostic test accuracy (DTA) reviews in schizophrenia. Challenges: (i) Despite using wide search parameters, we were unable to find many studies whose primary aim was to test diagnostic accuracy. This is perhaps not surprising for older checklists, but also newer, widely used checklists lack DTA studies, as do the very newest proposed diagnostic techniques. (ii) As in most mental health disorders, there is no objective reference standard for diagnosing schizophrenia. Instead, diagnosis is made by a psychiatrist based on clinical interviews and patients’ medical history, but disagreements are common among psychiatrists. We provided some insight into whether research should focus on a single test (clinical examination) or a combination of tests (e.g. clinical examination + checklists + MRI). (iii) Our main challenge was how to apply DTA methods, particularly quality assessment. Most of the included studies did not state whether all participants had psychotic symptoms; often a diagnosis of schizophrenia was retrospectively compared to the participant’s clinical examination. Some studies included all admissions to psychiatric close wards, and we assumed that the majority of patients would have psychotic symptoms. Often details of the reference standard were unclear, with studies simply stating that participants were diagnosed according to DSM or ICD diagnostic criteria. Conclusions: There is a lack of research assessing the diagnostic accuracy of tests in schizophrenia, which is probably due to a lack of understanding to what causes schizophrenia. In the real world diagnoses have to be made and the lack of appropriate studies might result in new tests being ‘crept’ into practice in the absence of evaluation and without regard to the consequences of erroneous diagnoses.

Anywhere Cochrane Reviews: responsive web design, the ‘connected article’ and Cochrane evidence Toler T1 , Joyce S2 , Pentesco-Gilbert D2 1 John Wiley & Sons, USA; 2 John Wiley & Sons, UK

Background: Responsive Web Design (RWD) is one of the prime concepts behind mobile internet access. What this provides to the user or reader is content that is enabled to be responsive to its device, optimizing their use of the information particularly to improve their mobile internet experience. After conducting extensive research to understand how users want to interact with HTML and PDF versions of articles on both large and small screens, Wiley developed two new

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

connected article formats: Anywhere Article (AA) and the ReadCube Enhanced PDF (ePDF). With the AA and ePDF, supporting information is integrated into the article environment, making for a less cluttered and clearer content. The focus is on presentation beyond the standard PDF format. Objectives: Wiley plans to introduce the AA and ePDF experience to systematic reviews in the Cochrane Database of Systematic Reviews (CDSR) in 2014 and to improve the readability, portability and adoption of Cochrane Reviews. In a nutshell, this will provide the reader with a satisfying experience without obstacles and clutter of traditional html and pdf publishing. Methods: A redesign of Cochrane Reviews along AA and ePDF lines will be implemented for the full CDSR alongside the current implementation of the HTML and PDF rendering. The resulting usage will be analysed and user feedback reviewed. Results: In 2012 and 2013, PDF was higher than HTML usage of full text Cochrane Reviews. The results of the introduction of AA responsive web design will be evaluated to consider the impact on the HTML usage and reader satisfaction, including portability. Conclusions: The key focus for the article HTML design and implementation is ease of use on the move and to be readable. The introduction of the AA experience is the start to improving, expanding and updating the Cochrane Review.

Open access and Cochrane Reviews Tovey D1 , MacLehose H2 , Pentesco Gilbert D3 , Binder L4 1 Editor in Chief, The Cochrane Library, UK; 2 Senior Editor, The Cochrane Library, UK; 3 John Wiley & Sons, UK; 4 Cochrane Central Executive, Germany

Background: ‘Open access’ is a way of publishing scholarly research that makes it available unrestricted online. ‘Open access’ for Cochrane means that authors can choose to make their Cochrane Review or Protocol open access in the Cochrane Database of Systematic Reviews (CDSR) immediately upon publication (gold open access), while authors of all other reviews and protocols will see them go free to view 12 months after publication (green open access). Cochrane is also committed to developing a roadmap for achieving universal open access to new and updated Cochrane Systematic Reviews by the end of 2016. On 1 February 2014, Cochrane had its first anniversary of providing ‘open access’. Objectives: The objectives of this presentation are to provide a background to open access and the evolving open access environment, report on the uptake of open access across Cochrane Reviews, and progress on the development of the open access roadmap for all Cochrane Reviews.

Cochrane Database Syst Rev Suppl 1–150 (2014)

35

Poster Session 1 P1: National priority setting of guideline development for chronic disease management in Korea Jo HS1 , Oh M2 , Kim DI3 , Lee YK4 1 Kangwon National University, Korean Academy of Medical Sciences, South Korea; 2 Kangwon National University Hospital, South Korea; 3 Yonsei University, South Korea; 4 Soonchunhyang Univerisity Buchein Hospital, Soonchunhyang University, Korea South

Background: The prevalence of chronic disease has increased rapidly in South Korea, but the level of appropriate management for patients with chronic diseases is low. Objectives: This study aimed to select the priority topics among chronic diseases in South Korea that needed guidelines to be developed to ensure the efficient allocation of resources. Methods: The Korean government, the Korean Center for Disease Control, has planned for development of clinical practice guidelines for chronic diseases through cooperation with the Korean Academy of Medical Sciences since 2013. Setting priority topics was the first step in the planning. For this selection, the following activities were carried out in order: (i) setting the criteria for selection of priorities, (ii) constitution of an advisory group, (iii) listing of 20 chronic disease candidates. (iv) yielding the statistical indicators, and (v) Delphi survey. The response rate showed 44%. The survey for priorities was performed with the sum of rankings approach, and the product of actual values approach, by targeted chronic diseases. Results: Six criteria for selection of priorities were selected through international case studies and literature review, these were: disease burden, relevance of national policy, demand for development, the status of developed guidelines, variation in treatment, and healthcare expenditure. The Delphi survey revealed the high priorities to be stroke, ischemic heart disease, back pain, hyperlipidemia, chronic renal failure, prostatic hypertrophy, chronic obstructive pulmonary disease, depression, dementia, asthma. These were variable depending on the experience of the clinical field; the diseases that need treatment guidelines to be developed were stroke, dementia, ischemic heart disease, and hyperlipidemia. Conclusions: It is necessary to use a rational process based on evidence to develop the priorities for guidelines for the management of chronic disease. The development of clinical pathways should reflect the opinions of the medical field such as the developers’ needs and the variation of care in addition to disease burden.

National Eye Institute, we expanded the project to include angle-closure glaucoma (ACG). Findings were published in 2012 and 2013 for POAG and ACG, respectively. Objectives: To assess the progress of the priority-setting project in terms of CEVG review completion. Methods: We surveyed specialists and asked them to rank the importance of having an answer to questions derived from clinical practice guidelines. For POAG, 105 American Glaucoma Society members ranked 9/45 questions, and for ACG, 96 clinicians from Asia-Pacific glaucoma societies ranked 15/42 questions as high-priority topics. We searched for existing CEVG systematic review (SR) topics on all 24 prioritized topics and submitted titles for new SRs to the CEVG editorial base as needed. We assessed the current status of each SR topic as of 20 March 2014. Results: For the nine POAG high priority topics (Table 1): four topics required a new SR, of which three topics were newly registered and one topic was not pursued because it cannot be ethically addressed by randomized controlled trials (RCTs); three topics had existing SRs with inconclusive findings and will be updated as new RCTs are conducted; two topics had high-quality SRs with conclusive findings and will be updated as needed. For the 15 ACG high priority topics (Table 2): four topics did not have registered SRs and were not pursued by the editorial base, of which three topics would be ’empty’ reviews and one topic cannot be ethically addressed by RCTs; 11 topics had existing Cochrane Protocols or SRs in progress. For

P2: Priority setting project for open-angle glaucoma and angle-closure glaucoma: where we are now? Wang X1 , Hui X1 , Lindsley K1 , Li T1 , Yu T1 , Wormald R2 , Dickersin K1 1 Cochrane Eyes and Vision Group, USA; 2 Cochrane Eyes and Vision Group, UK

Background: In 2007, the Cochrane Eyes and Vision Group US satellite was funded by Cochrane to undertake a priority-setting project for primary open-angle glaucoma (POAG). With funding from the Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

36 refraction and accommodation disorders were under-represented. Conclusions: These results prompt further investigation into why certain diseases are over or under-represented in CDSR relative to their DALY. With regard to ophthalmologic conditions, this study encourages that certain conditions should get more focus to create

the 36 POAG and 27 ACG non-priority topics; four topics have been undertaken in new SRs and three review topics have been updated for POAG; one topic has been undertaken in a new SR for ACG (Table 3). Conclusions: CEVG reviews do not cover all specialty society members’ priority topics for Cochrane SRs, especially in ACG. Funder priorities (e.g. no empty reviews) are a non-ignorable factor.

P3: Global burden of eye disease in 2010 and systematic reviews and protocols in Cochrane Database of Systematic Reviews Boyers L1 , Karimkhani C2 , Richheimer B3 , Dellavalle R4 1 Georgetown University School of Medicine, USA; 2 Columbia University College of Physicians and Surgeons, USA; 3 University of Colorado Anschutz Medical Campus, USA; 4 US Department of Veterans Affairs, Colorado School of Public Health, University of Colorado Anschutz Medical Campus, USA

Background: Disease burden should help guide research prioritization. The Global Burden of Disease (GBD) Study 2010 compiled data from 1990 to 2010 on 291 diseases and injuries, 1160 disease and injury sequelae, and 67 risk factors in 187 countries. The Cochrane Database of Systematic Reviews (CDSR) is the leading resource for systematic reviews in healthcare, with peer-reviewed systematic reviews that are published by Cochrane Review Groups. Objectives: To determine whether systematic review and protocol topics in CDSR reflect disease burden, measured by disability-adjusted life years (DALYs) from the GBD 2010 project. This is one of a series of projects mapping GBD 2010 medical field disease burdens to corresponding systematic reviews in CDSR. Methods: Two investigators independently assessed eight ophthalmologic conditions in CDSR for systematic review and protocol representation according to subject content. The eight diseases were matched to their respective DALYs from GBD 2010 (Table 1, Fig. 1). Results: All eight ophthalmologic conditions were represented by at least one systematic review in CDSR; 84% of the systematic reviews and protocols that focused on these conditions were from the Eyes and Vision Group. Comparing the number of reviews and protocols with disability, only cataracts was well-matched; glaucoma, macular degeneration, and other vision loss were over-represented. In comparison, trachoma, onchocerciasis, vitamin A deficiency, and Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

a better representation of what is causing the most disability and mortality within this research database. Our results provide good quality and transparent data to inform future prioritization decisions.

P4: Generalizability of findings from efficacy trials for chronic depression: an analysis of eligibility criteria ¨ ¨ von Wolff A1 , Jansen M2 , Holzel LP3 , Westphal A1 , Harter M1 , Kriston L1 1 University Medical Centre Hamburg-Eppendorf, Germany; 2 Institute for Educational Quality Improvement, Humboldt University Berlin, Germany; 3 Department of Psychiatry and Psychotherapy, University Medical Centre Freiburg, Germany

Background: Concerns have been raised that the rigorous eligibility criteria used to select patients for randomized controlled trials (RCTs) Cochrane Database Syst Rev Suppl 1–150 (2014)

37 limit the generalizability of trial findings.Compared with nonchronic depression, chronic depression is more often characterized by a longer duration of the depressive episode, a family history of mood disorders, and psychiatric comorbidity, and these characteristics are more often found among patients who are excluded from antidepressant efficacy trials. Objectives: The objectives of this study were to identify commonly used eligibility criteria in RCTs for chronic depression, to examine whether these criteria are met by patients with chronic depression who are in routine care, and to identify differences between patients who would and would not meet RCT criteria. Methods: Thirteen eligibility criteria were extracted from eight RCTs of combined psychotherapeutic and pharmacological interventions for patients with chronic depression. These criteria were then applied to a sample of patients with chronic depression receiving care in one of ten German hospitals (N = 231). Demographic, clinical, and treatment characteristics of those who met the RCT criteria and those who did not were compared in univariate and multivariate analyses. Results: Only 25% of the 231 inpatients met all RCT eligibility criteria. Patients were ineligible mainly because of suicide risk, low severity of depression at admission, and concurrent psychiatric or somatic disorders. No statistically significant differences were found between those who met the criteria and those who did not in demographic characteristics, length of inpatient stay, treatment outcome, and efficacy of certain antidepressants, except that slightly more patients meeting RCT criteria received selective serotonin reuptake inhibitors. Conclusions: Findings suggest that the generalizability of RCT findings to routine health care is less limited than frequently supposed.

P5: An approach to disinvestment: an Italian experience D’Amico R1 , Balduzzi S1 , Clivio L2 , Grilli R3 1 Italian Cochrane Centre, University of Modena and Reggio Emilia, Italy; 2 Mario Negri Institute, Italy; 3 Emilia-Romagna Regional Agency for Health and Social Services, Italy

Background: Disinvestment is the partial/complete withdrawal of resources from existing health care interventions that have questionable benefit. Different processes have been developed to identify these interventions, however there is not consensus on which one to use. In 2013, the Italian Ministry of Health established a working group in order to evaluate the actual use of the economic resources in oncology and identify potential areas in which it would be appropriate to start a process of disinvestment. The working group was asked to give an answer in 6 months. Objectives: To develop a simple, transparent, evidence-based process allowing the identification of a list of health interventions that have questionable benefit. Methods: The project’s steps were: (1) involvement of a regional oncologic commission composed of 30 members who were asked to provide a list of potentially dismissible interventions (PDIs); (2) bibliographic research to find evidence regarding the identified interventions –firstly Cochrane Systematic reviews (SRs), secondly non-Cochrane SRs, and thirdly guidelines indexed in PubMed were searched to support the decision; (3) involvement of a group of representatives of clinicians, policy makers and health economists to reach an agreement, through the Delphi technique, on PDIs. Results: The commission identified 37 PDIs (settings: 11 breast cancer; 5 colorectal cancer; 5 lung cancer; Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

5 prostate cancer; 11 other). Cochrane SRs covered 11/37 clinical questions, non-Cochrane SRs 4/37, guidelines 4/37; for 18/37 no evidence was found, and these were not considered in the following step. The representatives reached a consensus on the 19 interventions; on 18 of them there was agreement on their questionable effect, while for one there was still too much uncertainty. Conclusions: The process developed in this study allows a rapid identification of lists of health interventions that are used in clinical practice, but for which the efficacy has not been proven.

P6: Analysis of treatment description in RCTs in physiotherapy Sivaprakash J, Hariohm K, Saravan Kumar J MSAJ College of Physiotherapy, India

Introduction: One analysis found that 67% of drug treatment descriptions were adequate compared with only 29% of non-drug treatments. Physiotherapy is a non-drug health specialty that includes exercise, electrotherapy, thermal treatments etc. Such treatment procedures require clear descriptions of each element followed in the trials in order to be applicable in practice. Objectives: To discover whether description of procedures were improving in trials published in 2013 when compared to trials published in 2005 from the five top, core journals of physiotherapy. Methods: The top five core journals selected were: Physical Therapy, Journal of Physiotherapy, Archives of Physical Medicine & Rehabilitation, Clinical Rehabilitation, Journal of Orthopedic Sports & Physiotherapy. Studies for which full texts were not available were excluded. Randomised controlled trials (RCTs) from five journals were identified and 50% of these were randomly selected and analyzed for the seven items of Setting, Recipient, Provider, Procedure, Materials, Intensity and schedule, with the items being analyzed separately for experimental and control groups for the last four items. The items were marked as ‘Yes - if element of intervention clearly described’ or ‘No - if not adequately described’. An eighth item of ‘Overall’ was given ‘Yes - only if the all other items were marked ‘Yes’ and rated separately for experimental and control groups. Two raters individually rated all the checklists and disagreements were resolved by discussion. Results and discussion: Fifty-nine and 160 RCTs were identified from the five journals for the years 2005 and 2013 respectively. Thirty and 85 RCTs were selected randomly from 2005 to 2013, respectively, and included in analysis. Overall, 43% and 51% of the interventions were adequately described for the years 2005 and 2013, respectively. Conclusion: Description of interventions in physiotherapy literature is adequate. However, there is room for improvement especially in describing the procedure and clearly mentioning the schedule. Key words: non-pharmacology; procedure; methods

Cochrane Database Syst Rev Suppl 1–150 (2014)

38

P7: Methods of evidence mapping: a systematic review Schmucker C1 , Motschall E2 , Antes G1 , Meerpohl J1 1 German Cochrane Centre, Germany; 2 Institute of Medical Biometry and Statistics, University Medical Centre Freiburg, Germany

Background: Evidence mapping is an increasingly popular approach for the systematic evaluation of published research. While there are clear methodological standards for systematic reviews, variability exist between the terminology and methods used within evidence mapping. Objectives: The aim of this systematic review is to describe the methodology and terminology used in evidence mapping and to demonstrate the continuum between evidence mapping and traditional systematic reviews. Methods: A systematic literature search for methodological studies was conducted in 10 databases in order to obtain a comprehensive picture of the state of research standards for evidence mapping. In addition, websites of institutions that are conducting evidence mapping were searched. Results: The included study pool (n = 12) shows that the terms ‘evidence map’ and ‘scoping review’ are widely used within the field of evidence mapping. Evidence maps are an approach to depict both the number and characteristics of studies that exist in tabular form, as well as evidence gaps based on primary studies and systematic reviews of broad clinical questions. Scoping reviews also summarize the literature in a tabular form, but, in addition, give a descriptive narrative summary of the results. A quality assessment of the studies is generally not included. Conclusion: Evidence mapping allows the identification of research gaps. This aspect is particularly important for interventions that are used without sufficient evidence. In contrast, systematic reviews are mainly used to estimate effects for interventions and evaluate whether the included studies are reliable.

P8: Keeping up with the evidence: prioritizing systematic reviews on antiangiogenic therapy for age-related macular degeneration Lindsley K1 , Virgili G2 , Bacherini D3 , Dickersin K1 , Li T1 1 Cochrane Eyes and Vision Group, US Cochrane Center, USA; 2 Cochrane Eyes and Vision Group, Italy; 3 University of Florence, Italy

Background: Treatment for age-related macular degeneration (AMD) has been revolutionized in the past 20 years with the introduction of anti-vascular endothelial growth factor (anti-VEGF) therapies. The Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

field is fast-moving with an information overload in the medical literature. Objectives: To compare published systematic reviews (SRs) of randomized controlled trials (RCTs) of anti-VEGF therapies for AMD, and to assess the degree of overlap (i.e. extent to which SRs cite the same RCTs and evaluate the same interventions) and reasons for discordance among the SRs. Methods: We searched for eligible SRs in a database of eyes and vision SRs developed and maintained by the Cochrane Eyes and Vision Group US Satellite (updated as of 2012). We considered only RCTs included in the SRs when investigating the degree of overlap. We explored the reasons for discordance among RCTs included in SRs, such as the search date, interventions, and follow-up time. Results: We identified 10 SRs that examined the effectiveness and/or safety of anti-VEGF therapy for AMD published between May 2007 and July 2011 (searches conducted between September 2006 and June 2010). Twelve unique RCTs were included across all the SRs (Figure). With the exception of two RCTs published after nine of the SRs, there was high degree of overlap among SRs, with 10 of the RCTs included by at least three reviews (range: 3–8). In terms of specific citations, there was a total of 28 citations for the 11 included RCTs (range: 6–19 citations per SR). Earlier SRs included pegaptanib, whereas later SRs tended to focus on head-to-head comparisons of ranibizumab versus bevacizumab. Conclusions: Publication dates and restriction in eligibility criteria regarding interventions and follow-up times accounted for the majority of discordance among SRs. Concordance of effect estimates and conclusions is worth further investigation. Published within a 5-year span, the SRs added new RCTs as they were published, indicating this is a hot field. Keeping up with the evidence is important. Cochrane should discuss how to ensure timely updates of priority topics given competing demands on resources.

P9: Global burden of heart disease in 2010 and systematic reviews and protocols in Cochrane Database of Systematic Reviews Karimkhani C1 , Boyers L2 , Aksut B3 , Nasser M4 , Hilton J5 , Mavergames C6 , Forouzanfar M7 , Murray C7 , Gray W8 , Dellavalle R9,10,11 1 Columbia University College of Physicians and Surgeons, USA; 2 Georgetown University School of Medicine, USA; 3 Department of Internal Medicine, Columbia University Medical Center, USA; 4 Peninsula Dental School, Plymouth University, UK; 5 Cochrane Editorial Unit, The Cochrane Collaboration, UK; 6 Cochrane Informatics and Knowledge Management Department, Germany; 7 Institute for Health Metrics and Evaluation, University of Washington, USA; 8 Endovascular Service, Columbia University Medical Center, USA; 9 University of Colorado Anschutz Medical Campus, USA; 10 Colorado School of Public Health, USA; 11 US Department of Veterans Affairs, USA

Cochrane Database Syst Rev Suppl 1–150 (2014)

39 Background: Disease burden should help guide research prioritization. The Global Burden of Disease (GBD) Study 2010 compiled data from 1990 to 2010 on 291 diseases and injuries, 1160 disease and injury sequelae, and 67 risk factors in 187 countries. The Cochrane Database of Systematic Reviews (CDSR) is the leading resource for systematic reviews in healthcare, with peer-reviewed systematic reviews that are published by Cochrane Review Groups. Objectives: To determine whether systematic review and protocol topics in CDSR reflect disease burden, measured by disability-adjusted life years (DALYs) from Global Burden of Disease (GBD) 2010 project. This is one of a series of projects mapping GBD 2010 medical field disease burden to corresponding systematic reviews in CDSR. Methods: Two investigators independently assessed 10 cardiovascular conditions (CV) in CDSR for systematic review/protocol representation from February to March 2014. The ten CV diseases were matched to their respective DALYs from GBD 2010. Results: Eight of the 10 CV diseases were represented by at least one systematic review or protocol in CDSR, with the majority published by the Stroke Group, Heart Group, and Peripheral Vascular Diseases Group (56%, 20%, 19%) (Table 1, Fig. 1). Comparing review/protocol funding and disability, atrial fibrillation and flutter and aortic aneurysm were well-matched. Stroke and peripheral vascular disease demonstrated review/protocol over-representation when matched with corresponding DALYs. In comparison, ischemic heart disease, hypertensive heart disease, rheumatic heart disease, cardiomyopathy and myocarditis, vascular disorders of the intestine, and endocarditis were under-represented in CDSR when matched with corresponding DALYs. Conclusions: Degree of representation in CDSR is partly correlated with DALY metrics. The number of published reviews/protocols was well-matched with disability metrics for two of ten CV diseases, while two CV diseases were over-represented. The majority of studied diseases were under-represented. Our results provide good quality and transparent data to inform future prioritization decisions.

P10: Embedding research into research, SWAT-1 assesses the effectiveness of a ‘site visit’ on recruitment rates in a multicentre randomised trial Smith V1 , Clarke M2 , Begley C1 , Devane D3 1 Trinity College Dublin, Ireland; 2 Queens University Belfast, UK; 3 National University of Ireland Galway, Ireland

Background: There are many opportunities across tens of thousands of reviews and trials to resolve uncertainties about methodology through embedded research. The SWAR (Studies Within A Review) and SWAT (Studies Within A Trial) programme, established by the All-Ireland Hub for Trials Methodology Research working with the Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

40 Medical Research Council (MRC) Network of Hubs in the UK and others, is developing methods for such research. SWAT-1, the first in the series of these studies, provides an initial example, evaluating the effects of site visits by the principal investigator on recruitment rates in multi-centre randomised trials. Objectives: To evaluate the effects of a site visit on recruitment rates in a multi-centre randomised trial (the ADCAR trial). Methods: Using SWAT-1 methodology, a before-and-after comparison used the date of the site visit as the time point for the intervention, and for the comparison with control sites. Site A received the intervention. Site B and Site C did not receive it and acted as the controls. The primary outcome measures were difference in recruitment in each site from One month and 3 months pre-intervention to 1 and 3 months post-intervention. Results: Recruitment in Site A was significantly increased at 1 (risk ratio (RR) 2.25, 95% confidence interval (CI) 1.06–4.77, P = 0.03) and at 3 months (RR 1.69, 95% CI 1.19–2.42, P = 0.002) post-intervention. There was no significant difference in recruitment in Site B (RR 0.72, 95% CI 0.45–1.15, P = 0.17) or in Site C (RR 1.04, 95% CI 0.69–1.58, P = 0.85) at 1 month post-intervention. Comparing intervention (Site A) with controls (Site B+C); the change seen in Site A was significantly different to the change seen in B+C at 1 (P = 0.02) and 3 (P = 0.02) months post-intervention. Conclusions: This study is the first example of a study in the SWAT programme, using the SWAT-1 design. It shows how these simple low (or no) cost studies can be done and provides evidence that the site visit increased recruitment rates in a randomised trial. Further SWAT-1s are required to substantiate these findings and to examine the effects in different trials in different settings.

P11: The relationship between the external and internal validity of randomized controlled trials: a sample of hypertension trials from China Zhang X, Wu Y, Kang D, Liu X, Peng L, Hong Q West China Hospital, China

Objectives: To explore the relationship between the external and internal validity of hypertension randomized controlled trials (RCTs) conducted in China. Methods: Comprehensive literature searches were performed in MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials (CENTRAL) and others, and advanced search strategies were used to locate hypertension RCTs conducted in China between 1996 and 2009. The risk of bias in RCTs was assessed by a modified scale, and then studies with three or more grading scores were included for the purpose of evaluating external validity. A data extraction form that included four domains and 25 items was used to explore the external validity and relationship with internal validity. Univariate and multivariate analyses were done using SPSS software, version 21.0 (SPSS, Chicago, IL). Results: Two-hundred and twenty-six (226) hypertension RCTs were included for final analysis. RCTs conducted in university affiliated hospitals (P value < 0.001) or secondary/tertiary hospitals (P value < 0.001) scored higher for internal validity. Multicenter studies (median = 4.0, interquartile range (IQR) = 2.0) scored higher for internal validity than single-center studies (median = 3.0, IQR = 1.0) (P value < 0.001). Funding-supported trials had better methodological quality (P value < 0.001). In addition, reporting of inclusion criteria also leads to better internal validity (P value 0.004). Multivariate analyses indicated that sample size, industry-funding, the reporting of quality of life and a university Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

affiliated hospital setting had statistical significance (P value < 0.001, P value < 0.001, P value 0.001, P value 0.006, respectively). Conclusions: Several components of the external validity of RCTs related to the internal validity do not stand in easy relation to each other. Regarding the poor reporting, other possible links between two variables need to trace in the future methodological researches.

P12: DianaHealth.com: Dissemination of Initiatives to ANalyse Appropriateness in healthcare Osorio D1 , Quintana MJ1 , Balasso V1 , Sola` I2 , Urrutia G2 , Puig T1 , Bolibar I1 , Bonfill X2 1 Biomedical Research Institute Sant Pau (IIB Sant Pau), Barcelona, Spain; 2 Iberoamerican Cochrane Centre, Spain

Background: There is an increasing concern about the importance of avoiding practices of low value in healthcare (i.e. with uncertain or questionable benefit) in order to improve appropriateness, patient safety and alleviate financial pressure. Many worldwide initiatives have analysed the appropriateness of different interventions and, as a result, some lists of recommendations about overused, underused or misused interventions have been published by different means. A single source to consult these recommendations would be useful for decision-making. Objectives: To develop a website to disseminate worldwide initiatives that have analysed appropriateness in healthcare and their results. Methods: In order to create the contents of the website, we performed a wide search in PubMed and Google using different terms related to the topic. We also consulted experts in the field. We designed a free access website aimed at clinicians, health managers and researchers, available in English and Spanish. Results: DianaHealth.com is already online in www.dianahealth.com (Fig. 1). It has three sections. (i) A list of initiatives that have analysed appropriateness in healthcare, including a descriptive profile and links to the references. (ii) A database of recommendations about overused, underused or misused interventions. The database can be consulted using one or more searching criteria: initiative, specialty, type of intervention (e.g. diagnostic, pharmacologic, preventive), type of recommendation (in favour or against), publication year or search terms. (iii) A section open to the users, to suggest the inclusion of new initiatives or recommendations. DianaHealth.com contains 1267 recommendations from 11 initiatives at this moment, and it is constantly updated with more initiatives and recommendations. Conclusions: DianaHealth.com collects information about inappropriate, unnecessary or low value practices in healthcare. It is a potentially very useful tool to clinicians, health managers and researchers for using this information quickly and easily. It also allows comparison of results among initiatives and disseminates new analysis of appropriateness.

Cochrane Database Syst Rev Suppl 1–150 (2014)

41 P13: Endorsement of the reporting guidelines of clinical trials by journals of traditional Chinese medicine (TCM): a survey of journal editorial office and journal ’Instructions to Authors’ Wang Y, Wang Y, Zhu Q, Zhang L, Wei G, Su X, Ma B Evidence-Based Medicine Center, Lanzhou University, Gansu, China

Objectives: This study aims to assess the application of some important reporting guidelines of clinical trials in ’Instructions to Authors’ of journals of traditional Chinese medicine. Methods: Study 1: The latest version of the ’Instructions to Authors’ of each of the included journals of TCM was obtained. All text mentioning CONSORT, TREND and STROBE, PRISMA, ICMJE and clinical trial registration were extracted. Study 2: Each editor of the included journals was surveyed about their journal’s endorsement of the reporting guideline recommendations and its incorporation into their editorial and peer-review processes by telephone or email. Analyses were performed by SPSS 19.0. Results: Sixty-three journals of TCM were included. Study 1: in their ’Instructions to Authors’ only three journals recommended the CONSORT statement and one recommended its extension; only two journals referred to ICMJE and three to clinical trial registration; no journals mentioned PRISMA, MOOSE,TREND, STARD or STROBE. Study 2: Survey of journal editorial offices by telephone: 54 journals (85.7%, 54/63) responded to the survey. The majority (72.2%, 39/54) were journal editors, 16.7% were editorial directors, 5.6% were editors-in-chief. Eleven of the 54 responders said they knew the CONSORT statement: of these 27% (3/11) said that they just heard about it and did not understand the detail and content at all. Likewise, only 5 of 54 responders knew the CONSORT extension. As for other reporting guidelines, only 5 of the 54 responders said that they had heard or understood, but these papers were not mentioned in their ’Instructions to Authors’. Conclusions: The endorsement of the reporting guidelines in ’Instructions to Authors’ of journals of TCM was worrying. The reason might be that there are no mandatory standards for reports of medical research in China. So we would suggest: (i) Introduce reporting guidelines into higher medical education. (ii) Encourage medical journal editing staff to take part in training courses for reporting guidelines.

n w

a r d

ith

W

P14: Registered trials of traditional Chinese medicine in China: current status and issues Zhang L1 , Zheng R1 , Li Y1 , Wu T2 , Li Y2 , Shang H1 1 Evidence-Based Medicine Center, Tianjin University of Traditional Chinese Medicine, China; 2 Chinese Cochrane/EBM Centre, West China Hospital, Sichuan University, Chengdu, China

Background: Trial registration is considered to be an important measure to improve the quality of clinical trials, as well as promote the transparency. There is growing recognition that the quality of trials of traditional Chinese medicine (TCM) has become a focus internationally, but the registered status of trials is unclear to date. Objectives: To describe and summarize the current status of and existing problems with registered trials of TCM on the Chinese Clinical Trial Registry (ChiCTR). Methods: All trials on ChiCTR were searched and screened on 17 December 2013. The most extracted information included Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

study design, date of releasing the registration number, registration status, sources of funding, target disease and type of interventions. Results: Trials of TCM made up 15.46% (558/3610) of all registered trials on ChiCTR. The number increased from 8 in 2007 to 193 in 2013. Prospective registration accounted for 79.57% (444/558). Registered trials of TCM focused on cardiovascular and cerebrovascular diseases (118, 21.15%), digestive system disease (61, 10.93%), musculoskeletal diseases (55, 9.86%), respiratory problems (53, 9.50%), metabolic/endocrine diseases (39, 6.99%) and gynecological diseases (39, 6.99%). There were 449 randomized controlled trials (RCTs), and interventions were mainly TCM taken by oral administration (253, 45.34%), non-pharmaceutical therapies including acupuncture, massage and moxibustion (140, 25.09%), and TCM injection (28, 1.43%). The majority of trials were funded by government departments such as the Ministry of Science and Technology and Finance (207, 37.10%). Conclusions: The number of registered trials of TCM has increased with more funding from the government. However, this number is far less than the number of randomized controlled trials published in China. Full implementation of clinical trial registration will require more collaboration from participants in trials of TCM. Sufficient attention has been paid to chronic non-communicable diseases, which is the dominant therapeutic area of TCM.

P15: Cochrane Reviews in obstetrics and gynecology: what is the contribution of Indian studies? Radhika AG1 , Pradhan A2 , Singh N1 1 Guru Teg Bahadur Hospital, University College of Medical Sciences, Delhi, India; 2 Indian Council of Medical Research, Delhi, India

Background: Cochrane Reviews aim to produce evidence for healthcare practice globally including India. We wish to assess the contribution of studies that originated in India in developing these systematic reviews in the field of obstetrics and gynecology. Objective: To determine the number of studies of Indian origin in obstetrics and gynecology that provided conclusive data and hence, contributed towards formulating Cochrane Reviews in obstetrics and gynecology. Methods: A total of 930 Cochrane Reviews were accessed from the gynecological and pregnancy and childbirth list of Cochrane health topics. The details of included studies for each review were then accessed from references section. Studies of Indian origin were identified from this list and their numbers calculated. Results: A total of 930 Cochrane Reviews are available in the field of obstetrics and gynecology: in gynecology, 32 of a total 424 reviews - and for obstetrics 61 of 506 reviews - had Indian studies included for meta-analysis. Only 63 studies of Indian origin qualified for inclusion in gynecology and 137 in obstetrics. The focus in gynecology was subfertility and fertility regulation, each having 17 and 14 studies respectively, while there were 23 and 25 studies each for role of prostaglandins in obstetrics and in postpartum hemorrhage, with duplicating mentions. The overlapping studies were counted only once. Conclusions: There is an urgent continuing need for high quality research and its publication in India. Going by the miniscule contribution of the studies of Indian origin, it is evident that building awareness amongst the Obstetricians and Gynecologists is urgently needed. Better quality studies are especially required in specific areas that pose a challenge in our country to improve healthcare practices. Cochrane Database Syst Rev Suppl 1–150 (2014)

42 P16: Are public health strategies relevant to non-communicable disease in China covered in the Cochrane Library? Zhang F1 , Garner P2 1 Chongqing Medical University, China; Medicine, UK

2

Liverpool School of Tropical

Background: Non-communicable diseases are becoming more common in China, and relevant, evidence-based public health strategies are important for prevention. Objectives: We sought to appraise the adequacy of the Cochrane Database of Systematic Reviews in relation to coverage of priority health problems in China. Methods: We drew on a variety of data sources to identify the key non-communicable disease problems: national China epidemiological surveys, national health statistical yearbook, World Health Organization surveillance data and special studies. We then mapped this against public health policies that global policy makers have identified as potentially important to tackle these problems and their implementation status or effects. In the third step we assessed whether these public health policies were covered by reviews on The Cochrane Library. Results: The analysis is in progress. Cardiovascular diseases and diabetes mellitus are increasing in China. A variety of fiscal, regulatory, education and communication, and primary health care policies have been identified to minimise morbidity and mortality. Our early results of mapping these policies against systematic reviews in the Cochrane Library suggest good coverage in some areas of public health strategies (such as preventing smoking), but not in others where there is a complete absence of reviews. The full analysis will be complete by the time of the Colloquium. Conclusions: High level identification of priority public health topics probably will help the Cochrane Database of Systematic Reviews increase its relevance to public health in China.

P17: Development of a Patient Decision Aid using results of systematic reviews: transferring research and applying evidence in a clinical setting Mu W1 , Zhang B2 , Shang H1 1 Tianjin Center for Evidence-Based Medicine, China; 2 Tianjin University of Traditional Chinese Medicine, China

Background: Good clinical decisions are those made when patients are informed of the best available evidence and should conform to the patient’s own values. However, it is not easy to do this in clinical practice. Patient Decision Aids (PDAs) are tools to help patients make informed and shared medical decisions. They do this by providing educational information, presenting evidence and probabilities, clarifying values and guiding users through the decision process. Objectives: We aimed to develop a PDA for patients deciding whether to take Tongxinluo (TXL, a widely-used Chinese patent drug for angina) or isosorbide dinitrate (ISDN) for angina due to coronary heart disease using quality evidence generated from systematic reviews. Methods: We organized a tool development panel consisting of decision researchers, internal doctors, clinical researchers, methodologists and patients to work on a draft. The

n w

a r d

ith

W

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Ottawa PDA template was used for the tool. The 2005 International Patient Decision Aids Standards was applied to evaluate the quality of the PDA. Results: Our PDA product for angina patients contains four parts: (i) Background information, providing a plain-language introduction to angina, TXL and ISDN. (ii) Evidence and probabilities, showing the comparative effectiveness of TXL vs ISDN in improving angina symptoms and ECG outcomes using percentages, for example (92/100 vs. 78/100 for TXL vs. ISDN on symptom improvement). The comparative risk of developing side effects was also calculated. The evidence was translated from the results of two systematic reviews, one on efficacy and the other on safety. We used the GRADE Profiler to evaluate quality of evidence critically and marked the GRADE score for each evidence item on the PDA. (iii) Value clarification, by asking patients what factors influence their choice of drug options and matter most to them. They were required to rate the importance of each factor on a scale of 0–6. (iv) The SURE test (O’Connor 2008), a few questions to test if they are sure about which drug to use. Conclusions: This PDA for angina patients will be used in clinical practice and tested for its usefulness.

n w

a r d

ith

W

P18: Using evidence and other perspectives to inform health policy in low- and middle-income countries: nucleic acid amplification tests to detect transfusion transmissible infections in blood donors Tharyan P1 , David T1 , Daniel D2 , Mammen J2 , Chandran S2 , HTA Group2 1 South Asian Cochrane Centre, Vellore, India; 2 Christian Medical College, Vellore, India

Background: Nucleic Acid Amplification (NAT) tests detect and amplify viral nucleic acids of the Human Immunodeficiency Virus (HIV 1), Hepatitis B (HBV), and Hepatitis C viruses (HCV), present even in small amounts in infected blood donors, in the potentially infective time period between infection and appearance of an antibody response by an infected individual, that current serological tests cannot detect. Their high costs and doubts about their incremental cost-effectiveness have led to uncertainties about their inclusion as an additional screening method of blood donors in low- and middle-income countries. Objectives: To: (i) describe the process employed to inform the recommendations of the Health Technology Assessment (HTA) Group for NAT testing at a tertiary care hospital in India; (ii) briefly describe the outcomes of donor screening after implementation of the recommendations. Methods: The HTA group sought data regarding the: prevalence of Transmission Transmissible Infections (TTIs) in blood donors, globally and locally; accuracy of NAT versus other screening tests; incremental yield of NAT over other tests; cost of NAT testing; and the perspectives of the administration and users of blood products. The group factored these in with the ethical, moral, and legal obligations of the institution to frame recommendations; and used a consultative, and evidence-informed process. Results: The available data indicated that, unlike in developed countries, the NAT yield in countries such as India with a higher prevalence of TTIs in the donor population is likely to be higher. The HTA group estimated that that NAT would prevent about 50 infections annually with HBV and HCV and maybe HIV, if added to existing tests. The HTA group recommended implementing NAT screening of blood donors at Christian Medical College because of legal, Cochrane Database Syst Rev Suppl 1–150 (2014)

43 moral and ethical implications, and suggested methods to minimize the financial impact of this decision. Conclusions: Implementing resource-intensive technologies requires information not only about incremental cost-efficacy but also multiple other perspectives. Many more infections than estimated were averted.

P19: Rotavirus vaccine for Indian children: can latest evidence modify policy guidelines in India? Singh M, Thumburu KK, Jaiswal N, Jindal I, Agarwal A ICMR Advanced Centre for Evidence Based Child Health, Advanced Pediatrics Centre, PGIMER, Chandigarh, India

Background: Framing a health policy at global or national level for an intervention should be based on high quality locally-generated evidence and its cost effectiveness. Recently, an indigenously developed monovalent rotavirus vaccine (RV1) was tried in Indian children. Objectives: To evaluate the latest evidence on RV1 so as to modify Indian immunization policy guidelines. Methods: We conducted a meta-analysis that included efficacy studies of RV1 from low- and high-mortality countries during the first rotavirus season. We compared results from the meta-analysis to the results of the Indian indigenous rotavirus vaccine (116E strain) efficacy study. We applied the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach to find the strength of evidence. Results: The meta-analysis showed 80% and 89% reduction by RV1 compared to 53% and 31% reduction by indigenous vaccine (116E strain) in severe rotavirus gastroenteritis (RVGE) and hospitalizations respectively. The pooled meta-analysis including the Indian trial (116E strain) showed 78% and 87% reduction respectively. The evidence from the indigenous vaccine was moderately efficacious compared to the recommended monovalent vaccine. Since the burden of RVGE in Indian children is much higher, the modest efficacy of the indigenous rotavirus vaccine may prevent severe RVGE cases and its related hospitalizations, and decrease disease burden in infants and the burden for families. The evidence from the GRADE profiler assigned moderate quality to studies included previously in the meta-analysis in comparison to the trial on indigenous vaccine which was graded high. Moreover the indigenous vaccine is cost effective (INR1000/dose). Conclusions: Policy makers should consider the high-quality evidence of moderate efficacy and low cost of the indigenously developed vaccine (116E strain) against severe RV diarrhea and related hospitalizations in Indian children and consider its incorporation into the national immunization programme. Acknowledgement: Indian Council of Medical Research, New Delhi, India.

P20: Meta-analysis of non-randomized Indian trials confirming results of a Cochrane systematic review based on randomized trials Van Remoortel H1 , De Buck E1 , Vandekerckhove P1 , Agarwal SP2 1 Belgian Red Cross-Flanders, Belgium; 2 Indian Red Cross, India

Background: New malaria cases in India are estimated at 24 million/year by the World Health Organization, resulting in a high Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

socio-economic burden. A Cochrane systematic review, published in 2004, based on randomized controlled trials in non-Indian countries, showed that insecticide-treated bed nets (TN) are highly effective in reducing morbidity from malaria. Objectives: To investigate the effectiveness of TN on malaria in Indian families as part of the development of evidence-based Indian first aid and prevention guidelines. Methods: We performed a systematic literature review in MEDLINE and EMBASE to find Indian studies on the effectiveness of TN compared to untreated bed nets (UN) or no bed net (NN). Slide positivity rate [SPR; (total number of positive blood slides × 100)/total blood slides] served as malaria outcome. The overall effect of using TN compared to UN/NN on malaria (SPR) was investigated by grouping all studies in a meta-analysis and calculating the pooled risk ratios (RR). Results: We selected 32 references from 1602 relevant articles. Based on our selection criteria, we finally retained nine Indian non-randomized controlled clinical trials. Using TN decreased the risk of malaria in terms of SPR compared to UN [28% risk reduction, overall RR 0.72 (95% CI; 0.66–0.78)]. This risk reduction was even doubled when comparing the use of TN with NN [56% risk reduction, overall RR 0.44 (95% CI; 0.41–0.48)]. Quality of the evidence can be considered as moderate (level B according to GRADE) because of the non-randomized allocation procedure in all studies. The Cochrane systematic review revealed similar results by showing that TN reduced the incidence of uncomplicated malarial episodes by 39% compared to UN (three RCTs) and by 50% compared to NN (four RCTs). Conclusions: There is evidence from nine experimental Indian studies that using insecticide treated bed nets is an effective intervention to prevent malaria, which is in line with the findings of the Cochrane systematic review, performed outside India. The present findings support the current bed net use in the Vector-Borne Disease Control Project in India.

P21: Adherence to IMMPACT outcome reporting recommendations among trials assessing the effect of opioids for chronic non-cancer pain Busse J1 , Maqbool A1 , Sivananthan L1 , Lopes L2 , Schandelmaier S3 , Kamaleldin M1 , Hsu S1 , Riva J1 , Vandvik P4 , Tsoi L5 , Lam T6 , Ebrahim S1 , Johnston B7 , Oliveri L1 , Montoya L8 , Kunz R3 , Mulla S1 , Scheidecker A3 , Buckley N1 , Sessler D9 , Guyatt G1 1 McMaster University, Canada; 2 University of Sorocaba, Brazil; 3 University Hospital Basel, Switzerland; 4 Norwegian Knowledge Centre for the Health Services, Norway; 5 Tseung Kwan O Hospital, China; 6 Tuen Mun Hospital, China; 7 Hospital for Sick Children, Canada; 8 Santo Tomas University, Colombia; 9 Cleveland Clinic, USA

Background: Chronic non-cancer pain (CNCP) is common, but clinical trials exploring treatment options do not always report all outcomes of importance to patients. To address this issue, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) consensus has recommended nine core outcome domains when evaluating treatments for pain. Objectives: To examine the extent to which trials assessing the effect of opioids for CNCP adhere to IMMPACT recommendations. Methods: We searched several electronic databases systematically for English-language studies that randomized patients with CNCP to receive an opioid or a non-opioid control. In duplicate, and independently, teams of reviewers established the eligibility of each identified study, and Cochrane Database Syst Rev Suppl 1–150 (2014)

44 recorded all reported outcome domains from trials that proved eligible. Results: Out of a total of 23 109 citations, 161 proved eligible. Table 1 presents study characteristics. The proportion of trials reporting IMMPACT-recommended outcome domains was variable across domains: (i) pain (99%), (ii) physical functioning (46%), (iii) emotional functioning (29%), (iv) global improvement (44%), (v) adverse events (93%), (vi) participant disposition (75%), (vii) role functioning (17%), (viii) interpersonal functioning (8%), and (ix) impact on sleep/fatigue (31%). Although patients typically provided outcome data, many trials relied on clinician-reported outcomes or were unclear regarding the source (Table 2). With the exception of patient’s rating of global improvement, pain, and adverse events, our adjusted analyses found that all IMMPACT domains showed an increased rate of reporting over time (adjusted odds ratios all >2 for more recent date of publication, by decade). Conclusions: Trials evaluating the effect of opioids for CNCP show improved concordance with IMMPACT-recommended outcome domains over time; however, most recommended outcome domains are reported in fewer than half of published trials and the source of outcome information is often unclear.

P22: Meta analysis in China Li L, Tian J, Yang K Evidence-Based Medicine Center, Lanzhou University, Gansu, China

Background: Meta-analyses, as a key component of evidence-based medicine, have established evidence synthesis and become more widely accepted by clinicians, researchers and policy makers as a useful tool for critical assessment of the totality of evidence in a research question. Objective: To understand the trend and status of meta-analyses published by Chinese authors. Methods: PubMed was searched using [‘Meta-Analysis as Topic’(MeSH) OR ‘Meta-Analysis’ (Publication Type)] AND [China or Chinese (Affiliation)] without language and publication date restrictions on 22 March 2014. Results: Five-thousand, three-hundred and sixty-nine (5369) citations were obtained. The number of meta-analyses published by Chinese authors increased exponentially over the years. 18 697 authors were involved in conducting meta-analyses: 72.42% of them were involved in conducting one meta-analysis, and 1.21% were involved in conducting more than ten meta-analyses. Shanghai (1011), Beijing (492), and Chengdu (450) published the largest number of meta-analyses. The top three journals were PLOS One (435), Molecular Biology Reports (215), and the Cochrane Database of Systematic Reviews (198). Only 210 studies (3.72%) were produced in collaboration with other countries. Chinese authors published the fewest meta-analyses (11 citations) in journals with high impact factors (e.g. JAMA, BMJ, and the Lancet). Conclusion: Although the number of meta-analyses published by Chinese authors increased exponentially, most of them were published in regular journals and remained un-registered without collaboration.

n w ra

d h it

W

P23: Global burden of metabolic and endocrine diseases in 2010 and their representation in systematic reviews and protocols in Cochrane Database of Systematic Reviews Stephens P1,2,3 , Boyers L4 , Karimkhani C5 , Dellavalle R1,2,3 1 University of Colorado Anschutz Medical Campus, USA; 2 Colorado School of Public Health, USA; 3 US Department of Veterans Affairs, USA; 4 Georgetown University School of Medicine, USA; 5 Columbia University College of Physicians and Surgeons, USA

Background: Disease burden should help guide research prioritization. The Global Burden of Disease (GBD) Study 2010 compiled data from 1990 to 2010 on 291 diseases and injuries, 1160 disease and injury sequelae, and 67 risk factors in 187 countries. The Cochrane Database of Systematic Reviews (CDSR) is the leading resource for systematic reviews in healthcare, with peer-reviewed systematic reviews that are published by Cochrane Review Groups. Objectives: To determine whether systematic review and protocol topics in CDSR reflect disease burden, measured by disability-adjusted life years (DALYs) from the GBD 2010 project. This is one of a series of projects mapping GBD 2010 medical field disease burdens to corresponding systematic reviews in CDSR. Methods: Two investigators independently assessed seven metabolic and endocrine conditions in CDSR for systematic review and protocol representation according to subject content. The seven diseases were matched to their respective DALYs from GBD 2010. Results: All seven metabolic and endocrine conditions were

n w ra

d h it

W

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

45 represented by at least one systematic review in CDSR (Table 1, Fig. 1); 39.5% of the systematic reviews and protocols that focused on these conditions were from the Metabolic and Endocrine Disorders Group and 33% were from the Menstrual Disorders and Subfertility Group. Comparing the number of reviews and protocols with disability, only diabetes mellitus and uterine fibroids were well-matched, while endometriosis and polycystic ovarian syndrome were over-represented. By comparison, benign prostatic hyperplasia, premenstrual syndrome and glomerulonephritis were under-represented. Conclusions: These results prompt further investigation into why certain diseases are over- or under-represented in CDSR relative to their DALY. With regard to metabolic and endocrine conditions, this study suggests that certain conditions should get more attention in order to create a better representation of the highest causes of disability and mortality within this research database. Our results provide good quality and transparent data to inform future prioritization decisions.

n w

a r d

ith

W

P24: Using the GRADE approach to develop guidelines in low income countries: views of guideline panelists in Kenya Gathu M1 , Sinclair D2 , Opiyo N1 , Garner P2 , English M1 1 Kenya Medical Research Institute -Wellcome Trust Research Programme, Kenya; 2 Liverpool School of Tropical Medicine, UK

Background: In Kenya, as in many low-income countries, clinical guidelines are often formulated through direct adoption or adaptation of global guidance. We report the recent experiences of Kenyan policy makers using the GRADE approach to formulate evidence-informed recommendations around three priority pediatric topics. Methods: We invited participants from the 2013 guideline panels for in-depth interviews. All interviews were audio-recorded, transcribed, and analyzed using a framework analysis approach. Results: Seventeen participants were interviewed, drawn from public and private health facilities and the ministry of health. Ten had no prior experience of the GRADE approach. Our analysis of the transcripts identified five key emergent themes. (i) Debating the evidence increases confidence: participants linked clear understanding of the evidence with improved confidence that the right recommendations had been made, and felt able to justify the recommendations to others. (ii) Panel involvement facilitates ownership: participants considered broad inclusion on guideline panels essential to promote ownership of the recommendations, and ensure timely dissemination and implementation. (iii) Mixed views on capacity for wider adoption: participants valued the transparency and methodological rigour of the GRADE approach, but stated wider adoption would require further training and considerable investment of time and money. (iv) Consideration of national context important: participants universally stated that knowledge and understanding of the national context were essential for effective national policies, and some felt wider consultation in global guideline processes would improve their usefulness. (v) Implementation is the weakest link: participants gave examples of recommendations made years before that remain unimplemented, and noted that global policy often moved too quickly for countries to keep up. Conclusion: There is enthusiasm for rigorous, context specific, guideline development, but wider adoption will require investment of time and money. Regional collaboration, sharing information and resources, would reduce duplication of efforts.

P25: Implementation of prospective trial registration in the field of otolarynchology Stegeman I1 , Leeflang M2 , Hooft L3 1 Department of Otorhinolaryngology – Head and Neck Surgery, The Netherlands; 2 Department of Clinical Epidemiology, Biostatistics & Bioinformatics, Academic Medical Centre, The Netherlands; 3 Dutch Cochrane Centre, The Netherlands

Background: In an effort to make research more transparent and to diminish publication bias, the International Committee of Medical Journal Editors (ICMJE) requires all randomized controlled trials (RCTs) to be registered in a public database before enrolment of the first patient. All registered trials receive an unique trial number, if this number is also reported in the article, registered trial information Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

46 is easily accessible. Objectives: In this study we assessed the trial registration and the reporting of trial registration number in otolarynchologic studies. Methods: We examined RCTs published in the top-ten journals in otolarynchology. We assessed whether the articles reported a trial registration number, if not, we assessed whether those RCTs were registered in a trial register linked to the World Health Organization Search Portal (including ClinicalTrials.gov). Results: Seventy-nine publications of RCTs were identified, seven of which reported trial registration numbers in the manuscript. Additionally, five trials were found to be registered that did not report a trial number in their manuscript. In total 6.3% (11/79) of the RCTs were registered. Conclusions: In conclusion, clinical trial registration is not yet adopted in otolarynchology. That makes it impossible to check for selective reporting or publication for most of the trials. Further promotion of trial registration should be implemented by otolarynchologic journals.

P26: Engaging decision makers and researchers to identify priorities in health systems evidence synthesis for low-income countries: lessons learned from Uganda Obuku EA, Nabudere H, Semakula D, Sewankambo NK Africa Centre for Systematic Reviews and Knowledge Translation, Makerere University College of Health Sciences, Uganda

Background: Stakeholder involvement in priority setting may enhance the use of research in decision making, particularly in low resource settings such as sub-Saharan Africa. The Africa Centre for Systematic Reviews and Knowledge Translation identifies and synthesizes health systems evidence for decision making. Objective: To identify priority questions for health systems evidence synthesis for Uganda. Methods: We purposely selected decision makers (n = 15) from health related government departments, consumer advocacy groups and non-governmental organizations. The researchers (n = 29) were from universities and research institutes. We presented participants with background documents including the Norwegian EPOC satellite priority topics for low- and middle-income countries; and examples of research priority areas from World Health Organization health systems perspective and the Millennium Development Goals 4 (child health), 5 (maternal health) and 6 (infectious and non-communicable diseases), before completing two iterations of a mini-Delphi survey technique; followed by a feasibility scan of the PDQ-Evidence database. Outcome measures: Ten priority areas for health systems evidence synthesis ranked by: importance, availability of relevant research and viable options, uncertainty, opportunity for change and interest in deliberation. Results: Participants identified 34 priority areas for health systems research evidence synthesis. Most concerned delivery arrangements (18, 53%); followed by financial (9, 26%) and organizational (6, 18%) arrangements. However, financial arrangements (8, 80%) dominated the 10 priority questions that is; health insurance, health worker remuneration and resource allocation. We identified existing overviews (8) and systematic reviews (23) in PDQ-Evidence, addressing these topics. Conclusions: Our findings highlight financial arrangements as a resilient priority area for health system evidence synthesis in Uganda. The existing overviews and systematic reviews suggest a need to constantly engage decision makers about where and how to find relevant research evidence, and refine their information needs further. Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

P27: How does qualitative research contribute to evidence-based practice in nursing in China? Jin Y, Li Y, Shang H Evidence Based Medicine Center, Tianjin University of Traditional Chinese Medicine, China

Background: Due to the inability of quantitative research to explore many important health problems, over the past few decades, qualitative research has been increasingly valued as making an important contribution to medical science and clinical nursing. There has been a dramatic rise in the publication of qualitative research in nursing in China. Objectives: To determine the contribution of qualitative research to evidence-based nursing in China based on the quantity and quality of qualitative research and meta-synthesis of qualitative research. Methods: The Chinese Journal Full-Text Database, the Chinese Biomedicine Literature Database and the Wanfang Database were searched for qualitative research and meta-synthesis of qualitative research published in nursing journals, from inception to December 2013. Take a calculation analysis of all included studies. The Consolidated Criteria for Reporting Qualitative Research checklist (COREQ) was used to assess quality of qualitative research. Results: Seven-hundred and twenty-nine studies of qualitative research were identified; the number is far lower than the number for quantitative research. There was not one meta-synthesis of qualitative research. Since 2004, the number of qualitative research articles has increased markedly, especially in 2011 and 2012. Quality of qualitative research remains suboptimal. Key information in many published qualitative studies is missing, incomplete or ambiguous. Especially, inadequate reporting of methodology may mean that reader cannot tell exactly how the research was done. Conclusions: There is a need for future development of meta-synthesis of qualitative research to integrate findings from a number of different but inter-related qualitative studies, and to formulate a more comprehensive interpretation that then can be used as evidence in evidence-based practice. Qualitative studies should present sufficient information to allow readers to understand the design, conduct, and analysis of published studies better, and to facilitate the application of results. Qualitative studies make a minor contribution to the evidence-based practice in nursing in China.

n w

a r d

ith

W

P28: Tool of transferability: application on studies of cost-effectiveness ´ Velasquez M, Vallejos C, Bustos L, De la Puente C, Zaror C Universidad de La Frontera, Chile

Background: Studies of cost-effectiveness (CE) are conditioned by the local context, with different results between countries. Variables such as incidence of disease, severity, availability of resources, clinical practice variations, and relative prices, could modify the results. An economic evaluation (EE) is generalizable when its results can be applied without adjustment, and it is transferable if the results can be adapted to apply in other realities. There are several ways and methods to assess transferability. Objectives: To apply a tool of transferability in CE studies obtained from a systematic review directed to answer questions about health technology assessment (HTA) in Chile. Methods: A systematic review of the literature on transferability methods in CE studies was carried out. Alongside this resource a tool of transferability Cochrane Database Syst Rev Suppl 1–150 (2014)

47 was developed and implemented, which was compared with CE criteria defined by the Chilean Ministry of Health. Results: The transferability tool was developed from a study (Goeree 2011) selected in a systematic review, adopting the transferability criteria of Welte 2004 and including validation criteria of Drummond 2005. This tool was applied to 27 CE studies selected to respond to the evaluation of six health technologies in Chile; 68% of the articles were highly transferable. Compared with the ‘Chilean Guide to use CE evidence’, the correspondence was high. Conclusions: It is possible to transfer CE results and to avoid performing new CE studies. In the scope of HTA it is necessary when a rapid response is required or a new study is very expensive. It is noted that the main barrier to transferability is the wide variability of costs and willingness to pay in different countries.

P29: Changes in randomized controlled trials reporting quality over time: a systematic review of journals from China in traditional Chinese medicine Huang C, Zheng J, Zheng D, Yang J, Su X, Wei G, Wang X, Qin Q, Ma B Evidence-Based Medicine Center, Lanzhou University, Gansu, China

Background: Though a large number of RCTs of traditional Chinese medicine (TCM) have been published in China, there have been no systematic assessments of whether the reporting quality has risen since CONSORT was first introduced into China in 1997. Objectives: To evaluate the reporting quality of RCTs of TCM published in China. Methods: Nine core Chinese journals of TCM indexed in the SCI and MEDLINE/Embase [Chinese Journal of Integrative Medicine (English version), Journal of Traditional Chinese Medicine (English and Chinese version), Chinese Journal of Integrative Medicine (English and Chinese version), Zhenci Yanjiu, Chinese Traditional and Herbal Drugs, Chinese Acupuncture and Moxibustion, China Journal of Chinese Materia Medica, Journal of Chinese Integrative Medicine, Zhongyaocai] were searched from inception to December 2013. The CONSORT statement was first introduced into China in 1997, so the articles were grouped in two publication periods, i.e. before 1996 (pre-CONSORT) and after 1997 (post-CONSORT) for assessment of reporting. Data were collected using a standardized form. Analyses were performed using Excel and SPSS19.0. Results: A total of 1087 RCTs, 278 of which were published before 1997, were identified. Compared with literature published before 1997, there was significant improvement in some aspects such as title and abstract, introduction, trial design, statistical methods, recruitment, baseline data and ancillary analyses (P value < 0.05), etc. However, some important methodological components of RCTs were still incompletely reported, such as sample-size (0.3% versus 1.9%), sequence generation (1.6% versus 9.2%), allocation concealment (0% versus 2.1%), blinding (0% versus 1.2%). Besides, same as in funding (3.1% versus 5.6%) and protocol (0% versus 3.1%), and no trials reported any information about the registration. Conclusions: Overall, the reporting quality of RCTs in TCM has changed significantly in some of the aspects we analyzed since CONSORT was first introduced into China in 1997. However, there is room for improvement in some aspects, such as methodological components, registry, protocol and sample size.

n w

a r d

ith

W

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

P30: Acupuncture: summary of published systematic reviews in China Qian-ge Z1 , Wang Y1 , Zhang L1 , Wang Y1 , Zhao Y1 , Chen K1 , Chen Z1 , Ma B1 1 Evidence-Based Medicine Center, Lanzhou University, Gansu, China

Background: Systematic reviews (SRs) of acupuncture have become increasingly popular in China and have been published in large numbers in recent years. Objectives: To examine epidemiological and reporting characteristics as well as methodological quality of SRs of acupuncture based on PRISMA and AMSTAR guidelines in China. Methods: Four Chinese databases were searched [Chinese Biomedicine (CBM), Chinese Scientific Journals Database (CSJD), Chinese Journal Full-Text Database (CJFD) and Wanfang Database] for SRs of acupuncture, from inception through December 2013. Data were extracted into Excel spreadsheets. The PRISMA and AMSTAR checklists were used to assess reporting characteristics and methodological quality, respectively. Analyses were performed using Excel and SPSS 19.0. Results: A total of 138 SRs were identified in China. None of them had been updated, only 31% of these reviews were written by clinicians and one-third (33%) were published in specialty journals. The impact factors of 51% of the reviews were 0 so far. Information retrieval was not comprehensive in more than half (59%) of the reviews; 68% of the reviews did not reported information about quality assessment, while less than one-third (26%) reported assessing for publication bias. Statistical mistakes appeared in 33% reviews. Though 95% of the reviews used the terms ’systematic review’ or ’meta-analysis’ in the title, no reviews reported conflicts of interest or had been updated 2 years or more after they were published. Conclusions: While many SRs of acupuncture interventions have been published in China, the quality of these reviews is troubling. These are a potential key source of information for clinicians and researchers, but not only were many of these reviews incomplete, some contained mistakes or were misleading. Focusing on improving the quality of SRs of acupuncture, rather than continuing to publish them in great quantity, is urgently needed in order to increase the value of these studies.

n w

a r d

ith

W

P31: Analysis of randomized controlled trials published from India by Indian physiotherapists in PubMed and PEDro database Hariohm K1 , Prakash V2 , Saravan Kumar J1 1 MSAJ College of Physiotherapy, Chennai, India; 2 Ashok and Rita Patel Institute of Physiotherapy, Gujarat, India

Background: Increased scope of physiotherapy (PT) practice also has contributed to research in the field of PT. It is essential to determine the production and quality of the clinical trials from India, since it may reflect the scientific growth of the profession. These trends can be taken as a baseline to measure our performance and also can be used as a guideline for future trials. Objective: To quantify and analyze qualitatively RCTs from India that are indexed in PubMed and not-indexed in PubMed and to compare the descriptives. Methods: Articles were sought in PubMed and PEDro databases using the Cochrane Database Syst Rev Suppl 1–150 (2014)

48 keywords ’India’, ’Indian’ and ‘physiotherapy’ from 2000 to May 2013. Full texts were retrieved from both databases. Duplicates were removed in PEDro database. Articles identified were analyzed for PEDro score and items, CONSORT items such as randomised controlled trial (RCT) in title, flowchart, statement of primary outcome measure, a priori sample size calculation, reporting of adverse events, registration of protocol and year of publication. Results and discussion: PubMed and PEDro yielded 45 and 96 valid articles respectively that were included for analysis. A priori sample size calculations had been done and reported in 28.9% of the studies indexed in PubMed, but no other studies were found that reported it indexed by databases other than PubMed. The primary outcome measure was explicitly reported by 30.4% of the PubMed studies, but reported in only 2% of studies not indexed by PubMed (Table 1). Mean PEDro score of 5.5 (±1.2) and 3.4 (±1.2) were found in PubMed and PEDro (Table 2) and the t-test revealed a statistically significant difference in the means of PEDro scores between two database(P value < 0.005). Conclusion: Although the quantity of studies found via PubMed is lower than the quantity of studies not indexed by PubMed, the quality of the PubMed indexed studies was found to be better in relation to rigorous methodological design and better reporting. Keywords: physical therapy, South-East Asia

P32: Randomized controlled trials of acupuncture in China: an assessment of reporting quality with a CONSORT- and STRICTA-based instrument Chen K, Zhao Y, Zhu Q, Wang Y, Qin Q, Wang X, Chen C, Ma B Evidence-Based Medicine Center, Lanzhou University, Gansu, China

Objectives: To examine epidemiological and reporting characteristics as well as methodological quality of randomised controlled trials (RCTs) of acupuncture published in Chinese journals. Methods: Chinese databases were searched (CBM, CSJD, CJFD and Wanfang Database) for RCTs of acupuncture from inception. Data were extracted into Excel spreadsheets. CONSORT and STRICTA were introduced into China in 1997 and 2003, so the articles were grouped, and reporting assessed, in three publication periods, i.e. before 1996 (pre-CONSORT), 1997–2003 and 2004–2010 (post-STRICTA). Data were collected using a standardized form. Analyses were performed using Excel and SPSS. Results: A total of 1978 RCTs were identified, one-third of which could be identified as RCTs from the title. The number of citations of articles has increased over time but there existed no clear increase in highly articles (from 4.9% to 6.5% and to 7.0%). Informed consent has gained more and more attention (from 0.7% to 1.4% and to 21.8%) in contrast to the ethics review (from 0 to 0 and to 0.9%). The increasing trend of the reporting quality of randomized methods is not so obvious and the whole quality is troubling, such as the reporting of sequence generation (from 1.4% to 15% and to 26.3%), allocation concealment (from 0% to 1.4% and to 4.9%) and blinding (from 0% to 5.7% and to 9.1%). Another concerning problem was the risks of non-publication of entire trials, like sample size (from 0 to 0 and to 1.2%), participant flow (from 0% to 11.6% and to 20.6%). However, the reporting quality of other items in the STRICTA checklist remained poor, like acupuncture rationale (from 0% to 0.3% and to 1.0%) and practitioner background (from 0 to 0 and to 0.4%). Conclusions: The reporting quality of most CONSORT items has improved continuously while the reporting of acupuncture seem not to have been influenced by STRICTA. The effort of government to guide and add to the effort of editors to disseminate CONSORT and STRICTA by putting these checklists into instructions to researchers, is urgently needed in order to increase the reporting quality of these studies.

n w ra

d h it

W

P33: How many journals adopt reporting guidance for RCTs, systematic reviews and clinical practice guidelines in ’Instructions for authors’? A cross section survey from 150 medical journals Yao L, Chen YL, Wang Q, Wang XQ, Wei D, Cui RR, Li N, Yang KH Evidence-Based Medicine Center, Lanzhou University, Gansu, China

n w

Background: Empirical evidence suggests that active implementation of the CONSORT (Consolidated Standards of Reporting Trials) statement and other reporting guidance by journals can lead to improvements in the reporting of trials and other studies. Objectives: To investigate how many medical journals adopt reporting standards for randomised controlled trials (RCTs), systematic reviews (SRs) and clinical practice guidelines (CPGs) in their instructions for authors. Methods: Highly sensitive search filters were used to identify RCTs, SRs and CPGs in PubMed. Then we selected the top 50 journals that published each type

a r d

ith

W

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

49 of study and checked whether CONSORT, PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and reporting criteria for CPGs were adopted in their instructions for authors. Results: The results showed that 28 (56%) journals had adopted CONSORT, and 20 (40%) journals had adopted PRISMA. For practice guidelines, only one (2%) journal had adopted Conference on Guideline Standardization (COGS) as a reporting standard. In addition, two journals mentioned in their instructions that when authors develop guidelines, they should follow Appraisal of Guidelines for Research and Evaluation (AGREE) and the Institute of Medicine (IOM) standard. Conclusions: Compared with RCTs and SRs, very few medical journals endorsed reporting standards for clinical practice guidelines. We suggest that medical journals publishing clinical practice guidelines should introduce relevant reporting standards in their instructions for authors. Guidelines authors also need to report their guidelines clearly, transparently and in a standard way.

n w

a r d

ith

W

P34: Model construction of Health Technology Reassessment (HTR) in Korea using systematic review of HTR systems in four countries Lee SH1 , Seo HJ2 , Ji SM1 , Lee DH3 , Park JJ1 , Son SK1 1 National Evidence-based Healthcare Collaborating Agency, South Korea; 2 Department of Nursing, College of Medicine, Chosun University, South Korea; 3 College of Korean Medicine, Woosuk University, South Korea

Background: Health technology reassessment (HTR) is the process of withdrawing medical care resources partially, or completely, when existing health technology is regarded as being an inefficient use of healthcare resources that provides less health benefit for its cost. Obsolescence is a natural stage in the health technology cycle to promote optimization of health insurance finance use. Objectives: The purpose of this research is to construct an HTR system in Korea by examining the condition and methodology of the international HTR system in four countries. Methods: The Cochrane Library, Ovid MEDLINE, Ovid EMBASE, PubMed, and CINAHL were searched from January 2000 to February 2013. Grey literature was searched for via reference lists, bibliographies of relevant articles, and by searching the web sites of INAHTA and HTAi. Inclusion criteria were studies on HTR or reinvestment, studies reporting on the implementation method and theoretical knowledge of the HTR. Selection of papers and data extraction were performed by two independent review authors. Results: A total of 24 related papers were included. This report especially examined the countries that actually performed HTR through the government and local government, such as the United Kingdom, Australia, Canada and Spain. The overview of the HTR systems examined the reassessment performance method and the policy performance method. A reassessment of health technology was performed for optimization of use of health technology after its adoption and acceptance into practice. Apart from the difference of a legal basis, the reassessment system of theses countries are similar in HTR technology identification, and priority setting methods, assessment methods, and decision-making. We constructed an HTR system in Korea that is also composed of four steps on the basis of the results of this systematic review. Conclusions: HTR is one of the methods used to help the patient receive proper treatment to increase the efficiency and quality of treatment. Based on the lesson from foreign countries’ prior experience, we developed an HTR system in Korea considering the unique characteristics of Korea. Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Poster Session 2 P35: @EvidenceAid: exploring the social network of a humanitarian knowledge translation programme Mellon D1 , Clarke M1 , Allen C2 1 Evidence Aid; Queens University Belfast, UK; 2 Evidence Aid, UK

Background: Evidence Aid can be characterised as a knowledge translation project that uses a range of technologies, including social media, to improve the use of systematic reviews evidence before, during and after humanitarian emergencies. Evidence Aid has been using the Twitter platform to engage with partners and stakeholders since September 2011. There is little information available about the relative contribution of these digital media in supporting the Evidence Aid mission. Objectives: The aim of this project was to explore the use of Twitter by Evidence Aid though descriptive statistics of activity. Methods: Multiple third-party web applications were used to explore how Evidence Aid interacts with its followers on the Twitter platform. Summary statistics were calculated and a number of visualisation techniques were used to illustrate the global distribution of followers, trends in interactions and implications for enhancing stakeholder engagement. Results: In the 28 moths since joining Twitter, Evidence Aid has gained a considerable following: 940 unique accounts globally with followers in 69 countries. Just under half (46%) of all followers are located in the USA (39%) and the UK (17%), with Canada (5%) and Australia (5%) joint third in rank. There was little correlation between the distribution of followers and the incidence of disasters in each country. Estimates of the use of Twitter in each country, and the significantly larger follower-base of a number of partner organisations, suggests that there is potential to improve overall reach. The large majority (87%) of @EvidenceAid tweets contain original content, however a very high proportion of tweets are subsequently retweeted by followers. Thematic analysis of the content of @EvidenceAid tweets highlighted the mission focus of ’public health’ and ’humanitarian assistance’. Conclusions: Twitter is a valuable medium for communicating with partners and stakeholders in advance of disasters and humanitarian emergencies and Evidence Aid recognises its value and continues to work towards making optimal use of social media.

P36: Engaging the national mandate to promote evidence informed policy: using Cochrane methods and systematic reviews to move from idea to implementation Rana S1 , Aryal K1 , Mehata S2 , Dahal G3 , Pottie K3 , Lohani GR4 1 Nepal Health Research Council, Nepal; 2 Nepal Health Sector Support Programme, Ministry of Health and Population, Nepal; 3 Centre for Global Health, Institute of Population Health, University of Ottawa, Canada; 4 Ministry of Health and Population, Nepal

Background: Institutionalizing knowledge translation platforms can strengthen efforts to bring evidence-based Cochrane methods and Cochrane Database Syst Rev Suppl 1–150 (2014)

50 7.2 for aesthetic appeal. The critical appraisal was rated 7.8 for clarity, 7.6 for comprehensibility, and 5.0 for aesthetic appeal. Differences between the two formats were statistically significant for each factor. Regarding whether they would find the format of data presentation useful for their professional role, 90% responded yes for the critical appraisal vs. 70% for the infographic (P < 0.07). Conversely, 90% felt the infographic would be useful for patients and caregivers vs. 10% for the critical appraisal (P < 0.00). Respondents provided additional information on which formats may be preferred by different audiences: the infographic was engaging, attractive, easy to read, and captured a lot of information; however, some found it too busy, difficult to interpret, and difficult to determine the take-home message. Respondents found the critical appraisals to be clear, directive, professional, and concise; however, they found them more technical and less visually appealing. Respondents were divided over which format they preferred overall: 51% infographic, 49% critical appraisal. The majority (81%) preferred electronic over paper versions. Conclusions: This information can guide the development of user-friendly summaries of Cochrane SRs that may be the most effective for different audiences.

systematic reviews into health policy development. When the Nepal Health Research Council (NHRC) was established in 1991, it was given the central mandate to assess and communicate relevant health research to support national government work. However, two decades later effective mechanisms and partnership platforms for NHRC-Government of Nepal Ministry of Health and Population (MoHP) interaction still do not exist. Objectives: To share the approach of the NHRC to capitalize on current opportunities and establish its role in strengthening evidence-informed national policy-making. Methods: A conceptual framework informed by the Lavis (2003) and Straus (2009) approach and the organizational framework of programme planning will serve to plan, drive, and evaluate the actions of the NHRC to create a national center of excellence in research synthesis and knowledge translation with Cochrane support. Results: This poster will depict schematically the approach and actions of the NHRC and the NHRC Evidence Support Group to solidify evidence-informed policy-making partnerships in Nepal. Strategies and activities: enhance links between research producers and users; implement strategies to promote evidence uptake; training on priority setting, policy briefs, systematic reviews and meta-analyses; translate evidence into actionable messages; and identify and communicate health research gaps. Conclusions: A clear framework for knowledge translation efforts will support the NHRC and MoHP to bring evidence to decision-making. The results of this work are expected to form the basis for the NHRC to strengthen its institutional capacity in research synthesis and knowledge translation.

P38: How many network meta-analyses were cited by guidelines or Health Technology Assessments (HTA)?

References

Cui Q1 , Tian J1 , Song X1 , Huang D1 , Yang K1 1 Evidence-Based Medicine Center, Lanzhou University, Gansu, China

Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003; 81:221–222. Straus SE, Tetroe J, Graham ID. Knowledge translation in health care: moving from evidence to practice: Wiley-Blackwell, 2009.

P37: Translating knowledge from systematic reviews: evaluation of infographics and critical appraisal formats Hartling L1 , Crick K2 Child Health Field, Canada; 2 University of Alberta, Canada

1 Cochrane

Background: Systematic reviews (SRs) provide a rigorous synthesis of available evidence on a clinical question. Knowledge users need this information in a format that allows for quick reference with key messages highlighted. Objective: Compare two formats for summarizing the results of SRs for knowledge users. Methods: We developed an infographic and a critical appraisal to summarize the results of a Cochrane SR of drugs for treating acute migraine headaches in children. Knowledge users (n = 58) attending a national meeting completed a questionnaire regarding their impressions of the two formats. Wilcoxon signed rank test was used to compare knowledge users’ paired responses to the infographic and critical appraisal formats. Results: On a scale of 1–10 (10 most preferred), the infographic was rated 7.0 for clarity, 7.1 for comprehensibility, and Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: In recent years researchers have applied more focus on network meta-analysis (NMAs). More and more NMAs have been published in journals, but the citing situation of NMAs is not clear yet. Objectives: To investigate citations of NMAs. Methods: The Cochrane Library, PubMed, EMBASE, Web of Science were searched from inception to September 2013 for NMAs. A random sample of NMAs was selected. We used Excel for data description and analysis. Rate (%) was performed for measuring the result. Results: We sampled 222 NMAs out of 667 NMAs, of which 169 (76.1%) were cited in other articles. The number of times NMAs were cited ranged from 0 to 610 (Table 1). The total number of citations of NMAs was 4429, so each NMAs was cited 19.95 times on average. (i) NMAs were cited 4235 times in Web of Science, 1905 times in BIOSIS Citation Index, 52 times in SciELO Citation Index and 133 times in Chinese Science Citation Database (Table 2). (ii) There is a huge gap between the numbers of citations of NMAs in different types of publication:, books or book sections cited NMAs 16 (0.36%) times, while journals cited NMAs 4413 (99.63%) times. (iii) We particularly focussed on investigating how many guidelines or consensus, recommendation and health technology assessments had cited NMAs. NMAs were only cited 185 (4.17%) times by guidelines or consensus; cited 37(0.83%) times by recommendation and cited just one (0.02%) time by health technology assessment studies. Conclusions: This survey showed that few guidelines, consensus, recommendations and health technology assessment studies cited NMAs. In the future, we should pay more attention to using NMAs in clinical guideline development.

n w

a r d

ith

W

Cochrane Database Syst Rev Suppl 1–150 (2014)

51 P39: Stakeholder involvement in Cochrane Reviews of urological conditions Gudeloglu A, Neuberger MM, Sultan S, Parekattil S, Dahm P Cochrane Prostatic Diseases and Urological Cancers Group, USA

n w

a r d

ith

W

Background and objectives: Stakeholder involvement in Cochrane reviews (CRs) is increasingly being recognized as critically important. It promotes clinical relevance, dissemination and ultimately integration into individual patient and health policy decision-making. The focus of this study was to assess to what extent urologists participated in CRs related to their specialty. Methods: Based on a predefined written protocol, we retrieved the abstracts and author affiliations of all CRs published in The Cochrane Library (date of last search: 10 January 2014) and classified under ’urology’. Data points abstracted were the review title, date of publication, authors’ names, departmental, institutional and country affiliations, and Cochrane Review Group (CRG). We based the determination of the authors’ professional information on information provided in the published review supplemented by Internet searches. Data abstraction was performed by one member of the investigating team and double-checked by another. Descriptive statistical analysis was performed using SPSS version 22.0. Results: We identified 143 unique CRs published between 1998 and 2013. The Incontinence Group was responsible for the largest number of reviews (85; 59.4%) followed by the Prostatic Diseases and Urologic Cancers Group (44; 30.8%) and the Renal Group (8; 5.6%). The median number of: authors was four (IQR: 3; 5), institutions was two (IQR: 2; 3) and departments was three (IQR: 2; 4). A majority of reviews originated from a single country (88; 61.5%). Authors from 20 countries contributed to urology-related reviews; the UK had the greatest author contribution (82; 39.6%) followed by the USA (28; 13.5%) and Australia (19; 9.2%). Urologists contributed to approximately one-third of the reviews (52; 36.4%). This percentage (range: 36.5–40.9%) was similar for all three major contributing CRGs. Conclusions: Only one in three CRs related to urology includes authors that are urologists. There is concern that a lack of specialty-specific stakeholder involvement constitutes a barrier to the uptake of CRs; therefore, increased efforts to engage specialists appear to be indicated.

P40: Communication is everybody’s business: developing methods to improve the evaluation of communication strategies for childhood vaccination Kaufman J1 , Hill S1 , Lewin S2 , Ryan R1 , Heather A2 , Bosch-Capblanch X3 , Cartier Y4 , Cliff J5 , Glenton C6 , Muloliwa A7 , Oku A8 , Oyo-Ita A8 , Rada G9 , Synnot A1 1 Cochrane Consumers and Communication Review Group, Australia; 2 Norwegian Knowledge Centre for the Health Services, Norway; 3 Swiss Tropical and Public Health Institute, Switzerland; 4 International Union for Health Promotion and Education, France; 5 Universidade Eduardo Mondlane, Mozambique; 6 Norwegian Knowledge Centre for the Health Services, Norway; 7 Provincial Directorate of Health, Mozambique; 8 University of Calabar, Nigeria; 9 Catholic University of Chile, Chile

Background: A Global Polio Eradication Initiative report recently stated, ‘communication is everybody’s business.’ Health Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

52 communication encompasses a broad range of interventions beyond informing or educating. It includes advocacy, social mobilisation, health literacy and interpersonal communication, operating at societal and individual levels. Effective communication with parents and communities is critical to improving global vaccination coverage. To implement communication interventions about childhood vaccination, decision makers need to know what works. Currently, vaccination communication interventions are assessed using a variety of outcomes, making it difficult to compare intervention effectiveness across studies. Important process outcomes (e.g. knowledge, decisional conflict) are often not measured. To improve the evidence base and help decision makers focus their energy and scarce resources, we will develop a taxonomy of vaccination communication outcomes. This taxonomy will illustrate the range of potential outcomes and inform future work to define a core outcomes set. Methods: (i) We will search electronic databases for studies evaluating communication interventions directed to parents or communities about childhood vaccines. We will extract and code outcomes using content analysis and categorise the outcomes into a taxonomy, building on existing frameworks. (ii) To gather feedback on the taxonomy’s completeness and appropriateness, we will convene a consultation panel of researchers, policy makers, health professionals and experts in program delivery and evaluation. Using focus group methodology, the panel will provide feedback and discuss outcome prioritisation and causal pathways. Transcripts will be analysed thematically and the results will inform the taxonomy finalisation and subsequent development of the core outcomes set. Conclusions: This taxonomy will illustrate the range of potential outcomes and form the basis of a consultation to establish a core outcome set, encouraging consistent outcome measurement and selection by trialists and systematic reviewers. It may also guide the development of intervention logic models.

P41: An evaluation of harvest plots to display results of meta-analyses in overviews of reviews Hartling L1 , Wingert A1 , Crick K2 , Fernandes R3 , Thomson D1 , Williams K4 1 Cochrane Child Health Field, University of Alberta, Canada; 2 School of Public Health, University of Alberta, Canada; 3 Cochrane Child Health Field, Lisbon Academic Medical Centre and Portuguese Branch of the Iberoamerican Cochrane Centre, Portugal; 4 Cochrane Child Health Field, University of Melbourne, Australia

Background: Harvest plots are used to display evidence from complex and diverse studies or results graphically. They group studies based on whether they demonstrate a positive, negative or no effect, and can display additional information, such as quality of evidence. Objectives: To: (i) determine the feasibility of using harvest plots to depict complex results in overviews of reviews; (ii) survey end-users to determine their preferences for graphical display of data, and their understanding of data presented as harvest plots. Methods: We developed harvest plots for two overviews. We conducted a survey of 279 members of the Cochrane Child Health Field, which includes pediatric healthcare providers and researchers. We asked whether harvest plots, used in conjunction with tables typically presented in overviews of reviews, are superior to tables alone. Results: Fifty-three members responded. On a scale from 0 to 100 (where 100 was most favourable), average responses showed little difference between harvest plots and standard Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

tables with respect to the following features: well suited to display results from meta-analysis [51.1 (SD 26.9) harvest plots, 55.8 (SD 24.6) tables; aesthetically pleasing (57.5 (SD 28.5) harvest plots, 44.5 (SD 25.0) tables]; easy to understand [53.2 (SD 26.3) harvest plots, 50.4 (SD 26.1) tables]; format intuitive (49.6 (SD 25.2) harvest plots, 44.7 (SD 24.2) tables). Twenty-two (48.9%) respondents indicated that harvest plots could be used in conjunction with tables to display results of meta-analyses; 20 (44.4%) respondents indicated they could be used with some improvements. Conclusions: Our goal was to explore a novel form of data presentation to promote uptake of evidence from overviews of reviews. Suitability and ease of understanding the harvest plots were similar to standard tables. Harvest plots were more aesthetically pleasing; however, standard tables were slightly better suited to display results. Neither presentation format was ranked highly on any variable. Further work is needed to identify and evaluate methods for presenting results from overviews to ensure they are readily understood by end-users.

P42: Use of Cochrane bladder cancer reviews: confirming the need for Wikiproject Medicine and assessing what is ‘UpToDate’ Shepherd A1 , Bain E2 1 Royal Adelaide Hospital; University of Adelaide, Australia; 2 Robinson Research Institute, University of Adelaide, Australia

Background: Wikipedia and other commonly used sources of online medical information, e.g. UpToDate, have become central pillars of medical education for patients and providers –it is estimated that Wikipedia medical articles are viewed over 180 million times per month; UpToDate receives a further 18 million monthly topic views. While UpToDate uses a ‘rigorous editorial process to synthesise the most recent medical information,’ less than 1% of Wikipedia articles have passed through a formal peer-review process. Objectives: To determine use of the Cochrane Prostatic Diseases and Urologic Cancer Group’s bladder cancer reviews on Wikipedia; and to compare use of these reviews in ‘UpToDate’ articles. Methods: Relevant Wikipedia and UpToDate articles were reviewed to assess use/non-use (determined by citation) of the eight Cochrane Reviews in the area of bladder cancer management (published 2000–2012), and to determine the impact of non-use. Results: None of the eight Cochrane Reviews were cited on relevant Wikipedia page(s). Relevant UpToDate articles however cited three of the Cochrane Reviews, and ‘Other published versions’ (published in the Lancet and Eur Urol) of two further Cochrane Reviews were cited. Where no Cochrane Review was cited, UpToDate was more likely than Wikipedia to reference an alternative review/meta-analysis, or individual trials (including randomised trials conducted/published post-publication of the relevant Cochrane Review). In most cases, the absence of reference to the relevant Cochrane Review led to missing/incomplete information, rather than inaccurate recommendations/conclusions. Conclusions: Evidence from bladder cancer management Cochrane Reviews is under-utilised on Wikipedia, leading to the transfer of incomplete healthcare information; as anticipated, use on UpToDate is more encouraging. The Cochrane Collaboration’s announcement in February 2014 of a partnership with Wikiproject Medicine represents an important step towards providing patients and providers with Cochrane Database Syst Rev Suppl 1–150 (2014)

53 accurate and clinically relevant online medical information; ensuring that Cochrane Reviews are ‘up-to-date’ will assist in this ongoing challenge.

P43: Evidence-based Indian first aid guidelines De Buck E1 , Van Remoortel H1 , Geuvens H2 , Vande Veegaete A2 , Vandekerckhove P2 , Agarwal SP3 1 Centre for Evidence-Based Practice, Belgian Red Cross Flanders, Belgium; 2 Belgian Red Cross Flanders, Belgium; 3 Indian Red Cross Society, India

Background: Training first medical responders in India has been considered a very cost-effective intervention for frequently occurring diseases and injuries. In 2013, the Belgian Red Cross-Flanders together with the Indian Red Cross, launched a project to develop evidence-based first aid guidelines and prevention advice specifically adapted to the Indian context. As a basis for these guidelines, scientific evidence was sought to decide which first aid and preventive interventions are effective. Objectives: Development of evidence-based guidelines to train lay people on how to manage and prevent emergency situations in India. Methods: In a previous first aid guideline project, we performed evidence-based reviews to study the effectiveness of several first aid and preventive interventions relevant for laypeople. Additionally, we identified evidence on effectiveness, safety, and feasibility of various first aid and preventive procedures from Indian studies, and on alternative interventions that have been used by Indian lay people. The quality of the scientific evidence was determined according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology. Results: After developing 68 specific search strategies in PubMed, 48 studies were selected that were in favor of 10 different first aid interventions (such as using rice oral rehydration salts/rice water for diarrhoea) and 16 different preventive interventions (such as burning neem oil in a kerosene lamp to prevent malaria, or yoga to promote a safe pregnancy), relevant for India. Based on the evidence conclusions and expert opinions of a multi-disciplinary panel of 12 Indian experts who met in New Delhi, final recommendations were formulated. This resulted in the Indian First Aid Guidelines. Conclusions: By searching for evidence from Indian studies and on alternative Indian interventions, we were able to develop evidence-based first aid guidelines adapted to the Indian context. In a next step, didactical materials based on these contextualized guidelines will be developed and tested in a pilot implementation phase in different districts of India.

They are less likely to be biased than narrative reviews (NRs), though NRs are historically more established among the medical community. A previous study comparing both review types found that, in numbers, the NR is by far the dominating type of review to be published, concluding that ‘journal publishing of non-systematic reviews [. . .] has far outstripped the growth of SRs’ (Bastian 2010). Objectives: We aimed to find out whether the advent of EBM is reflected in SRs having a larger increase in publication rates than NRs - are SRs beginning to draw level with NRs in terms of growth in publishing numbers? Methods: We conducted a literature search in MEDLINE in 2013 to assess the number of published NRs and SRs from 1990 to 2012. We used a filter to identify the number of SRs and then subtracted this number from the number of all publications indexed with the tag ‘review’ to obtain the number of NRs. The retrieved references were grouped according to year of publication and their numbers fitted to a parabola, with the quadratic term year2 indicating whether the increase is declining or growing. Results: Our analysis shows that from 1990 to 2012, the increase in number of published NRs per year is clearly slowing down (P value < 0.001), while the numbers of published SRs indicate an accelerated growth (P value < 0.001), thus confirming our hypothesis that SRs show a larger increase in publication rates than NRs. This leads to some alignment in publishing numbers: For instance, in 2004, the number of published NRs was approx. 14 times the number of published SRs, whereas the same ratio was approx. 5 in 2012 (Fig. 1). Conclusions: We agree with Bastian 2010, in that the staple of medical literature synthesis remains the non-systematic NR. However, our results also suggest a trend towards the SR as the scientific community’s preferred type of review: growth in published SRs is now exceeding growth in published NRs, which might eventually lead to the SR drawing level with the NR in terms of publishing numbers.

P44: Systematic reviews (SR) versus narrative reviews: is the SR drawing level? Schell L1 , Motschall E2 , Ruecker G2 , von Elm E3 , Meerpohl J1 1 German Cochrane Centre, Germany; 2 Department of Medical Biometry and Medical Informatics, University Medical Centre Freiburg, Germany; 3 Institute for Social and Preventive Medicine, University Hospital Lausanne, Switzerland

Background: Since their emergence in the 1980s, systematic reviews (SRs) have been the cornerstone of evidence-based medicine (EBM). Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

54 P45: Is the definition of ’Cochrane consumer’ well understood by the public for non-English speaking countries? Challenges and barriers in China Jia P, Zhang L, Mao X, Zhang M Chinese Cochrane Centre, China

Background: The Cochrane Consumer Network (CCNet) has always referred to its members as ’consumers’ who are patients, family members, and patient advocates. However, little is known about how the term ’Cochrane consumer’ is understood by people in non-English speaking countries. Objectives: To investigate the number, coverage and application of term ’consumer’ in China and to see whether the use of ’consumer’ in healthcare content matches the CCNet consumer definition. Methods: We searched four main Chinese databases including CBM, VIP, CNKI and WanFang using the search term ’consumer’, ’health care’ etc. from 1977 to March 2014. Then, we classified and analysed the searching results into different sections. Results: A total of 557 301 articles were associated with the term ’consumer’; 326 750 (65%) articles referred to economic consumers while 937(0.2%) related to healthcare of which 196 were health economics consumers, 186 were health policy consumers, 147 were pharmacy consumers and 144 were preventive medicine consumers. The number of articles increased from 2005 to 2008, with 57, 58, 90 and 94 respectively. However, the term ‘consumer’ in the above had nothing to do with the CCNet consumer definition. Only 28 (2%) medical articles referred ‘consumer’ to ‘patient’. Of 28 medical articles, 25% was about nursing care, and 21% was about doctor-patient relationship. Conclusions: There has been a considerable debate within Cochrane about a preferred label for people receiving healthcare, and there is no size shoe that matches all internationally. Very few articles in China indicate that ‘consumer’ used in a healthcare context matches the CCNet definition. ’Consumer’ in China mostly means that ‘one purchases some commodity or service’. Consequently, there is a great potential challenge about how exactly to interpret ’Cochrane consumer’ in the Chinese context.

for pregnant women in Germany were obtained from potential holding organizations, institutions, gynecologists and internet searches. Brochures addressing the risk of salmonellosis, toxoplasmosis and listeriosis were included. Two researchers independently analyzed brochures according to EBPI-criteria. Additionally, a systematic review about the risk of salmonellosis, toxoplasmosis and listeriosis during pregnancy was carried out to evaluate if nutrition brochures provide the current available evidence. Results were synthesized in a narrative way. Results: One-hundred and fifty-five (155) nutrition brochures for pregnant women were obtained. From these, 50 brochures that provided information on the risk of infection were included in the analysis. Most brochures did not include literature citations and only few brochures gave a risk description (Table 1). Advertising for supplementary products was present in 22% of the analyzed brochures. Results of the systematic review showed an incidence rate of toxoplasmosis of up to 4% in susceptible pregnant women. No studies analyzing the risk of salmonellosis or listeriosis were identified. Conclusions: Nutrition brochures for pregnant women in Germany should be adapted to comply with EBPI criteria to achieve a better quality of disseminated information. This is relevant not only to pregnant woman, but also to those responsible for creating brochures, and to physicians in charge of patient information.

P46: Evidence-based patient information: an analysis of nutrition brochures for pregnant women in Germany ¨ Kullenberg de Gaudry D1 , Grede N1 , Motschall E2 , Lins S1 1 German Cochrane Centre, University Medical Centre Freiburg, Germany; 2 Centre for Medical Biometry and Medical Informatics, University Medical Centre Freiburg, Germany

Background: Pregnant women are often unsure how to adapt their nutritional habits to meet the optimal nutrient supply for their unborn child and to prevent food-borne infections. Evidence-based patient information (EBPI) about an adequate nutrition during pregnancy is extremely important. EBPI can diminish the high uncertainty most women experience during this time and support them in their personal risk assessment. Nutrition brochures for pregnant women are available in Germany, but not all of them comply with EBPI criteria, which is a prerequisite for informed patient choice. Objectives: To evaluate nutrition brochures for pregnant women in Germany based on EBPI criteria. Methods: Nutrition brochures Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

P47: Experience of capacity building for evidence-based child health Agarwal A1 , Singh M2 , Jaiswal N1 , Thumburu KK1 , Jindal I1 , Mathew JL2 , Dutta S2 , Chadha N3 1 ICMR Advanced Centre for Evidence Based Child Health, Department of Pediatrics, Advanced Pediatric Centre, PGIMER, Chandigarh, India; 2 Department of Pediatrics, Advanced Pediatric Centre, PGIMER, Chandigarh, India; 3 Dr Tulsi Das Library, PGIMER, Chandigarh, India

Background: There is a growing need for evidence to inform policy and practice in India. The Post Graduate Institute of Medical Education Cochrane Database Syst Rev Suppl 1–150 (2014)

55 and Research, Chandigarh under the aegis of the Indian Council of Medical Research (ICMR), New Delhi established an Advanced Centre for Evidence-Based Child Health. This Centre conducted workshops and short courses to inform healthcare providers and researchers about the principles of systematic reviews (SRs). Objectives: To build capacity, conduct systematic reviews and promote practice and training of evidence-based health care in children. Methods: Twelve short courses on ‘How to practice evidence-based child health’ and six workshops on ‘Protocol development’ were conducted in Chandigarh and two satellite centers, one each in north and north east India, by the tutors from the ICMR Advanced Centre for Evidence-Based Child Health. We followed the module developed by Royal College of Paediatrics and Child Health. Pre-tests and post-tests with similar questions were conducted during each course to evaluate the knowledge of evidence-based child health. Short courses included lectures followed by small group interactive sessions on critical appraisal of randomised controlled trials, diagnostic test accuracy and conduct of systematic reviews. Results: Four-hundred-and-fifty participants from different parts of India were educated and sensitised to the Cochrane Collaboration and systematic reviews. More than half of the participants were not aware of Cochrane and SRs before the course. More than 50% indicated interest in further training in SRs. Most participants preferred small group interactive sessions. There was a significant increase in post-test marks compared to pre-test. Conclusions: Capacity building workshops and short courses in India are avenues for increasing awareness and contribution to Cochrane and systematic reviews. There is a growing need to continue to train individuals and develop programmes to support use of evidence for policy making and clinical practice in India.

P48: Translation initiative of the Brazilian Cochrane Centre: experience of the first 8 months Silva AA, Parra MT, Porf´ırio GJM, Freitas CG, Martimbianco ALC, Torres MFS, Pesavento T, Reiter M, Torloni MR, Riera R, Atallah AN Brazilian Cochrane Centre, Brazil

Background: The Cochrane Library is one of the main information sources for evidence in health worldwide. In Brazil and Latin America, access to The Cochrane Library is free, but language is still a barrier. The Cochrane Collaboration, along with the Brazilian Cochrane Centre (BCC), are trying to increase the access of health professionals and consumers to information through the translations of Cochrane Review abstracts. Objectives: We present the experience of translating of Cochrane Abstracts into Brazilian Portuguese by volunteers. Methods: Two BCC research assistants voluntarily agreed to translate Cochrane Review abstracts. A third BCC research assistant was trained by Cochrane translation support officers on how to prepare and upload translated texts to Archie. A five-step translation process was created: (i) a BCC coordinator selected a monthly list of priority review abstracts to translate; (ii) each volunteer selected and translated the abstracts and plain language summaries from that list; (iii) the translated texts were reviewed by the coordinator; (iv) the translations were transformed to html format and uploaded to Archie by a third volunteer; (v) translations were posted in The Cochrane Library and Cochrane Summaries. Results: Between May and December 2013, the BCC team translated 101 Cochrane Review abstracts and plain language summaries. Each volunteer translated an average of one abstract and Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

plain language summary per week. Conclusion: The initial experience of the BCC in translating Cochrane Review abstracts was successful and the initiative is growing. This year we intend to increase the number of volunteers and to optimize the translation process.

P49: Selecting Cochrane Reviews to be developed as Cochrane Clinical Answers: what selection criteria should we use? Pettersen K1 , Tort S2 1 Wiley, UK; 2 Cochrane Editorial Unit, UK

Background: Cochrane Clinical Answers (CCAs) (http://cochrane clinicalanswers.com/) aims to create a user-friendly entry point to the high-quality data from Cochrane Reviews to increase their clinical accessibility and thus improve patient care. To date we have developed 300 CCAs and we aim to create 600 CCAs by the end of 2014. However, we have over 5000 reviews to choose from. What mechanisms can we use to select reviews for CCAs? Objectives: To develop robust selection criteria for CCAs, ensuring appropriate coverage of key current clinical questions across a range of specialties, with a focus on internal medicine. Methods: (i) Market research conducted by Wiley established that hospital-based physicians were a key target market for CCAs, as Cochrane market penetration was not strong among this audience. (ii) Initial subject areas suitable for this audience decided in collaboration with Cochrane Review Groups (CRGs) based on prevalence and high morbidity of condition including high-hospitalization rates. (iii) Initial criteria for selection of reviews for CCAs established, including recently updated reviews and CRG-recommended reviews. (iv) After 250 CCAs had been created, coverage audited and subject areas decided for a projected core product containing 600 CCAs. (v) Initial selection criteria validated against list of ‘most accessed’ Cochrane Reviews identified by work done by Wiley in relation to The Cochrane Library impact factor calculation. (vi) Additional mechanism developed to create CCAs when practice-changing new reviews are published. Results: In many subject areas, the initial selection criteria for CCAs matched well with reviews cited in the ‘most accessed’ list, suggesting that many CRGs are prioritizing updating and developing of ’Summary of findings’ tables for their most accessed reviews. Conclusions: Selecting Cochrane Reviews on which to base CCAs is a challenge, as wider coverage will always be the ideal. However, CRGs can facilitate the process by prioritizing updating based on review usage, creating new reviews based on known unanswered clinical questions and encouraging authors to add ’Summary of findings’ tables.

n w

P50: Stories versus (and) evidence used in legislative testimony

a r d

Apollonio D1 , Bero L2 1 University of California San Francisco, USA; 2 University of Sydney, Australia

ith

Background: Public health researchers focus on evidence-based presentations to policymakers, while political scientists claim that legislators respond primarily to constituent demands. Objectives: We sought to determine how policymakers used evidence in making policy decisions. Our goal was to determine the extent to which

W

Cochrane Database Syst Rev Suppl 1–150 (2014)

56 policymakers relied on research, such as systematic revise, to make decisions, and whether there were effective strategies for increasing the use of evidence in policymaking. Methods: We interviewed 21 policymakers from 50 American states at the Rocky Mountain Evidence Based Health Care Workshop in June 2005. In the United States, state governments retain independent control over many public health issues and hear testimony and review evidence in support of or opposition to proposed laws. The interview guide asked them to reflect on what they and their colleagues found compelling when hearing testimony, and how evidence was used in decision making. Results: Although legislators claimed to use evidence in decision making, they found personal anecdotes most convincing, and their assessment of what constituted evidence was very broad, encompassing anecdotes as well as research studies. Policymakers did not feel that the presentation of numerical results or systematic reviews per se was compelling. Conclusions: These findings offer validation of the value of testimony, and suggest that public health advocates should go beyond a numbers-only approach in introducing scientific evidence into policymaking. Some of policymakers’ reliance on anecdotes appears to reflect structural constraints such as limited time to make decisions and the absence of staff members qualified to find and interpret research findings. Policymakers proposed that researchers giving testimony intended to influence policy should provide simplified information, specific examples, and personal stories, even sometimes working in tandem with advocates to strengthen the appeal of traditional evidence.

n w

a r d

ith

W

P51: Translating research into practice: current challenges in pediatric sleep medicine Khan S1,2 , Heussler H3 , Flenady V3 , McGuire T3 , Pache D3 , Norris R4 , Charles B5 1 Griffith Health Institute, Griffith University, Australia; 2 Mater Research Institute, University of Queensland, Australia; 3 Mater Research Institute, University of Queensland, Australia; 4 St Vincent Hospital, Sydney, Australia; 5 University of Queensland, Australia

Background: Therapeutic guidelines in the management of circadian rhythm sleep disorders in the pediatric population are currently non-existent, which leads to adverse disease burden and encourages off-label prescribing. Objectives: To discuss the methodological issues experienced while we developed a Cochrane Review for the management of orphan disorder and to recommend a safe and effective dosing regimen and monitoring parameters of melatonin in a pediatric population. Methods: A mixed method study approach was followed. Ongoing phase 2 trial of melatonin evaluated the primary and secondary outcomes for improvement in sleep characteristics in children with neurodevelopmental disorders and visual impairment. Outcome from the trial was compared to the evidence (a systematic review and protocols developed by our team in collaboration with Cochrane Developmental, Psychosocial and Learning Problems Group and non-RCTs excluded from the Cochrane Review). Results: No studies met the inclusion criteria for our Cochrane Review. The quality of the excluded studies was acceptable but not on a par with methods determined by Cochrane. From our ongoing phase 2 trial of melatonin, an improvement in total sleep time, reduction in sleep onset latency, and fewer night-time waking was observed with low dose melatonin (0.5–2 mg; n = 11) compared with baseline/placebo. This is in contrast with the ongoing practice of high dose melatonin usage in Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

pediatric sleep medicine in Australia and elsewhere. Conclusions: For highly-specialized therapeutic areas, a null-evidence report, based on strict inclusion criteria of Cochrane, runs the ethical risk of excluding potentially relevant practice-guiding evidence by determining that current practice lacks sufficient evidence (highly relevant for orphan disorders and drugs). Although melatonin is approved for its use in primary insomnia in adults (>55 years of age) in Australia and is used as hormonal supplement and/or complementary medicine elsewhere in the world, there is no clinical protocol or best practice tools for guiding GPs, pediatricians, and the family/carers on rational use in pediatric population.

P52: Development of plain-language summaries of knowledge syntheses and other published research on zoonotic and environmental public health issues in Canada Young I1 , Waddell L2 , Greig J1 , Pham M2 , Mascarenhas M1 , Michel P1 1 Public Health Agency of Canada, Canada; 2 Public Health Agency of Canada/University of Guelph, Canada

Background: The application of systematic reviews and other knowledge syntheses (e.g. scoping reviews) is increasing to address various high-priority issues at the human, animal and environmental health interface in Canada. However, knowledge syntheses are primarily disseminated in formats that are highly technical, lengthy, and are not easily accessible and understandable by research end-users such as policy- and decision-makers, policy analysts and advisors, and other public health professionals. Previous research indicates that these audiences would prefer to receive research information in shorter, more appealing formats that are written in plain language. Objectives: We developed a series of one-page, plain-language summaries of published knowledge syntheses and other research knowledge (e.g. risk assessments) at the Public Health Agency of Canada (PHAC) to improve the transfer and exchange of this knowledge to inform decision-making. Methods: We developed a standardized summary template that was informed by reviewing and evaluating plain-language summary formats from other organizations and sectors. In addition, we piloted a strategy to produce these summaries in a consistent and timely manner using trained knowledge brokers. Currently, the template has been applied to seven recently published (2012–2013) journal articles authored or co-authored by PHAC researchers working in the areas of food safety, zoonotic infectious diseases, and environmental health. Results: The initial summary template has received enthusiastic and positive preliminary feedback from our research end-users. The summaries appear to help address some fundamental reporting issues for knowledge synthesis evidence, mainly articulating findings in key ’take-home’ messages that can be easily reported to end-users in one or two short sentences. A formal evaluation of the summaries is planned to evaluate further their accessibility and utility for end-users. Conclusions: Summaries are a promising tool to enhance the uptake of research knowledge by senior decision-makers and those who support them in Canada about zoonotic and environmental public health issues.

Cochrane Database Syst Rev Suppl 1–150 (2014)

57 P53: Consumer support and education beyond national borders: phase 3 Walsh M1 , Vanderheyden A1 , Rader T2 , Pardo Pardo J2 1 Cochrane Upper Gastrointestinal and Pancreatic Diseases Group, Canada; 2 Cochrane Musculoskeletal Group/Canadian Cochrane Centre, Canada

Background: When diagnosed, consumers sometimes turn to patient organizations for support; however, the quality of treatment-related information is variable. Phase 3 of this project illustrates how consumers with Cochrane have enabled the dissemination of Cochrane Summaries for gastroenterology-related conditions to consumers and compares results with phases 1 and 2 that involved the dissemination of The Cochrane Library within English-speaking arthritis-related patient organizations for conditions under the remit of the Cochrane Musculoskeletal Group. Objectives: To disseminate Cochrane Summaries to English-speaking gastroenterology-related patient organizations worldwide. Methods: An internet search was performed to locate English-speaking gastroenterology-related not-for-profit national and international organizations representing conditions within the remit of the four gastroenterology-related Cochrane groups (Upper Gastrointestinal and Pancreatic Diseases, Inflammatory Bowel Disease and Functional Bowel Disorders, Hepato-Biliary and Colorectal Cancer). These organizations were provided with an article about Cochrane and Cochrane Summaries and asked to promote both via newsletter/Facebook/Twitter. Results: In total, 64 organizations were located and contacted. Twenty-five confirmed receipt of the article. Of those, six (five national within the USA, UK, Canada and Australia and one international) agreed to promote Cochrane Summaries in some way. Conclusions: Consumers can effectively disseminate Cochrane Reviews by promoting The Cochrane Library and Cochrane Summaries through patient organizations. The three phases of this project resulted in 21 arthritis and gastroenterology-related organizations agreeing to promote Cochrane Reviews or Plain language summaries in some way, thereby, potentially reaching thousands of consumers previously unaware of The Cochrane Library. This presentation will illustrate editor perceptions of Cochrane Summaries/The Cochrane Library and attempt to explain differences in responses between groups, benefits and limitations of this system and how it might translate to other Cochrane groups.

P54: Testing Treatments interactive (TTi): promoting better research for better healthcare D’Amico R1 , Formoso G2 , Antes G3 , Chen Y4 , Ravaud P5 , Badenoch D6 , Gaxiola GP7 , Glasziou P8 , TTi E9 1 Italian Cochrane Centre - TTi Italiano, Italy; 2 Emilia-Romagna Regional Agency for Health and Social Services - TTi Italiano, Italy; 3 German Cochrane Centre - TTi Deutsch, Germany; 4 Lanzhou University - TTi Chinese, China; 5 French Cochrane Centre - TTi Francais, France; 6 Minervation Ltd, Oxford TTi English, UK; 7 Hospital Pediatrico de Sinaloa - TTi Espanol, Mexico; 8 Bond University - TTi English, Australia; 9 Editorial Alliance, TTiEA

Background: The internet is now used widely as a source of information about the effectiveness and safety of healthcare interventions. Web and social media bear both great potential for knowledge transfer - and great dangers, because they also help Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

to diffuse distorted messages, often affected by conflicts of interests, presented uncritically and driven by sensationalism. If it is to stand out, reliable health information should not only be understandable, but also be presented in a practical, interesting and target-oriented fashion. Built on the success of Testing Treatments - a book for the public originally published in English –‘sibling’ websites in many languages have been established to promote better research for better health care. Methods: We will analyze some patterns of navigation of these sibling websites - called Testing Treatments interactive (TTi) - to provide data, which could help address their further development. Although there are certain common features across all of the sibling websites, the editor of each website is free to develop his/her specific website in ways judged to be appropriate for the target language group. In each website the texts of Testing Treatments (in the English original or in translation) are available for free download. Many other open access resources, such as videos, audios, cartoons, e-learning materials, and hypertexts of references from scientific journals, are available through the websites. TTi English has also started to differentiate target user groups, such as students and teachers, journalists and science writers, as well as patients and the public. Visitors of the websites will be asked to answer a short survey and to rank their navigation experience for each specific topic they were interested in. Expected results: We will identify preferences of navigation and kinds of materials considered most informative and/or interesting. We expect such information to provide useful insights for developing the TTi initiative and, more generally, for public health professionals and researchers who wish to improve strategies for promoting better research for better healthcare.

P55: The Chinese patients’ autonomy, preference and perceived involvement in clinical decision making Mu W1 , Zhang B2 , Shang H1 1 Tianjin Center for Evidence-based Medicine, China; 2 Tianjin University of Traditional Chinese Medicine, China

Background: Since 1970s, continuous efforts have been made to implement shared decision making (SDM) in the health system of western countries, but little has been done to promote it in mainland China, where a large portion of its population are now paying their own medical bills. Objectives: We aim to examine the extent of Chinese patients’ willingness to embrace higher autonomy by way of SDM, and its implementation in current practice. Methods: We searched four Chinese-language databases for studies examing the Chinese patients’ role in clinical decision making. Both the patients’ and care providers’ perspectives were provided. Pre-specified inclusion and exclusion criteria were used for study selection. The Cochrance risk of bias assessment tool and the Critical Appraisal Skills Programme were used for quality evaluation of quantitative or qualitative studies. Results: Our literature search identified 711 articles. After screening, five cross-sectional questionnaire surveys involving 1510 patients and 1208 doctors were included. (i) Patient’s autonomy preference: 42–55.3% of patients preferred a SDM model, while 40.5–53.5% enjoyed a more dominant role.(ii) Perceived involvement in care: patients’ perceived involvement ranged from 66.99% to 73.08% on a 100% scale. (iii) Information needs: 80.8–95% needed to be informed on treatment selection. (iv) A survey on general practitioner’s attitudes towards patients’ engagement showed 56.1%, 69.3% and 41.2% of respondents supported the idea of empowering patients

n w ra

d h it

W

Cochrane Database Syst Rev Suppl 1–150 (2014)

58 in making diagnosis, choosing medication and determing treatment options. However, doctors’ perceived patients’ involvement were 43.9%, 24.6% and 24.6%, respectively. (v) Predictors of patients’ involvement included trust in the doctor, feeling respected, and a friendly relationship with the doctor. Doctors’ perceived obstacles to eliciting patient engagement included time limit, communication problems and fear of de-authorization. Conclusions: Chinese patients generally held favorable attitudes towards SDM. While the patients perceived a mildly higher than average degree of participation, their doctors perceived it differently.

n w

a r d

ith

W

P56: Adaptation of rapid evidence review methodology Haynes J, Robbins C, Tom G Kaiser Permanente, USA

Background: Policy makers and other stakeholders often require evidence overviews conducted within an expedited time frame to inform decision making on relatively urgent matters impacting operations or clinical care. Objectives: We adapted an established methodology for rapid reviews (created by the Ottawa Hospitals Research Institute, OHRI), tailoring it for use with clients in our organization. Methods: The development of evidence-based guidelines requires systematic evidence review, which often takes months to produce. In an effort to equip decision-makers with accelerated evidence syntheses, our Evidence Services analytic unit examined rapid review protocols from other organizations and adapted the methodology created by the OHRI Knowledge to Action Knowledge to Action (KTA) team. Since adapting the methodology in 2013, we have conducted four rapid evidence reviews: (i) the effectiveness of group psychotherapy for depression and anxiety, (ii) effective social care models for the elderly, (iii) effective complex care models for the elderly with multiple chronic conditions and (iv) novel oral anticoagulants for the prevention of stroke in patients with non-valvular atrial fibrillation (AF). Results: An eight-step process was adapted from the KTA team, including: (i) needs assessment, (ii) clinical question development and refinement, (iii) proposal development and approval, (iv) systematic literature search, (v) screening and selection of studies, (vi) narrative synthesis of included studies, (vii) report production, and (viii) ongoing follow-up and/or evaluation with requestors. Conclusions: Based on client feedback, rapid evidence reviews have proven to be useful tools for synthesizing and sharing information on emergent issues in an expedited manner. Our clients have noted that rapid reviews provide a concise synthesis of the literature that is easily understood, with information not only on the conclusions reached, but also the methodology. The information then serves as the focus of the discussion with clinical experts and stakeholders, and helps them arrive at clear operational and clinical care recommendations.

n w

a r d

ith

W

P57: How was patient involvement in shared decision making covered in Cochrane Review? Jia P, Zhang L, Mao X, Zhang M Chinese Cochrane Centre, China

Background: Patient involvement has been emphasized by Cochrane Collaboration. However, little is known about the extent to which Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

patient involvement in shared decision making has been covered in Cochrane systematic reviews (CSRs). Objectives: To investigate patient involvement in shared decision making in The Cochrane Library including their coverage of the publication, country of origin and disease distribution. Methods: We searched the online version of The Cochrane Library for systematic reviews using the term ‘patient\ consumer involvement’, ‘patient\ consumer participation’ and ‘decision making’ from 1989 to February 2014. Microsoft Excel 2007 was used to perform data extraction and analysis. Results: In total there were 8217 CSRs published, of which only 111(1.4%) reviews were associated with patient involvement in shared decision making. Within the 111 reviews, less than 10 reviews were relevant to ’patient involvement’ from 1998 to 2006. However this number increased from 2010 to 2013, with 12, 15, 15 and 22 reviews respectively. The distributions of CSRs were mainly in the Consumer and Communication Group (15), Stroke Group (10), Effective Practice and Organization of Care Group (8), Musculoskeletal Group (7), and Multiple Sclerosis and Rare Diseases of the Central Nervous System Group (5). Among the total of 111 CSRs, 102 (92%) first authors were from developed countries, 41 of whom were from UK. Only six first authors were from developing countries including Brazil (4), Venezuela (1), Chile (1), China (1), India (1) and Thailand (1). Meanwhile, the institutions of the first authors mostly were universities (52%), colleges (10%) and hospitals (8%). Conclusions: Although patient involvement in both research and clinical practice is emphasized by the Collaboration, there is not much evidence to support this. Hopefully ‘patient involvement research’ will be focused in future CSRs.

P58: Potentially misleading reporting of comparisons in Cochrane systematic reviews and in major general medical journals Ciapponi A, Ariel B, Glujovsky D Instituto de Efectividad Cl´ınica y Sanitaria, Argentina

Background: P values and confidence intervals (CIs) provide different information. P values provide information about statistical significance while CIs add information on the direction and magnitude of the effect. Considered in isolation, P values can be misleading, and lead to health decisions based exclusively on statistics. Objectives: To assess the number of studies using the term ’statistically significant’ but not mentioning any CI for reporting comparisons in abstracts in The Cochrane Database of Systematic Reviews (Cochrane DSR) and in the five major general medical journals during the last decade. Methods: We searched in PubMed (15 March 2014) by year and by each of the top five journals according to their 2013 impact factor [New England Journal of Medicine (NEJM), Lancet, Plos Medicine (PM), BMJ, JAMA] and Cochrane DSR, from the year 2004 to 2014 using the search string: [(statistically significant[tiab]) NOT (‘Confidence Interval’[tiab] or ‘Interval, Confidence’[tiab] or ‘Intervals, Confidence’[tiab] or 95% CI[tiab])] AND [Jour]. Results: There was not a clear temporal trend in the number of abstracts that used the term ’statistically significant’ without corresponding CIs in the Cochrane DSR, although there was a negative trend in relative terms (Table 1, Fig. 1). In the five major general medical journals the problem was less frequent in absolute numbers, but bigger in relative terms than the Cochrane DSR in NEJM, Lancet and PM (Fig. 2). Conclusions: The potential misleading interpretation of reporting statistical significance without CIs is a Cochrane Database Syst Rev Suppl 1–150 (2014)

59 problem that shows a slight downward trend over the last decade in the Cochrane DSR, but not a clear pattern among the top-five rated general medical journals. There have been clear recommendations about this problem in the Cochrane Handbook for Systematic Reviews of Interventions since September 2008, but statements are not so clear in the other journals. Probably clearer guidelines for authors would contribute to a solution.

collaborators implemented activities from the communication and marketing strategies plan. Results: The actions resulted in: (i) 18-year identity: the BCC logo has been modified to celebrate the Centre’s eighteenth anniversary. (ii) We standardised the eighteenth anniversary logo in all the materials used in virtual and paper communication. (iii) Improvement of language used on the BCC webpage, in continuous updating of events, and promotion of workshops and activities (including links to new translations and online courses). (iv) Social media: in its first year the BCC Facebook account had 314 followers. On 27 March 2014, it had grown to 607 followers; 135 posts were published and the top 10 posts had 14 708 accesses. Linkedin, YouTube and Pinterest accounts are being developed. (v) Development of a new institutional website: the layout was created to facilitate access to health professionals and consumers. We elaborated the new content by organizing new sessions. The newest version offers more information, it is more functional and allows more communication with other Cochrane Collaboration websites. (vi) The press: we hired a press adviser to inform the general public and opinion formers about BCC actions and evidence-based health culture. We held an Evidence-Based Health workshop for journalists with more than 20 communication professionals attending. Conclusions: The communication and marketing plan activities developed by volunteer collaborators promoted visibility and interaction between the BCC and different sections of the public. It increased recognition of the BCC as a research pioneer center in Brazil. It disseminated the concepts of evidence-based health in Brazil when making health related decisions.

P60: Translation and validation of the ’Risk of bias’ (RoB) tool to Brazilian Portuguese Martimbianco ALC, Porf´ırio GJM, da Silva AA, Freitas CG, Parra MT, Torres MFS, Reiter M, Pesavento T, Riera R, Torloni MR, Atallah AN Brazilian Cochrane Centre, Brazil

P59: The Brazilian Cochrane Centre new communication and marketing strategies Reiter M, Fioretti B, Torloni MR, Riera R, Martimbianco ALC, Torres MFS, Porf´ırio GJM, Parra MT, Silva AAD, Pesavento TF, Freitas CG, Atallah AN Brazilian Cochrane Centre, Brazil

Background: In 2013, the Brazilian Cochrane Centre (BCC) implemented activities for professionals and consumers in order to increase its dissemination. We emphasized considering systematic review (SR) information for health decision making. We report these activities after a year of work. Objectives: To report the results of 1 year’s work on new communication and marketing strategies implemented by the BCC. Methods: The BCC volunteer Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: Methodological misconducts of randomized clinical trials (RCTs) may affect the results of studies, either overrating or underrating the effectiveness of an intervention. The Cochrane ’Risk of bias’ (RoB) tool was developed to improve the accuracy of evaluating bias in RCTs. Many Cochrane authors are non-native English speakers and frequently struggle to understand the meaning of the terms and words used in this tool. The translation and validation of the RoB to other languages would facilitate the use of this tool by all review authors. Objectives: Present our experience in translating and validating the RoB tool into Brazilian Portuguese. Methods: The Brazilian Cochrane Center (CCB) team has been authorized by Handbook editors to conduct this study. We are following the process of translation and validation recommended by Beaton et al. (2000). Results: The RoB tool is being translated into Portuguese by two independent Brazilian translators: (i) a Brazilian health professional experienced in evidence-based medicine and fluent in English, and (ii) a professional English-Portuguese translator. Neither of them have been informed about the objectives of the study. The translations will be compared by a committee composed of translators, health professionals and researchers. The first version will be translated back into English by two native translators with good expertise in Brazilian Portuguese. The two versions will be compared and discussed by the committee. The pretest version will be evaluated by a group of 20 individuals who will check the clarity of the text. The committee will Cochrane Database Syst Rev Suppl 1–150 (2014)

60 adapt the final version accordingly. Conclusions: The translation and validation of the Cochrane Collaboration RoB tool is ongoing. We expect that final version will be ready in 2014. This tool will help Brazilian authors in the conduct of future Cochrane Reviews.

P61: Community symposiums: an innovative method to share evidence at field level Perera M1 , Guruge D2 , Peiris K3 , Samarasinghe D4 1 Department of Public Health, University of Kelaniya, Sri Lanka; 2 Health Promotion Unit, Rajarata University of Sri Lanka, Sri Lanka; 3 Plan Sri Lanka, Sri Lanka; 4 Department of Psychological Medicine, University of Colombo, Sri Lanka

Background: Health Promotion, defined by the World Health Organization in 1986 as a ‘process of enabling people to increase control over, and to improve, their health’, is still in its innovative phase. New methods of evaluation and knowledge sharing too are being developed as the field progresses. ‘Community symposiums’ that utilize a narrative method to present health promotion processes is an example. Objectives: To explore the usefulness of community symposiums as a tool to share evidence in promoting community health in Sri Lanka. Methods: In-depth interviews and focus group discussions were conducted using structured protocols to find out the perceptions of identified partners, healthcare workers, community members and other stakeholders involved in the community symposiums held up to May 2012 in Sri Lanka. In-depth interviews were carried out among 24 participants recruited by judgment sampling after community symposiums held at district and national level. Eight focus group discussions were conducted among community members and field health staff separately in each district. Constant comparative analysis method was used to analyze data. Results: The main benefits of community symposiums reported were in evaluation, community empowerment, advocacy and knowledge sharing. The symposiums have also contributed to scaling-up the interventions within and between districts, building a supportive network among people of different community settings and making the processes more sustainable. The main weakness perceived by the informants was the lack of ability and experience to utilize community symposiums optimally. Conclusions: Community symposiums enhance the effectiveness of health promotion interventions by effective sharing of evidence at field level.

n w ra

d h it

W

P62: Online interest in Cochrane: data from Google Trends Timimi L1 , Adams C2 1 The Priory Academy, Lincoln, UK; 2 Schizophrenia Group, UK

Background: In 2012, 1.8 billion Google searches were made. ‘Google Trends’ is a free online tool that is able to track the number of times a specific search is made in each month from 2004 until the present date. The Cochrane Collaboration website (www.cochrane.org) is ranked 120 472 in the world in terms of internet traffic and 27% of traffic is from Google. Objectives: To investigate the frequency of Cochrane-related Google searches to date and to investigate the frequency of Google searches for other leading medical publications (2004 to date). Methods: One post-General Certificate of Secondary Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Education (GCSE) school student on a 1 week placement searched specific phrases in Google Trends associated with the Cochrane Collaboration, and Cochrane Reviews and the top four ’Medicine, General and Internal’ journals in the ISI Journal Citation Report. Data downloaded from Google Trends, and generated line graphs for relevant comparisons. Results: Searches related to the Cochrane Collaboration and The Cochrane Library consistently show downward trends. Many popular Cochrane reviews peak in search frequency annually. Interest is at its highest during the winter months, whilst through (northern hemisphere) summer the number of searches drop to an immeasurable amount. The top four medical journals all display similar downward trends. The UK’s National Institute for Health and Care Excellence (NICE) exceeded the search frequency of Cochrane, with an upwards trend in searches. Conclusions: Cochrane’s decline is part of the overall decline in online interest in leading medical journals. Whilst it is reassuring that Cochrane is not falling behind the top medical journals, online interest in Cochrane may be falling behind broader guidance publications such as those produced by NICE. The annual peaks in searches for specific Cochrane reviews would seem to suggest that interest may be driven by the Cochrane Colloquium.

P63: Assessment of attitudes towards participation, willingness to participate, and competence to participate in Randomized Controlled Trials: a cross-sectional qualitative and quantitative survey of psychiatric patients and key family members George D, Dholakia S, Tharyan P Christian Medical College, Vellore, India

Objectives: To understand the perspectives of Indian patients undergoing treatment for major psychiatric disorders, and of their relatives, regarding participation in randomised clinical trials (RCTs), and to assess their competence to consent. Methods: A cross-sectional quantitative and qualitative study using the ‘Prospective Preference Assessment’ method on moderately-ill, consenting, psychiatric inpatients, and their relatives, assessed their attitudes to participation in RCTs; their comprehension of information regarding two hypothetical trials; their willingness to participate; and the barriers and facilitators to participation. Clinical assessments of the capacity to consent were supplemented, in a sub-set, by independent assessments using the MacArthur Competence Assessment Tool for Clinical Research (MaCAT-CR). Quantitative data were presented as frequencies and qualitative data from audio-recorded, verbatim transcripts, were analysed for themes using the ‘Grounded theory’ framework. Results: All 20 participants (9 patients, 11 relatives) endorsed the need for RCTs and their methods; and altruism as a motive for participation. Only 50% were willing to participate in the hypothetical trials. Comprehension of information sheets was sub-optimal. Trust in doctors and organisations, and the opinions of family members’ facilitated participation. Unfavourable risk/benefit ratios, the use of placebos, distrust in doctors and organisations, financial and other hardships, and opinions of family members were barriers. The majority of participants judged competent on clinical assessment failed formal tests of competence to consent.

Cochrane Database Syst Rev Suppl 1–150 (2014)

61 P64: Workshop for journalists as a communication tool to disseminate evidence-based health to the general public Reiter M, Torloni MR, Riera R, Fioretti B, Martimbianco ALC, Torres MFST, Porf´ırio GJM, Parra MT, Silva AAD, Pesavento TF, Freitas CG, Atallah AN Brazilian Cochrane Centre, Brazil

Background: In March 2014, the Brazilian Cochrane Centre (BCC) held a workshop for journalists to shorten the distance between evidence-based health and the communication vehicles in the country. Objectives: To present evidence-based health and to teach journalists to find evidence-based health information sources. That way, the communication vehicles would act as dissemination agents of reliable information to the general population. Methods: A free 4-hr walk-in workshop was given in conjunction with the ABRACOM (Brazilian Association for Communication Agencies). The lectures’ contents included: presenting the Cochrane Collaboration and its main study designs, basic concepts of evidence-based health and systematic reviews, evidence-based pediatrics, and journalistic cases in the health field published without scientific evidence. Results: (i) More than 20 health-communication professionals from public relations agencies, big media and specialized vehicles attended. (ii) The workshop content had a viral effect and as a result, two open television news story about the topic (TV Globo and RedeTV, the first has the biggest audience in the country). (iii) Ongoing: the newspaper Folha de S.Paulo is preparing important story news about health judicialisation in collaboration with the BCC. The Science UOL which is the main health and science Brazilian portal is preparing a research series that will include the Cochrane Collaboration findings. Conclusions: The workshop achieved its goal to approximate findings between the Brazilian media, the BCC and the Cochrane Collaboration. All results from this action are being analyzed. We expect more communication vehicles to start publishing evidence-based health content material. This was the first step to introduce the CCB collaborators and the Cochrane Collaboration studies as information sources for the Brazilian media; this resulted in up to date published content reaching the general public and those who form opinions.

P65: Cochrane vignettes: use and satisfaction of Cochrane Reviews in a Cochrane learning continuing medical education program Moja L1 , Pussegoda K2 , Kwag K3 , Grad R4 , Bjerre L5,6,7 , Lewin G8 , Becker L9 , Cowlard S10 , Renigan-Boyle V10 , Ni Ogain O11 , Tort S11 , Moayyedi P12 , Schaafsma ME2 , Pentesco-Gilbert D10 , Tugwell P13 , Ueffing E2 , Urquhart B10 , Tovey D11 , Grimshaw J14 1 University of Milan, Italy; 2 Canadian Cochrane Centre, Ottawa Hospital Research Institute, Canada; 3 IRCCS Orthopedic Institute Galeazzi, Italy; 4 Department of Family Medicine, University of McGill; Herzl Family Practice Centre, Canada; 5 Department of Family Medicine, University of Ottawa, Canada; 6 Department of Epidemiology and Community Medicine, University of Ottawa, Canada; 7 Bruyere Research Institute, Canada 8 Department of Family Medicine, University of Ottawa, Canada; 9 Cochrane Innovations, Canada; 10 Wiley-Blackwell, UK; 11 Cochrane Editorial Unit, UK; 12 Department of Medicine, McMaster University, Canada; 13 Cochrane Musculoskeletal Group, University of Ottawa, Canada; 14 Cochrane Centre, Ottawa Hospital Research Institute, Canada Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: Although Cochrane has emphasised the role of online continuing medical education (CME) in dissemination of systematic review (SR) findings, it has not proposed specific strategies for the incorporation of SR contents into CME programs. Objectives: We developed a suite of online CME modules targeting health professionals based upon Cochrane SRs. Methods: CME modules are based upon published Cochrane SRs addressing gastrointestinal, back, inflammatory bowel, and musculoskeletal conditions. Each module includes five multiple choice questions plus a fictional and memorable vignette featuring ’Dr. Cochrane’. Vignettes are produced through a multi-step editorial process to ensure scientific, editorial, and educational rigour. The CME modules are accredited by The Royal College of Physicians and Surgeons of Canada, the College of Family Physicians Canada, and The Accreditation Council for Continuing Medical Education of the United States of America. Since September 2013 the program is accessible worldwide. A five-point rating scale [from 0 (poor) to 5 (excellent)] was used to evaluate satisfaction, quality of educational content and changes in knowledge and practice. Results: Sixty Cochrane vignettes have been published on the Cochrane Learning platform to date. Between inception and January 2014, a total of 600 accesses were recorded by diverse health professionals across nearly 70 countries. The average overall evaluation and quality of the educational content was rated as 3.91. The majority of users strongly agreed (4.19) that they would apply the knowledge and skills derived from the Cochrane vignettes to practice while expressing confidence (3.74) that the activity will improve their professional effectiveness. The rate of access considerably increased in December, reflecting the need for clinicians to complete their annual credit requirements. Conclusions: ‘Dr Cochrane’ is an innovative program that provides health professionals with the opportunity to improve patient outcomes and the efficient use of healthcare system resources. CME programs represent an opportunity to improve the relevance and accessibility of Cochrane SRs.

P66: Nursing faculties’ knowledge and awareness of systematic reviews and Cochrane Balusamy P1 , Prakash D2 , Seema S1 , Balusamy P1 1 Navodaya College of Nursing, Raichur, Karnataka, India; 2 Navodaya College of Physiotherapy, Raichur, Karnataka, India

Background: Much confidence has been placed in the nursing profession’s potential to have a positive impact on the health care system. Evidence-based practice is regarded as a core competence to improve health care quality. Good systematic reviewing is the top most strategy in the hierarchy of levels of evidence and the core focus of Cochrane is the systematic review. Descriptive study was done to describe nursing faculties’ knowledge of systematic reviews and awareness of Cochrane. Objectives: To assess nursing faculties’ knowledge of systematic reviews and awareness of Cochrane. Methods: The study was conducted among 100 nursing faculties working in selected nursing colleges in selected universities of health sciences in south India. A convenient sampling technique was used to collect the data. The self-administered questionnaire consisted of a demographic profile, questions to determine knowledge about systematic reviews, and an awareness question about Cochrane. Results: The study results showed that nearly half of the nursing faculties had low knowledge (56%) about systematic reviews, however, Cochrane Database Syst Rev Suppl 1–150 (2014)

62 more than half of the nursing faculties (65.7%) had heard about Cochrane, 66.7% knew of The Cochrane Library, but only 48% had heard about the Cochrane Nursing Care Field. Only 18% had used the Cochrane Database of Systematic Reviews, and 13% knew that Cochrane in India is located in Vellore. A statistical relationship was found between the level of knowledge and academic qualification among nursing faculties. Conclusions: Nursing faculty must involve themselves in educational and research activities to make the students aware about the reliable data base for evidence based practice .The low awareness of systematic review among Nursing Faculties creates a need for educational interventions about systematic review for the benefit of effective nursing care services.

P67: The Cochrane Library iPad edition: is it meeting the needs of users? Helmers M1 , Stewart G2 , Abaring T1 1 Wiley, USA; 2 Wiley, UK

Background: The Cochrane Library iPad edition was launched in December 2012. The app was devised to provide a new platform for the dissemination of Cochrane Systematic Reviews and is free to download via the Apple iTunes Store. Each issue contains an editorial and up to 12 specially abridged and enhanced Cochrane reviews chosen by the Editor-in-Chief. Objectives: To analyse data collected from visitors to the app. To survey users of the app to find out if the features available are meeting their expectations. To survey non-users of the app using The Cochrane Library social networks and email contacts to find out whether there are features that may make the app more desirable and useful to new potential users. Methods: Google Analytics has been used to collect data on visitors to the app since just after launch (9 January 2013). A survey using the online survey platform Qualtrics will be promoted via the app, email messaging, and The Cochrane Library’s social media channels. Results: The app was downloaded just under 18 000 times in 2013 and has experienced unexpected popularity with users in Colombia and other countries that are not traditionally associated with high usage of The Cochrane Library. The results of the user survey will be analysed in June 2014. Conclusions: The app is currently appealing to a wide audience. We wish to continue to develop the app to meet the expectations of users and provide features that match their needs. Conclusions will be drawn from the analysis once the data are available.

P68: Understanding use of evidence for health policy decisions: three case studies Chaudhary N Futures Group, India

n w ra

Background: Health policy makers’ decisions are influenced by many factors including the subjective manner in which they individually process decision-relevant information. Objective: To understand the role of the evidence in influencing existing health policy decisions. Methodology: This is a descriptive study of three selected interventions for addressing maternal and child health under the National Health Mission (NHM), India. The evidence for the intervention, its relevance to policy decisions and health outcomes

d h it

W

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

were studied using secondary data. Results: Case Study (CS) 1; Cochrane Review (CR) 2011 reconfirms WHO recommendation that folic acid (FA) supplementation in periconceptional period prevents neural tube defects. Under NHM, Auxiliary Nurse and Midwives received FA in the drug kits supplied to subcentres, but lack of any policy direction, capacity building efforts and monitoring, resulted in ineffective use of FA. CS 2: The National Iron+Initiative 2013 expanded the target group for biannual deworming to children of 1–5 years and adolescents of 10–19 years. WHO recommends biannual albendazole in endemic areas with >50% prevalence of soil transmitted helminths (STH).This decision was taken in the absence of data on prevalence of STH. However, CR (2012) suggests lack of evidence regarding suggested benefits. CS 3: WHO (2007) recommended use of Ready to Use Therapeutic Food (RUTF) for home-based management of Severe Acute Malnutrition (SAM). India (has 8 million SAM cases) has often debated the use of indigenous foods versus RUTF. CR 2013 states either RUTF or flour porridge can be used to treat children at home. Lack of rigour and evidence on outcomes in studies using indigenous food led to government investing in research pending which no conclusive decision taken. Conclusions: Inclusion of evidence-based intervention unaccompanied by policy decision and health system action results in ineffective coverage. Conflicting evidence on interventions is likely to promote subjective policy decisions. Contextual relevance, quality and conflict of interest in evidence impedes decision making. which impacts health of mothers and children.

n w

a r d

ith

W

P69: The snowball effect: promoting Cochrane evidence via social media Littlewood A, Riley P Cochrane Oral Health Group, UK

Background: Social media allows the exchange of information and ideas in virtual communities and networks and could be a useful tool to promote the use of Cochrane evidence. Objectives: To track a recently published Cochrane Oral Health Group Review, and see how it has been promoted via social media channels (e.g. blogs, Twitter, etc.). Methods: The Cochrane Library is now linked to Altmetrics, which shows article level metrics for individual Cochrane Reviews. The Oral Health Group actively promoted the review ‘Triclosan/copolymer containing toothpastes for oral health’, and then tracked the Altmetrics score to see how often it was retweeted on Twitter, how often it was blogged, and how this compared to other articles of the same age. We also looked at where in the world it was mentioned on social media. Results: According to Altmetrics, the review was blogged three times, and tweeted 58 times by 47 people. It also appeared on two news sites. It was tweeted in Spanish, Dutch and Japanese as well as English. The Twitter activity came primarily from the UK, but also Australia, Spain, Japan, the Netherlands, Peru, Chile and Canada. The article has an Altmetrics score of 74, which means it is in the top 5% of all articles tracked by Altmetrics (1.9 million articles). Compared to other articles in the Cochrane Library, it has scored higher than 95% of the other reviews published around the same time. Conclusions: The results of this experiment show that social media can attract attention to Cochrane evidence. However, we need to know more about how often this results in more people reading the full review, and how great an impact it could have on practice or informing patients. There are limitations to Altmetrics, as it did not initially pick up all blog posts about the review. Cochrane Database Syst Rev Suppl 1–150 (2014)

63 P70: Patients and doctors working together to improve health service: a survey analysis of shared decision-making between clinicians and patients Jia P, Zhang L, Mao X, Zhang M Chinese Cochrane Centre, China

Background: The doctor-patient dispute has been increasingly prominent in recent years in China. Furthermore, openness in communication is seriously jeopardised by the lack of trust between patients and doctors. Objectives: To conduct a survey of the possible factors that cause the tension between patients and doctors; to provide an effective way to encourage shared decision-making between them so as to alleviate the current tense relationship. Methods: A survey including 16 items (eight for patients and eight for doctors) was conducted in one of the Level A hospitals in the southwest region of China. A total of 853 questionnaires were provided, 580 for patients and 273 for doctors respectively. Microsoft Excel 2007 was used to perform data extraction and analysis. Results: The survey yielded 822 questionnaires with a 96.37% response rate. Most of the patients (93.19%) want doctors to seek their opinions when providing the treatment, while 61.48% of doctors indicated it was difficult to share decision-making with patients. Over half the patients (60.47%) would like to obtain information from doctors, whereas, 52.75% of doctors said that it was difficult to provide patients with high quality and reliable evidence or information. Twenty percent of the patients were not fully satisfied with the existing treatment, while 63% of doctors indicated that it was difficult to meet patients’ desires. The reasons for the difficulties in communication were mainly about: short communication time (50.48%) and improper communication methods (37.07%). Only 14.61% of doctors said that they had enough time to communicate with patients. Conclusions: Lack of openness in communication was identified as a major factor that hindered shared decision-making. A successful bidirectional way of encouraging shared decision-making to alleviate the current tense relationship for both sides is needed.

P71: Tamil language translation of Cochrane summaries Hariohm K1 , Prakash V2 , Saravankumar J1 , Vasanthan R3 , Jeyanthi S1 1 MSAJ College of Physiotherapy, India; 2 Ashok & Rita Patel Institute of Physiotherapy, Charotar University of Science and Technology University, India; 3 Oxford College of Physiotherapy, India

Background: Billions of people living in low- and middle-income countries speak native languages other than English. Limited availability of Cochrane Reviews in languages other than English can undermine its potential to reach populations with the highest disease burden that need (evidence-based) quality health care. Tamil is a language spoken by about 70 million people around the world and is the official language of five Asian countries including India. Hence, there is a need to translate Cochrane summaries into Tamil. Methods: A team of five physiotherapists and two non-medical people with Tamil as their first language formed to translate Cochrane summaries. In the first step, three Cochrane summaries including the Abstract and the Plain language summaries related to physiotherapy were selected and independently translated by two physiotherapists of the team. In the Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

second step, all the team members reviewed the translated material and the final version was prepared by consensus. In the third step, the final version of the translated summaries was presented to the potential target audience: native Tamil-speaking physicians, physiotherapists, patients and caregivers. They were asked to provide feedback on clarity and comprehensibility of the summaries. To facilitate easy understanding of the translation, we also prepared a glossary of common terminologies used in Cochrane Plain language summaries. Results: In general, participants provided positive feedback and perceived all the translated Tamil summaries as simple and easily understandable. Having a glossary was considered helpful for reading the summaries. The review team considered, translation of a few English nouns such as ‘treadmill’ into Tamil to be a potential source of confusion and misunderstanding. Hence we decided to use the English word within the bracket along with the Tamil translated word. Conclusion: Translation of Cochrane Plain language summaries into Tamil is feasible and can be valuable. It can help Cochrane efforts to disseminate high quality, relevant and up-to-date synthesized research evidence to more than 70 million people around the world.

Poster Session 3 P72: Non-comparable patients in non-randomised comparative studies: misleading study design? Armstrong N, Deshpande S, Riemsma R, Kleijnen J Kleijnen Systematic Reviews Ltd, UK

Background: Non-randomized comparative studies can be a source of effectiveness data, particularly in the absence of randomized studies. However, given the risk of selection bias, it seems reasonable to expect such studies to make some attempt at comparability i.e. similar populations allocated to each treatment. One would not expect treatments to have been explicitly selected based on patient prognosis. Objective: To assess the extent to which non-randomized comparative studies are subject to explicit selection on the basis of prognosis, using a systematic review of fenestrated or branched endovascular abdominal aortic aneurysm (fEVAR or bEVAR). Methods: A systematic review was conducted following published guidance. Bibliographic databases such as Medline (OvidSP), EMBASE and CENTRAL were searched on 1 October 2013 without date limits. To be included, non-randomized comparative studies had to demonstrate no explicit intent to select the treatment based on prognosis. Results: Twenty-four studies were ordered for full text screening, but none satisfied our inclusion criteria. Nineteen of these studies were excluded due to study design; five of these were found to have selected treatment on the basis of prognosis despite the fact that they were described as comparative studies. Therefore, these studies were considered to have selection bias and thus non-comparable groups. Moreover, the studies had included patients in the fEVAR/bEVAR group who were actually ineligible for the comparator, open surgical repair (OSR) due to, for example old age (Table 1). Conclusion: In fEVAR/bEVAR not one non-randomized study attempted comparability of patients. Indeed, they explicitly used the same method of selection i.e. according to prognosis. Indeed, some patients treated with fEVAR/bEVAR were not even eligible for OSR. This is only one example and it seems unlikely that it is unique. We recommend research to investigate comparability in other clinical areas, Cochrane Database Syst Rev Suppl 1–150 (2014)

64 particularly where randomized trial evidence is scarce. Researchers and publishers should take particular care when describing or assessing what can be informed from this type of study design.

focuses on trial-level imputation from non-parametric summaries, but also considers algebraic recalculation from parametric summaries, trial-level imputation from external data sources or another treatment group internally, trial-level imputation of correlation and solutions at the meta-analysis level. For each method, we note whether it calculates the mean or the SD, list which summary statistics are needed to implement it and summarise any assumptions.

P74: Improvement was needed for flow diagram in network meta-analysis Lou L, Xie W, Liu L, Wei Y, Mao L, Tian J Evidence-Based Medicine Center, Lanzhou University, Gansu, China

P73: Meta-analysis of continuous outcomes: systematic review of methods available for dealing with missing mean and standard deviation values Butcher I1 , Brady M2 , Lewis SC1 , Murray GD1 , Langhorne P3 , Weir CJ1 1 Centre for Population Health Sciences, University of Edinburgh, UK; 2 Nursing, Midwifery and Allied Health Professions Research Unit, Glasgow Caledonian University, UK; 3 Institute of Cardiovascular and Medical Sciences, University of Glasgow, UK

Background: Over a third of stroke reviews in the Cochrane Database of Systematic Reviews include a continuous primary outcome; three-quarters contain a continuous secondary outcome. These continuous outcome measures are often of crucial relevance to patients, carers and healthcare professionals (e.g. quality of life measures) or are pivotal in economic evaluations (e.g. hospital length of stay). Many continuous outcomes in this context are not normally distributed, and resulting analysis strategies and reporting vary. Omission of mean or standard deviation (SD) data from clinical trial reports, perhaps due to the skewed distribution of the outcome, can prevent the inclusion of the trial in a standard meta-analysis, potentially causing bias in that analysis. Objectives: To identify current approaches and develop improved methods for handling missing summary statistics for continuous outcomes within meta-analysis, enabling best use to be made of available clinical trial evidence. Methods: We investigate how best to impute the mean or SD where either of these has not been reported. In certain circumstances these are suitable summary statistics to analyse in a meta-analysis, regardless of the underlying continuous distribution. We report on a comprehensive review of the literature to identify all methods of deriving missing means and standard deviations. We searched electronic resources (including MEDLINE, EMBASE, Web of Science, BioMed Central and The Cochrane Library), relevant journals and grey literature from inception up to March 2014. This updates a previous review (Wiebe 2006) of methods used to determine the SD and extends it to include methods for imputing the mean. Our search Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Background: A flow diagram can be very useful; it should depict all the included studies that fulfilled the eligibility criteria, whether or not data have been combined for statistical analysis. Objective: To investigate the usage frequency and complete degree of flow diagram in network meta-analysis (NMA). Methods: PubMed, EMBASE, ISI Web of Knowledge, and The Cochrane Library were searched, to identify published NMAs. Two reviewers independently extracted data according to following items related to flow diagram reporting: whether flow diagrams were used, whether the databases that were used in retrieval were reported, whether reasons for inclusion and exclusion of studies, etc. were reported. Analysis was performed using Excel 2007. Results: Two-hundred and five (205) NMAs were included in the analysis of frequency of use of flow diagrams: 71.22% (145/205) of the NMAs reported a flow diagram; 21.46% (44/205) described the screening process using text; and 7.32% (15/205) did not describe relative information of screening process including text and flow diagram. For the searching phase, 71.22% (146/205) reported the databases searched, and 16.10% (33/205) reported other resources, the searching records were reported with a median of 720 (range: 28 to 29 773). For the screening phase: 66.83% (137/205) screened the literature by reading the title and abstract, and 33.66% (69/205) reported reasons for inclusion and exclusion; 68.29% (140/205) screened the literature by reading the full text, and 56.10% (115/205) reported reasons for inclusion and exclusion. Finally, the median of included studies was 16 (range: 4–437). Conclusion: The reporting quality of flow diagrams was fair in NMAs. The reasons for inclusion and exclusion of papers should attract more attention.

n w ra

d h it

W

P75: Reporting of perinatal mortality and its components in Cochrane Reviews: still room for improvement? Bain E, Middleton P, Crowther C Robinson Research Institute, School of Paediatrics and Reproductive Health, University of Adelaide, Australia

Background: Expert opinion favours the separate reporting of stillbirth and neonatal death in favour of (or as well as) the combined measure of perinatal mortality. Stillbirths and early neonatal deaths now often differ substantially with respect to their principal known or suspected causes; in high-income countries in particular, aetiological differences exist. Cochrane Database Syst Rev Suppl 1–150 (2014)

65 While a presentation at the 2010 Cochrane Colloquium highlighted this need for ‘clearer reporting of perinatal deaths in Cochrane Reviews... particularly separate reporting of fetal and neonatal deaths... ’ progress towards this goal is unknown. Objectives: To review recent approaches to the reporting of perinatal mortality, fetal and neonatal death in Cochrane Reviews, and to assess the utility of the reviews for identifying promising interventions. Methods: We reviewed new intervention reviews published by the Pregnancy and Childbirth Group in 2012–2013, assessing the inclusion of pre-specified primary and secondary review outcomes relating to perinatal mortality, and the separate reporting of its components; we also reviewed reporting of outcome data by included trials. Results: Fifty of the 74 reviews included pre-specified review outcomes relating to stillbirth and/or neonatal death; 31 pre-specified a primary review outcome (perinatal mortality was the most common). Only one-third of the reviews pre-specified separate components of perinatal death as outcomes. For almost half of the reviews (24/50) there were no data reported by trials for these outcomes; where data were reported, reviews were unable to confirm or refute reductions in deaths due to insufficient trials or participants. Conclusions: A more consistent approach to the reporting of perinatal mortality in Cochrane Reviews is required –particularly the separate reporting of fetal and neonatal death, and the standardisation of review and trial outcome definitions. While Cochrane Reviews are essential for assessing promising interventions for reducing perinatal mortality (and recognise this through pre-specification of outcomes), their ability to do so is often compromised by deaths not being reported in trials.

an emphasis on ‘phases’ in the process of a review of complex interventions we move away from a procedural focus on ‘mixing’ evidence synthesis approaches. We encourage review authors to consider which configurations of methods, techniques and approaches make sense to answer predefined, as well as emerging, research questions during the review process.

P76: A typology of questions for systematic reviews on complex interventions Hannes K Cochrane Qualitative and Implementation Methods Group, Belgium

Background: Several authors have pleaded for the careful planning of mixed method reviews. However, the focus on the procedural or design related aspects of conducting a mixed method review leaves potential review authors with the impression that differences between quantitative and qualitative approaches to synthesizing research evidence are merely technical. Decisions about how to approach complex interventions reviews should be guided by the type of questions that emerge during the review process. Objectives: To present a typology of questions assisting review authors in selecting appropriate review questions for different phases in a mixed method review. Methods: A research inquiry into the most beneficial promotion program to stimulate breast feeding will be used as an example to illustrate how questions related to effectiveness, feasibility, appropriateness and/or meaning may reveal themselves to an author team in the context of a review on complex interventions (Table 1). Results: Complex intervention reviews are contingent in nature and tend to move from one particular phase in a review to another. Different research questions may be generated in different phases in the iteration process (Figure 1). Consequently, review authors’ decisions (i) tend to grow organically, and (ii) are influenced by different beliefs about what needs to be covered in a review. A phenomenon under review may appear the same across quantitative and qualitative evidence syntheses strands. However, the distinction between inquiries into ‘meaning’ and ‘effectiveness’ reconciles a phenomenon to an exclusive quantitative or qualitative research paradigm. Conclusions: With Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

P77: Challenges in conducting economic evaluation informed by Cochrane Reviews: case study of adherence in epilepsy Plumpton C1 , Reuber M2 , Brown I2 , Marson A3 , Hughes D1 1 Bangor University, UK; 2 University of Sheffield, UK; 3 University of Liverpool, UK

Background: Cochrane Reviews typically focus on efficacy and effectiveness and rarely address efficiency which is important for informing resource allocation (1). Working with the Cochrane Epilepsy Group we aimed to conduct an economic evaluation of evidence sourced from the Cochrane Review of strategies for improving adherence to antiepileptic drugs (AED) (2). Methods: The review included five trials that differed appreciably hence we considered it inappropriate to combine data. We focused on the trial most closely resembling current clinical management of patients with epilepsy, which measured adherence with an objective and least biased method and for which we were able to obtain patient-level data to improve the accuracy of the economic evaluation (3). We calculated an indirect estimate of the impact of the intervention on costs and quality-adjusted life years (QALYs) by matching patients to those recruited to the SANAD trial Cochrane Database Syst Rev Suppl 1–150 (2014)

66 for which we also had access to patient-level data (4). Adherence outcomes from the trial were matched to the self-report adherence assessment in SANAD. Generalised linear models adjusted cost and utility for demographics and adherence. Results: Fifty-eight (of 69) patients from the adherence trial matched 450 (of 747) patients from SANAD. The mean cost of the intervention group, £740 (95% CI 615–929) is higher than that of the control group £681 (95% CI 515–954). QALYs in the intervention group were significantly higher than the control, 0.82 (95% CI 0.80–0.84), compared with 0.80 (95% CI 0.77–0.83). The incremental cost-effectiveness ratio is £3230/QALY. Conclusion: The model indicates that an intervention in which patients note when and where they intend to take their AEDs, and what they would be doing at the moment they will take their medications would be cost-effective in a UK setting, at a threshold of £20 000/QALY. As a source of evidence, Cochrane Reviews proved limited; nevertheless this exercise highlights the importance of identifying economic evidence when conducting a Cochrane Review.

References 1. 2. 3. 4.

Syst Rev 2013, 2:83. Cochrane Database Syst Rev 2011, 1. Epilepsy Behav 2009, 16:634–639. Lancet 2007, 369:1000–1015.

P78: Update of the Cochrane systematic review of interventions for preventing obesity in children: a research program Waters E, Steele E, Pettman T Cochrane Public Health Group, Australia

Background and objectives: The Cochrane Systematic Review ‘Interventions for preventing obesity in children’ (E Waters 2011) synthesises evidence from childhood obesity prevention activities. To ensure the next update reflects recent developments in obesity prevention activities, meets the needs of review stakeholders and does not duplicate existing reviews, it will be informed by a two-staged research program. Methods: The first stage aims to identify evidence-gaps regarding the effectiveness of childhood obesity prevention activities, from a policy perspective. First, a document analysis of international obesity policy documents will be conducted to identify priorities for childhood obesity prevention research. To validate this list of priorities, a web-based survey with a select group of international obesity policy experts will be conducted. The second stage is an overview of existing systematic reviews of interventions aimed at preventing childhood obesity. This exercise will identify gaps and limitations in the scope and methods of existing reviews. Results and conclusions: This presentation will present results from each stage of the research program, and discuss how the results will inform the development of the Cochrane Review update.

P79: Challenges and considerations involved in using AMSTAR in overviews of reviews Foisy M, Hartling L Alberta Research Centre for Health Evidence, University of Alberta, Canada

Background: Overviews (of reviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. Overview authors should report the methodological quality of the included SRs; this can be done using the Assessment of Multiple Systematic Reviews (AMSTAR) tool. Objectives: We examined methodological considerations involved in using AMSTAR in overviews. We assessed: (i) what differences exist in AMSTAR scores across Cochrane and non-Cochrane reviews, (ii) challenges involved in using AMSTAR to assess quality of included SRs, and (iii) feasibility of using the AMSTAR score as an inclusion criterion. Methods: We selected a sample of seven overviews in different clinical areas and searched for all SRs meeting each overview’s inclusion criteria. Two reviewers conducted AMSTAR assessments for each SR and resolved discrepancies through consensus. Challenges and issues surrounding the use of the AMSTAR tool were noted and discussed. Results: To date, we have assessed 72 SRs in four clinical areas (acute otitis media, bronchiolitis, eczema, procedural sedation). Average AMSTAR scores (/11) were much higher for Cochrane (9.95 ± 1.08) compared with non-Cochrane (4.95 ± 2.25) SRs (Table 1). Average agreement between reviewers was high overall, but slightly higher for Cochrane (90.9%) compared with non-Cochrane (85.1%) SRs. AMSTAR assessments were difficult when SRs were broader in scope than the overview’s clinical question and when SRs examining similar interventions made different methodological decisions or differed in conclusions. Conclusions: The relatively high agreement between reviewers when using AMSTAR for Cochrane and non-Cochrane SRs suggests that AMSTAR can be used successfully in the context of overviews. Cochrane SRs are often higher quality than non-Cochrane SRs; this seems to be largely driven by reporting and may not be associated with methodological quality. Nevertheless, non-Cochrane SRs with low AMSTAR scores may be missing important information and may be challenging to use in overviews. A minimum AMSTAR score may be useful as an inclusion criterion for overviews.

References Waters E, de Silva-Sanigorski A, Burford BJ, Brown T, Campbell KJ, Gao Y, et al. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

67 P80: The benefits and harms of methylphenidate treatment for attention deficit hyperactivity disorder (ADHD) in children and adolescents: preliminary results Storebø OJ1 , Krogh HB1 , Ramstad E1 , Danvad Nilausen T1 , Rosendal S1 , Skoog M2 , Groth C3 , Holmskov M1 , Løgstrup Magnusson F1 , Forsbøl B4 , Buch Rasmussen K1 , Kirubakaran R5 , Zwi M6 , Gillies D7 , Gauci D8 , Riegl M9 , Maia CR10 , Simonsen E1 , Gluud C2 1 Psychiatric Research Unit, Region Zealand Psychiatry, Denmark; 2 Copenhagen Trial Unit, Centre for Clinical Intervention Research, Copenhagen University Hospital, Denmark; 3 Paediatric Department, Herlev University Hospital, Denmark; 4 Child Psychiatric Clinic, Child and Adolescent Psychiatric Department, Denmark; 5 South Asian Cochrane Network & Centre, Prof. BV Moses & ICMR Advanced Centre for Research & Training in Evidence Informed Health Care, Christian Medical College, India; 6 Richmond Royal Hospital, South West London & St George’s NHS Mental Health Trust, UK; 7 Western Sydney Local Health District Mental Health Service, Cumberland Hospital, Australia; 8 Department Health Information and Research, Ministry of Health, Malta; 9 Medicines and Healthcare Products Regulatory Agency, Special Populations Unit (Paediatrics), UK; 10 ADHD Outpatient Program (ProDAH), Hospital de Clinicas de Porto Alegre, Federal University of Rio Grande do Sul, Brazil

Introduction: In this large Cochrane systematic review we will include about 150 randomised clinical trials (RCTs) and 220 non-randomised studies (NRS) for assessment of benefits and harms. The organizational and logistical challenges in conducting this review and synthesizing data were tremendous. Aim: To assess the benefits and harms of methylphenidate treatment for attention deficit hyperactivity disorder (ADHD) in children and adolescents. Methods: The work is being conducted by an international research group of 19 people, and the organisation of references, data extractions, and author correspondences to obtain additional data from about 250 studies, are organised using the online reference software Mendeley and Google Drive. The benefits are measured by parallel and cross-over RCT studies and these are combined in large meta-analyses as well as case control studies. The harms are assessed by both RCTs and NRS studies. The cohort studies are synthesised by the use of weighted means and case reports reported qualitatively. We are using a hierarchy structure for the different study designs in relation to each adverse outcome and thereby showing all the possible evidence from RCTs to case studies. Different subgroup analyses are conducted including risk of bias, types of rating scales, duration of treatment, parallel compared to first period cross over trials, sex, dose, and age. Results: Approximately 370 included studies are being processed systematically and the review is scheduled to be completed mid-2014. The logistical challenges, the organisation of the large amount of data and preliminary results are presented.

Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

P81: An extension of the Proportional Hazard Model for SROC modelling in meta-analysis of diagnostic studies ¨ Charoensawat S1 , Bohning D2 1 Health Sciences Program, Udon Thani Rajabhat University, Udon Thani, Thailand; 2 School of Mathematics and Southampton Statistical Sciences Research Institute, University of Southampton, UK

The number of meta-analyses of diagnostic studies is increasing and the models that deal with the summary receiver operating characteristic (SROC) have become popular, for example, the bivariate random effects model (BREM) and the hierarchical summary receiver operating characteristics (HSROC). A model named the proportional hazard model (PHM) has been developed. The PHM has a simple form, as it has only one parameter of interest, which is ’diagnostic accuracy’. The PHM is also easy to interpret, as the smaller the value of theta, the higher the diagnostic accuracy. The improved PHM structure is largely enriched by allowing a random-effects component. Based on a simulation study, the adjusted profile maximum likelihood estimator (APMLE) was considered to be the best of several estimators to estimate theta. The preferred model can be applied under several conditions. Especially observe heterogeneity because it occurs frequently in meta-analysis and ignoring it may lead to bias. Also, observed heterogeneity explain the variation in study outcome between studies. This leads to motivation to improve the extension of the PHM for SROC modelling in meta-analysis of diagnostic studies. A mixed model and a general linear mixed model are used for generalising the PHM for unobserved and observed heterogeneity. In the example we present, the analysis of meta-analysis data set by using an extension of the PHM for SROC modelling.

n w ra

d h it

W

P82: Should Cochranites adopt a specific hierarchy of outcomes? Lawrence K1 , Beaumier J4 , Mintzes B1 , Jauca C2 , Tejani A1 , Kwok C3 , Musini V2 1 UBC Therapeutics Initiative, Canada; 2 Cochrane Hypertension Group, Canada; 3 Queen’s University Internal Medicine, Canada; 4 Vancouver Health Authority, Canada

Background: Cochrane authors may not be choosing the most appropriate outcomes for their reviews. A survey conducted in 2013 found that Cochrane users disagreed with the review authors’ selection of primary outcomes 71% of the time. Participants indicated the authors’ selection did not reflect the most important outcomes. One root of this problem may occur during outcome selection in randomized controlled trials (RCTs). Objectives: A specific hierarchy of outcomes for Cochrane review authors would serve to create essential knowledge for Cochrane consumers with respect to the benefit versus harm of an intervention. Methods: A number of ’worst case scenarios’ are explored in order to learn from previous, harmful mistakes. Drawing on evidence in Therapeutics Letters that use Cochrane methods to assess treatment outcome, we point out where flaws occur in outcome selection of RCTs and discuss how a specific hierarchy of outcomes could increase the value and impact of Cochrane Reviews. Conclusions: A focus on key patient outcomes within a specific hierarchy allows assessment of net treatment benefits. This attention to outcome hierarchies, rather than a dichotomized approach that separates expected beneficial from unexpected harmful effects, could greatly benefit systematic review methodology. Cochrane Database Syst Rev Suppl 1–150 (2014)

68 P83: Lack of originality in non-Cochrane systematic reviews

P84: Systematic reviews of risk factors: methodological challenges and implications for evidence reviewers

Fusco N, Saldanha I, Gresham G, Li T Cochrane Eyes and Vision Group, US Cochrane Center/Johns Hopkins Bloomberg School of Public Health, USA

Pauwels NS, Dieltjens T, Van Remoortel H, Vandekerckhove P, De Buck E Belgian Red Cross Flanders, Belgium

Background: Lack of originality in biomedical literature is most problematic when there is substantial unattributed identical text. The standardized format of systematic reviews (SRs) may put them at higher risk for this. Objective: To evaluate lack of originality in non-Cochrane SRs and determine the frequency of text identical to Cochrane SRs Methods: We searched PubMed on 18 March 2014 (Search Strategy) to identify the 50 most recent non-Cochrane SRs of intervention  effectiveness. We used TurnItIn c , a web-based originality detection software, to find text within each section (e.g. Methods, Results) identical to a source identified by the software. We defined identical text as ≥20 continuous words from another source under any of the following scenarios: (1) reference but no quotation marks; (2) quotation marks but no reference; or (3) neither quotation marks nor reference.  Using Google Forms c , an investigator extracted the following data for each section: number of instances of identical text and, for each instance, the scenario, and whether the source was a Cochrane SR. We calculated the proportion of SRs with at least one instance of identical text and the median number of instances by section. We performed sensitivity analyses using 15 and 10 word cutoffs. Results: Forty-four SRs (88%) included at least one instance of identical text [median = 3 (interquartile range (IQR) 2–6.5) instances per SR]. Three SRs (6%) included at least one instance of identical text from a Cochrane SR [median = 2 (IQR 1–18) instances per SR; Table]. Two of those three had text identical to an earlier Cochrane SR on a similar topic. Most identical text was in the Methods section (75/228 instances, 32.9%). Most identical text did not have either references or quotation marks (66.7%). Conclusions: The high proportion of non-Cochrane SRs with non-original text is a concern. The fact that the wording may have originated from a similar Cochrane SR is worth further investigation. The SR community, including Cochrane, should discuss acceptable lack of originality in Methods and Results. Future research should evaluate if Cochrane SRs themselves include non-original text.

Background: Blood services play a critical role in healthcare by providing a safe, adequate and cost-effective supply of blood components. To achieve this, donors demonstrating certain characteristics that are considered to increase the risk of complications in the transfusion recipient, are deferred. Therefore, systematic reviews examining risk factors for blood safety (e.g. having tattoos, vaccination or blood disorder) are essential to set appropriate, up-to-date and evidence-based donor eligibly criteria. However, such reviews are challenging to conduct. Objectives: To give an overview of the methodological challenges related to the conduct of systematic reviews of risk factors. Methods: Literature reviews on risk factors were performed. Results: The following challenges were encountered during multiple stages of the review process. Firstly, we encountered selective reporting of statistically significant risk factors in the abstract. This can be solved by the inclusion of other risk factors than the factor of interest in the search strategy, and the incorporation of general wording about risk factors. As a result, the search will reach high sensitivity but specificity will be negatively influenced. Secondly, the study designs used for risk factor analysis (e.g. cross-sectional studies) differ from experimental designs and are more prone to bias. As a consequence, the initial level of evidence will be low, according to the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach. Finally, different approaches to control for confounding (e.g. by study type and/or by post-hoc statistical analysis) are being used. Therefore, the evidence reviewer will often not be able to pool data or to make final conclusions about independent risk factors. Conclusions: To inform policy makers in transfusion, highly sensitive literature searches concerning risk factors for blood safety are needed. Due to reporting issues, experimental flaws and statistical differences, systematically reviewing of risk factors is not only subjected to higher risk of bias but is also time- and cost-intensive.

P85: Characteristics of cross-over trials in dentistry and oral health reviews Kumbargere Nagraj S, Eachempati P Faculty of Dentistry, Melaka Manipal Medical College, Malaysia

Background: The two common problems associated with cross-over trials are carry-over effect and risk of drop-out. Due to these reasons, it is often difficult to extract suitable data from a cross-over trial (Higgins 2009). We undertook this study to see the common problems associated with dentistry and oral health reviews in relation to cross-over trials. Objectives: (i) To determine the number of published Cochrane dentistry and oral health reviews that include cross-over trials. (ii) To assess how many of these cross-over trials have considered sufficient washout period as one of their inclusion criteria. (iii) To determine how many reviews have used paired results in meta-analysis. (iv) To establish the risk of drop-outs in the included/excluded cross-over Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

Cochrane Database Syst Rev Suppl 1–150 (2014)

69 trials. Methods: We included all 154 published (updated) Cochrane dentistry and oral health reviews. Full texts were downloaded from The Cochrane Library. Titles, inclusion/exclusion criteria, number of cross-over trials included, washout period, drop-outs and data used in meta-analyses were extracted from each review. Results: Out of 154 full reviews, 13 had included 49 cross-over trials. In these 13 reviews, eight studies had defined the duration of washout period. Studies in three reviews did not need any washout period and studies in two did not mention washout period. Only five reviews mentioned a minimum washout period as one of their inclusion criteria. Twelve reviews excluded a total of 35 cross-over trials due to a variety of reasons that included insufficient/no washout period (58.3%), pooled data (33.3%) and other issues (8.4%). Meta-analyses were not done in eight reviews. Paired data were combined with parallel group data using generic inverse variance method in two reviews. Two reviews used the data from first half of a study, and one review analysed the cross-over data separately. Risk of drop-out was not a common problem in our study. Conclusions: Insufficient washout period and pooled data are the common problems faced by dentistry and oral health reviewers in dealing cross-over studies.

P86: Faster may not be better: data abstraction for systematic reviews Gresham G, Matsumura S, Li T Cochrane Eyes and Vision Group, US Cochrane Center, USA

Background: Data abstraction is at the core of a systematic review (SR). Earlier studies have examined error rates and level of experience, yet data on factors associated with the quality of data abstraction using modern data systems such as the Systematic Review Data Repository (SRDR) are limited. Objectives: Estimate the time taken for data abstraction using SRDR and examine factors associated with errors in data abstraction. Methods: We developed and pilot-tested a data abstraction form in SRDR to capture the characteristics of study design (68 items), participants (12 items), and results (15–25 items) of a trial report. Eight data abstractors (DA), four experienced and four less experienced, performed data abstraction for four trial reports each. DAs recorded time taken for completing each section of the form. We compared each datum entry against the reference value and calculated the proportion (%) of errors. Linear regression was used to examine the association between time and % errors. Results: Experienced DAs had an average of 3.3 years (range 2–4), and less experienced had an average of 9 months (range 0–12) working in SRs and participating in an average of 4.3 (range 3–8) and 0.75 (range 0–1) reviews, respectively. The overall mean time for data abstraction was 47.8 min (range 38.0–58.1 min) and the overall error rate was 17.0%. Less experienced DAs took less time to complete data abstraction and made 14.2% more errors across all types of questions (95% CI: 7.2–21.2%) (Table). Highest errors were observed in questions collecting numerical results (27.2%), followed by questions about participant characteristics (10.3%), and design (6.2%). For every 5-min increase in time, 2.4% fewer errors occurred on average (95% CI: 1.3–3.6%, P value < 0.001) (Figure), regardless of experience level. Conclusions: These are preliminary data, but we found evidence that the faster that data are abstracted, the greater the proportion of errors, for both experienced and less experienced DAs. More research is needed to determine whether quality assurance and rigorous training prior to data abstraction may reduce risk of errors, while maximizing efficiency. Copyright c2014 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

P87: Meta-analyzing surveillance studies to estimate the burden of vaccine-preventable infectious disease Thumburu KK1 , Jaiswal N1 , Agarwal A1 , Jindal I1 , Singh M2 , Bharti B2 , Mathew JL2 1 ICMR Advanced Centre for Evidenced Based Child Health, Advanced Pediatric Centre, PGIMER, Chandigarh, India; 2 Department of Pediatrics, Advanced Pediatric Centre, PGIMER, Chandigarh, India

Background: The policy-makers and epidemiologists usually estimate the infectious disease burdens due to a specific organism by using statistical remodelling or extrapolation of the data from different settings. Though proven for usefulness, these are complex and difficult to replicate. For this reason there is a need to create strategies to make it simple and easy to replicate. Objectives: To describe a simpler method of using the data available from surveillance of infectious diseases. Methods: A systematic review and meta-analysis was carried out on surveillance studies to discuss the impact of the clinical evidence available. The proportions were calculated from individual studies: (i) proportion of invasive pneumococcal diseases (IPD) from suspected children (