Journal of Internal Medicine 2006; 260: 488–490
doi:10.1111/j.1365-2796.2006.01707.x
LETTER TO THE EDITOR
Efficacy or effectiveness? Dear Sir, Seemingly similar in meaning, ÔefficacyÕ and ÔeffectivenessÕ express distinctly different concepts. A medical intervention is ÔefficaciousÕ if it works under strictly controlled (laboratory) conditions and it is considered ÔeffectiveÕ if it works under Ôreal lifeÕ conditions. Efficacy (or fastidious) trials test for efficacy and effectiveness (pragmatic) studies for effectiveness of a therapy. Discussion continues over which of the two approaches are more valuable. The main characteristics of the two types of clinical investigation are summarized in Table 1. The typical efficacy study tests whether the experimental therapy generates specific therapeutic effects, while the typical effectiveness study is aimed at quantifying the sum of specific effects and other contributors to the total therapeutic effect. In theory therefore, the effect size measured in an efficacy study is a pure product of the specific effect of the experimental treatment, while that of an effectiveness study can be due to additional factors such as placebo- or context effects, effects of undeclared concomitant therapies and social desirability (Fig. 1). Thus causal attribution between the treatment and the clinical outcome is usually possible in efficacy trials, while this is rarely the case in effectiveness studies. Efficacy trials will tell us whether a treatment works in principle and under optimal, ÔlaboratoryÕ conditions. In some cases, however, Ôreal life situationsÕ might differ from the ideal. For instance, a drug that normalizes cholesterol levels might not be acceptable to Ôreal lifeÕ patients if it causes adverse events. Similarly, an anti-hypertensive drug might work for a very narrowly defined homogenous group of patients (e.g. those who have no other risk factors and take no other medication), but in Ôreal lifeÕ where patients are heterogenous the effect could be smaller or totally lost. Whenever there is reason to suspect that this could be the case, effectiveness 488
trials are required to test whether the results of efficacy trials also hold true in practice. Would it then make sense to rely on the findings of effectiveness studies in the absence of efficacy data? If the results of effectiveness trials suggest a therapeutic benefit of the experimental therapy, we would not normally be able to be sure whether this is due to the treatment itself or whether it was caused by one of the many other factors that could have contributed to the overall therapeutic effect (Fig. 1). Effectiveness trials are therefore most valuable to test whether the results of efficacy trials are applicable to Ôreal lifeÕ. In the absence of efficacy data, the results of effectiveness trials can be difficult to interpret and may therefore be of little value. What if both efficacy and effectiveness data are available but do not agree with each other? In many conventional settings, this would correspond to situations similar to the ones outlined above: a treatment is efficacious yet in Ôreal lifeÕ there are obstacles, e.g. due to problems with compliance. It is, however, also conceivable that an effectiveness trial shows a benefit while an efficacy trial does not. This scenario is frequently encountered in the area of complementary/alternative medicine. The most logical explanation would then be that the experimental treatment is devoid of specific effects but generates benefit through context effects. Such a treatment would be associated with positive outcomes without directly causing them. An example could be homoeopathy, which seems to help many patients [1, 2] while rigorous trials of homoeopathic remedies have repeatedly been shown them to be no different from placebos [3, 4]. Is this sufficient justification for adopting such a (placebo) treatment into routine health care? We don’t think so. We do not need a placebo to generate a placebo effect: even treatments with specific effects generate placebo- or context effects. In most instances, the debate about the relative value of efficacy and effectiveness trials distracts from the fact that both address different research Ó 2006 Blackwell Publishing Ltd
LETTER TO THE EDITOR
489
Table 1 Characteristics of efficacy and effectiveness studies Efficacy studies
Effectiveness studies
May have design features such as randomization and blinding Often use placebo controls Tightly defined inclusion/exclusion criteria All aspects of intervention are strictly standardized
Can be randomized but are rarely blinded Rarely use placebo controls Less tightly defined inclusion/exclusion criteria Aspects of the intervention may be only loosely defined, e.g. concomitant treatments Tend to employ subjective outcome measures Compliance usually not closely monitored Bias and confounders can be substantial Intention-to-treat analysis High external validity Research question: does it work?
Tend to employ objective outcome measures Compliance usually closely monitored Bias and confounders are minimized Per protocol analysis High internal validity Research question: can it work?
Efficacy study*
Effectiveness studyº
7
7
ES
6
6
ES
6
Total 5
5
perceived
5
4
4
therapeutic
4
3
3
effect
2 1
Control
3
3
2
2
2
1
1
1
Experimental Group
Fig. 1 Schematic difference between effects captured by efficacy and effectiveness studies. *Example: a placebo-controlled, double-blind, randomized trial with two parallel groups. OExample: a randomized trial of the experimental treatment versus usual care. 1 ¼ Effect of natural history of disease, 2 ¼ Hawthorn effect, 3 ¼ Regression towards the mean, 4 ¼ Effect of concomitant treatments, 5 ¼ Social desirability, 6 ¼ Placebo effect, 7 ¼ Specific effect of experimental treatment, ES ¼ Effect size measured as difference between experimental and control group.
questions. Both types of study undoubtedly have their place. We should, however, be vigilant that effectiveness trials are not (mis)used (in the absence of convincing efficacy data) to convince policy makers of the value of treatments which have no specific effects at all. E. Ernst, M. H. Pittler Complementary Medicine, Peninsula Medical School, Universities of Exeter & Plymouth, Exeter, UK
References 1 Witt C, Brinkhaus B, Jena S et al. Acupuncture in patients with osteoarthritis of the knee: a randomised trial. Lancet 2005; 366: 136–43. 2 Spence DS, Thompson EA, Barron SJ. Homeopathic treatment for chronic disease: a 6-year, university-hospital outpatient observational study. J Altern Complement Med 2005; 11: 793– 8. 3 Shang A, Huwiler-Mu¨ntener K, Nartey L et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of
Ó 2006 Blackwell Publishing Ltd Journal of Internal Medicine 260: 488–490
490
LETTER TO THE EDITOR
placebo-controlled trials of homoeopathy and allopathy. Lancet 2005; 366: 726–32. 4 Ernst E. A systematic review of systematic reviews of homoepathy. Br J Clin Pharmacol 2002; 54: 577–82.
Correspondence: Edzard Ernst MD, Complementary Medicine, Peninsula Medical School, Universities of Exeter & Plymouth, 25 Victoria Park Road, Exeter EX2 4NT, UK. (fax: +44(0)1392 427562; e-mail:
[email protected]).
Ó 2006 Blackwell Publishing Ltd Journal of Internal Medicine 260: 488–490
