ROBERT MOLES, LOSING. THEIR GRIP: THE CASE OF .... Kathryn Campbell & Clive Walker, Medical Mistakes and Miscarriages of Justice: Perspectives on the ...
FORENSIC SCIENCE AND MISCARRIAGES OF JUSTICE: SOME LESSONS FROM COMPARATIVE EXPERIENCE Kent Roach* ABSTRACT: This paper provides a critical assessment of the National Research Council’s (NRC) 2009 report in light of comparative experience in Australia, Canada, and the United Kingdom. It suggests that the NRC’s proposals for federal regulation of the forensic sciences are more appropriate for a unitary state than a federal system. The NRC report could have been strengthened by examining the British experience with a forensic regulator and a 2008 report on forensic pathology in the Canadian province of Ontario. The Ontario pathology report has already produced tangible reforms to the practice of forensic pathology within coroners’ systems while the NRC unrealistically calls for the abolition of all coroner systems. The dangers of superficial reforms are examined, including a Canadian example of published research being misapplied in a manner that contributed to a wrongful conviction. Finally, the NRC’s pessimistic conclusions about judicial exclusion of unreliable forensic science are contrasted with recent and more optimistic reform proposals in Canada and the United Kingdom. CITATION: Kent Roach, Forensic Science and Miscarriages of Justice: Some Lessons from Comparative Experience, 50 Jurimetrics J. 67–92 (2009). The recent report of the National Research Council (NRC) on strengthening forensic science focuses exclusively on the American experience.1 This orientation follows the mandate that the committee was given by Congress. It also reflects the extensive American experience with the contribution of forensic science to wrongful convictions.2 Nevertheless, Strengthening Forensic Science in the United States: A Path Forward (the NRC report) could have been enriched by attention to comparative experience.3 *Prichard and Wilson Chair in Law and Public Policy and Professor of Law, University of Toronto, and Former Director of Research for the Inquiry into Pediatric Forensic Pathology in Ontario (2008). I thank the organizers of the Forensic Science in the 21st Century conference and particularly Michael Saks for inviting me to give a preliminary version of this paper at the conference at Arizona State University in April 2009. I also gratefully acknowledge the financial support of Canada’s Social Science and Humanities Research Council, which has supported my ongoing work on the comparative study of wrongful convictions. 1. COMM. ON IDENTIFYING THE NEEDS OF THE FORENSIC SCI. CMTY. ET AL., NAT’L RESEARCH COUNCIL OF THE NAT’L ACADS., STRENGTHENING FORENSIC SCIENCE IN THE UNITED STATES: A PATH FORWARD (2009) [hereinafter, NRC REPORT]. 2. See Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science Testimony and Wrongful Convictions, 95 VA. L. REV. 1, 8–11 (2009), for an examination of the role of forensic science in cases involving 156 DNA exonerees in the United States. 3. Comparative experience refers to similarities and differences revealed by the study of comparative law or jurisprudence. This is generally defined as “the comparative study of the intellectual conceptions that underlie the principal institutions of one or more foreign legal systems.” William Ewald, Comparative Jurisprudence: What Was it Like to Try a Rat?, 143 U. PA. L. REV. 1889, 2114 (1995).
FALL 2009
67
Disclaimer: This information or any portion thereof may not be copied or disseminated in any form or by any means or downloaded or stored in an electronic database or retrieval system without the express written consent of the American Bar Association.
Roach Attention to comparative experience could have strengthened the case for reform by providing some support for the idea that the United States lags behind some other democracies in the delivery of reliable forensic science.4 Comparative analysis might have revealed the unique challenges faced by the United States. These challenges are related to the radical decentralization of the American criminal justice and forensic science systems and the heavy use in the United States of the criminal sanction. Comparative analysis also would have revealed numerous common challenges that many countries face with respect to reform of the forensic sciences. An important common challenge is the disciplinary fragmentation of the forensic sciences into separate disciplines without much thought to scientific and regulatory issues that are common to all forensic sciences. Federalism adds jurisdictional fragmentation to the disciplinary fragmentation of the forensic sciences in Australia and Canada, as well as in the United States. Attention to comparative reform proposals might have revealed the viability of more incremental reforms that stop short of the NRC’s proposed abolition of coroner systems and of laboratories affiliated with policing and prosecutorial agencies. Finally, comparative analysis would have indicated that other jurisdictions, rightly or wrongly, are moving towards Daubert-style judicial gatekeeping5 with respect to unreliable science even while the NRC eschews reliance on such measures. Although this paper will mention experience in other countries, most notably Australia and the United Kingdom, I will focus on the comparative Canadian experience with forensic science. In particular, I will suggest that the NRC report missed an opportunity by not examining the 2008 report of the Inquiry into Pediatric Forensic Pathology in Ontario (the Ontario pathology report), as well as other commission of inquiry reports in Canada that have examined the role of forensic sciences in a number of wrongful convictions.6 4. See generally Paul C. Giannelli, Wrongful Convictions and the Need to Regulate Crime Labs, 86 N.C. L. REV. 163 (2007) (discussing various examples of unreliable forensic science in the United States). Crime labs in the United States have greater levels of decentralization than in many other countries. 5. Daubert requires judges to determine whether expert evidence is reliable enough to be admitted as evidence. In making this determination of threshold reliability, trial judges are encouraged to examine whether the theory or technique has been tested and subjected to peer review; whether its known or potential error rate is known; whether proper standards have been observed; and the degree of acceptance of the theory or technique. Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 593–97 (1993). 6. Public inquiries in Canada are investigative bodies appointed at the discretion of the executive but usually headed by sitting judges or retired judges. They hold both adjudicative and systemic hearings with wide participation from affected parties and groups before issuing comprehensive reports on matters of public controversy. Three Canadian inquiries have devoted considerable attention to issues involving forensic science. See generally STEPHEN T. GOUDGE, INQUIRY INTO PEDIATRIC FORENSIC PATHOLOGY IN ONTARIO (2008), http://www.goudgeinquiry.ca/; PATRICK J. LESAGE, REPORT OF THE COMMISSION OF INQUIRY INTO CERTAIN ASPECTS OF THE TRIAL AND CONVICTION OF JAMES DRISKELL (2007), http://www.driskellinquiry.ca/pdf/final_ report_jan2007.pdf; FRED KAUFMAN, REPORT OF THE KAUFMAN COMMISSION ON PROCEEDINGS INVOLVING GUY PAUL MORIN (1998), http://www.attorneygeneral.jus.gov.on.ca/english/about/ pubs/morin/. On the Canadian inquiry process, see generally Kent Roach, Inquiring into the
68
50 JURIMETRICS
Forensic Science and Miscarriages of Justice Part I of this paper will examine the dangers of disciplinary and jurisdictional fragmentation of the forensic sciences in a comparative perspective. Forensic sciences in many countries have developed in an insular disciplinary fashion with little attention to overarching common scientific issues. The greatest strength of the NRC report is its unified vision of forensic science. This unified vision will be contrasted with the fragmented disciplinary and jurisdictional approach that has so far characterized Canada’s experience with public inquiries into the forensic sciences. Indeed, the Canadian experience which has focused on elaborate inquiries into specific forensic errors reveals the shortcomings of attempting to reform the forensic sciences one discipline and one jurisdiction at a time. This section of the paper will also argue that federalism aggravates jurisdictional fragmentation of the forensic sciences. Although the NRC report avoids this problem by calling for significant federal regulation of forensic science, it remains unclear whether its centralized vision is workable in any federal system. Part II of the paper examines some of the dangers of the NRC report’s neglect of the comparative experience with forensic science with particular attention to forensic pathology. This Part will suggest that the NRC’s work would have benefited from consideration of the nascent British experience with a Forensic Science Regulator. It will also suggest that the NRC would have benefited from consideration of the Ontario pathology report, particularly with respect to the questions of whether reform of forensic pathology is possible within coroner systems and whether triage principles should be used to ensure that qualified forensic pathologists perform the most complex and contentious autopsies. The Ontario pathology report explores the difficulties of subjecting forensic pathology to the quantifiable measures of reliability and uncertainty that the NRC recommends be developed for all the forensic sciences.7 Additionally, this section will examine how one Canadian inquiry has considered, but rejected, the NRC’s proposal that all police crime laboratories should be abolished in favor of more targeted recommendations about the need for better training, quality assurance, and procedures at all crime laboratories.8 The NRC makes ambitious recommendations for a more scientific, standardized, and regulated approach to forensic sciences. Few would object to these reform goals, but the report can be faulted for not dealing with some likely obstacles and shortcuts that will be encountered in reaching these goals. The third part of this paper will examine some possible dangers when attempts are made to reform the forensic sciences. One danger is superficial standardization and regulation that is not supported by underlying scientific research.
Causes of Wrongful Convictions, 35 CRIM. L. BULL. 152 (1999) [hereinafter Roach, Wrongful Convictions]; Kathryn M. Campbell, Policy Responses to Wrongful Conviction in Canada: The Role of Conviction Review, Public Inquiries and Compensation, 41 CRIM. L. BULL. 145 (2005); Kent Roach, Canadian Public Inquiries and Accountability, in ACCOUNTABILITY FOR CRIMINAL JUSTICE: SELECTED ESSAYS (Philip C. Stenning ed., 1995) [hereinafter Roach, Canadian Inquiries]. 7. NRC REPORT, supra note 1, recommendation 3, at 190. 8. LESAGE, supra note 6, at 185.
FALL 2009
69
Roach Another danger is that the introduction of a more robust research base into the forensic sciences can also introduce additional sources of error and misinterpretation. I will illustrate this potential problem with a concrete example taken from a Canadian inquiry, which found that published and peer reviewed research was improperly used in a wrongful conviction to bolster the credibility of fiber analysis.9 The fourth part of this paper will examine some of the dangers associated with the NRC’s pessimistic conclusion about the limited contribution that courts can make under Daubert v. Merrell Dow Pharmaceuticals, Inc.10 in reforming the forensic sciences. This will be done by examining movements in both Britain and Canada towards Daubert as a means of ensuring threshold reliability11 before forensic evidence and other expert evidence is admitted into criminal trials. Again, this is an area where comparative analysis might have expanded the scope of the NRC report because it would have provided a concrete example of how Canadian courts inspired by Daubert have excluded some forensic evidence offered by the prosecutor because of concerns about threshold reliability. It also would have revealed the importance of ensuring adequate legal aid funding so that forensic science evidence offered by the prosecution can be challenged effectively.
I. THE DANGERS OF DISCIPLINARY AND JURISDICTIONAL FRAGMENTATION OF THE FORENSIC SCIENCES Disciplinary fragmentation refers to the fact that the forensic sciences have generally operated as separate and somewhat insular disciplines, such as fingerprint comparison, firearms identification, handwriting identification, hair and fiber comparison, and forensic pathology. The forensic sciences have not generally been studied as a whole. Hence, little attention has been paid to common issues that run through all or most of the forensic sciences. Such issues would include the dangers of potential confirmation and observer bias and the difficulties of communicating scientific knowledge to lay audiences.12 Jurisdictional fragmentation refers to the fact that forensic sciences are often not subject to common forms of governance. For example, medical examiners or coroner systems of death investigation are often outside the jurisdiction of crime laboratories. Forensic sciences in many countries are not
9. KAUFMAN, supra note 6, at 83. 10. 509 U.S. 579 (1993). 11. Threshold reliability refers to decisions made by judges in preliminary proceedings that expert evidence has sufficient reliability to be admissible in legal proceedings. Determinations of threshold reliability are distinguished from determinations of ultimate reliability made by juries and other triers of fact in deciding guilt or innocence. 12. See generally Michael Saks & David Faigman, Failed Forensics: How Forensic Science Lost Its Way and How It Might Yet Find It, 4 ANN. REV. L. & SOC. SCI. 149 (2008) (discussing how scientific principles sometimes do not apply in forensic science despite the misconception that scientific principles generally prevail).
70
50 JURIMETRICS
Forensic Science and Miscarriages of Justice subject to national regulation and are subject to a variety of local and state regulations. The combination of disciplinary and jurisdictional fragmentation of the forensic sciences presents serious challenges for the reform of all the forensic sciences. A controversy in one discipline or in one jurisdiction may draw awareness to scientific and regulatory deficiencies, but the lessons and reforms that arise out of such controversies will not necessarily be transferred to other disciplines and jurisdictions. As will be seen, comparative experience and federalism confirm the dangers of disciplinary and jurisdictional fragmentation of the forensic sciences.
A. The National Research Council’s Unified Vision of Forensic Science The NRC report impressively attempts to break down discipline-bydiscipline and jurisdiction-by-jurisdiction fragmentation of the forensic sciences by examining common cross-cutting scientific and methodological issues that affect all the forensic sciences. The NRC’s unified vision of the forensic sciences is reflected in the very structure of its report. Most of the report’s chapters are devoted to scientific and regulatory issues that are common to all the forensic sciences. For example, chapters are devoted to principles of science,13 improving methods, practice and performance,14 and education and training,15 while only a few chapters describe or assess the forensic sciences on a discipline-by-discipline basis.16 The NRC proposes to overcome the fragmentation of the forensic sciences by recommending a common system of federal certification and regulation for all the forensic sciences. Thus a number of chapters of its report are devoted to common regulatory issues, such as the need for integrated governance,17 strengthening oversight,18 and the admissibility of all forensic science in court.19 The NRC’s emphasis on scientific and regulatory issues that are common for all the forensic sciences is perhaps the greatest intellectual strength of the report. That said, the difficulties of implementing such an integrated and unified approach should not be underestimated. Some of these difficulties can be seen by examining comparative experience with reform of the forensic sciences, as well as by consideration of the particular challenges of reform in countries with federal systems.
13. NRC REPORT, supra note 1, ch. 4. 14. Id. ch. 6. 15. Id. ch. 8. 16. Id. chs. 5, 9 (describing various forensic sciences and death investigation services). 17. Id. ch. 2. 18. Id. ch. 7. 19. Id. ch. 3.
FALL 2009
71
Roach
B. The Canadian Experience of Fragmented Reform The unified approach towards the forensic sciences taken in the NRC report has clear advantages over the approaches taken in Canada, which have reflected both disciplinary and jurisdictional fragmentation. An important driver of forensic science policy and reform in Canada has been the conduct of a series of public inquiries, headed by judges, that have examined the contribution of forensic science to a series of wrongful convictions. Although these inquiries have produced increased awareness about the frailties of some forms of forensic sciences and some reforms, they have generally focused on the problems in particular disciplines of the forensic sciences and in particular jurisdictions. As will be seen, the Canadian experience demonstrates the difficulties of attempting to reform the forensic sciences one discipline and one jurisdiction at a time. In 1998, the Report of the Kaufman Commission on Proceedings Involving Guy Paul Morin was issued in Ontario after an extensive inquiry relating to a murder conviction proven to be false because of DNA testing.20 The Kaufman Commission examined the work of the Centre of Forensic Sciences, located in Toronto, Ontario, which had provided evidence that hair and fiber evidence linked Mr. Morin with a nine-year-old girl who was murdered. The Commission found that the Centre’s contributions to Morin’s wrongful conviction were substantial. The findings of the analysts were overstated and misunderstood by police, prosecutors, and the courts.21 Contamination problems were not disclosed.22 The Commission recommended sweeping changes involving better resources, training, quality assurance, research, governance, and independence for the Centre.23 The Commission also recommended the following: more attention by the Centre to complaints about forensic scientists; monitoring of their courtroom testimony, including the use of misleading language such as “consistent with”; better education of lawyers and judges about forensic sciences; and more critical and restrictive approaches to the admissibility of forensic evidence.24 This inquiry initiated significant reform in Ontario’s Centre of Forensic Science.25 Unfortunately, these reforms were limited to the forensic sciences that were represented at the Centre (biological, firearms, handwriting, toxicology) and did not extend to forensic pathology. As will be seen, it took another major inquiry in Ontario a decade later to initiate similar re20. KAUFMAN, supra note 6, at 1. 21. Id., Executive Summary, at 4–5. 22. Id., Executive Summary, at 6. 23. See generally id., Recommendations. 24. See generally id., Recommendations. Recommendations 1–35 of the Commission all are related to forensic science at the Centre of Forensic Science and in the courts. For a more detailed account of this inquiry, see Roach, Wrongful Convictions, supra note 6. 25. See Jeffrey R. Manishen, Wrongful Convictions, Lessons Learned: The Canadian Experience, 13 J. CLINICAL FORENSIC MED. 296, 297–98 (2006) (detailing post inquiry reforms including DNA testing of hair, recording of preliminary reports, increased training, new protocols for reports and complaints, documentation of contamination, monitoring of courtroom testimony, and creation of advisory board and quality assurance unit).
72
50 JURIMETRICS
Forensic Science and Miscarriages of Justice forms to forensic pathology in Ontario. This experience underlines the challenges presented by the disciplinary and institutional fragmentation of the forensic sciences. Forensic science is fragmented not only by discipline, but also by jurisdiction. Many of the Morin inquiry reforms were limited to the province of Ontario. In 2007, an inquiry report from the neighboring province of Manitoba revealed problems with the way that hair and fiber analysis had been conducted in that province.26 Analysts employed in laboratories of the Royal Canadian Mounted Police (RCMP) resisted some of the reforms proposed and adopted in Ontario as a result of the Morin Inquiry.27 A decade after the Kaufman Commission issued its two-volume report, another Ontario Commission of Inquiry issued a four-volume report on pediatric forensic pathology called the Inquiry into Pediatric Forensic Pathology in Ontario.28 This inquiry was appointed by the Ontario government in the wake of concerns raised by the work and testimony of Dr. Charles Smith. Five international experts reviewed some of Dr. Smith’s work and raised concerns about 20 of 45 cases reviewed, including 12 cases that had resulted in convictions.29 Although pediatric forensic pathology is a very different field than hair and fiber comparison analysis, the similarities between the recommendations made following the Morin inquiry and those made following Ontario’s subsequent pathology inquiry are striking. The Ontario pathology report called for better resources, training, quality assurance, research, independence, governance, and monitoring of courtroom testimony, just as the Kaufman Commission on Morin had a decade earlier.30 It also called for more attention to complaints, monitoring of courtroom testimony, better education of lawyers and judges about science, and more restrictive and critical attention to the admissibility of scientific expert evidence, including the language used by expert witnesses to express their findings.31 The similarity of the findings and recommendations by the two Ontario commissions underlines a real fragmentation between the various disciplines of forensic science. A similar fragmentation can be observed in Australia where reforms to some forensic sciences in 26. LESAGE, supra note 6, at 146–85. 27. In this case, an RCMP hair analyst testified in a 1991 trial that three hairs found in the accused’s van were “consistent with” those of the victim. Id. at 149. The analyst defended the use of “consistent with” despite the fact that the term had been specifically criticized by the Morin inquiry in 1998. Id. at 151. DNA analysis subsequently determined that the hairs could not have come from the victim and indeed likely came from three different people. Id. at 155. The Commission’s report criticized the RCMP lab for not having another analyst verify the “matches” at the time and for the analyst testifying that the hairs were “consistent with” those of the victim despite the varying interpretations of that phrase. Id. at 163, 166. The report also affirmed the conclusions of the Kaufman Commission that hair comparison analysis, because of its significant error rates, should only be used to exclude suspects. Id. at 169. The Driskell Commission, however, followed the Kaufman Commission in stopping short of recommending a complete ban on hair microscopy evidence in criminal cases. Id. at 172. 28. See GOUDGE, supra note 6. 29. Id., Executive Summary, at 17. 30. Id. at 590–631. 31. Id.
FALL 2009
73
Roach light of two public inquiries also did not seem to affect the practice of forensic pathology.32 Without the type of unifying vision seen in the NRC report, there is a real danger that the hard lessons that the discovery of wrongful convictions reveals about weaknesses in the forensic sciences will be learned slowly on a discipline-by-discipline and jurisdiction-by-jurisdiction basis. The similarities between the recommendations made by the Morin inquiry and the Ontario pathology report also provide support for the conclusion of the NRC that there are common problems and common solutions in the forensic sciences. The NRC report, like the two Ontario reports, gravitates towards common recommendations about the training and certification of forensic scientists and quality assurance of their work both in the laboratory and in the courtroom.33 Although specialists in each discipline can point to unique problems in their particular discipline, the common issues that run through the forensic sciences are important to address. Thus, two of the greatest strengths of the NRC report are its unified vision of the forensic sciences and its focus on problems and remedies that are common to all the forensic sciences. The Canadian experience is a warning about the dangers of a fragmented approach to forensic science, which, at the extreme, will require reforms to be fashioned in each separate discipline and in each separate jurisdiction.
C. Federalism as a Force of Fragmentation The United States and Canada both face challenges in improving forensic science because of federalism, which in both countries assigns the administration of criminal justice to state or provincial governments. A striking feature of the NRC report is its aggressive advocacy of a large federal role in forensic science reform.34 The report recommends the creation of the National Institute of Forensic Science as a federal entity that would not only establish but also enforce best practices, standard terminology, and mandatory certification for all forensic scientists who practice or testify.35 The federal spending power can, as the experience with DNA demonstrates, have powerful incentives.36 Nevertheless, it is questionable whether direct federal regulation of forensic science and forensic scientists proposed by the NRC is workable. The heavy federal role contemplated by the NRC is driven by the report’s intellectual 32. For commissions of inquiry in Australia that were critical of forensic sciences, see generally Royal Commission of Inquiry Concerning the Conviction of Edward Charles Splatt (1984) and Royal Commission into the Chamberlain Convictions (1987). “A great many recommendations were made by the two Commissions. As a consequence, the forensic scientific services in South Australia were given an overhaul and procedures were significantly improved. However, forensic pathology was not seen to have been implicated in either of those cases” and controversy subsequently emerged about forensic pathology in South Australia. ROBERT MOLES, LOSING THEIR GRIP: THE CASE OF HENRY KEOGH 251–52 (2006). 33. See generally NRC REPORT, supra note 1. 34. See id., recommendation 1, at 90. 35. Id., recommendation 7, at 215. 36. Innocence Protection Act of 2004, Title IV of the Justice for All Act of 2004, Pub. L. No. 108-405, § 413, 118 Stat. 2260, 2285 (2004) (providing federal incentive grants to the states for DNA testing). See also Brandon Garrett, Claiming Innocence, 92 MINN. L. REV. 1629, 1673–75 (2008) on post conviction DNA statutes enacted by the vast majority of states.
74
50 JURIMETRICS
Forensic Science and Miscarriages of Justice vision of unified and reformed forensic science. As suggested above, this intellectual vision is helpful, but it also creates a risk of ignoring more mundane and practical jurisdictional realities. The experience in Australia and Canada offers little support for the idea that the federal government can, through the might of its spending power, effectively regulate forensic science. Australia is in some respects the best comparison with the United States because the states in both countries have jurisdiction over most criminal law matters. In Australia, as in the United States, forensic science is still organized and regulated by the states.37 To be sure, controversies in one state can have some effects in other states, but much legislative reform must still be implemented at the state level. Canada represents a more centralized model of criminal justice because criminal law and procedure is a matter of exclusive federal jurisdiction, even though the administration of justice is a matter of provincial jurisdiction. Nevertheless, Canada, like the other federal systems, has resisted national regulation of forensic science. Manitoba’s Driskell inquiry was asked by Mr. Driskell and the Association in Defence of the Wrongfully Convicted to recommend a national audit of criminal cases where hair microscopy evidence was used to sustain convictions.38 The province of Manitoba had already started an audit process for such cases, but the flawed evidence in Driskell’s case came from a laboratory run by the national police force, the Royal Canadian Mounted Police.39 Commissioner LeSage expressed concerns “that the problems identified relating to hair microscopy evidence in Driskell’s case are not unique to his case or unique to Manitoba”40 but nevertheless did not make a formal recommendation for such a national audit or for a review of the forensic labs of the national police force.41 Effective national regulation of forensic science will take great political will and cooperation in any federation. The country that seems most amenable to the central regulation proposed by the NRC is the United Kingdom. For example, the United Kingdom already has a national Forensic Science Regulator and a national registry of forensic pathologists.42 Alas, however, the United Kingdom is, even after devolution, basically a unitary state. Some comparative research might have served as a reality check on the NRC’s recommendations for federal regulation of the forensic sciences. At the least, more exploration of how the federal spending power can interact with the regulatory jurisdiction of the states would have been helpful. 37. W. Tilstone, Reflections from Abroad on the Occasion of the Forensic Science Society’s 40th Anniversary, 40 SCI. & JUST. 76, 76–78. See Nat’l Inst. of Forensic Sci. Austl. (NIFS), Gov’t Forensic Serv. Providers, http://www.nifs.com.au/F_S_A/FSA_frame.html?Providers.asp&1 (last visited Nov. 30, 2009), for a list of forensic science providers in the various Australian states. 38. LESAGE, supra note 6, at 181. 39. Id. at 146. 40. Id. at 182. 41. Id. at 182–84. 42. ANDREW RENNISON, FORENSIC SCIENCE REGULATOR, MANUAL OF REGULATION PART ONE: POLICY AND PRINCIPLES 9–16 (2008), http://police.homeoffice.gov.uk/publications/operationa lpolicing/Manual_of_Regulation_22.9.08.pdf?view=Binary.
FALL 2009
75
Roach
II. THE DANGERS OF NEGLECTING COMPARATIVE EXPERIENCE The more or less exclusive focus of the NRC report on the American experience with forensic science is in large part related to the daunting task that it was assigned by Congress. The American experience is vast, and the problems that wrongful convictions and other controversies have revealed about forensic sciences severe. Nevertheless, the NRC report could have gained insight from an examination of forensic science experience and reforms of other countries.
A. British Lessons The report ignored the recent British experience with the regulation of forensic science. Traditionally, the Forensic Science Service had a virtual monopoly on the provision of forensic science, but this has changed. The Service now has the status of a government company that competes in a competitive market.43 In part in response to this development, a Forensic Science Regulator position was created in 2007.44 This person is assisted by a staff and various advisory councils.45 The regulator is responsible for quality standards and accreditation for those who provide forensic science for the criminal justice system.46 He also hears complaints in relation to quality standards and provides independent advice to the government on forensic science issues.47 The regulator has published a draft manual of regulations48 and has undertaken a review of the accreditation of forensic practitioners.49 Both initiatives place considerable stress on the need for broad consultation. The Forensic Science Regulator is trying to play a somewhat similar role to that the NRC contemplates for its proposed National Institute of Forensic Science, but the regulator is not discussed in the NRC report.50 There is also no discussion in the report of British attempts to deal with severe shortages of forensic pathologists.51 The shortage of trained forensic pathologists is a global problem and there is a global market for the services of trained forensic pathologists.52 If only to this limited extent, the NRC ignores 43. Id. at 10. 44. Id. at 17. 45. Id. at 32–35. 46. Id. at 8. 47. Id. at 42–49. 48. Id. 49. See generally FORENSIC SCIENCE REGULATOR, A REVIEW OF THE OPTIONS FOR THE ACCREDITATION OF FORENSIC PRACTITIONERS (2009), http://www.homeoffice.gov.uk/documents/ cons-2009-forensic-science?view=Binary. 50. See NRC REPORT, supra note 1, at 312 app. B, for a showing that the NRC panel did hear some evidence about the U.K. Forensic Systems, although a transcript of the discussion does not appear in the report. 51. HOME OFFICE, REVIEW OF FORENSIC SCIENCE PATHOLOGY SERVICES IN ENGLAND AND WALES, EXECUTIVE SUMMARY 3–4 (2003), http://police.homeoffice.gov.uk/publications/operationalpolicing/forensic_pathology_report.pdf?view=Binary. 52. 3 GOUDGE, supra note 6, at 298–307.
76
50 JURIMETRICS
Forensic Science and Miscarriages of Justice comparative experience at its peril. Additionally, there is no discussion of the British Home Office’s experience with a registry of forensic pathologists or national protocols for the conduct of pediatric autopsies.53 Although time limits and a focus on making recommendations amenable to the particular American situation help explain the NRC’s lack of engagement with the comparative experience of other countries, the lack of a global perspective remains unfortunate. Engagement with those who practice similar forensic sciences in other countries can help in the transmission of research, best practice protocols, and exposure to different and perhaps better ways of conducting forensic work. Global engagement can also help create connections that can facilitate peer review and quality assurance programs, the recruitment of talent, and the development of a common sense of professionalism.
B. Forensic Pathology and Death Investigation Systems The NRC report devotes a chapter to forensic pathology in the context of medical examiners and coroner systems.54 This chapter notes that the NRC had first recommended in 1928 that coroner systems be replaced by medical examiners run by trained and properly paid pathologists.55 Coroner systems persist in part because they are mandated by some state constitutions and in part because there are 2,342 separate death investigation jurisdictions.56 It is difficult to conceive of any other country that would face such a “hodgepodge and multiplicity of systems,”57 and to this extent the exclusive focus on the United States in this chapter of the NRC report may be justifiable.
C. The Possibility of Reform within Coroner Systems The NRC report could have benefited from an examination of other death investigation systems if only to underline the unique challenges faced by the United States. Comparative information about death investigation systems is not difficult to obtain. A recent leading medicolegal text, Death Investigation and the Coroner’s Inquest, provides an accessible survey of death investigation systems in the United States, Canada, Australia, Europe, and Asia.58 The book poses interesting questions about the involvement of police, judicial officers, and forensic pathologists in each death investigation system.59 The two volumes of Independent Research Studies Prepared for the Inquiry into Pediatric Forensic Pathology in Ontario feature articles on both American and Australian death investigation systems including the innovative Victorian 53. Kathryn Campbell & Clive Walker, Medical Mistakes and Miscarriages of Justice: Perspectives on the Experiences in England and Wales, in 2 INDEPENDENT RESEARCH STUDIES PREPARED FOR THE INQUIRY INTO PEDIATRIC FORENSIC PATHOLOGY IN ONTARIO, PEDIATRIC FORENSIC PATHOLOGY AND THE JUSTICE SYSTEM 325, 361–62 (Kent Roach ed., 2008). 54. NRC REPORT, supra note 1, at 241–68. 55. Id. at 242. 56. Id. at 243, 245, 247–50. 57. Id. at 246. 58. See generally IAN FRECKELTON & DAVID RANSON, DEATH INVESTIGATION AND THE CORONER’S INQUEST (2006). 59. See generally id.
FALL 2009
77
Roach Institute for Forensic Medicine that served as a model for the Ontario Commission’s reform recommendations.60 Comparative analysis has to be conducted with care, and it may be that the lessons of comparative experience are less easily transferred to the particular political and legal structure of the United States. Even if this were the case, however, comparative analysis would have helped underline the unique challenges of reforming American death investigation systems. The NRC recommends that federal funds be provided “to establish medical examiner systems, with the goal of replacing and eventually eliminating existing coroner systems.”61 Even taken on its own terms, this recommendation is problematic given the report’s recognition of the constitutional status of coroners in some states and that not all medical examiner systems are staffed by trained forensic pathologists. This may be an example of the danger, recognized elsewhere in the NRC report,62 of the perfect being an enemy of the good. An examination of the Ontario pathology report would have revealed a different approach that did not recommend the elimination of coroner systems but rather focused on ensuring that trained forensic pathologists would have the necessary resources and independence within coroner systems to perform medicolegal autopsies and to do quality assurance of their own work.63 Many of the recommendations made in the Ontario pathology report have already been embraced in legislation to recognize a forensic pathology service within the coroner system.64 The legislation assigns responsibility to the Chief Forensic Pathologist with respect to the maintenance of a registry of qualified forensic pathologists and any complaints that may be made about forensic pathologists.65 The Chief Forensic Pathologist, along with the Chief Coroner, will be subject to oversight by a Death Investigation Oversight Council.66 Some might argue that Ontario’s system of appointed coroners who are medical doctors is more benign than American systems of elected coroners without medical training. At the same time, however, the NRC report does not advert to the danger that a medical doctor, within either a coroner or a medical examiner system, might provide erroneous opinions about cause of death because that person does not have specific forensic pathology training or expertise.67 In any event, the Ontario pathology report made a judgment that it 60. See, e.g., Stephen Cordner et al., A Model Forensic Pathology Service, in 1 INDEPENDENT RESEARCH STUDIES PREPARED FOR THE INQUIRY INTO PEDIATRIC FORENSIC PATHOLOGY IN ONTARIO, CONTROVERSIES AND MODELS IN PEDIATRIC FORENSIC PATHOLOGY 149 (Kent Roach ed., 2008). 61. NRC REPORT, supra note 1, recommendation 11, at 267. 62. Id. at 183. 63. See 3 GOUDGE, supra note 6, ch. 12 (discussing the Ontario coroner system and trained forensic pathologists within the system). 64. Coroners Amendment Act, 2009 S.O., ch. 15 (Can.), http://www.ontla.on.ca/web/bills/ bills_detail.do?locale=en&BillID=2073&BillStagePrintId=4371&btnSubmit=go. 65. Id.§ 7, 7.1. 66. Id.§ 8, 8.1. 67. The NRC report notes that some medical examiners are medical doctors without forensic pathology training. NRC REPORT, supra note 1, at 256. In contrast, the Ontario pathology report was critical of a medical doctor and experienced coroner who purported to review and validate Dr.
78
50 JURIMETRICS
Forensic Science and Miscarriages of Justice was not necessary to abolish coroners to achieve the same end as that recommended by the NRC, namely that “all medicolegal autopsies be performed or supervised by a board certified forensic pathologist.”68 Sometimes incremental reforms are better than no reform at all. Abolition of coroner systems seems no more realistic than when the NRC first made the recommendation in 1928.69
D. The Need to Prioritize Reforms and other Limited Resources An important reform goal shared by the NRC and the Ontario pathology report is that all medicolegal autopsies should be done by board certified forensic pathologists. Nevertheless, there are questions about the realism of this important reform in the short-term. The NRC provides important but alarming data that only about 400 to 500 forensic pathologists practice full time in the United States even though there is an estimated need for 1000 forensic pathologists.70 There are only about 70 residencies and 70 new positions for forensic pathologists available each year.71 Moreover, 30 percent of the positions go unfilled.72 One factor is that forensic pathologists, who endure inadequate work facilities and regularly have their work subject to adversarial challenge in court, make substantially less than hospital pathologists.73 The Ontario pathology report recognized the shortage of trained forensic pathologists as a fundamental restraint on any reforms to forensic pathology. To that end, it recommended adequate and sustainable funding for fellowships in forensic pathology, an increased emphasis on forensic pathology in medical schools, and sufficient resources in the meantime to allow Ontario to recruit appropriately credentialed forensic pathologists from abroad.74 Both the NRC and the Ontario pathology report recognized the shortage of forensic pathologists and the neglect of forensic pathology in North American medical schools as a fundamental restraint on the ability to reform death investigation systems. The Ontario pathology report identified a triage principle that is absent in the NRC report. The triage principle was that the most complex and important autopsies—pediatric deaths in criminally suspicious circumstances—be conSmith’s pathology work because “[a] proper review required expertise in forensic pathology.” 2 GOUDGE, supra note 6, at 245. 68. NRC REPORT, supra note 1, recommendation 11, at 268. 69. See NATIONAL RESEARCH COUNCIL, THE CORONER AND THE MEDICAL EXAMINER (National Research Council 1928). Another example would be the recommendation for the accreditation of medical examiners, whereas only 54 systems now are accredited by the National Association of Medical Examiners (NAME). NRC REPORT, supra note 1, at 258. The Ontario pathology report considered, but did not recommend, that the rigorous NAME standards be implemented. 3 GOUDGE, supra note 6, at 360. 70. NRC REPORT, supra note 1, at 257. 71. Id. 72. Id. 73. In addressing this issue, the NRC recommends the funding of student loan forgiveness and fellowship support, but it does not address the salary gap between forensic and other pathologists. Id. The Ontario pathology report recommended that forensic pathologists be included in the Laboratory Medicine Funding Framework to ensure that they are paid on a similar scale as hospital pathologists. 3 GOUDGE, supra note 6, at 305. 74. 3 GOUDGE, supra note 6, at 301–02, 306.
FALL 2009
79
Roach ducted at centralized locations and only by certified forensic pathologists with pediatric forensic experience.75 It is surprising that the NRC report does not articulate such triage principles given the controversies over shaken baby syndrome and pediatric deaths in many countries including the United States.76 A research paper prepared for the Ontario pathology report by Dr. Randy Hanzlick, a leading authority on death investigation systems in the United States and the Chief Medical Examiner for Fulton County, Georgia, who also testified before the NRC committee, established a helpful triage for reforms.77 Dr. Hanzlick set the training and certification of forensic pathologists and the performance of suspicious infant death autopsies by such persons as the highest priority reforms.78 He identified the establishment of an independent medical examiner system only as a third-level priority.79 Without proper triage, there is a danger that the most urgent reforms may be held back while more difficult reforms are debated.
E. Difficulties of Quantification of Error Rate and Degrees of Certainty The NRC recommends the establishment “of quantifiable measures of reliability and accuracy of forensic analyses” and “quantifiable measures of uncertainty in the conclusions of forensic analysis.”80 Leaving aside the practical difficulties of establishing such measures in other forensic sciences,81 there are real questions about how workable these recommendations will be in the field of forensic pathology and other interpretative forensic sciences. The senior judge who ran the Ontario pathology report—Justice Stephen Goudge of the Ontario Court of Appeal—closely questioned many leading forensic pathologists about how they would quantify and express the degree of certainty behind their opinions about the cause of death. Justice Goudge concluded that “there is no common understanding of how forensic pathologists think about their level of confidence or certainty in their opinions; how they articulate this level, if at all, when communicating their opinion . . . .”82 He found that the legal standard of proof beyond a reasonable doubt “has no correlation with science or medicine”83 and recommended that multidisciplinary 75. Id. at 324–27. 76. See, e.g., State v. Edmunds, 746 N.W.2d 590 (Wis. Ct. App. 2008) (admission of new evidence about the evolving science on shaken baby syndrome and finding of reasonable probability that the new evidence would have affected guilty verdict ). 77. Randy Hanzlick, Options for Modernizing the Ontario Coroner System, in 1 INDEPENDENT RESEARCH STUDIES PREPARED FOR THE INQUIRY INTO PEDIATRIC FORENSIC PATHOLOGY IN ONTARIO, CONTROVERSIES AND MODELS IN PEDIATRIC FORENSIC PATHOLOGY 271, 307–09 (Kent Roach ed., 2008). 78. Id. at 308. 79. Id. at 309. 80. NRC REPORT, supra note 1, recommendation 3(b)(c), at 190. 81. The NRC has taken a different position on the measurement of error rates in DNA testing. JAY D. ARONSON, GENETIC WITNESS 189 (2007); see also Erica Beecher-Monas, Blinded by Science: How Judges Avoid the Science in Scientific Evidence, 71 TEMPLE L. REV. 55, 66 (1998). 82. 3 GOUDGE, supra note 6, at 410. 83. Id. at 414.
80
50 JURIMETRICS
Forensic Science and Miscarriages of Justice work be done to help forensic pathologists better understand and express the level of certainty with which they hold their opinion.84 These conclusions were reached after considerable probing by the Ontario pathology report. They suggest that the NRC may be overly optimistic in recommending that quantifiable measures of certainty be developed, at least with respect to forensic pathology. To the extent that quantifiable standards can be developed, there is also the problem of whether jurors will properly understand them given the NRC committee’s own observations about the difficulties jurors have with statistical evidence85 and how such standards will be integrated into legal systems based on proof of guilt beyond a reasonable doubt.
F. Abolishing Police Crime Laboratories versus the Need for Quality Assurance in All Laboratories The NRC’s approach to forensic sciences is focused on increasing the reliability of forensic science that is supplied in state institutions. One of the NRC’s recommendations is that federal incentive funds be used to remove laboratories from the control of law enforcement and prosecutors.86 This may be another example of the perfect being the enemy of the good. It runs counter to Manitoba’s Driskell inquiry, which rejected proposals that the RCMP’s crime laboratories be separated from the RCMP, in part because the RCMP laboratory had admitted its mistakes, cooperated with the inquiry, and formed its own external Advisory Group.87 The commissioner noted that “the forensic science panelists generally agreed that in their experience, formal affiliations between a forensic laboratory and a police force are not always a cause of concern.”88 The NRC’s recommendations about the need for increased awareness and further research on confirmation bias89 are more important than its recommendations about the need to remove all forensic laboratories from the administrative control of police and prosecutors. Although independent laboratories may be desirable, it is more important that all laboratories engage in quality assurance and address the dangers of confirmation bias. The separation of crime laboratories from the police would provide no guarantee against error. A report prepared by a forensic science expert, Douglas Lucas, for Manitoba’s Driskell Commission concluded that there was nothing in the case under review that suggested the forensic opinion was influenced by the laboratory’s relation to the police.90 It also stressed that the Kaufman Inquiry revealed that crime laboratories that were independent from the police could also produce flawed opinions.91 Dr. Lucas also noted
84. Id. at 413. 85. NRC REPORT, supra note 1, at 237. 86. Id., recommendation 4, at 190–91. 87. LESAGE, supra note 6, at 185. 88. Id. 89. NRC REPORT, supra note 1, recommendation 5, at 191. 90. DOUGLAS LUCAS, REPORT IN FORENSIC SCIENCE MATTERS TO THE COMMISSION OF INQUIRY RE JAMES DRISKELL 29–30 (2006), reprinted in LESAGE, supra note 6, app. G. 91. Id.
FALL 2009
81
Roach that, regardless of their official status, most forensic laboratories would work for police and prosecutors, and the ultimate issue was the objectivity and impartiality of the laboratory regardless of its affiliation.92 A leading article by Professor Paul Giannelli similarly stresses the need for quality assurance and protocols to control various forms of cognitive bias more than separation from policing and prosecutorial organizations.93 Professor Giannelli concludes that separating laboratories from the police is not politically viable and that a better solution is to have the laboratories subject to a forensic science commission with outside representation.94 In short, quality assurance, blind review, proficiency testing, and the documentation of what information analysts receive from police would more directly address the likelihood of error than formal separation of all crime laboratories from the police. The NRC’s recommendations both to abolish coroner systems and police crime laboratories will be difficult to implement and will engender much resistance from the targeted institutions. The absence of priorities for reforms in the NRC report creates the danger that resistance to some of the most unpopular and unrealistic recommendations may jeopardize the implementation of other needed and attainable reforms. In other words, there is a danger that the perfect may be the enemy of the good.
G. The Missing Defense Experts There is very little said in the NRC report about the role of defense experts as a check on flawed forensic science, including the concerns raised in the United States about the paucity of defense experts in many criminal cases.95 In contrast, the Ontario pathology report had the benefit of studying a 1991 case that involved a successful adversarial challenge to Dr. Smith’s testimony.96 Ten defense experts effectively rebutted Dr. Smith’s testimony that the cause of a child’s death was nonaccidental. They were retained when the family of the accused 16-year-old baby sitter sold their family home and cashed in their retirement savings to defend their daughter.97 The Ontario pathology report recommended that increased legal aid funding be provided for defense of cases involving complex pediatric forensic evidence.98 Although the NRC was correct to focus on the basic problem of unreliable forensic science provided to the state, it neglected the contributions that effective adversarial challenge could make in revealing some inaccurate forensic science. Indeed, the financial difficulties that most accused have in challenging foren92. Id. 93. “The key to laboratory performance is quality assurance.” Giannelli, supra note 4, at 213. 94. Id. at 228. 95. Paul C. Giannelli, Ake v. Oklahoma: The Right to Expert Assistance in a Post-Daubert, Post-DNA World, 89 CORNELL L. REV. 1305, 1338–42 (2004); Brandon L. Garrett & Peter J. Neufeld, Invalid Forensic Science and Wrongful Convictions, 95 VA. L. REV. 1, 89 (2009) (defense expert evidence presented in only 19 of 137 trials of subsequent DNA exonerations). 96. 1 GOUDGE, supra note 6, at 11–13; see supra note 29 and accompanying text. 97. 1 GOUDGE, supra note 6, at 12; R. v. M.(S.), [1991] O.J. 1383 QUICKLAW (O.C.J. July 25, 1991) (Can.). 98. 3 GOUDGE, supra note 6, at 460–63.
82
50 JURIMETRICS
Forensic Science and Miscarriages of Justice sic science offered by the state is critically important in understanding why the accused, unlike corporate defendants in civil cases, have generally been unable to use Daubert to exclude evidence on reliability grounds.99
III. SOME REFORM DANGERS As suggested in the first part of this paper, the NRC report establishes an ambitious and inspiring vision of reform of all of the forensic sciences. Few would disagree with such a goal. What could be criticized, however, is the report’s failure to discuss the obstacles that likely will be encountered along the way. As discussed above, the report does not grapple with the practical difficulties of implementing unifying reforms in a federal system. The report also largely ignores the dangers of quick fixes that will attempt to provide the forensic sciences with a veneer of increased scientific engagement and standardization. Finally, the report neglects the danger of misinterpretation when the research base for forensic sciences is improved and relied upon as part of a more scientific and evidence-based approach to forensic science.
A. The Dangers of Superficial Reforms and Quick Fixes At some points in the report, the NRC seems aware of the dangers of superficial reforms that can create an illusion that forensic sciences are adequately regulated and that forensic opinions have consistency and scientific support. For example, the NRC notes that while the Association of Firearm and Tool Mark Examiners have published protocols for making identifications, the protocols depend on the experience and judgment of the examiner.100 They do “not even consider, let alone address, questions regarding variability, reliability, repeatability, or the number of correlations needed to achieve a given degree of confidence.”101 Similarly, guidelines provided by the American Board of Forensic Odontology, a certifying board since 1976, do not address the degree of probability. “There is no science on the reproducibility of the different methods of analysis that lead to conclusions about the probability of the match.”102 These examples demonstrate the danger of faux standardization and faux science. This can occur when a professional organization, as a defensive measure, provides standard terminology and guidelines for examiners in anticipation of possible challenges to their expertise. In the absence of scientific research and support, however, such guidelines may only provide legitimizing cover for the subjective and experiential-based judgments of the examiner. Certification measures without adequate education and enforcement 99. Erica Beecher-Monas, Paradoxical Validity Determinations: A Decade of Antithetical Approaches to Admissibility of Expert Evidence, 6 INT’L COMMENTARY ON EVIDENCE 1, 21–22 (2009), http://www.bepress.com/ice/vol6/iss2/art2; Keith A. Findlay, Innocents at Risk: Adversary Imbalance, Forensic Science, and the Search for Truth, 38 SETON HALL L. REV. 893, 901 (2008). See also supra text accompanying note 5. 100. NRC REPORT, supra note 1, at 155. 101. Id. 102. Id. at 174.
FALL 2009
83
Roach could cause more harm than good by creating the illusion of proper training. The same is true with respect to standardization measures that lack scientific support and mask subjective assessments. The NRC report cryptically notes that “some certification organizations appear to lack stringent requirements.”103 The NRC’s ultimate recommendation that a new National Institute of Forensic Science establish certification requirements for forensic science professionals and standard terminology104 would presumably address the problem of shallow or faux scientific standards. Nevertheless, it would have been helpful if the report had provided some interim strategies for determining the validity and reliability of standards that have or will be developed as a wide variety of professional organizations respond to the report and related critiques of the forensic sciences. The articulation of interim standards to expose inadequate regulation and standardization is especially important if, as suggested above, there are difficulties in imposing federal regulation on the forensic sciences.
B. The Importance of Building a Research Base for the Forensic Sciences In one of the few examples of a comparative perspective, the NRC notes that pathology departments in American medical schools usually do not conduct research in forensic pathology and that “[a] review of the forensic literature for basic research in forensic pathology reveals that efforts are largely originating in Europe, Scandinavia, and Japan. In other countries, universities house a department of legal medicine . . . where forensic pathologists have the time, expertise and funding needed to perform basic forensic research.”105 The NRC report recommends that Congress should appropriate funds “to support research, education and training in forensic pathology” and to encourage “collaborative research to be conducted by medical examiner officers and medical universities.”106 The Ontario pathology report similarly recommended that teaching and research be seen as an integral part of a forensic pathologist’s job and that funding be provided to that end.107 It also recommended that funding be made available to establish a Centre for Forensic Science and Medicine at the University of Toronto.108 This Centre has been established.109
103. Id. at 209. For a more critical examination of attempts by the American Board of Forensic Anthropology to certify forensic anthropologists, see Kathy Gruspier, Pediatric Forensic Pathology as Forensic Science: The Role of Science and the Justice System, in 2 INDEPENDENT RESEARCH STUDIES PREPARED FOR THE INQUIRY INTO PEDIATRIC FORENSIC PATHOLOGY IN ONTARIO, PEDIATRIC FORENSIC PATHOLOGY AND THE JUSTICE SYSTEM 151, 168 (Kent Roach ed., 2008). 104. See NRC REPORT, supra note 1, ch. 7 (discussing the problems pertaining to standards and offering recommendations). 105. Id. at 261. 106. Id. at 267. 107. 3 GOUDGE, supra note 6, at 298. 108. Id. at 298–99. 109. For information concerning the activities of the Centre, see Centre for Forensic Sci. & Med., Univ. of Toronto, http://www.forensics.utoronto.ca/site6.aspx (last visited Nov. 8, 2009).
84
50 JURIMETRICS
Forensic Science and Miscarriages of Justice Both the NRC and the Ontario pathology report support an evidencebased approach to the forensic sciences, which makes greater use of published research. The link to research also explains the common focus in both reports on promoting the research base for the forensic sciences and bringing the rigor of the scientific method and peer review to the forensic sciences. There is much that is attractive in this vision of evidence-based forensic science, especially as contrasted with an older experience-based approach to forensic science. In particular, the experience-based approach can be resistant to adversarial challenge or concerns about observer or confirmation bias. Commitment to research is also an attractive way of encouraging forensic scientists to be more independent and to recognize legitimate areas of controversy.
C. Dangers in Misinterpreting Research: A Cautionary Tale from Canada Although there is much to be said for increasing the research base for all forensic sciences, even this reform presents some dangers. Ontario’s 1998 Kaufman Commission into the Guy Paul Morin case provides an important warning about how published and peer reviewed studies can be misused in court. In Morin’s second trial, a study by two British forensic scientists about the likelihood of random fiber transfer was cited by the prosecution as support for the conclusion that the fiber evidence indicated contact between the accused and the victim.110 The authors of the study, however, testified at the inquiry that their study had no relevance to Morin’s case because of differences in methodology and context.111 The forensic scientists who testified in Morin’s case had misunderstood the published study, and the prosecutors who relied on the study were not in a position to correct such scientific misunderstandings. In the end, the Commission concluded that “the study, properly understood, did not support the case for the prosecution” and was “seriously misused at Mr. Morin’s trial and likely misled the jury.”112 This finding is a warning that the introduction of published research in a trial can introduce another source of error and misinterpretation. Such risks do not justify less research, but they do suggest that a more research-based culture in the forensic sciences will require greater sophistication among forensic experts and their target audiences.
110. Graham Jackson & Roger Cook, The Significance of Fibres Found on Car Seats, 32 FORENSIC SCI. INT’L 275 (1986). 111. The researchers concluded that their study should not have been introduced in the case because: (1) it involved only two target fibers while there were 153 to 463 target fibers in the Morin case, yielding a much higher likelihood of “coincidental ‘matches’”; (2) because the accused and the victim were next door neighbors, there were environmental links in the Morin case that were not present in the study; (3) the study only examined coincidental matches on the front seats, whereas the Morin case involved an examination of the entire car; and (4) their study relied on a more discriminating test, thin layer chromatography, than the one used in the Morin case. KAUFMAN, supra note 6, at 120–31. 112. Id. at 119.
FALL 2009
85
Roach
IV. THE DANGERS OF GIVING UP ON DAUBERT WHEN OTHER COUNTRIES ARE STARTING TO EMBRACE IT The NRC devoted a chapter to the admissibility of forensic science evidence in litigation.113 It found that “[m]uch forensic evidence—including, for example, bite marks and firearm and toolmark identification—is introduced in criminal trials without any meaningful scientific validation, determination of error rates or reliability testing to determine the limits of the discipline.”114 The report was pessimistic about the contributions of judicial gatekeeping to improved forensic science, concluding that “Daubert has done little to improve the use of forensic science evidence in criminal cases.”115 Unlike the Ontario pathology report,116 the NRC report did not make recommendations designed to assist the accused in making Daubert challenges to forensic science. The NRC seems close to giving up on Daubert117 at a time when many other countries are embracing its promise. A committee of the House of Commons in the United Kingdom and the Law Commission of England and Wales have both called for stricter admissibility standards patterned on Daubert.118 The Ontario pathology report similarly recommended that “trial judges should be vigilant in exercising their gatekeeping role” with respect to the admissibility of scientific evidence and exclude such evidence if it “does not satisfy standards of threshold reliability . . . whether or not the science is classified as novel or not.”119 Threshold reliability requires an assessment of “the reliability of the proposed witness, the field of science and the opinion offered in the particular case”120 with attention paid to the limits of the proposed witness’s expertise and the language used to express the opinion.121 The Ontario pathology report stressed that judges needed to pay close attention to the methodological and reliability issues identified in Daubert.122 Moreover, judges should examine the details of the proposed expert evidence.123 It also
113. NRC REPORT, supra note 1, at 85–110. 114. Id. at 107–08. 115. Id. at 106; see supra text accompanying note 5. 116. 3 GOUDGE, supra note 6, at 413–14, 460–63. 117. NRC REPORT, supra note 1, at 3–20. 118. HOUSE OF COMMONS SCI. & TECH. COMM., FORENSIC SCIENCE ON TRIAL: SEVENTH REPORT OF SESSION 2004–2005 H.C. 96-l, at 75–76 (2005), http://www.publications.parliament. uk/pa/cm200405/cmselect/cmsctech/96/96i.pdf; THE LAW COMM’N, CONSULTATION PAPER NO. 190, THE ADMISSIBILITY OF EXPERT EVIDENCE IN CRIMINAL PROCEEDINGS IN ENGLAND AND WALES: A NEW APPROACH TO THE DETERMINATION OF EVIDENTIARY RELIABILITY ¶¶ 1.10, 3.16– 3.17 (2009), http://www.lawcom.gov.uk/docs/cp190.pdf. Of particular interest to an American audience, the Law Commission argues that the reliability of expert evidence is a matter of law and that no particular deference is owed to the trial judge’s decision. Id. ¶ 4.20. This latter consultation paper was published after the release of the NRC report. 119. 3 GOUDGE, supra note 6, at 487. 120. Id. at 496. 121. Id. at 493. 122. Id. at 483–84. 123. Id. at 495.
86
50 JURIMETRICS
Forensic Science and Miscarriages of Justice stressed that the judge was not limited to the binary decision whether to include or exclude expert evidence.124 Judges can also place limits on the expert’s opinion evidence because of concerns about reliability, error rates, and misleading language.125 The Ontario pathology report’s optimism about the contribution that judges could make to the reliability of expert forensic science evidence finds some support in the evolution of Canadian law on the subject of expert evidence. As in some jurisdictions in the United States, the Canadian courts have moved away from tests that focus on whether the expert has special knowledge and whether there is general acceptance of such knowledge to more rigorous tests.126 The Canadian tests require that the expert opinion evidence be 1) relevant, 2) necessary to assist the trier of fact, 3) given by a properly qualified expert, and 4) that it not be excluded by an exclusionary rule, such as the general rule that the prejudicial effect of the evidence should not outweigh its probative value.127 In 1994, the Supreme Court of Canada warned about a danger that expert evidence will be misused and will distort the fact-finding process. Dressed up in scientific evidence which the jury does not easily understand and submitted through a witness of impressive antecedents, this evidence is apt to be accepted by the jury as being virtually infallible and as 128 having more weight than it deserves.
The Court subsequently recognized the relevance of the Daubert factors relating to whether the theory behind the testimony has been tested and subjected to peer review and inquiry into its known or potential error rate.129 The Court has stressed that science that might be accepted in a clinical setting to treat a patient may have too high an error rate to justify its use as forensic evidence in criminal proceedings.130 Two of the main reasons the NRC was pessimistic about judicial gatekeeping were its findings that appellate courts defer to admissibility decisions made by trial judges and that stricter admissibility standards, when applied in criminal cases, generally exclude expert testimony offered by the accused.131 Until recently, similar criticisms could have been made of the Canadian law.
124. Id. 125. Id. at 500. 126. Id. ch. 18. 127. R. v. Mohan, [1994] 2 S.C.R. 9, 16 (Can.), http://csc.lexum.umontreal.ca/en/. 128. Mohan, 2 S.C.R. at 17 (Can.). 129. R. v. J.-L.J, [2000] 2 S.C.R. 600, ¶¶ 28, 33, 2000 SCC 51 (Can.), http://csc.lexum. umontreal.ca/en/. See also supra text accompanying note 5. 130. J.-L.J., 2 S.C.R., ¶ 55 (Can.); R. v. Trochym, [2007] 1 S.C.R. 239, ¶ 55, 2007 SCC 6 (Can.), http://csc.lexum.umontreal.ca/en/. 131. NRC REPORT, supra note 1, at 96 (citing Peter Neufeld, The (Near) Irrelevance of Daubert to Criminal Justice, 95 AM. J. OF PUB. HEALTH S109 (Supp. 1.) (2005)). Some in the United States argue that if applied strictly, Daubert can be a barrier to the accused’s right of defense and confrontation. See Christopher Slobogin, The Structure of Expertise in Criminal Cases, 34 SETON HALL L. REV. 105, 109 (2003); see generally Janet C. Hoeffel, The Gender Gap: Revealing Inequities in Admission of Social Science in Criminal Cases, 24 U. ARK. LITTLE ROCK L. REV. 41 (2001) (discussing the exclusion of expert testimony in criminal cases).
FALL 2009
87
Roach In 1993, the Supreme Court of Canada deferred to the trial judge when he refused to strike the testimony that a burn expert gave about child abuse and the testimony that a child abuse expert gave about burns.132 In subsequent years, however, the Canadian courts have placed greater reliance on the Daubert factors, but these factors were generally applied only to novel science and were used in leading cases to exclude expert evidence presented by the accused.133 Refusal by Canadian courts to admit evidence called by the defense, with respect to the reliability of eyewitness identification134 and the phenomena of false confessions,135 also supports the idea that the accused does not often benefit from stricter standards for the admissibility of expert evidence. Some recent Canadian decisions, however, demonstrate that courts are capable of using Daubert tools to take a harder look at the reliability of evidence offered by the prosecution. A significant factor in these decisions has been increased attention to the contribution that unreliable evidence makes to wrongful convictions. The most important case in this respect is the Supreme Court of Canada’s 4:3 decision in R. v. Trochym.136 In that 2007 decision, the Court excluded posthypnosis testimony of a witness who purported to provide eyewitness identification of the accused at a site of a murder. The majority stressed the importance of determining the threshold reliability of the evidence and started its judgment by noting: In recent years, a number of public inquiries have highlighted the importance of safeguarding the criminal justice system—and protecting the accused who are tried under it—from the possibility of wrongful conviction. As this Court has previously noted, “[t]he names of Marshall, Milgaard, Morin, Sophonow and Parsons signal prudence and caution in a murder case”: United States v. Burns, [2001] 1 S.C.R. 283, 2001 SCC 7, at para. 1. In the case at bar, we consider once again the need to carefully scrutinize evidence presented against an accused for reliability and prejudicial effect, and to ensure the basic fairness of the criminal process. More specifically, the decisions at trial to admit post-hypnosis evidence and similar fact evidence in this case constitute 137 errors of law that are neither harmless nor negligible.
In deciding that posthypnosis testimony should be excluded, the Court was not deterred by the fact that it had been accepted in previous cases or by the trial judge. The Court was not overly influenced by the fact that the evidence might 132. R. v. Marquard, [1993] 4 S.C.R. 223, 227, 229–30 (Can.), http://csc.lexum.umontreal. ca/en/. 133. See J.-L.J., 2 S.C.R., ¶ 61 (Can.) (exclusion of results of a penile plethysmograph offered by a person accused of sexual assault). Note, however, that another leading case decided the same year held that expert evidence about child abuse accommodation syndrome offered by a psychologist called by the prosecutor should not be admitted. R. v. D.D., [2000] 2 S.C.R. 275, ¶¶ 70–71, 2000 SCC 43 (Can.), http://csc.lexum.umontreal.ca/en/. 134. R. v. McIntosh, [1997] 102 O.A.C. 104, ¶.26 (Can.). 135. R. v. Osmar, [2007] 220 O.A.C. 186, ¶¶ 70–71 (Can.); R. v. Phillion, [2009] 246 O.A.C. 317, ¶ 12 (Can.). 136. [2007] 1 S.C.R. 239, 2007 SCC 6 (Can.), http://csc.lexum.umontreal.ca/en/. 137. Id. ¶ 1.
88
50 JURIMETRICS
Forensic Science and Miscarriages of Justice not be characterized as novel science, noting that “the admissibility of scientific evidence is not frozen in time.”138 The court stressed that what was “most troubling” about posthypnosis evidence was “the potential rate of error in the additional information obtained through hypnosis when it is used for forensic purposes. At the present time, there is no way of knowing whether such information will be accurate or inaccurate. Such uncertainty is unacceptable in a court of law.”139 Based on the NRC report, similar conclusions could be made about much forensic science evidence. In other Canadian cases, courts have been prepared to exclude or discount forensic evidence offered by the prosecutor because the evidence was only grounded in the expert’s experience and not in science or the evidence. In the recent Stephen Truscott appeal, the Ontario Court of Appeal disregarded two opinions offered by expert witnesses called by the prosecution—a forensic pathologist and an entomology expert—about the time of death.140 The Court of Appeal was concerned that the opinions were based simply on the considerable experience of the Crown’s experts and did not engage with the scientific literature or other evidence.141 Whether Canadian courts will continue the trend of applying Daubert threshold reliability to prosecution evidence remains to be seen.142 Defense lawyers will need adequate training, resources, and funding to make effective Daubert challenges, and judges will have to be prepared to exclude or limit forensic evidence even though it has been previously admitted in some cases. A recent Ontario Court of Appeal decision, however, suggests that the application of Daubert standards to all forms of expert evidence will be controversial. The Court of Appeal overturned the decision of a trial judge to exclude expert evidence offered by the prosecution about the meaning of a teardrop tattoo in a gang-related murder case,143 The Court of Appeal warned that it was inappropriate to apply Daubert standards that require scientific methodology and the calculation of error rates to sociological evidence.144 It relied on Kumho Tire Co. v. Carmichael145 to justify a more flexible approach 138. Id. ¶ 31. 139. Id. ¶ 55. 140. Re Truscott, [2007] 226 O.A.C. 200 (Can.). 141. Id. ¶¶ 165–69, 313–14. 142. For cases excluding prosecution evidence, see, e.g., R. v. Wood, [2007] 79 Alta. L.R.4th 358, ¶ 85 (Can.) (drug recognition evidence in impaired drivers); R. v. Klymchuk, [2008] 244 O.A.C. 210, ¶¶ 15–16 (Can.) (evidence based on crime scene reconstruction); R. v. Hughes, [2008] O.J. 2421, ¶ 21 QUICKLAW (June 9, 2008), aff’d, [2009] ON.C.A. 268 (Can.) (evidence of sniffer dog detecting fire accelerant). But stricter threshold reliability standards also have been used to exclude defense evidence. R. v. Chalmers, [2009] 247 O.A.C. 250, ¶¶ 78–79 (Can.) (forensic kinesiology evidence excluded). In addition, some suspect forms of forensic evidence also continue to be admitted. See, e.g., R. v. Shaffer, [2006] 210 O.A.C. 104, ¶¶ 20–21 (Can.) (expert in firearm identification offering evidence on sight lines); R. v. B.-H.R., [2008] ON.C.J. 369, ¶ 10 QUICKLAW (July 7, 2008) (firearm evidence admitted); R. v. VanEindhoven, [2007] Nu.J. 1, ¶ 77 QUICKLAW (Jan. 9, 2007) (bite-mark “match” admitted). 143. R. v. Abbey, [2009] ON.C.A. 624, ¶¶ 174–76 (Can.),, http://www.canlii.org/en/on/onca/ doc/2009/2009onca624/2009onca624.pdf. 144. Id. ¶¶ 104–17. 145. 526 U.S. 137 (1999).
FALL 2009
89
Roach that does not require the application of Daubert criteria to non-scientific expert evidence based on the experience of the expert.146 This decision has the potential to allow much forensic science offered by the state’s experts to continue to be admitted without regard to its use of scientific methodology or attempts to determine its error rate. If this happens, the NRC’s pessimism about Daubert will be validated in another jurisdiction. Nevertheless, the Ontario Court of Appeal’s decision has some potential to limit the damage of faulty or overbroad forensic science expert testimony. It suggests that trial judges should be careful to limit the scope and language of an expert’s opinion, particularly in cases where the testimony may relate to the ultimate issue of guilt.147 It also follows the recommendations of the Ontario pathology report in stressing that trial judges are not limited to the remedy of excluding expert evidence. In other words, trial judges can, in their gatekeeper role, place restrictions on the language used by the expert witness in order to reflect uncertainty and controversies in the science and the limits of the witness’s expertise.148 Professor Gary Edmond of the University of New South Wales has argued that a higher standard of demonstrable reliability should be applied to prosecution evidence.149 Edmond relies on the state’s ability to fund the type of research and verification procedure that can affirm the reliability of the evidence that it presents in criminal cases.150 This argument should not be confused with one that necessarily would allow the defense to introduce evidence of suspect reliability.151 The Ontario pathology report adopted many of Professor Edmond’s recommendations, including the importance of determining threshold reliability and establishing guidelines that judges could use to accomplish this task.152 But the Commission did not accept his proposal that forensic science sought to be introduced by the prosecutor be held to a higher standard of demonstrable reliability than would be applied to defense evidence.153
146. Abbey, ON.C.A. 624, ¶¶ 110–12 (Can.). 147. Id. ¶ 103. 148. Id. ¶ 64 (citing 3 GOUDGE, supra note 6, at 499–500). 149. Gary Edmond, Pathological Science? Demonstrable Reliability and Expert Forensic Pathology Evidence, in 2 INDEPENDENT RESEARCH STUDIES PREPARED FOR THE INQUIRY INTO PEDIATRIC FORENSIC PATHOLOGY IN ONTARIO, PEDIATRIC FORENSIC PATHOLOGY AND THE JUSTICE SYSTEM 91, 140–45 (Kent Roach ed., 2008). 150. Id. at 117; see also MICHAEL REDMAYNE EXPERT EVIDENCE AND CRIMINAL JUSTICE 126 (2001) (the state as a repeat player has resources to carry out research on the expert evidence it uses). 151. Professor Edmond proposes that the defense should still have to satisfy existing standards, but that the state “as an exemplary litigant” should be bound by higher standards. Edmond, supra note 149, at 117. The Law Commission, however, elides this distinction when it concludes: “[W]e believe that it would be quite wrong to create different standards of reliability for prosecution and defence expert evidence. Our view is that the defence should not be able to divert the jury’s attention from reliable prosecution evidence by being allowed to adduce inherently unreliable expert evidence . . . .” THE LAW COMM’N, supra note 118, ¶ 6.63. 152. See generally GOUDGE, supra note 6. 153. Id.
90
50 JURIMETRICS
Forensic Science and Miscarriages of Justice It is unfortunate that the NRC did not consider a demonstrable reliability standard for prosecution forensic evidence. Such an approach has the potential to provide the strongest incentives for the federal government to fund, and the states to accept, increased regulation and reforms in forensic science. Such federal regulation and reforms would be designed to test and increase the reliability of the many forensic sciences that the state routinely relies upon to convict its citizens. The stricter demonstrable reliability approach to the admission of forensic science evidence offered by the state in criminal trials has the virtue of conceiving of forensic science as an interdependent system. This system includes the laboratories and analysts used by the state, the reliance that police and prosecutors place on forensic evidence produced by the state, and the state’s unique burden of proving guilt beyond a reasonable doubt. The demonstrable reliability standard also takes into account the state’s unparalleled resources and the possibility that such resources can be used to test and perform quality assurance on forensic science used by the state in criminal cases. Finally, a requirement that the state’s forensic evidence be demonstrably reliable recognizes that the criminal justice system is designed to prefer wrongful acquittals to wrongful convictions.154 The NRC’s conclusions that Daubert has been unable to keep unreliable forensic science out of the courts suggests that courts would be reluctant and perhaps unable to apply even stricter standards for the admissibility of forensic evidence offered by the prosecution. Even if this is correct, the NRC should have been sensitive to the danger that routine admissibility by the courts of forensic science evidence of untested reliability and validity could hamper the strong case it makes for fundamental reforms to the practice and oversight of forensic science.
In conclusion, the NRC report is an important and bold call for reform of the forensic sciences based on common scientific research principles and on aggressive federal funding and regulation of the forensic sciences. If only as an intellectual exercise, the report has demonstrated a way to surmount the disciplinary and jurisdictional fragmentation of the forensic sciences. Nevertheless, there are real questions about the workability of the NRC’s unified vision of the forensic sciences in any federal system, let alone one as decentralized as the United States. Robust use of the federal spending power can produce powerful incentives, but direct federal regulation of forensic science and forensic scientists may be unrealistic. Although the United States’ radically fragmented approach to forensic sciences poses unique challenges, and the NRC’s focus on the United States was related to the mandate it received from Congress, it is regrettable that the 154. The extent of the preference for wrongful acquittals over wrongful convictions is a matter of dispute. See generally Alexander Volokh, N Guilty Men, 146 U. PA. L. REV. 173 (1997) (examining various formulations of the precept that it is better for 10 guilty people to go free as opposed to an innocent person being convicted).
FALL 2009
91
Roach report makes no attempt to assess relevant comparative experiences with forensic sciences. A comparative approach could in some cases have strengthened the NRC’s case for reform, and in others, it could have served as a reality check on its recommendations. Consideration of the recent Ontario pathology report for example, would have suggested that considerable reform and professionalization of death investigation systems is possible within coroner systems. In the wake of this report, Ontario has adopted a system in which forensic pathologists will have responsibility for devising best practices, education, and quality assurance standards within a system run by coroners. Significant reform is possible and urgently needed without the political and constitutional change necessary to abolish coroner systems. Consideration of the Ontario pathology report also would have underlined the common challenges that the shortage of trained forensic pathologists pose for all jurisdictions and the likelihood of increased international competition for such persons. It would have cast doubt on the idea that quantifiable standards of reliability and certainty can easily be developed for interpretative forensic sciences such as forensic pathology. Finally, the Ontario pathology report, combined with the British experience with shaken baby syndrome, would have identified suspicious infant deaths as one category of cases in which it is particularly important to have autopsies performed by properly trained forensic pathologists. Both the NRC and the Ontario pathology report stress the need to increase the research base of the forensic sciences to move towards a more scientific and evidence-based approach. This is an attractive vision that should be pursued, but a research-based approach should not be idealized. Mistakes can and will be made in interpreting and relying on the much needed research base. Ontario’s 1998 Kaufman Commission provides a valuable account of how forensic scientists, prosecutors, and jurors misunderstood the application of published and peer-reviewed research in a manner that played a role in Guy Paul Morin’s wrongful conviction. Although a more robust research base will help improve the forensic sciences, it will also increase opportunities for misinterpretation and miscommunication of the true meaning of forensic evidence. The NRC report contains a largely negative account of the contribution that Daubert has made to the reliability of forensic science. This account can be contrasted with much more optimistic accounts provided by British and Canadian reformers. British and Canadian reformers can learn much from the NRC’s pessimistic evaluation of the effects of Daubert in the criminal justice system. At the same time, the NRC may be premature to give up on Daubert. It might have learned something from the Ontario pathology report’s proposals for increased legal aid and better training for defense lawyers and judges to allow the accused to make better use of Daubert. Finally, the NRC’s pessimism about Daubert also demonstrates some complacency about the role of the courts in the multifaceted effort that is required to improve the forensic sciences.
92
50 JURIMETRICS
