How Should We Organize to Do Informatics? - CiteSeerX

3 downloads 12 Views 119KB Size Report
versity of Pittsburgh Medical Center, Suite 8084, Forbes Tower,. 200 Lothrop Street ..... call these two perspectives ''external'' and ''internal.'' Let us first examine ...

Journal of the American Medical Informatics Association

Volume 5

Number 3

May / Jun 1998

293

Forum Paper 䡲

How Should We Organize to Do Informatics? Report of the ACMI Debate at the 1997 AMIA Fall Symposium ORGANIZER

AND

MODERATOR : CHARLES P. FRIEDMAN, PHD

Participants: MARK E. FRISSE, MD, MS, MBA, MARK A. MUSEN, MD, PHD, WARNER V. SLACK, MD, WILLIAM W. STEAD, MD

The continuing development of the field of medical informatics has raised new questions and placed before us new dilemmas. Spurred by the proliferation of information systems to support the broad missions of our institutions, and the evolution of these systems from luxuries to necessities, organizational issues have assumed increasing prominence. Among a dazzling array of organizational issues now before us is the tension between the long-standing academic role of informatics groups within medical centers and the ever-expanding service role. In the academic role, we seek the knowledge to create improved technology and to train the next generation of informatics researchers. In the service role, we seek to put existing technology, developed internally or purchased from vendors, to best use across the full scope of medical center activities. The dilemma before us is not whether both roles are important — the answer to that is clear—but rather how to organize ourselves within our institutions to address both of them. How much organizational distance should exist between the people who carry out these different roles, and who should direct their ef-

Affiliations of the moderator and participants: University of Pittsburgh, Pennsylvania (CPF); Washington University, St. Louis, Missouri (MEF); Stanford University, Stanford, California (MAM); Harvard University, Boston, Massachusetts (WVS); Vanderbilt University, Nashville, Tennessee (WWS). This debate was part of the program of the AMIA Annual Fall Symposium, Nashville, Tennessee, October 25 – 29, 1997. Correspondence and reprints: Charles P. Friedman, PhD, University of Pittsburgh Medical Center, Suite 8084, Forbes Tower, 200 Lothrop Street, Pittsburgh, PA 15213. e-mail: 具[email protected]典. Received for publication: 1/16/98; accepted for publication: 1/16/98.

forts? Most academic medical centers are actively searching for answers to these organizational questions, and many AMIA members are engaged in this pursuit. The answers obtained will be of profound consequence for our field. The salience of this issue directed its selection as the focus of the ACMI Debate at the closing session of the 1997 AMIA Fall Symposium. The purpose of the debate was not to generate a universal answer, for no such answer exists, but rather to illuminate the many factors that must be considered as our institutions search for an appropriate organizational model. To frame the debate, we intentionally polarized the issue around a specific proposition: Resolved: Academic medical centers should have a single unit responsible for information systems supporting the clinical and academic missions and also should be charged to carry out high-quality education and research in medical informatics.

The polarity is such that the affirmative team would argue in favor of one group under one leader who would carry out all roles. The negative team would argue for a significant level of separation.

Debate Format and This Report We adapted the standard high school and college debate format to fit the available time and to use competition as a device to promote deeper understanding of key issues. There were no judges and no declared winners. Each team had two members: Warner Slack and William Stead for the affirmative, Mark Frisse and Mark Musen for the negative. The format included eight-minute constructive statements, twominute cross-examinations, and three-minute closing (rebuttal) statements in this order:

294

First affirmative constructive statement, by Slack Cross-examination of Slack, by Musen First negative constructive statement, by Frisse Cross-examination of Frisse, by Slack Second affirmative constructive statement, by Stead Cross-examination of Stead, by Frisse Second negative constructive statement, by Musen Cross-examination of Musen, by Stead Closing (rebuttal) statements by Frisse, Slack, Musen, and Stead, in that order In preparing this summary, we sought to convey the substance and spirit of the debate in a manner suited to printed text. This narrative follows the order of the debate as it occurred on October 29, 1997, at the AMIA Fall Symposium in Nashville, Tennessee. The constructive statements included here were edited from the notes the debaters used to prepare their statements. The cross-examinations and closing statements were edited from the debate transcripts and retain much of the colloquial language used in the event. We include bibliographic references only to direct quotations and citations used by the debaters themselves. The format of this and any debate, and most notably the polarization of a multifaceted issue, often requires participants to take extreme positions. The debaters’ views, in reality, overlap more than this report would suggest. Some of the debaters believed that they could, if asked, argue with equal effectiveness in support of their opponents’ position. Statements of the debaters may be at variance with their own personal beliefs and do not represent official policies of the institutions for which they work. With the following, we began the 1997 ACMI debate.

FRIEDMAN

ET AL.,

ACMI Debate

—that tensions arise. When the research laboratory is in the hospital proper, the academic physician will clash with the administrator from time to time, but as long as he or she stays mostly in the laboratory, twisting DNA and manipulating ribosomes, the conflict will be minor. But when doctors and doctors in training, who are both medical school faculty, with their students, and hospital staff, with their apprentices, move onto the floors and into the clinics, an intramural town–gown conflict of epidemic proportions is almost assured. From the Dean’s perspective, the hospital and its administrators are there to serve the faculty and their students and thereby serve the patients. The hospital should be a safe haven for patient care and teaching. From the CEO’s perspective, on the other hand, the doctors are there to serve the patients by serving the administrators. And the conflict is often unresolvable. In years gone by, the doctors had the power and all too often treated administrators with an arrogant attitude that ranged from condescension to disdain. Now the balance of power has shifted. More and more, the administrator is in charge. And this is nowhere more evident than in clinical computing. The medical school faculty still control the clinical departments within their hospitals—up to a point. The chief of clinical pathology is both a professor and a pathologist. And what machines to buy is still, for the most part, the decision of the pathologist. In the early days, x-ray machines were invented in the academic setting. Now, of course, industry makes the machines, but the radiologist, also a professor, still determines the functional specifications and how they’re used. Radiology and clinical pathology are good precedents for clinical computing as an academic discipline.

First Affirmative Constructive Statement, by Warner Slack

The computer itself was invented in an academic setting. Now industry makes the machines, but the computer scientist in the academic setting still specifies their function and use.

Academic medical centers in modern times have two functional entities, each with its master. On the one hand, there is the medical school with its academic hierarchy, from dean on down; on the other hand, there is the hospital with its administrative hierarchy, from the president or chief executive officer (CEO) on down. In some instances, the territorial imperatives are distinctly defined and well established. Anatomy belongs in the medical school, as does the professor of anatomy; housekeeping and room service belong in the hospital, as does the vice president in charge of facilities management. It is when functions overlap— when the academician and administrator must interact

Computing in the teaching hospital, however, is a different story. Early machines were better suited for finance than for clinical applications, and doctors were wary. By the time doctors woke up to the potential of the computer in medicine, territorial imperatives were firmly established. The chief financial officer (CFO) or chief information officer (CIO) and his or her staff were in charge. And this is the situation in most hospitals today, where the clinical computing, if there is any, is poor. Computing companies market to administrators and their consultants. More often than not, the commercial systems are of little help to the doctor and his or her students.

Journal of the American Medical Informatics Association

Volume 5

The exceptions to this situation—teaching hospitals that have good clinical computing—have developed their computing under the auspices of an academic department — for example, Vanderbilt, LDS Hospital, the Regenstrief Institute, Geneva University Hospital, Columbia Presbyterian Hospital, Duke, and several of Harvard’s teaching hospitals. But, where the administrative, non-academic approach encroaches on the academic approach, clinical computing and the related teaching programs deteriorate. By clinical computing, I mean computing designed primarily to help the clinician and patient in the practice of medicine —computing that gives the results of diagnostic studies immediately on request; offers access to the biomedical literature; offers advice, consultation, alerts, and reminders; assists with communication by electronic mail; and assists in the day-to-day practice of medicine. In other words, clinical computing is a medical discipline, and in the medical school environment it belongs in an academic department. The functions of an academic department of clinical computing — functions that cannot be performed well with the administrative approach—are, first, to do basic research with computers in medicine; second, to invent, design, develop, study, and implement computing systems that help clinicians care for their patients, or else to guide the purchase of computing systems of proven quality that help clinicians care for their patients (If academic clinicians insist on good computing, companies that market what are now bad computing systems will change their ways.); third, to be responsible for the day-to-day operation of the clinical computing systems used in the care of patients; and fourth, to teach the principles and practice of clinical computing through real-time experience, courses in the medical school and graduate school, and programs for residents and fellows. An academic orientation toward clinical computing is essential to a good teaching program. In summary, as I survey the current state of computing in medicine, I find that with the administrative approach, hospital computing is designed primarily to help the administrator, the CFO, and the consultant. Clinical computing and education get low priority. With the academic approach, where clinical computing and education are the goals, there are benefits to administrators as well as to patients, students, and faculty physicians. There are demonstrated precedents for success with the academic approach to clinical computing. I know of no precedents for success with the administrative approach.

Number 3

May / Jun 1998

295

Cross-examination of Warner Slack, by Mark Musen Musen: You talked about academic units in your opening remarks but you did not state what you view as an academic discipline. What are the features of an academic discipline? Slack: Within the medical school environment, I consider the first and most important mission of an academic unit such as cardiology, radiology, or clinical computing, to be the care of the patient. Then, as a corollary of this, to teach students how, in the future, to care wisely and well for their patients. Then, finally, to do research both in the applied world as well as the theoretic world. In my view, all of these must be toegether in one academic unit for the program to be successful. Musen: You mention patient care and medical research. Are both of those necessary conditions for an academic unit? Is it necessary that an academic unit do patient care? Slack: Within a medical school, definitely. You certainly would not want an administrator doing cardiology, although I recently read an article where the title was ‘‘Why the Administrator Should Participate Directly in Patient Care.’’ Musen: Does Harvard Medical School have any basic science departments? Slack: Yes. Musen: What are some of them? Slack: People object to the word ‘‘basic science,’’ so that is a bit of a loaded question. I would consider areas such as anatomy, which are clearly defined as academic departments within the sciences and don’t directly relate to hospital care, to be basic sciences. Musen: Let me just ask about that. Anatomy is an academic department. Would you believe that anatomy does patient care in the usual sense of the word? Slack: No. I hope not. Although there may be an occasional exception. Musen: Let me pursue another line of questioning. You are particularly well known for work that you did early on in your career in the area of automated patient history-taking. Have we resolved all the questions related to automated patient history-taking? Slack: Definitely not. Musen: Then why did you stop doing that work? Because of your operational computing responsibilities?

296

Slack: I haven’t actually stopped that work. I’m pressing on with vim and vigor.

First Negative Constructive Statement, by Mark Frisse Our opponents give effective arguments based on our health care systems’ critical need for informatics professionals who can bring much-needed aid to the pressing problems of information systems in health care delivery, teaching, and research environments. We, speaking against the proposition, do not deny the need, nor do we discourage individual trainees from jumping into the fray and doing what they can to aid in this critical area any more than we discourage our internists from entering practice or our doctoral students from entering commercial sectors. But we think that our colleagues’ arguments miss the point: the proposition is about academic units as training grounds —not about the endgame of careers. And so we frame our arguments along the traditional lines of debate that have taken place in medical education since the introduction of the German model for medical education decades before the Flexner era.1 We base our arguments not on the endgame but on the very propositions that have led to the rise of the university as a unique and treasured resource and the importance of the truly academic life to the well-being of future generations. We base our arguments on both the high expectations for academic life and the more realistic appraisal of academic life in health sciences centers. We call these two perspectives ‘‘what should be’’ and ‘‘what is.’’ We also frame these different perspectives from the view of an impassioned observer and from the view of an active participant: someone like you, here in the audience, who is either engaged in a medical informatics career or considering one. We call these two perspectives ‘‘external’’ and ‘‘internal.’’ Let us first examine the world as we believe it should be. In this world, the university is the crucible for a commitment to the long-term investment in knowledge — a commitment made in the belief that such knowledge will ultimately lead to quantum leaps in technology and behavior and, hence, will be to the long-term benefit of society. Where informatics in the academic medical center is concerned, this investment is routinely tested by the practical application of innovation in the real-world setting. Such application, we believe, is best achieved not by a single quasi-academic unit but instead by a seamless relationship between an academic unit and a different group given the responsibility for implementation and support of

FRIEDMAN

ET AL.,

ACMI Debate

information technology. The currency of exchange is a ‘‘marketplace of ideas.’’ We find many of the arguments for a single unit to be based not on principles but, instead, on the cynical belief that this is the only way for the research and service communities to interact effectively, to share mutual respect, and to communicate. Let us turn now to the impact the proposed organizational model has on the career of the academic informatics faculty. In the idealistic, ‘‘what should be’’ context, faculty seek an organization that places a long-term commitment to sustained and productive intellectual growth. The commitment is an agreement between the university and the individual faculty member: If one pursues the very hard labor of focused academic work and exemplified excellence in teaching and research, one will be compensated both for the potential long-term benefit conferred to society and to the university. This is, we believe, what the academic informatician seeks. He or she is not reticent in engaging in a critical examination of current operations and systems, but the faculty does this—not to fix the current system or even develop the ‘‘next system’’—but instead to focus on what Tony Gorry has called ‘‘the system after next.’’2 In the ideal, university-based academic informatics unit, faculty members find an acceptable balance between research, teaching, and service. Excellence is achieved by having time protected to think, and think critically, about the critical issues facing our field. To some, such an ‘‘ivory tower’’ view might reflect a disregard for the current and highly problematic state of information systems in biomedicine. This is not our intent. Rather, we take the view that successful realization of many of the aims of medical informatics — aims that are as yet seldom realized—requires a wide range of skills and temperaments. Where skills are required, we believe strongly that trainees engaging in academic careers must have a strong suite of basic computer science, statistics, epidemiology, decision analysis, and managerial skills. In our view, these skills require formal, rigorous, uninterrupted training. If one wants only warm bodies, one should hire them. If one wants leaders, one should train them well over several years. Scholarship is a life where most satisfaction must come from ones’ peers. To paraphrase Peter Senge,3 one’s ideas are adopted only after they become the ideas of others and—as a corollary — after they are not widely associated with their originator. Those who effect change, then, often do not see it. Truly important work often requires more than one lifetime to be realized.

Journal of the American Medical Informatics Association

Volume 5

Let us now examine the issue from a more realistic perspective. When looking at the world as it is rather than as it should be, the arguments opposing the proposition are even more compelling. First, a stark and depressing fact: Despite years of substantive contributions and excellent service, many of the country’s leading applied medical informatics groups are seeing their systems unplugged and replaced by commercial products that will not provide the same degree of immediate value but which, it is claimed, will confer offsetting benefits of better organizational management. The CIOs initiating these changes are acting in what is believed to be the best interests of their organization, but given the enormous pressures they face from their constituents, they spend little, if any, time on research. Nor do their overextended commitments allow them a budget to create a research infrastructure. Although ‘‘medical informatics’’ is a hot topic in academia, it does not appear that most institutions really have made a commitment to confer on the discipline the academic legitimacy characteristic of the traditional clinical and ‘‘basic science’’ departments. The leadership of our institutions, in other words, while often voting with their speech in favor of the proposition, have voted with their wallets against the proposition supported by Drs. Stead and Slack. From a purely pragmatic view, then, it seems critical for the academic researchers to ‘‘stay in the lane’’ and do what they do best, while maintaining strong bonds of communication with individuals equally committed to the more immediate aspects of information system operations. Let us finally examine the question from the perspective of a clinician or researcher who is either pursuing or contemplating a career in informatics—people like most of us in the audience. When preparing for this debate, I thought often of one of our more prominent colleagues who once told me how he ran from the clinic to fix a modem bank and, later in the day, tried to sit down and work on his very complex research. Lives such as these are characterized by interruption and conflicting goals. We believe it is the rare individual who can rise above these conflicts and meet all his or her obligations with the excellence expected from a university. We ask you who are in this situation: Which position would you take on this proposition? Do you believe that most institutions will create a uniform environment equally capable of opportunities in both basic research and applied activities? Can you name, please, five institutions in the country who have adopted this model and sustained it with success for more than a few years? Do you believe that our best informatics

Number 3

May / Jun 1998

297

research institutions would be capable of maintaining their excellence if they were given all information system support functions? Conversely, examine the institutions that are providing world-class support for information systems and ask yourself whether or not these institutions and their leaders are the most likely candidates for the creation of intellectual environments that will generate the ideas that will influence the next generation of computer systems? We who speak in opposition to the proposition have common training roots at Stanford but have embarked on very different paths. One of us has maintained an almost exclusive focus on pure medical informatics research, and most believe his efforts will have an impact for years to come. One of us have moved from a primarily research position to a position that emphasizes administration. His successes have been due not to a single organizational model but instead to the effective communication and coexistence between those who do the day-to-day things and those who think in the long term. In summary, we who speak in opposition to the proposition argue that a single reporting structure is nothing more than an often ill-conceived effort to provide coherence when different groups do not communicate effectively. We argue that a single source of leadership is essential, but it should come at the level of our deans, chairs, presidents, and CEOs. Organizational structures cannot compensate for a lack of knowledge about information technology on the part of those responsible for the overall prosperity of the enterprise. While our leaders are trying to learn, our CIOs are trying to make flawed systems work successfully, and our faculty are engaged in a discouraging debate about the future of academic medicine, we urge everyone in attendance to ask: ‘‘What is it I really love doing and how can I make a contribution doing this work?’’ We suggest that when individuals frame the issue this way, their futures will be more secure in organizations with peaceful coexistence and common respect; not in an organization wholly dependent on the ‘‘person at the top’’ for coherence.

Cross-examination of Mark Frisse, by Warner Slack Slack: Your comments imply that you have in mind some hospitals where administrative management has resulted in good clinical computing. What hospitals can you point to where the administrative approach has resulted in computing that helps the physician and patient in the educational system within a medical school environment?

298

Frisse: That is not what I meant to convey. I was trying to claim that good research comes first and that good medical informatics research will be the major driver for good clinical computing. I’m not trying to argue for an exclusively administrative approach. Slack: If a cardiologist is doing basic research in his or her laboratory, should the application of that research in clinical cardiology then be turned over to the administrator as it moves out into the ward? Frisse: Administrators have to get better. If the problem is that the administration is weak, then it is okay for an informatician to fix the problem by becoming an administrator. I am claiming that, in the general case, if we put the research and operational computing together, we are placing our research mission at risk. I would claim that you two have not talked about real research yet. Slack: Will you grant me that clinical computing—as I’ve defined it, helping the physician in the practice of medicine — is a medical discipline? Frisse: Yes, a clinical discipline. Slack: And are you comfortable with having the CFO in charge of this discipline, which is so often the case? Frisse: Actually, in my own institution, I am just as comfortable with the CFO leading the effort as I would be with anyone else right now. But it is an unsatisfactory state of affairs.

Second Affirmative Constructive Statement, by William Stead The field of medical informatics is at a crossroads. Forty years is long enough. We must prove that we can generate ideas that translate on a wide scale into working systems that improve health care, health education, or biomedical research. Unless the ideas are useful, and the translation occurs, we will be judged irrelevant. An integrated unit with responsibility for research, training, and provision of the enterprise operation and decision-support infrastructure has the best chance to meet this dual challenge. Each of these three legs must be robust, but integrating them organizationally is a win–win–win. Operational experience and requirements can guide the research agenda, provide a laboratory and a source of funds. Training can alleviate the critical shortage of talent. Research can provide the needed breakthroughs to provide an infrastructure that really works. Operational successes will in turn breed increased basic research funding.

FRIEDMAN

ET AL.,

ACMI Debate

Critics argue that the combination of responsibilities spreads everyone too thin—that research and basic training will be shortchanged or that applications will be built when more leverage could be obtained through vendors. Not so. Those bad outcomes can happen, but such failures are not a byproduct of an integrated organization. The problems stem from a failure to support each of the legs of the integrated organization adequately or from the lack of a management structure that can make the required business decisions. Balancing priorities and tradeoffs is a daunting challenge but one that must be faced. It is no different from learning how to manage our multiple missions as we bring our health science schools together with their affiliated hospitals. As those challenges are overcome, an integrated organization enables six critical success strategies. First, a medical informatician can provide a human bridge between informatics and other biomedical disciplines. Having the idea, and getting an innovation to the stage that it can be tried in practice, is just part of the job. In some cases the informatician must use the innovation for others, or help them use it before it is ready for them to use it on their own. In other words, the combination of the system and the informatician can be used before the system by itself would be seen as a win by its non-informatics users. This is what Randy Miller means when he says: ‘‘Informatics is not a spectator sport—you have to get your hands dirty to make a difference.’’ When I interview graduates of informatics training programs, and they tell me that they have built applications but that no one would use them, I ask whether they used them themselves in practice to understand what the problems were. If they say no, I do not offer them a job. Second, just as in other clinical disciplines, hands-on practice is also a chance to learn what is actually needed from informatics. Biomedicine’s current work processes were designed when communication was difficult. They consist of a set of sequential operations, each using a largely self-contained buffer of information, with a small amount of communication between steps (e.g., a copy of a discharge summary going to a referring physician). When you place information systems into one of these processes, such as order capture, and try to make them interact with other systems that have been placed in another process, such as pharmacy dispensing, things often get worse. To succeed you need blow up the old way of working and design new ways of working that would not be possible without information technology, such as distributed parallel work processes enabled by constant high-bandwidth communication and asynchronous linkages.

Journal of the American Medical Informatics Association

Volume 5

Third, it is almost impossible to figure out how a new process might work, and what type of information technology would be needed to enable the process, without first implementing and trying something. This problem can be reduced by involving the informatics research effort in an iterative process involving rapid prototyping in operational settings. Real wins happen when a medical informatician has the skills and authority to use a system, identify what is needed, make the necessary operational changes on the fly, and redirect the research effort. It is no accident that the first four recipients of the Morris Collen Award got their starts building clinical systems in operational settings. Fourth, an informatics innovation cannot be moved into practice unless it is supported by the infrastructure that supports practice. This problem is acute for trainees who must work in a limited time horizon. The problem is no less for established investigators, because the rapid change in technology can outdate an innovation if dedicated infrastructure must be implemented to support it. Success comes when an innovation can be handled like a light bulb—when it is ready, you plug it into the pre-existing lighting system — and it comes to life. This type of fit requires collaborative planning between individuals responsible for the research program and those responsible for implementation of infrastructure. Fifth, Chuck Friedman has described a tower of innovation in medical informatics.4 It begins with hypothesis generation and then moves through model development and implementation to evaluation. Research funding is available for the first and fourth stage. Operational funding is available for the middle two stages. An integrated organization allows an investigator to play different roles in the effort and to move back and forth between funding sources as their work evolves. Finally, the traditions of stand-alone operational units are rooted in data processing. Data processors feel they have done a good job when they bastardize the database of a complicated system to make it do something it wasn’t designed to do—for example, by storing the ICD code of the diagnosis in the field that was intended to hold the mother’s maiden name. Medical informatics is the exact opposite: recording data and knowledge in structures that allow very simple programs to make complicated relationships clear. We need staff who understand both informatics and operations. We need researchers who understand realworld problems and constraints. The cultural barriers that exist today will not be overcome unless the individuals who are responsible for academic informat-

Number 3

May / Jun 1998

299

ics link hands with their counterparts who are responsible for the operation and decision-support infrastructure — a linkage that is much easier with an integrated organizational structure. In closing, I think that the problem was summed up yesterday, at the panel about integration of informatics into vendor products. One CEO said that 60 percent of the implementations of their stable clinical products occurred on time and budget and that only 50 percent of available clinical function was used in the average implemented site. All agreed that the success rate for leading-edge projects is significantly less than these disappointing numbers. When you consider the rate of technologic change, these statements suggest that the industry does not know how to build products that can be implemented in time to meet our business needs. Three of the four CEOs said that implementation problems stemmed from the clients’ inability to incorporate the product into their processes. I will not accept such an answer from the faculty and staff of the Informatics Center at Vanderbilt. We own responsibility for successfully meeting the business need of the enterprise—buying when we can, building when we must, always focusing on what it takes to create a win. We can establish that level of accountability because of our integrated organization structure.

Cross-examination of William Stead by Mark Frisse Frisse: Is your model essential for the advancement of academic medical informatics research? Stead: Yes. Frisse: In other words, I can’t do good research unless I work for you? Stead: No. You can do good research as long as you work in an enterprise that has a model that allows your research to be productive. Frisse: How do you define productivity? I’ve been very impressed by your short-term horizons, which are very characteristic of a CIO, and I’d offer you the job at my place in a heartbeat. But what if I say that I’m interested in five years down the road? Will you hire me if I’ve got a good idea that may take five years to achieve fruition? Stead: First, my own time horizon is closer to 30 years than it is to two. Second, you are welcome to pursue a good idea if you can find research funding to support development of that good idea and if you do not need input from the operational world to let you know how to go about working on that idea.

300

Frisse: So I cannot have input from the operational world unless I work for you? Stead: It has been very hard for researchers to get that input unless they are part of an integrated unit that also has responsibility for operational computing. For example, when one of our informatics faculty wanted to enter orders as a unit clerk to gain an understanding of how the system might be improved, the response was that a physician could not have a sign-on that was coded as a unit clerk’s. I was in a position to point out that the individual was assuming the role of a unit clerk and that they could have such a signon despite their advanced training. Frisse: You were functioning as a CIO, and you understood the situation. The CIO needs to understand the requirements of the medical informatics research unit; they do not have to lead that unit. Would you say that the fundamental leadership structures of academic medical centers are dysfunctional? Stead: I think that academic medicine is moving toward integrated organizations on all fronts. Your partner suggested the applicability of the basic science model. At Vanderbilt, we are focusing less on departments. Our Cancer Center, for example, brings together basic sciences such as cell biology and clinical sciences such as oncology into an integrated unit. I think that informatics needs to be handled similarly. Frisse: Does modern management theory suggest that the right way to reorganize and to let go and embrace change is by having a hierarchic model similar to the one you suggest? Stead: I am not suggesting a hierarchic model. It is a model of integrated strong team leaders working toward a common objective under common mentorship. Frisse: So there is no boss at Vanderbilt? Stead: I do not think of myself as a boss.

Second Negative Constructive Statement, by Mark Musen We are here today to discuss academic medical informatics. Our goal is not to solve the problems of clinical computing in academic medical centers, although those problems are quite significant and very important. As academicians, we are concerned about each of the issues that Bill Stead laid out in his talk. We are concerned about getting access to clinical data. We are certainly concerned about funding. Nevertheless, we

FRIEDMAN

ET AL.,

ACMI Debate

are not here to debate how best to address the problems of information infrastructure in academic medical centers; we are here to debate how best to address the needs of medical informatics as an academic discipline. Mark Frisse and I believe very firmly that medical informatics has both an academic component and a service component. These two components are quite distinguishable in their content. In the model that the affirmative team advocates, a single organization has responsibility for teaching and research in medical informatics and for maintaining the computing infrastructure of the medical center. That model, I believe, is becoming increasing untenable. As clinical computing evolves, the academic component and the service component of medical informatics necessarily must become more separate. It is our conviction that academic research and institutional service have their own complexities, and that the training and skill set required for excellent work in one area does not by any means ensure that one can perform excellent work in the other area. It is important to take a step back and consider what academics is all about. Earlier in this debate, I asked Warner Slack to enumerate the features of an academic unit in a school of medicine. Warner said that he viewed those components to include clinical care, scholarship, and research. Bill Stead later identified training as an important contribution of an academic unit. Although the affirmative team acknowledges these essential features, they continue to ignore the question of how academic units in medical informatics can best meet their objectives for research and training. The emphasis in this debate has been on improving patient care and on installing better clinical computing systems in certain hospitals. Although these are important goals for modern medical centers, they are quite separate from the goals of training, scholarship, and research that we all agree are essential to the academic mission. I would like to direct our attention to the research questions that need to be addressed within academic medical informatics. Academic medical informatics concerns scholarship in areas such as medical concept representation, computational algorithms for decision support, and evaluation of new technology for information management. Simply put, the issue is whether these research questions can be optimally pursued by workers who spend the majority of their time attending to activities that are best performed by CIOs and by information system vendors. I look at my own institution and think about the various academic departments at Stanford. My university

Journal of the American Medical Informatics Association

Volume 5

Number 3

May / Jun 1998

301

has many important and illustrious academic units. Within my own medical school, the Department of Genetics does extremely important work in the area of understanding the molecular basis of inheritance. Do I want those faculty members to be doing genetic counseling? Probably not. I think of our Department of Biochemistry. Do I want the Biochemistry faculty to manage the pharmacy in Stanford Hospital? Probably not. Do I want the Department of Civil Engineering to be putting up our new buildings? Do I want the Department of Mechanical Engineering to be putting in our sewer systems? Conversely, do I want the architects of Stanford’s sewer system to be applying for NSF grants to study fundamental principles of fluid mechanics?

terprises; trying to deal with the engineering of systems that, unlike the simple programs that ran in 4K of memory in the 1960s, now entail layer upon layer upon layer of various software components whose multiple interactions are extremely complicated and difficult to understand. There has become a new kind of programming necessary to deal with the client – server systems that are now pervasive in enterprise computing. I believe that researchers in academia are probably among the least qualified individuals to be building the robust systems that we need in the clinical workplace. Those of us who are in academic centers simply tend not to have kept pace with rapidly changing software-engineering practices in the commercial arena.

Wherever we look in any kind of enterprise, there is an important service role that needs to be filled. Wherever we look, there are related academic questions that need to be addressed. The people who perform these two kinds of activities are not necessarily the same. The skill sets required for the two classes of work are quite different.

Unlike the gentlemen on the affirmative side, I believe that downstairs in the AMIA Fall Symposium vendor exhibition there is reason to have considerable hope in the commercial sector. Information system vendors increasingly recognize the importance of the clinical dimensions of computing. The off-the-shelf information systems available for installation in the nation’s medical centers are only getting better. The vendor community does not have to rely on those of us in academia to tell them what the market demands. Indeed, market forces will have a key role in ensuring that information systems that meet Warner Slack’s criteria for clinical utility will become increasingly commonplace. Furthermore, the more that information system vendors can incorporate the results of medical informatics research into their products, the more successful those products will be.

Bill Stead asserted that the previous winners of ACMI’s Morris Collen Award are all investigators who started out performing clinically important service computing within their institutions. I agree completely and I do not wish to diminish the significant service work that those awardees have done. Yet each of these giants of our field started a research career when the world was a very different place—when there was no clinical computing in academic centers, when there were no hospital information systems that were available commercially, and when it was the responsibility of those who were on the academic side to recognize the failure of those on the administrative side to deal with the clinical questions that were at hand. The recipients of the past three Collen Awards had to address significant service issues early in their careers because, at the time, there was no one else to do it. It is precisely because of the efforts of these early pioneers that there is now a vast and successful industry that can address the information technology needs of modern medical centers, an industry with which those of us in academia would be foolish to try to compete. The past recipients of the Collen Award were doing their work at a time when the whole notion of building software systems was very different from what it is now. In the last five to ten years, the nature of software engineering has changed dramatically. Suddenly, we are trying to understand the subtleties of requirement analysis; trying to understand how computing can scale to the needs of huge, distributed en-

We are here, however, to debate the nature of academia and the role of medical informatics in the academic community. Many of us in medical informatics can point to stunning achievements with respect to the systems that we have installed and to the way those systems are being used in our home institutions. Many of us, on the other hand, have been less successful in convincing our colleagues in academia that our activities constitute a basic scientific discipline that is in the same ranks as, for example, mechanical engineering or even biostatistics. The reason for this discrepancy is that we tend not to write our scientific papers in ways that make our hypotheses clear to academicians outside the medical informatics community. One of my greatest concerns about our discipline is that, in those situations where medical informatics faculty must deal with the uninterrupted servicing of the computing enterprise, it is much harder to frame the underlying research questions in ways that allow investigators outside our field to understand our scientific contributions.

302

Previously, I asserted that those of us in academia are not ideally suited to implementing the kinds of enterprise solutions that Bill Stead and Warner Slack would like us to implement. I now maintain, conversely, that the field of medical informatics faces substantial challenges because many of our scholars are so preoccupied with the day-to-day management of such systems, that they appear to have lost track of how our particular discipline might inter-digitate with other academic areas. For example, the medical informatics community can point to a host of seemingly successful IAIMS sites, such as the one at Vanderbilt. Yet how does work on IAIMS relate to widely published research in the industrial engineering community on the effects that information systems have had on large organizations and on the ways in which workers interact with such systems? How can we relate the IAIMS achievement to investigation by scientists outside medical informatics who have examined testable hypotheses in the area of enterprise computing? How does the work we do in the area of clinical vocabulary speak directly to the hundreds of people in the computer science community who investigate formal knowledge representation and ontology? We in medical informatics have exciting results to offer to scientists outside our discipline, but we usually have not been successful in demonstrating the generalizability of our contributions. As academicians, we need to develop faculty who have both the skills and the time to publish papers that communicate the hypotheses we are exploring and the contributions we are making to science and to medicine. Work in medical informatics is important because it proposes and tests significant hypotheses and because it promotes the understanding of medical concepts and medical practice. It is only as a side effect that our research, we hope, makes life better for people who work in hospitals.

Cross-examination of Mark Musen, by William Stead Stead: As a medical informatician at Stanford, do you think it is acceptable for your CEO to sign a contract for a multimillion-dollar purchase for an information system that has a significant chance of failure without informed consent? Musen: I don’t think it is acceptable with respect to the enterprise for my CEO to do that. But the real question is what role should academic medical informatics play in terms of providing good computing within the academic center?

FRIEDMAN

ET AL.,

ACMI Debate

Stead: If you were a cardiologist, and you saw something bad going on in the catheterization laboratory, would you bring it to someone’s attention? Musen: Certainly. Stead: How long has the medical informatics research program at Stanford been in place? Musen: Decades. Stead: How many ideas have translated into practical things that have created a substantial change in health care or education? Musen: I think it is fair to say that the work we have done has had enormous effects on the work that other people have done. You are pointing out that we have had limited influence in applying our work in our own institution, but I don’t think that diminishes the broad applicability of the kind of results that we have had in our research. Stead: What makes you think that the research at Vanderbilt is suffering when we have a ‘‘best paper’’ at this meeting and a major research grant has just been awarded? Musen: Your researchers have all my respect, but what I am saying is that they are overly busy with, for example, order entry systems. I wish the people in, say, the computer–human interaction community could understand the scientific principles demonstrated by your order entry system and the importance these have outside medicine.

Closing Rebuttal Statements Mark Frisse: What Bill and Warner are saying is a valid position. Based on my personal experience and observations, I would be the first to agree with their dark views about the woeful state of clinical information systems in this country. I would be the first to mention Paul Strassman’s comment that 31 percent of the major software systems in the country will never even be finished, much less deployed.5 In health care, I am rather convinced that many of the systems we are starting to deploy will simply not meet expectations. The problem with Bill and Warner’s argument is that their model is not practical. Their arguments are phrased as if change is only ‘‘top down’’ and the result of a war of conquests. Having run a library and information service organization for five years, I know I cannot do it all with excellence. It takes a team, not a single leader. I know that I can’t run a library ‘‘hands on’’ and do leading edge research. I can certainly advise and challenge; I can certainly create an environment for others to do research;

Journal of the American Medical Informatics Association

Volume 5

and I can certainly be a participant in the research community or I can choose to delegate many administrative tasks. I can do one very well and champion the other, but I can’t be the end-all and be-all for everyone. What Bill and Warner are talking about assumes that someone put in charge of every aspect of information systems can meet all expectations. We should only be so lucky that we would have 50 or 100 of these ‘‘someones’’ who can run a complex service organization, conduct research, create a vision, and effect change— all in an atmosphere where the only constancy is fiscal austerity. But there simply aren’t enough well-trained people to do it all. Clearly, if the leadership of our medical centers believed in the importance of informatics as much as we do, and if we practiced what Bill and Warner preach, we would have an entire generation of health professional technology managers capable of rising to the occasion. And if this were the case, academic medicine would not be in its current troubled state. So overall, their hypothesis is defensible. It is, in my view, as satisfactory a short-term fix for the problem as any other model. But it is merely that—a quick fix for a problem that is the result of organizational dysfunction in a turbulent health care delivery climate. I’m just saying that in the average case, the researcher — the person who is looking for a life that is going to make a contribution ten years down the road — is pursuing a destiny different from the one proposed as the subject for this debate. It is possible to claim that too many service responsibilities can come only at the expense of our long-term investment in intellectual capital. And, despite what has been raised as a lack of results in ‘‘pure’’ informatics research, I believe such results are everywhere. For example, when you use the ‘‘help wizards’’ in some Microsoft products, you are using a Bayesian network architecture composed and produced by Stanfordtrained physician medical informatics researchers who are now in the private sector but maintain ties to their academic medical informatics colleagues. I maintain that it is still possible to do basic medical informatics research. I believe that if you want to make a long-term contribution, despite the funding problems, despite the trials and tribulations, you can do so. This career choice is not for the faint-of-heart, but it has its satisfactions. You do not have to be joined at the hip with a service organization to make it happen, but you must communicate and work effectively with your organization’s service arm. Indeed, I would close by saying that if you cannot work well with your colleagues providing service and sup-

Number 3

May / Jun 1998

303

port, you should not be worrying about these broader issues. Instead, you should think about finding an organization where such communication is the norm rather than the exception. Communication failures between research and service are a recipe for disaster; but in the typical case, so is the complete integration of these two distinctively different forms of work. Warner Slack: First of all, I think the reason that Bill and I didn’t dwell much on basic research is because most of us would agree that basic research belongs in an academic department. It is in the clinical arena where the disagreement occurs, and I feel very strongly that clinical computing belongs in an academic department just as cardiology belongs in an academic department. And just as I would agree that a theoretic geneticist with an academic appointment might not be qualified to do genetic counseling, I would argue that within the academic medical center we would want the qualified genetic counselor to have an academic appointment. To isolate clinical computing from an academic department is, in my view, to eliminate a very important component of the teaching program as well as to compromise the quality of medical care. In conclusion, I would like to congratulate Mark Frisse and Mark Musen for having defended so ably what is basically an indefensible position. Mark Musen: In examining the role of academic units in medical informatics, perhaps we should go back to the Flexner report, which surveyed all academic medicine at the beginning of this century.6 Flexner argued that academic units within medical schools should not simply be administering apprenticeship programs, but instead should be linked to the research performed by scholarly faculty. Flexner believed that the training of clinicians who could think creatively about new problems required medical students to be guided by scientists who made contributions to new knowledge, rather than by practitioners whose role was to aid the institution or to aid patients. In medical informatics, there is an analogous need for academic units where faculty concentrate their efforts on scholarship rather than on practice. When academic units are not encumbered by the dayto-day running of some service organization, the result is an enhanced environment for training — not a deficient environment, as Warner Slack suggested. The training opportunities are better because faculty suddenly have the time to address broad scientific issues. The emphasis moves from the immediate problems of a particular organization to the scientific questions that need to be solved now, next year, and in 30 years. As Tony Gorry has said,2 a central problem in medical informatics is that often our best people are filling service roles, thinking about what the next sys-

304

tem is going to be. Gorry suggests that what the field really needs are scientists thinking about the capabilities of the ‘‘system after next.’’ Academic medical centers need faculty in medical informatics who can be working on that system after next, faculty who are not constrained by the problems of getting today’s already outdated order entry system to work. If there were better scholarship in medical informatics, many of the problems of clinical computing that were raised earlier in this debate would disappear. We would have better clinical computing not because faculty would have the ability to control in some topdown fashion what happens in our academic medical centers; rather, our academic groups would create such good science that the market would drive the vendor community to incorporate our results into their products. As members of medical school faculties, we still would be able to influence the workers who have the decision-making authority regarding computing in our institutions—authority and responsibility that rightfully belongs outside academic circles. We believe that the business of academic units in medical informatics should be education and basic research. It is not necessary to either of these activities that faculty or students implement, administer, and maintain computing systems for the clinical enterprise. If we can develop departments of medical informatics with more intensive, more focused attention to scholarship, then our trainees will have better skills and our trainees will be more sought-after, both in academic centers and in the commercial sector. William Stead: This debate hinges on two issues. The first is the assumption that to have an integrated model, you have to have one person who does it all. That is not what we are proposing. We are talking about an integrated team in which people can fluidly move across roles. Different individuals can focus on each of the important areas while still moving toward a common purpose. The second issue is what responsibility the people in academic informatics should have for ensuring that their institution has excellent operational systems. I do not think we can abdicate the responsibility for achieving a working infrastructure that provides a competitive edge, because in the final analysis that is the only reason for an academic health sciences center to invest in informatics.

FRIEDMAN

ET AL.,

ACMI Debate

If you look at General Motors, you will find that they don’t build cars any more. They design cars and they understand their customers. They have converted themselves into a team of engineers and market analyzers; everything else has been outsourced. Medical informatics is a natural analog to General Motors’ engineering. We’ve got to take responsibility for that component, or we will not have any role in the game.

Conclusion The statements of and interactions among the four participants have left us with much to consider. Had the debate generated a clear winner, or had the views of the participants converged to a consensus compromise position, there would be a clear recommendation from this exercise. This did not occur. Instead, the persistent defensibility of both positions underscored the dilemmas we, as a field, face and also legitimated the different ways we currently organize ourselves to do our work. This debate will prove successful, in the long term, if the debaters’ arguments bring our organizational dilemmas into sharper focus and if, thereby, institutions examining the ways to organize their informatics activities find these arguments helpful to their deliberations. The organizer of the debate (CPF) wishes to thank the participants for their careful preparation and for the articulate presentation and defense of their positions. He also thanks the ACMI Scientific Affairs Committee (Paul Clayton, Bob Greenes, Ed Hammond, and Ted Shortliffe) for its help in generating the theme of the session and phrasing the proposition.

References 䡲 1. Ludmerer KM. Learning to Heal: The Development of American Medical Education. New York: Basic Books, 1985. 2. Gorry GA. Envisioning the system after next. EDUCOM Rev. 1997:30 – 3. 3. Senge PM. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday, 1990. 4. Friedman CP. Where’s the science in medical informatics? J Am Med Inform Assoc. 1995;2:65 – 7. 5. Strassman PA. The Squandered Computer. New Canaan, Conn: Information Economics, 1997. 6. Flexner A. Medical Education in the United States and Canada: A Report to the Carnegie Foundation for the Advancement of Teaching. Boston, Mass: DB Updike, Merrymount, 1910.