to clinicians' workflow. Overuse of ... Other channels such as email are much less invasive .... workflow. Conversely, email has a higher time latency (clinicians read email intermittently), but it is .... Inferencing strategies for automated alerts on.
Optimal Use of Communication Channels in Clinical Event Monitoring William R. Hogan, M.D. Michael M. Wagner, M.D., Ph.D. Center for Biomedical Informatics, University of Pittsburgh Medical Center ABSTRACT We argue that the optimal use of communication channels in clinical event monitors is an important design consideration for these systems. We review the state-of-the-art in selection of communication channels, including our current approach—allowing users to choose the communication channel by which the event monitor sends each notification. We describe a new approach that we are in the process of developing. In this new approach, we view event monitoring as the decision of whether and how to send new patient data to a clinician and apply the principle of maximum expected utility to this decision problem. Our initial experience with this approach suggests that notifying clinicians of normal patient data may be of high utility. We also found that methods for explanation in uncertain reasoning may be necessary in this approach. BACKGROUND A clinical event monitor may use more than one communication channel for sending notifications to clinicians.1 Ideally, the event monitor makes effective use of these channels. Some communication channels such as alphanumeric pagers notify clinicians rapidly but may be disruptive to clinicians’ workflow. Overuse of these channels may hinder instead of help the clinician care for patients. Other channels such as email are much less invasive, but may take longer to notify the clinician.
Thus, such channels may not be suitable for urgent situations. The availability of multiple communication channels in clinical event monitors thus raises questions about how to make optimal use of available channels. In what situations is one channel preferred over another? Why is one channel preferred? What criteria should be used to select among channels? Should the institution set a policy that dictates under what circumstances a particular channel is used? Should the recipients of notifications have a choice? Should the event monitor include in its knowledge base a set of desiderata for choosing the channel at runtime? Existing event monitors described in the literature have only one channel available or send each notification by only one channel (table 1). Our current approach to selection of communication channels differs from these systems in that each notification may go by more than one channel. Presently, our solution to efficient use of communication channels is to allow users to decide the channel by which each individual notification is sent. Limitations with this approach, as we shall discuss, led us to consider alternatives. In this paper, we discuss a new approach in which the event monitor sends each notification by the optimal communication channel (which can be any of the available channels). We discuss a model that embodies this approach (and define optimality) after first describing our current method for selection of communication channels.
Table 1. Communication channels used in clinical event monitors Available channels Method of selection CPR screena N/A Email N/A b CPR screen One channel is specified in the Email action slot of medical logic Fax modules c 5 Critical lab values by both routes, CPR screen Latter Day Saints (LDS) Alphanumeric paging all others by CPR screen onlyc CPR screen Certain conditions to PDAs, all Cedars-Sinai6 Palmtop PDAs others by CPR screen Clinical event monitor Brigham & Women’s2 Beth Israel3 Columbia-Presbyterian4
a
Doctors are paged to “8888” to indicate that there is an alert for them to view. A view alerts option is available in the main menu upon logging into the CPR. c Doctors are sometimes informed of alerts by other health-care personnel who read them on a CPR screen b
USER PREFERENCE Our clinical event monitor (named CLEM) is capable of sending notifications by either 2-way alphanumeric pager or email. CLEM chooses the channel for each notification based on a table of clinicians’ preferences with individual notifications as rows and individual communication channels as columns. Users may change their preferences at any time, using either paper or web-based forms. One limitation of this approach is the effort required on the part of the user. Our experience suggests that, as the system grows, this effort may become greater than users are willing to expend. If new channels and new types of notifications are added to the system, the preference-selection form takes longer to complete. Also, house staff have different preferences depending on the service to which they are assigned (e.g., intensive care unit versus wards). Users may thus have to reconfigure their preferences every time they change services. Users’ preferences may also change over time, so for example, their ward-specific preferences may be different for the first versus the last month of the year. Finally, users have different preferences for their current set of inpatients and the outpatients they see in clinic, suggesting that they may need to maintain two sets of preferences at any given time. Another, more fundamental, drawback is that users may not make optimal use of communication channels. For example, a user may choose email for critically high potassium levels, and as a result not take action as quickly as if the result had been paged. Worse yet, users may subvert the system. For example, a user may have all notifications sent to an email account that she never uses. A NEW APPROACH To address the limitations of our current approach, a new approach should automatically select communication channels without the need for users to maintain a list of their preferences. Also, the channel by which the event monitor sends notifications should be the channel that maximizes benefit to patient care but also minimizes inconvenience to clinicians. In this approach, the event monitor acts as an intelligent agent to select the optimal communication channel for a notification at runtime. In our interpretation of this approach, we view clinical event monitoring as the decision problem of whether and how to send new patient data to the clinician. To implement this approach, we chose decision theory. Despite the fact that current event
monitors are rule based, we decided against a rulebased approach for several reasons. First, the number of rules for each type of data may grow quickly beyond what is maintainable and feasible to encode. For example, the are numerous contexts in which we may want to send a serum potassium level, such as values that are critically high or low, values that are high when the patient is on medications that cause high potassium, values that are low when the patient is on medications that cause low potassium, and values that are low when the patient is on medications that exacerbate the effects of low potassium. We must specify in advance a channel for each context. Addition of a new communication channel then requires manual reevaluation of every context for every type of data to determine whether and when we should send it by the new channel. Another drawback to the rule-based approach is a lack of representation of uncertainty. For example, a serum potassium result can be falsely elevated due to hemolysis or blood-sample clotting in the presence of leukocytosis or thrombocytosis. We therefore chose a decision-theoretic model, which allows us to apply the principle of maximum expected utility. The event monitor chooses the channel that maximizes the expected utility (to the patient) of notifying the clinician of a new datum. For example, by causing the clinician to rectify an abnormal patient condition that might otherwise result in harm to the patient, the event monitor increases expected utility. Conversely, the event monitor may negatively influence expected utility by using a channel that disrupts the clinician’s workflow, potentially causing her to take her attention away from other actions that have greater utility. Figure 1 is an influence diagram that illustrates this model. The decision node Channel may include a do not send option as well as the various channels available to the event monitor. Note that in figure 1 we distinguish between the presence of the patient condition and whether the notification is appropriate. The reason is that even if the condition is present with certainty, it may still be inappropriate to notify the clinician about it. For example, we may not want to notify the clinician of a new smoking cessation program if the patient has terminal lung cancer. The Heeded node represents whether the clinician will take action based on the notification as a result of receiving the notification. Not shown in the influence diagram are variables (lab results, problem list items, medications, etc.) that influence whether patient conditions are present. To enhance the flexibility and ease of maintenance of the decision-theoretic approach, we can model the decision as a choice among
.
.
.
Figure 1. A decision-theoretic model for selecting among communication channels. PtCond = patient condition, PGY = level of training, INvsOUT = inpatient vs. outpatient. types of communication channels rather than the actual, individual channels that are available. We therefore place channels into categories based on their properties. The event monitor must then have available a mapping from the properties of channels to actual channels. The addition of a channel is then straightforward: we add it to the list of available channels and specify its properties. Important properties of channels to model may include time latency and invasiveness. Time latency is the interval from the time the message is sent to the time that the clinician reads the message. Invasiveness describes how disruptive to clinicians’ workflow the channel is. For example, pagers have low time latency, but can disrupt clinicians’ workflow. Conversely, email has a higher time latency (clinicians read email intermittently), but it is less disruptive to workflow. A key issue in decision-theoretic models is knowledge acquisition; that is, the specification of probability distributions for each variable in the network, and utilities for each utility node. Fortunately, many belief-network learning algorithms exist to help alleviate this knowledge-acquisition bottleneck.7 For example, Wagner has previously described how we may automatically determine the probabilities in a decision-theoretic system by learning them from utility feedback.8 Our use of 2way SkyTel pagers potentially allows us to obtain immediate feedback from clinicians about whether the event monitor sent the notification by the proper channel. We currently ask (but do not require) users to reply to many notifications stating whether they were emailworthy, pageworthy, or both. Thus, we
have the capability to obtain immediate feedback for machine learning, but issues such as whether and how often users will respond must be addressed. EXPERIENCE TO DATE We have built two notifications based on a version of the decision-theoretic approach without decision or utility nodes (figures 2 and 3). They remain experimental and are not used to send patient data to users. The belief-network in figure 2 models the decision of whether and how to notify a physician about a new hematocrit result. The discrete values that each node may take are given in the box overlapping the node. Note that the Channel node has three values: email, pager, and do not send. Because the admission ICD-9 code at our institution is not always a true reflection of current patient state, we use it as evidence to infer the probability that the patient actually has a gastrointestinal bleed (GI bleed), dehydration (hypovolemia), or neither. The HctDrop node represents the change in hematocrit level since the last value was performed. We specified the probabilities for the network using our own subjective assessments. At runtime, CLEM instantiates the ICD9Code and HctDrop nodes from patient data and then runs an exact belief-network inference algorithm to compute the probability distribution of the Channel node. CLEM sends the notification by the channel with the highest posterior probability of being appropriate. The process of constructing this belief network according to the decision-theoretic model led us to the insight that notifying clinicians of
normal lab values may be of high utility. Such notifications are contrary to conventional wisdom in event monitoring which tends to ignore context and thus finds only abnormal values interesting. The belief network in figure 2 sends all hematocrits by pager in patients with a GI bleed, regardless of the change since the last value. This behavior seems intuitive—physicians typically follow the hematocrit vigilantly in patients with a GI bleed, thus having the results in a timely manner enables them to react to adverse changes quickly.
GI Bleed Hypovolemia Neither
GI Bleed Hypovolemia Neither
Figure 2. results.
The belief network in figure 3 consolidates at least five rules in CLEM’s KB: high K, high K/Kspring diuretic, high K/K-supplement, low K, and
Less than 4 Between 4 & 6 Greater than 6
Email Pager Do not send
A belief network that distributes hematocrit
The belief network in figure 3 controls the distribution of serum potassium (K) results. The Channel node is the same as the one in the hematocrit network. Note in figure 3 that we model probabilistically the relationship between the true serum K and its measurement (SerumK and SerK_Test nodes) because there is not an exact relationship between the two. The K result may be artificially high, for example, if the blood sample is hemolyzed. In constructing this belief network, we found it more logical to consolidate five rules from our rule-based knowledge base (KB) into one belief network than to construct five separate networks, because the belief network easily models interactions among variables and does not require ad hoc mechanisms to avoid redundant notifications. Current event monitors (including CLEM) avoid redundant notifications by passing explicit state variables among rules. For example, to avoid sending two notifications in a patient with high K who is taking spironolactone (i.e., both high K and high K/K-sparing diuretic notifications), we add a precondition to the antecedent of the high K rule that checks for a state variable asserted by the consequent of the high K/K-sparing diuretic rule. We assign the high K/K-sparing diuretic rule a higher priority so that if it fires, it will suppress the high K rule and CLEM will send one notification.
Figure 3. A belief network that sends serum potassium results. Ksparing, K_Suppl, and Digoxin refer to whether the patient is taking a potassium-sparing diuretic, a potassium supplement, or digoxin, respectively. SerumK = true serum potassium concentration. SerK_Test = measurement of serum potassium. Arrhythmia = likelihood patient will have an arrhythmia without corrective action.
low K/digoxin. The visual representation of knowledge provided by the graphical portion of the belief network should facilitate knowledge acquisition and maintenance by alleviating the need to review multiple knowledge structures for potential variable interactions when adding to, deleting from, or modifying the representation. The belief network approach requires some form of explanation so that we include relevant data with the serum K result in the notification message itself. For example, in a patient with a low K who is also taking digoxin, we want to include both the low K result and the fact that the patient has a digoxin prescription in the notification. One approach to explanation is to use free-text templates with instantiation of variables at runtime (also known in the explanation literature as canned text). Canned text requires either a single message that handles every unique situation or context in which we want to send a result like serum K or, alternatively a separate message for each context. In figure 3, the serum K level might take three values (low, normal, or high), and there are three relevant binary variables (K_Suppl, Ksparing, Digoxin). Thus, there are 24 possible combinations of these variables representing 24 contexts in which we send serum potassium levels. Constructing one template for all 24 contexts is not likely to be feasible and one message per context would require significant resources to create and maintain (for all types of data). Generation of
notifications at runtime is therefore preferable to canned text. Fortunately, belief-network explanation algorithms exist9 and they generate messages that are acceptable to users.10 We plan to experiment with explanation in the next phase of this research. DISCUSSION The optimal use of communication channels is an important problem in clinical event monitoring. We have found that our current approach of eliciting users’ preferences has drawbacks. We have described a decision-theoretic model for addressing the problem of making optimal use of communication channels and our preliminary experience with developing it for use in our event monitor. Our experience has led to new insights about event monitoring—namely, that notifying clinicians of normal patient data may be of high utility. Another insight is that rule-based systems often use multiple rules for notifying clinicians of one patient datum (such as serum K), and a belief-network approach can consolidate such rules into one representation without the need for ad hoc mechanisms to avoid redundant notifications. Two obstacles to implementing the decision-theoretic model include knowledge acquisition and explanation, but these difficulties can be addressed. Future work includes incorporation of an explanation facility, assessment of whether a decision-theoretic approach is generally applicable to all event-monitoring situations, and evaluation of the potential advantages of the approach.
ACKNOWLEDGEMENTS This work was supported by grants T15 LM/DE 07059 and G08 LM06625-01 from the National Library of Medicine.
References 1. Hripsack G, Clayton PD, Jenders RA, Cimino JJ, Johnson SB. Design of a clinical event monitor. Comp Biomed Res. 1996;29:194-221. 2. Kuperman GJ, Teich JM, Bates DW, et al. Detecting alerts, notifying the physician, and offering action items: a comprehensive alerting system. JAMIA Symposium Supplement. 1996:704-8. 3. Rind DM, Safran C, Phillips RS, et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Archives of Internal Medicine. 1994;154:1511-7. 4. Hripcsak G, Cimino JJ, Johnson SB, Clayton PD. The Columbia-Presbyterian Medical Center decision-support system as a model for implementing the Arden Syntax. Proc 15th SCAMC. 1991:248-52. 5. Tate KE, Gardner RM, Scherting K. Nurses, pagers, and patient-specific criteria: three keys to improved critical value reporting. Proc 19th SCAMC. 1995:164-8. 6. Shabot M, LoBue M, Leyerle B, Dubin S. Inferencing strategies for automated alerts on critically abnormal laboratory and blood gas data. Proc 13th SCAMC. 1989:54-57. 7. Buntine W. A guide to the literature on learning probabilistic networks from data. IEEE Trans Knowl Data Eng. 1996;8:1-17. 8. Wagner M. Reminder systems that learn from utility feedback. PhD Thesis, Intelligent Systems Program. Pittsburgh: University of Pittsburgh; 1996. 9. Haddawy P, Jacobson J, Kahn CE, Jr. BANTER: a Bayesian network tutoring shell. Artificial Intelligence in Medicine. 1997;10:177-200. 10. Suermondt HJ, Cooper GF. An evaluation of explanations of probabilistic inference. Comp Biomedical Res 1993;26:242-54.
