This is a title

SAFER Maternity: A Clinical Decision Support System with an Authoring Tool for Clinicians

Yaëlle Chaudy School of Computing University of the West of Scotland [email protected]

Thomas M. Connolly School of Computing University of the West of Scotland [email protected]

Brian Magowan Consultant Obstetrician NHS Borders [email protected] .nhs.uk

Mario Soflano School of Computing University of the West of Scotland [email protected]

Clinical Decision Support (CDS) is a growing field and the technology is increasingly used by both clinicians and patients. In maternity care, numerous guidelines exist on risk assessment and proposed care plans during pregnancy and labour. However, as new evidence arise, these guidelines are subject to change. It is time consuming for clinicians to (i) compile all this information and (ii) keep it up to date. This paper will present our approach to overcome these two issues: the SAFER (Safe Assessment Form to Evaluate Risk) maternity system. This CDS system contains rules extracted from current guidelines on maternity care, they allow care plans to be generated at various stages of gestation based on patient data. The system includes a mobile application and a web interface. The mobile application can be used to visualise care plans and edit patient data, it is available both online and offline with a synchronisation option. The web interface allows clinicians to manage their patients and superclinicians (clinicians with administrator role) to edit the logic used to generate the care plans. That way, the rules can be kept up to date. The system also caters for different sets of rules to be created and used by different health boards since sometimes the guidelines can be interpreted differently. The SAFER rules were initially created and used in an Excel file format. This paper will present the results of a formative evaluation of the CDS system performed with the clinician who developed the initial Excel file and also tested against the Excel file. The evaluation concluded that the SAFER platform was more robust and that its rules engine was able to represent all the previous rules; based on the decision trees for each care plan rule, 3640 unit tests were generated automatically to make sure that the results provided by the system were the ones expected for all possible combinations of patient data. All the tests were passed successfully. This paper concludes that SAFER maternity is a robust platform that can be used successfully by (i) clinicians to enter patient data and generate reliable care plans, (ii) patients to visualise their care plans and (iii) superclinicians to update the system’s rules. Keywords: Clinical Decision Support Systems, Pregnancy, Maternity, Risk assessment, Authoring tool.

1. INTRODUCTION Early identification and management of the risks associated with pregnancy is essential to providing appropriate treatment to pregnant women (National Institute for Health Care Excellence, 2008). In Scotland there are six areas that have been targeted for improved maternal care: post-partum haemorrhage, stillbirth, sepsis, venous thromboembolism, smoking cessation and appropriate induction of labour. Effective identification of maternal risk and its subsequent management is guided by nationally recognised evidence and best practice (e.g. National Institute for Health and Care Excellence NICE, Scottish Intercollegiate Guidelines Network SIGN). However, with so much evidence and practice guidelines available, health care staff can be overwhelmed by © Chaudy, Connolly, Magowan and Soflano. Published by BCS Learning and Development Ltd. Proceedings of BCS Health Scotland Conference

1

bureaucracy and can miss key clinical factors. Risk assessment is frequently carried out in an ad hoc manner, and is not always undertaken. Clinical decision support systems (CDSS) are used in various health areas such as HIV testing (Chadwick et al., 2017), head trauma (Goldberg et al., 2016) and cardiovascular risk factor management (Sperl-Hillen, Crain, Ekstrom, Margolis, & O'Connor, 2016). Two key challenges when creating a maternity CDSS are (i) compiling multiple guidelines from numerous sources into usable rules and (ii) keeping the rules up to date should these guidelines change or new ones emerge. This paper will present the SAFER (Safe Assessment Form to Evaluate Risk) maternity project. SAFER is a risk assessment and risk management system that has been developed and tested by a small team of Scottish

SAFER Maternity: A Clinical Decision Support System with an Authoring Tool for Clinicians Y Chaudy ● T Connolly ● B Magowan ● M Soflano

clinicians in a rural setting. It is used to assess antenatal risks and develop a comprehensive clinical management plan. This project aims to improve early identification and management of risks associated with pregnancy. SAFER maternity is aimed at both clinicians and patients and is to be used throughout pregnancy and labour. This paper will present the previous research related with the project in Section 2. Section 3 will introduce the SAFER maternity project and will detail two of its key elements: a rules editor to amend the rules used to generated care plans and the patient view of questions and care plans. Section 4 will then discuss the results of a preliminary evaluation of the tool summarising the expert formative comments received and explaining our testing strategy. Section 5 will draw conclusions and discuss future directions of our research.

However, challenges to CDSS development, management and use are complex and sociotechnical in nature, involving people, processes and technology (Ash et al., 2015). This complexity has also led to a view that evidence for the role of CDSS in improving safety, quality and efficiency has been mixed. For example, a meta-analysis by Lobach et al. (2012) examined 323 papers for evidence of process or clinical outcomes improvement and/or cost reductions. They found that CDSS can improve process outcomes, including increased preventive services and increased ordering of appropriate medical treatment. They found moderate evidence that CDSS improves the ordering and completion of appropriate clinical studies and can decrease morbidity. However, there was less evidence that CDSS lowers mortality, costs or adverse events, although they did note that there were fewer studies in this area. Nuckols et al. (2014) examined 16 pre-post and quasi-experimental studies of computerized physician order entry systems with and without CDSS and found no differences in the incidence of preventable adverse medication events. Jeffery, Iserman, and Haynes (2013) found that CDSS in diabetes management may marginally improve clinical outcomes, but felt that confidence in the evidence was low because of risk of bias, inconsistency and imprecision. Bayoumi et al. (2014) performed a metaanalysis with 36 studies to evaluate the effectiveness of computerised drug-lab alerts to improve medicationrelated outcomes and found no reductions in adverse drug events or lengths of stay in hospital although there was an increased likelihood of prescribing changes in accordance with the alerts.

2. PREVIOUS RESEARCH Healthcare is a heterogeneous environment consisting of different types of information systems, such as electronic health record (EHR) systems, picture archiving and communication systems (PACS), laboratory information systems (LIS) and clinical decision support systems (CDSS) (Loya, Kawamoto, Chatwin, & Huser, 2014). In this paper we focus on CDSSs, which provide clinicians with actionable, patient-specific recommendations or guidelines for care at the point-of-care to enhance patients’ health (Moja et al., 2016). CDSS has been broadly defined as software that provides “clinicians, patients or individuals with knowledge and person-specific or population information, intelligently filtered or presented at appropriate times, to foster better health processes, better individual patient care, and better population health” (Osheroff et al., 2007). Some studies investigating the effectiveness of CDSSs demonstrate their potential to assist with issues raised in clinical practice, decrease the rate of medication errors, increase clinicians’ adherence to guideline- or protocolbased care and improve the overall efficiency and quality of healthcare delivery systems (Bright et al., 2012; Chaudhry et al., 2006; Holbrook et al., 2009; Sequist et al., 2005). For example, in a systematic review of 100 studies, Garg et al. (2005) found that CDSSs improved practitioner performance in 64% of the studies assessing this outcome, including 4 of 10 diagnostic systems, 16 of 21 reminder systems, 23 of 37 disease management systems, and 19 of 29 drug dosing or prescribing systems. However, only 13% of studies improved patient outcomes.

Furthermore, it has been found that simply implementing a CDSS is not sufficient to ensure its use. Several studies highlight that certain forms of CDSS are turned off or ignored (Shah et al., 2006). Poor adoption and use has been attributed to a number of barriers, including but not limited to poor integration of CDSS into clinical workflow, lack of technical support to address hardware and software issues, limited training of clinicians on how to use and apply CDSS when making decisions, context-insensitive alerts and reminders, distrust of clinical guideline-based CDSSs as quality improvement tools, and an overwhelming number of prompts requiring physician attention (“alert fatigue”) (Kesselheim, Cresswell, Phansalkar, Bates, & Sheikh, 2011; Moxey et al., 2010). In a meta-synthesis study by Miller et al. (2015), 56 qualitative studies involving physicians and registered/advanced practice nurses’ experience of CDSS use in clinical practice were reviewed against a set of quality criteria based around four themes: credibility, transferability, dependability and confirmability, broadly analogous to internal validity, external validity, reliability and concurrent validity, respectively. Nine studies were of sufficiently high quality for synthetic analysis. Five major themes were identified from the quality studies: user interface usability issues, clinician-patient-system integration, the need for better ‘algorithms’, system maturity and patient safety. Under usability, three subthemes were

A study by Kawamoto, Houlihan, Balas, & Lobach (2005) found that CDSSs significantly improved clinical practice in 68% of trials and five system features were significantly more likely to improve clinical practice than interventions lacking the feature: automatic provision of decision support as part of clinician workflow, provision of recommendations rather than just assessments, provision of decision support at the time and location of decision making and computer based decision support. 2


identified: (i) nuisance issues (such as repetitive or redundant alerts or inappropriate alert presentation), (ii) user control and inefficiency issues (such as users’ inability to actively manage alerts) and (iii) data entry associated with alerts was found to be rigid and inflexible. Examples of clinician-system integration issues included reliance on clinicians to integrate CDSS and information across different Electronic Health Record (EHR) modules, such as medication prescription and laboratory test ordering and CDSS requirements for immediate action despite the need for further information or consultation. CDSS algorithms were found to be lacking in maturity; for example, algorithms may not fully accommodate a work task or problem or may not accommodate more complex patient scenarios. System immaturity issues included interoperability issues (e.g., information flow between infusion pumps and EHRs and between different departments or healthcare agencies), system response times, the requirements for multiple logins and general system crashes. Patient safety issues included threats or perceived threats to patient safety, for example, junior clinicians were thought to inappropriately accept CDSS recommendations without critical evaluation.

(hydroxychloroquine before pregnancy, aspirin during pregnancy and azathioprine before/during pregnancy). However, no evaluation of the system has been provided. García-Sáez et al. (2014) discuss the development of the MobiGuide CDSS for gestational diabetes patients based on computerised clinical guidelines and adapted to a mobile environment. The system supports four types of personalized advice delivered through a mobile app at home: (i) therapy, to help patients to comply with medical prescriptions; (ii) monitoring, to help patients to comply with monitoring instructions; (iii) clinical assessment, to inform patients about their health conditions; and (iv) upcoming events, to deal with patients’ personal context or special events. The system also supports different types of reminders that patients can configure to help comply with monitoring instructions (eg, blood glucose monitoring) or administration of insulin. Again, no evaluation of the system has been provided. Horner et al. (2013) discuss the evaluation of a CDSS named Bacis (Basic Antenatal Care Information System) that was developed to help health workers comply with the maternity guidelines at primary health care clinics in South Africa. Bacis uses patient data entered during an antenatal visit to perform risk classification, identification of patients for referral and scheduling of maternity care interventions. The evaluation consisted of a control group of patients (n=25) who were treated before the introduction of Bacis and an experimental group of patients (n=100) who were treated after the introduction of Bacis. The method of assessing compliance was a record review. The review measured compliance to the guidelines for 18 antenatal protocol items such as: fetal heart rate, severe symptoms (eg. severe headache, abdominal pain or discomfort, reduced fetal movement, abdominal bleeding), pregnancy infections, blood pressure, proteinuria, maternal weight and height, haemoglobin screening and prophylaxis (calcium, iron and folic acid). There was an improvement from 85.1% to 89.3% in the compliance of nurses when using the CDSS, although this was not statistically significant, although compliance for specific groups of patients did show statistically significant improvement: compliance at booking, patients younger than 18 years and patients booking after week 20.

CDSS in Maternity Care While there is significant research on CDSSs generally, there is not an abundance of empirical evidence on the use of CDSS in maternity care. Sukums et al. (2015) evaluate the QUALMAT (Quality of Maternal Care) CDSS for maternal care in rural primary health care in Tanzania and Ghana. The evaluation consisted of a questionnaire and semi-structured interviews with 61 and 56 participants at the midterm and final assessment point, respectively. The system was used in 2703 out of 3798 (71%) antenatal care cases and 14,189 out of 24,204 pregnancies (59%) in Tanzania and Ghana respectively, while it was also used in 1185 out of 1427 (83%) and 1435 out of 2144 (67%) of all deliveries in Tanzania and in Ghana, respectively. It was found that training and regular support were important success factors whereas barriers were unreliable power supply and perceived high workload, although the researchers believed that the barriers did not substantially hinder adoption and utilization of the system during patient care. Paydar, Kalhori, Akbarian, and Sheikhtaheri (2017) discuss the development of a CDSS to predict pregnancy outcomes for systemic lupus erythematosus affected women, a condition which is highly associated with poor obstetric outcomes. Based on the values for 16 factors the CDSS predicts the outcome of the pregnancy as either spontaneous abortion or live birth. The factors are divided into three areas: (i) lab tests (complement C3 before/during pregnancy, platelets before/during pregnancy, proteinuria before/during pregnancy, C-reactive protein before pregnancy, haematuria during pregnancy and anti-ds DNA and anticardiolipin antibodies IgG in the first trimester), (ii) problems (anti-phospholipid antibody syndrome, anaemia and leucopoenia before pregnancy and flare up of lupus six months before pregnancy) and (iii) drugs

Bakibinga et al. (2017) discuss the development of a CDSS for community health volunteers (CHVs) to identify and track mothers and children in a bid to reduce pregnancy-related complications and newborn deaths in the urban slums of Kamukunji subcounty in Nairobi, Kenya. The intention is that CHVs will be provided with a mobile phone running a CDSS app for data collection with data being relayed to a central server. When visiting households, the app will automatically find the patient’s record using the geolocation of the household. The CHV will complete the appropriate health form on the app and submit the completed form to the central server over the Internet. Where the app detects that Internet connectivity is not available, the completed forms will be stored on the 3


device and only be relayed to the server when the connectivity is established. The app is preconfigured with danger signs for both mothers and newborns as defined by the World Health Organisation 1. When any danger sign is identified by the system, the CHV is immediately prompted to refer the patient for further investigation and the CHV will not be allowed to proceed until the referral has been made. If the patient has not been seen within 24 hours, an SMS is sent to the respective CHV and the CHV should follow up with the patient to check their condition. However, no evaluation of the system is provided in the paper.

synchronise when possible. The project can be used by three different types of users:

3. OUR APPROACH: THE SAFER MATERNITY CDSS

Additionally, there is an Administrator role allowing a user to perform more technical tasks such as modifying the source code for the rules engine, running tests or editing the list of health boards. An admin user can also create clinicians and superclinicians.

 



As discussed in Section 1, early risk assessment and management is essential during pregnancy. The two main challenges when providing pregnancy advice on care plans are i) the need to compile guidelines from various different sources to make informed decisions and ii) the difficulty in keeping these guidelines up to date and sharing them. To address these issues, the SAFER (Safe Assessment Form to Evaluate Risk) project was created. The aim of the SAFER project is to provide a risk assessment and risk management system for pregnant women and to engage them at the earliest stage of their pregnancy in the identification and management of pregnancy associated risk. It was originally created as a suite of calculators for maternal risk assessment, held in Excel spreadsheets. In its original format, it has been evaluated by Scottish clinicians in a rural setting. The SAFER form has reduced the use of up to eight risk assessment forms into one and preliminary analysis has shown an improvement in the assessment of venous thromboembolism, foetal growth and post-partum haemorrhage. This paper presents how the original Excel spreadsheet was converted into a web and mobile app solution. The architecture of the SAFER CDSS is shown in Figure 1. The project can be decomposed into three main components:  



Clinicians: They can view and update patients’ details and care management plans for patients in their own health board. Superclinicians: They can view and update patients’ details and care management plans for patients in their own health board. They can also create and modify the SAFER rules (list of questions and rules for generating care plans). Patients: They can only view their own care management plan.

Figure 1: The SAFER software architecture

This section will first explain how rules are modified and detail the SAFER rules editor in Section 3.1. Then, Section 3.2 will present the patient view showing how questions are displayed and care plans generated. The following subsections will use the diabetes screening as an example for illustration purposes. The decision tree for this plan, extracted from the Excel formula, is shown in Figure 2.

User management: Storage and retrieval of clinical staff and patient details. Rules management: Storage and retrieval of the questions to display in the patient form and the rules for generating care management plans based on the answers to these questions. Care plans management: Storage and retrieval of patient data (answers to questions) and generated care plans.

All three components are connected to a global database in the SAFER server. Both clinicians’ and patients’ mobile apps also have their own local databases allowing the system to work offline and

Figure 2: Rules for screening for diabetes

1

www.who.int/maternal_child_adolescent/news/events/.../ Mother_and_Baby_card.pdf 4


 

3.1. Rules editor The rules editor is only available in the web interface, its complexity made it too difficult to adapt to a mobile version. It allows a superclinician to edit the logic behind the generation of care plans for pregnant patients. This can mean modifying the list of questions needed as input and/or changing the way care plans are generated based on the answers. These two tasks will be presented here.



A superclinician can add other questions. These new questions can be of two types: numbers (whole or floats) or enums (string selected from a list of possible values). A number can have a range (minimum and maximum value) and an increment (number of decimals). An enum have a list of possible values and can have a default value for prefilling the patient questionnaire. A superclinician can also specify a helptext to give more information on the expected answer. There is no restriction in number, at the time of writing, the latest version of the rules contains 53 questions. Questions are grouped in sections such as “Booking appointment”, “Antenatal”, “Labour Ward”. Sections can also be created and deleted. Figure 3 shows some of the questions in the “Booking appointment” section.

3.1.1. Patient information There are eight predefined questions. They were defined by our consultant and cannot be modified or deleted:     

EDD (Estimated Due Date): date Gestation in days: number, calculated from the EDD Gestation: string (number of weeks and days).

CHI (Community Health Index): number Age: number, calculated from the CHI Height: number in meters Weight: number in kilograms BMI (Body Mass Index): number, calculated from height and weight

Figure 3: SAFER rules editor - Questions tab



3.1.2. Care plans generation From a patient’s answers to the questions discussed previously, SAFER generates a number of care plans for the patient. Care plans can be created, deleted and modified and are composed of:  



A name (e.g. “Screening for diabetes”) Intermediate calculations in case some answer need to be aggregated with AND or OR before being used (e.g. when the rules look for one risk factor from a list)



5

The rules as a decision tree with a test condition and rules to follow if the condition is true and false (e.g. if the patient has diabetes no need to screen, otherwise, look for risk factors if she has any, screen her early, otherwise routine screening) A list of evidence links, guidelines justifying the logic behind the care plan’s generation A list of information links, for the patient use.


3.2. Patient view Once a set of rules has been created and published, it can be used in the patient view to generate her care plans. Patients can be created by clinicians and superclinicians, once created they are visible in the health board’s list of patients and new data can be entered. An email is also automatically send to the patient explaining the SAFER system and providing an invitation link to use should that patient wish to register for a patient account and visualise her care plans. The patient view is available in both the web interface and the mobile application for clinicians and patients. It is composed of: 



The set of rules to use: a drop down allows the clinician to select from a list of published rules. Each option specify the version, who created the rules and when. Only clinicians can change its value, patients do not see it. Patient information: Some basic information (CHI, name, EDD and gestation) is always visible and the rest of the sections are shown as tabs. Figure 5 shows Mary Smith’s data with the “24 weeks and After” section open. Clinicians can update all the information displayed. Patients have only a read only access; they can visualise their record but cannot alter it.

Figure 4: SAFER rules editor - Screening for diabetes care plan tab

Figure 4 presents the “Screening for diabetes” care plan view in the editor. The latest version of the rules have 13 care plans: Folic Acid, Aspirin, Screening for diabetes, Smoking cessation, Iron treatment, Growth assessment, Antenatal Heparin Prophylaxis, Inpatient Heparin Prophylaxis, Postnatal Heparin Prophylaxis, PPH prevention, Pyrexia, Intrapartum Group B strep Rx and Bladder after delivery. Superclinicians can save their changes as draft or published rules. A draft can be modified at a later stage and is for the superclinician use only. A published set of rules can be seen by all clinicians from the same health board and can be used for patients when generating care plans. Once published, rules can be used to create new ones (copy and edit) but they cannot be modified. A third status exists for rules: outdated. A superclinician can specify that a new set of rules should be used by publishing one and marking the previous one as outdated.

Figure 5: SAFER patient view – questions 

6

Management plans: All the care plans specified in the rules are shown here as accordion panes. Default plans are shown in blue and risk in red for quick identification of


action needed. For more information on the care plan, a user (clinician or patient) can click on the accordion pane. Inside, it displays the detail of the intermediate calculation used; every risk factor is listed and a ticked/unticked box specifies whether the statement is true for the patient. Evidence and information links are also shown. Figure 6 shows some of the care plans for Mary Smith who has a BMI over 30 and a history of gestational diabetes. The “Screening for diabetes” pane is open.

The SAFER rules editor  The editor was found easier to use than Excel formulas.  The editor was made easier to use for nontechnical users: o The initial wording of the rules “if, then, else” was changed to “test, yes, no”. o The care plans were made editable in the decision tree view by clicking on a node. o The initial “deprecated” status was changed to “outdated”.  Initially, sections, questions and care plans were added at the end of the list when created. After review, they can now be ordered by drag and dropping them to the desired place. User management  Create new pregnancy records: Because we keep records of patients and their pregnancy data, we needed to add an option for clinicians to start a new care plan for a new pregnancy without deleting or overwriting the patient information.  An issue was identified and fixed: when a patient was moved to another health board the rules were not updated.  The patient and clinician list tables were simplified (unnecessary fields were hidden). Patient view  Risk care plan should be highlighted: risk plans are now specified in the editor and shown in a red background in the patient view.  The section “Basic information” is now displayed on top of the other questions and always shown.  The question “Gestation in days” is now hidden but still available to use in the editor  Because some plans are associated with others, they should not be displayed as two separate care plans: plans can now be linked in the editor and are displayed together in the patient view as a result (e.g. “Heparin prophylaxis recommendation” and “Heparin prophylaxis dose”).

Figure 6: SAFER patient view - care plans

4. EVALUATION AND RESULTS At the time of writing, the SAFER maternity CDSS has been approved for use in NHS Borders at an information governance meeting. However, it hasn’t been deployed. This section will present the preliminary results from a formative evaluation of the tool in term of usability and usefulness in Section 4.1 and the result of extensive unit testing confirming the accuracy of the rules engine in Section 4.2.

4.2. Quantitative evaluation: Testing the new CDSS against the previous Excel solution In CDSS, accuracy and reliability are key aspects of any evaluation. We performed extensive testing of the SAFER CDSS. First, automated tests were run to answer the question: “Can all the rules of the initial Excel solution be reliably represented in the CDSS?” the results will be discussed in Section 4.2.1. Second, the two solutions were compared in Section 4.2.2 to answer: “What are the advantages of using the new CDSS compared to the previous Excel one?”.

4.1. Formative evaluation: Usability and usefulness of the tool The SAFER maternity project was initially introduced to the consultant obstetrician and gynaecologist, creator of the Excel solution. His formative feedback was taken into account and changes were made to the system as a result. Conclusions from the expert review and a summary of the changes include:

7


4.2.1. Automated tests For each one of the 13 care plans present in the Excel file, a decision tree was created. Based on the decision trees, unit tests were generated automatically to verify that all the results provided by the system were the ones expected. The tests were designed to generate every possible combination of patient data leading to each possible end values of the care plan. For example, the “screening for diabetes” care plan has three possible end values:  



developed and integrated into the interface. With this tool, users can specify their expected plan and possible values for patient data by editing a test configuration file. Figure 7 shows the configuration file for the “Screening for diabetes” plan. Tests can then be run dynamically and the result is presented to the user as green or red text as shown in Figure 8.

“Has diabetes, Screening not required” o Diabetes this pregnancy = Yes “HbA1C at booking and an OGTT at 24-28 weeks” o Seven tests generated: one for each risk condition: BMI over 30, Asian, black Caribbean, Middle Eastern, family history of diabetes, previous baby over 4.5Kg and previous gestational diabetes. “Fasting blood glucose at 28 weeks” o 32 tests generated: for every combination of non-risk factors (e.g. family origin could be ‘Caucasian’ or ‘Other’, family history of diabetes could be ‘No’ or ‘Don’t know’ etc.)

A total of 3,640 tests were ran, and all passed successfully confirming that the SAFER CDSS is indeed able to represent all the existing rules.

Figure 8: Screening for diabetes tests run

4.2.2. Differences inherent to the format used in both solutions Due to the format used for both the original and the new solutions, a number of differences can be outlined. The new CDSS offers various advantages over the previous Excel file. First, it allows patient data and a history of patient care plans to be stored securely. The Excel file was previously stored on the clinicians’ devices and patient data was overwritten after every change, no history was available and it was not possible to visualise the care plans advised at an earlier stage. Having a global repository also allows for data mining to be performed making it easier to evaluate the effect of the system. Moreover, it allows patients to visualise their information on their own device and in real time. It avoids spelling errors and some mapping mistakes. In the previous version, some inconsistencies were found. As the Excel file evolved significantly over time, some questions were modified. Co-morbidities for example was created as a yes/no question but later, the ‘Yes’ was replaced to detail the co-morbidities in question. However, a field was still checking for “comorbidities = Yes” which was, as a result, always false. A field was also checking for a ‘yes’ answer (all lower case) instead of ‘Yes’. Finally, a cell was labelled “multiple pregnancy” but was checking another question due to a mapping issue (cell D19

Figure 7: Tests configuration - screening for diabetes

Since rules can be modified at a later stage by superclinicians using the editor, a testing suite was

8


6. REFERENCES

was checked instead of B19). These were errors that were very difficult to identify due to the complexity of the spreadsheet and the lack of automated testing tools. The rules editor uses question names in rules so mapping issues are avoided and dropdown for enums values so spelling errors cannot be made.

Ash, J. S., Sittig, D. F., McMullen, C. K., Wright, A., Bunce, A., Mohan, V., . . . Middleton, B. (2015). Multiple perspectives on clinical decision support: a qualitative study of fifteen clinical and vendor organizations. BMC medical informatics and decision making, 15(1), 35. Bakibinga, P., Kamande, E., Omuya, M., Ziraba, A. K., & Kyobutungi, C. (2017). The role of a decision-support smartphone application in enhancing community health volunteers’ effectiveness to improve maternal and newborn outcomes in Nairobi, Kenya: quasi-experimental research protocol. BMJ open, 7(7), e014896. Bayoumi, I., Al Balas, M., Handler, S. M., Dolovich, L., Hutchison, B., & Holbrook, A. (2014). The effectiveness of computerized drug-lab alerts: a systematic review and meta-analysis. International journal of medical informatics, 83(6), 406-415. Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., . . . Musty, M. D. (2012). Effect of clinical decision-support systemsa systematic review. Annals of internal medicine, 157(1), 29-43. Chadwick, D., Hall, C., Rae, C., Rayment, M., Branch, M., Littlewood, J., & Sullivan, A. (2017). A feasibility study for a clinical decision support system prompting HIV testing. HIV medicine, 18(6), 435-439. Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., . . . Shekelle, P. G. (2006). Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of internal medicine, 144(10), 742-752. García-Sáez, G., Rigla, M., Martínez-Sarriegui, I., Shalom, E., Peleg, M., Broens, T., . . . Hernando, M. E. (2014). Patient-oriented computerized clinical guidelines for mobile decision support in gestational diabetes. Journal of diabetes science and technology, 8(2), 238-246. Garg, A. X., Adhikari, N. K., McDonald, H., RosasArellano, M. P., Devereaux, P., Beyene, J., . . . Haynes, R. B. (2005). Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. Jama, 293(10), 1223-1238. Goldberg, H. S., Paterno, M. D., Grundmeier, R. W., Rocha, B. H., Hoffman, J. M., Tham, E., . . . Deakyne, S. J. (2016). Use of a remote clinical decision support service for a multicenter trial to implement prediction rules for children with minor blunt head trauma. International journal of medical informatics, 87, 101-110. Holbrook, A., Thabane, L., Keshavjee, K., Dolovich, L., Bernstein, B., Chan, D., . . . Investigators, C. I. (2009). Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial.

Finally, when a new version of the spreadsheet was created, it had to be sent to and downloaded by everyone using the system. Patient care plans created with an older version of the spreadsheet were either not moved onto the new one or all the patient data needed to be re-entered. With the new system, version management is as easy as changing a value in a dropdown box. Patients can be moved to a new version or back to an older one seamlessly. The overall conclusion of the evaluation of the new CDSS is that SAFER maternity is robust and reliable. It offers various technical advantages over the previous solution and can be easier to use for clinicians and patients. 5. CONCLUSION This paper presented a CDSS: SAFER maternity. This project aims to improve early identification and management of risks associated with pregnancy. SAFER maternity is aimed at both clinicians and patients and is to be used throughout pregnancy and labour. The system also caters for changes in the pregnancy care guidelines. This paper introduced a rules editor that can be used by clinicians with special privileges to amend the rules used to generate care plans. It also presented the patient view of questions and care plans. This paper presented our preliminary evaluation of the tool and concluded that SAFER maternity platform can be used successfully by clinicians to enter patient data and generate reliable care plans, patients to visualise their care plans and superclinicians to update the system’s rules. The system was found capable of replicating all the rules present in the previous Excel solution, while being more robust and easier to use. Moreover, it allows patient data and a history of care plans to be stored securely. Further work will include piloting the tool for usability and usefulness evaluation as well as collecting data for a quantitative evaluation of its effect on pregnancy and positive outcome improvement in crucial areas such as post-partum haemorrhage, stillbirth, sepsis, venous thromboembolism, smoking cessation and appropriate induction of labour. Further development of the tool includes improving the test suite by providing a visual interface to replace the test configuration files, which are not easy to use for non-technical users.

9


Canadian Medical Association Journal, 181(1-2), 37-44. Horner, V., Rautenbach, P., Mbananga, N., Mashamba, T., & Kwinda, H. (2013). An e-health decision support system for improving compliance of health workers to the maternity care protocols in South Africa. Applied clinical informatics, 4(1), 25. Jeffery, R., Iserman, E., & Haynes, R. (2013). Can computerized clinical decision support systems improve diabetes management? A systematic review and meta‐analysis. Diabetic Medicine, 30(6), 739-745. Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005). Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. Bmj, 330(7494), 765. Kesselheim, A. S., Cresswell, K., Phansalkar, S., Bates, D. W., & Sheikh, A. (2011). Clinical decision support systems could be modified to reduce ‘alert fatigue’while still minimizing the risk of litigation. Health Affairs, 30(12), 2310-2317. Lobach, D., Sanders, G. D., Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., . . . Hasselblad, V. (2012). Enabling health care decisionmaking through clinical decision support and knowledge management. Evid Rep Technol Assess (Full Rep), 203(203), 1Y784. Loya, S. R., Kawamoto, K., Chatwin, C., & Huser, V. (2014). Service oriented architecture for clinical decision support: a systematic review and future directions. Journal of medical systems, 38(12), 140. Miller, A., Moon, B., Anders, S., Walden, R., Brown, S., & Montella, D. (2015). Integrating computerized clinical decision support systems into clinical work: a meta-synthesis of qualitative research. International journal of medical informatics, 84(12), 1009-1018. Moja, L., Friz, H. P., Capobussi, M., Kwag, K., Banzi, R., Ruggiero, F., . . . Nyberg, P. (2016). Implementing an evidence-based computerized decision support system to improve patient care in a general hospital: the CODES study protocol for a randomized controlled trial. Implementation Science, 11(1), 89. Moxey, A., Robertson, J., Newby, D., Hains, I., Williamson, M., & Pearson, S.-A. (2010). Computerized clinical decision support for prescribing: provision does not guarantee uptake. Journal of the American Medical Informatics Association, 17(1), 25-33. National Institute for Health Care Excellence, N. (2008). Antenatal care for uncomplicated pregnancies. NICE clinical guidelines. Updated edition. London. Nuckols, T. K., Smith-Spangler, C., Morton, S. C., Asch, S. M., Patel, V. M., Anderson, L. J., . . . Shekelle, P. G. (2014). The effectiveness of computerized order entry at reducing

preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Systematic reviews, 3(1), 56. Osheroff, J. A., Teich, J. M., Middleton, B., Steen, E. B., Wright, A., & Detmer, D. E. (2007). A roadmap for national action on clinical decision support. Journal of the American Medical Informatics Association, 14(2), 141-145. Paydar, K., Kalhori, S. R. N., Akbarian, M., & Sheikhtaheri, A. (2017). A clinical decision support system for prediction of pregnancy outcome in pregnant women with systemic lupus erythematosus. International journal of medical informatics, 97, 239-246. Sequist, T. D., Gandhi, T. K., Karson, A. S., Fiskio, J. M., Bugbee, D., Sperling, M., . . . Bates, D. W. (2005). A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. Journal of the American Medical Informatics Association, 12(4), 431-437. Shah, N. R., Seger, A. C., Seger, D. L., Fiskio, J. M., Kuperman, G. J., Blumenfeld, B., . . . Gandhi, T. K. (2006). Improving acceptance of computerized prescribing alerts in ambulatory care. Journal of the American Medical Informatics Association, 13(1), 5-11. Sperl-Hillen, J. M., Crain, A., Ekstrom, H. L., Margolis, K. L., & O'Connor, P. J. (2016). A Clinical Decision Support System Promotes Shared Decision-Making and Cardiovascular Risk Factor Management. Journal of PatientCentered Research and Reviews, 3(3), 218. Sukums, F., Mensah, N., Mpembeni, R., Massawe, S., Duysburgh, E., Williams, A., . . . Blank, A. (2015). Promising adoption of an electronic clinical decision support system for antenatal and intrapartum care in rural primary healthcare facilities in sub-Saharan Africa: The QUALMAT experience. International journal of medical informatics, 84(9), 647-657.

10