How to make decisions with algorithms - orbit rri

______________________________________________________ _________

How to make decisions with algorithms Ethical decision-making using algorithms within predictive analytics Persson, Anders University of California.

Kavathatzopoulos, Iordanis Uppsala University.

Corresponding Author: Anders Persson, [email protected]

Abstract The use of automated decision-making support, such as algorithms within predictive analytics, will inevitably be more and more relevant, and affecting society. Sometimes it is good, and sometimes there seems to be negative effect, such as with discrimination. The solution focused on in this paper is how humans and algorithms, or ICT, could interact within ethical decision-making. What predictive analytics can produce is, arguably, mostly implicit knowledge, so what a human decision-maker could, possibly, help with is the explicit thought processes. This could be one way to conceptualize the interactive effect between humans and algorithms that could be fruitful. Presently there does not seem to be very much research regarding predictive analytics and ethical decisions, concerning this human-algorithm interaction. Rather it is often a focus on pure technological solutions, or with laws and regulation. Keywords: predictive analytics, algorithms, ethical decision-making, implicit bias, critical thinking, automation, fourth paradigm;

Introduction Who will make decisions that affect us in society, and how should these decisions be made? Technology, IT, robots and algorithms are increasingly used when making decisions that affect environment, society and individuals. There is a very good way that technology, math, and statistics can help people to make better decisions, where humans show limitations. A calculator will give you the square-root of any number in an instant, while it would take much longer for most humans. The motivation from organizations and institutions that are doing data analysis of large data quantities, so called Big Data, is often similar; to find better knowledge and to be more

accurate than a human could looking at the same amount of data. However, there is a risk that technology enhances our human short-comings, like having pre-conceptions and generalizations about groups of people enhanced, which will have ethical implications. For example, within recruitments today you can be up to a thousand applicants to one position, and there is an efficient way that you can train algorithms into looking for specific abilities and properties of applicants. It might be necessary with some automation, just to be able to handle the sheer amount of data; it might demand a, so called, Fourth Paradigm of scientific discovery. But there are some ethical risks involved with this, like companies and public institutions that are using credit scores as a measure for candidate’s success. The intuition is that those with poor ratings lack responsibility and will perform worse on a job (O’Neil, 2016). But there are a lot of reasons why you might have a bad score, and especially in the United States it is related to having experienced accidents and gaining large (unpaid) hospital bills. For individuals having an accident, in other words, the punishment is doubling up by making it harder to get a job. There is also the risk that you will lose your job you already have. Washington State included a mathematical model for assessing their teacher workforce, by looking at how well their students performed1. Despite excellent personal recommendations from colleagues and their own bosses, all the lowest-scoring teachers got fired. These are some examples of how the use of predictive analytics and machine-learning algorithms affect people in society, when relying on them too heavily. The motivation to use computers and data analysis to improve decision-making is in itself sound and reasonable. Colleagues, managers, and recruiters alike, are after all only human, and can make a positive review, and judgment of someone based on his own personal preferences, and they can be affected enough to give a bad review, simply from they themselves having a bad day. Tht is surely not a fair and objective assessment. If we could avoid human shortcomings like these, and to harness some cold hard objectivity from data and mathematics, that could be good thing. The main problem with the examples mentioned above is, arguably, not that there are negative effects for people, it is rather that it is seen as discriminatory, unfair, or simply not based on reasonable and sound knowledge. It is also because of the fact that the decision-maker simply seems to have been fooled, like by that insufficient mathematical model assessing teachers, or that recruiter giving a candidate a bad review simply because he had a bad day. If a decision would have been critically reflected upon, evaluated, analyzed, with consequences taken into account, and weighted against other alternatives, and still ending up affecting us negatively, it would probably be easier for us to accept. Also, if alternatives and their consequences has been taken into account, been reasoned, and reflected upon, then we could hold that decision-maker more accountable. Hence, part of the problem with “automated” decision-makers in the form of machines and algorithms, is that we want some accountability, as has been highlighted in the literature before (Gumbus and Grodzinsky, 2016). One possible solution to this, that will be discussed briefly, is to use laws and regulation to restrict unethical use, and to place accountability where it is deemed to belong. The investigation in this paper is rather focused on how decisions-makers can be supported, and what they need, to make critically analyzed, reasoned, and well informed decisions. How do we make ethically, and critically, sound decisions? Is it so, that if the process that is investigating the problem at hand,

resulting in the best available knowledge and alternatives to make a judgment from, then the conclusion will be a satisfactory ethical decision? To what extent can this process benefit from Information and Communication Technology (ICT), as well as algorithms and predictive analyses, and when is it constrained?

What Algorithms and Predictive Analyses Do An algorithm is generally defined as something like “a limited sequence of actions that is performed”. It could be simple calculations, or more complex forms of automated reasoning. Predictive analytics is very much related to data mining, data profiling, and so called Big Data; to analyze large data sets to find correlations and statistical patterns. It is also related to machine learning, where a computer can learn from data input, without being programmed. This is where algorithms often come in, in the context of predictive analytics and predictive modelling; the end result of machine learning software, after being trained on data sets, is an algorithm that tries to predict the outcome of new data sets as input. Another way of putting it is that algorithms are based on pattern recognition, and is looking for these patterns in the new data. Based on correlations in the pattern from the training process, it attempts to predict what kind of values is most likely associated to the new data. In the case of applicant screening, it usually attempts to predict who of the candidates are most likely to succeed, based on certain values to look for. There is strength in algorithms that it can siphon through large amounts of data, and to be able to estimate, with a probabilistic certainty, about future outcomes. Recruiters that got 100, or 500, or 1000 applications and CVs for a single position to hire, were quickly swamped and overwhelmed of information, and it is difficult dealing with that in any constructive fashion without a lot of effort and research. An unconstructive way would, for example, be to simply throw 50% of em in the trash and read the remaining. Still, just to keep everything in the head when weighing different aspects, of a large amount of candidates, against each other, would be taxing for a human recruiter. This is of course something an algorithm can do better and help with, and it is understandable why it is popular, and almost mandatory, for bigger agencies and companies, to have some kind of computerized Applicant Tracking System (Rosenblat et al., 2014), to electronically handle information; often a web-based CV-uploading service, followed by a score, based on a match of certain values and attributes, like academic degree for example.

Limitations and risk with algorithms Three concepts to illustrate some limitations of predictive analytics and algorithms, and their use, are; the nature of probabilistic estimations on humans, a distinction with science, and a likeness to apprehension. In the most basic sense, a limitation of algorithms and predictive analytics is simply that they are estimations; probabilistic outcomes, based on historical data. In some areas it might be easier to ORBIT Journal DOI:

3

make use of statistics and predict phenomenon, like in chemistry, or materials sciences, that can make good use of large amounts of data to look for correlations in (Agrawal and Choudhary, 2016). In some sense, materials, atoms and molecules mostly behave based on certain principles, that we have discivered. Also humans, as we like to think about it, behave based on certain, predictable, principles, but social phenomenons, and human behavior, tend to be continuously developed, changing over time, and in that sense it is less predictable. For example, the statistician Nate Silver has become famous the past US elections, when he and his team successfully have been able to predict the result of 49-50 of 50 states since 2008, each election. However, the 2016 election his team failed to predict the outcome of Hillary loosing (although, granted, they had a higher probability chance of it happening than other major forecasters) (Wikipedia, 2017). One reason why forecaster like Silver failed to predict Trump winning is because this kind of campaign strategy had never been seen before. Hence, there were no historical data to be able to, accurately, and completely, predict the future result on. In this sense, historical data about materials tend to not change over time as human behavior, and society, that is continuously developing, and in many ways changing. Another limitation and criticism, that predictive analytics may face, is illustrated by Gotterbarn’s (Gotterbarn, 2015) distinction between Big Data Science and Science. The former is seeking correlations and recognize patterns, but unlike the latter, without seeking understanding and casual relationships; while you look for correlations in both instances, science has a focus on understand the underlying causes for their correlation, while Big Data Science tend to not. What the former seeks and establishes is, according to Gotterbarn, a kind of pseudo-facts, which is used and fed back into the loop to make further correlations and conclusions with, without having deeper knowledge of the phenomenon observed in the correlations. Gotterbarn also writes about a limited kind of reasoning, that we humans tend to do; that we gather and select a limited amount of data and facts, draw a conclusion from it, and act on it. This action in turn, changes future, related events, and creates a positive feedback loop in confirming our prior belief and conclusion. This could also be called a cognitive confirmation bias, which is our tendency to seek out information that confirms our prior beliefs. This in turn, also relates to implicit biases, to unconsciously confirm generalizations of, for example, categories of people, which we come to in the next section. Third and final, a concept illustrating limitations with machine learning and algorithms used for decision-making is Teresa Scantamburlo’s (Scantamburlo, 2016) distinction between judgments and apprehension. A judgement is an act of reasoning, choosing between two alternatives, or motivating the choice, at least. Apprehension is to simply perceive an object or alternative, and without never motivating the choice of action, neither affirming, nor denying it, is acted upon. Machine learning, Scantamburlo concludes, is very much the latter, of arriving at an answer, perceiving an object, an alternative as the most likely and probable (based on previous experience/data), and act upon it. This is very much a similar critique as mentioned by Gotterbarn, that Big Data Science does not search for casual relations and understanding why things happen the way they do, they (often) merely accept the simple correlation (apprehension) and pattern that is found.

Implicit knowledge and biases All of the mentioned conceptual limitations of predictive analytics mentioned above also seems to be able to be related to human knowledge, and short-comings, and also seem to end up similar risks of discrimination and negative treatment of groups of people. A definition of two different kinds of knowledge in cognitive science is explicit and implicit knowledge, and learning processes (Sun et al., 2001). While explicit knowledge is consciously obtainable, easily verbalized and transferable bits of knowledge and information, implicit is unconscious, related to action and tacit knowledge; as in how to achieve motor-skilled goals. There is an intricate dynamics between these different types of knowledges and processes for learning, and they are inter-related. However, what we verbalize is the explicit, while the implicit enables us to perform very complex motor-skills and estimations; for example, it is what makes us able to drive a car. We start of being very much conscious of the affair or moving sticks, pedals and driving wheel around, and get into all kinds of sticky situations in the traffic around us. An experienced driver rarely thinks of these things; trying to over-take you will check the side-mirror, move your foot on the pedal, and turn the wheel without thinking of the complex dynamics between all different movements. The truly expert drivers, like Formula 1 drivers, have been observed wearing eye-tracking glasses. They only need a flick of the gaze in the side mirror, for as short as 100 milliseconds, to see if, for example, someone is trying to over-take, to know how to react (Schrader, 2016). Similar kind of knowledge and information that humans use to drive cars, machines and computers use to drive cars, autonomously; like Google Self Driving Cars. Both use knowledge close and hand to action, that does not need reflection, or complex calculations. Perhaps the quick gaze of the side-mirror by the Formula One driver can be called an “actionable insight”; clear road behind, no obstacle coming up on your side. Actionable insights is an attractive term used in data analytics, like is business, defined as when you have information for enough insight into future events that it can be informative for decision makers. A note of caution can be brought up, in making too strong of a likeness between humans and machine learning; even though the latter often is inspired from the former, with some kind of computerized neural networks, machine learning has gone beyond the human structures of the brain with their networks. Interestingly enough, the kind of problems machines end up in when they are classifying, defining and predicting the future, is still very much like the human problems; specifically related to the implicit process and knowledge. It is with implicit knowledge that we find a lot of the negatively discriminatory effects in human behavior; what is often referred to as implicit biases. For example a study examining recruiters that receive CV’s from applicants are shown to be affected by the name of the applicant; if it is native sounding, compared to if it is foreign sounding, or is associated to a underprivileged minority group. Both in the US as well as Sweden, there was a different respond in how many got called to an interview, with identical CV’s sent out with only the name being changed (Jost et al., 2009). ORBIT Journal DOI:

5

There is also a more general implicit bias in our use of all kinds of lexical words and language use, that through experience becomes semantically associated differently. This can be investigated in Implicit Association Tests (IAT), that for example show that most people, in a western context, associated “dangerous”, more to “black” than to “white”. Similar negative associated can be found to females, compared to males. What research seems to suggest is that predictive analytics can have similar kind of discriminatory effects as mentioned above. For example, Caliskan-Islam (Caliskan et al., 2017) has shown that machine learned algorithms, that use some kind of statistical keyword method, have similar kind of biases with words associated to underprivileged groups when they are run thought IAT’s like the ones above. As Barocas and Selbst (2016) explains, an algorithms is never better than the data set it is using; if the data and words (from human lexical use) is laden with biases and is discriminatory skewed, so will the resulting algorithm be. Related to recruiting, and the example of credit score mentioned in the introduction, an algorithm is trained on of previous workers employed. In the statistical pattern related to a performance measure, there is a correlation to the credit score. When the company receives applicants for a new position, the algorithm receives their data, including credit score, and gives some sort of resulting prediction of how successful this candidate would be. There are different ways of how this comes to affect the selections of candidates that proceed in the process, but generally speaking it seems to be used as an initial cut of applicants, where some proceed to be offered interviews. The problem, then, in terms of possible unfair treatment with the use of algorithms, is that the data set used to train the algorithm might have been filled with implicit biases; or the resulting model the algorithm is using simply does not apply to the new applicants (data set). This can be interpreted as a problem with invalid knowledge, and just as humans can make unfair decisions and judgments based on limited knowledge, so can algorithms.

Solutions proposed There are some problems and risks associated to using predictive analytics in decision making, as we briefly have introduced so far, and tried to give a just as brief insight into the underlying causes. These problems are not new and have been investigated, and solutions are suggested to, at least, mitigate the risks that can be seen with things like discriminatory effects. Mainly the solutions are based on, either (1) better technological and statistical analysis, or containing the possible analyses and actions, or (2) using laws and regulations to constrain possible actions. A less investigated approach are solutions with (3) promoting and supporting explicit thinking with humans using algorithms for decision making. First off, there are machine learning techniques that could attempt to solve some of the prediction problems, like over-fitting on data sets that does not follow the pattern of the model, or introducing better statistical measurements like statistical power and effect size; and not only correlations of statistical significance. More to the point of implicit biases, there are attempts of including diversity measures, and checks for excluding categories that could be discriminatory in

the resulting algorithm. There is a risk here, to exclude the sensitive categories for discrimination, that you have no way of controlling if you end up with a discriminatory pattern, or not (Dwork et al., 2012). So, excluding information, and to limit knowledge, may have the opposite result of treating individuals fairly. Another way would be to anonymize personal and sensitive data. This is often the proposed approach when it comes to CV’s, and names, like the example in the last section, as well as with submissions to conferences, like ETHICOMP 2017. This would be a constraint, and similarly like above, you would risk losing insight to possible unfair treatments, like we have seen can exist not only in personally identifiable name, but in any kind of implicit use of words and sentences. A second option is to use laws and regulations to restrict some types of algorithms. Currently, laws are not proficient to stop the risky type of predictive analysis, but there are updates on the way. EU has a “General Data Protection Regulation” (GDPR) that is supposed to come into effect during 2018. For example, Article 22 is a section regarding decision-making of individuals using profiling and data mining, based on personal data (International Association of Privacy Professionals, 2012). There is some hope that institutions and companies are not going to be able to, just as freely as today, be able to use personal data that affects people negatively. However, there might be cause to be concerned that the new regulations will be tooth-less. For example, the restrictions to data profiling and using algorithms, is an opt-out implementation where user will have to actively restrict companies and institutions form the (potentially) harmful use. Examples in other spheres show that there will only be a small portion that actively will make use of this restriction. Overall, though, the solution proposed with laws and regulation is to try to contain the use of predictive analytics, and there is reason to be cautioned about how that is implemented, as well. A third, and non-exhaustive, and non-contradicting, alternative is to help the human decisionmaker to have more relevant knowledge when making the decisions. What kind of thinking, and what kind of process, is desirable in a decision-maker? The focus on the analysis in this paper is with knowledge; what kind of knowledge we get with most predictive analytics, what we humans have, and what kind of problems they run into. So how can we make better decisions with algorithms? An area that seems less investigated is into the process of decision-making, and getting reliable, and valid knowledge, to support your decision. There are two alternatives to how this third, decision-maker-focused, option could be applied, that will be discussed further down, but first something to clarify what this third option could entail.

Scientific thinking as a process of inquiry What we as a society have decided is a good methodology for reliable and valid knowledge, is with a scientific methodology. Just as Gotterbarn asks for (see section 3), as he problematizes the uses of big data that you often see, we would need not only a partially inspired methodology of science, but a rigorously applied one.

ORBIT Journal DOI:

7

Some hallmarks of scientific methodology are: Testability/Falsifiability, Replicability, Generalizability, Precision and confidence. For example, a key mantra in science is that correlation does not entail causation, otherwise it is easy to get lost in spurious correlations; such as a significant correlation between the (a) Number of people drowned by falling into a pool, and (b) Films Nicolas Cage has appeared in, from 1999-2009 (Vigen, 2017). However, it is unlikely that there is a causal relationship, and that the correlation will be generalizable to the next coming years, say 2010-2015; this would be a hypothesis to test and check. What we would also do in that instance, is to test and falsify the supposed causal relationship, that was theorized; between (a) and (b). This is one way to counteract the confirmation bias we humans tend to go by. Most of the time we try to prove what we already think is true, instead doubting, being uncertain, and trying to get what we think is true disproved. What the methodology seeks to instill in researcher and scientists is a scientific way of thinking, an inner dialogue with you yourself, doubting, and questioning what you believe to be, and not to be, the case. This is a dialogue you, of course, also can have with others. The kind of dialogue we are looking for here is not much unlike philosophical dialogues, like the ones seen put down by Plato, depicting Socrates; Socratic dialogues. They have the central theme of a protagonist that is interrogating others understanding of a presented (moral, or any) issue. This could also be called evidence-based thinking, deductive thinking, or (a sort of) critical thinking. The latter is often defined to in broader terms, like for example also including some inductive thinking, as well as openness and flexibility of the mind, creativity, and imagination, to fully be able to question and find answers. It is often concluded that; to be critically thinking, it is not enough to merely be able to dissect an argument, and be able to think rational and logically. Rather, it is a process including intuition, inductive reasoning, or implicit knowledge, combined with explicit knowledge, reasoning, rational analysis. This seems to be the only way of making right decisions. This also seem to go very much in line with what Scantamburlo is asking for when it comes to decision-making with machine-learning; less apprehensions, and more judgements (see section 3 for details). As was concluded about implicit knowledge, that humans and algorithms alike seems to operate on, is limited and has some inherent problems with (implicit) biases. So what we need is explicit knowledge and processes, and solely the explicit? Not necessarily, much for the same reason mentioned above with critical thinking; that being logical and reasoned does not seem to be enough to be a critical thinker. Same goes with explicit knowledge processes; consciously verbalizing, albite in a reasoned, rational manner, is not enough to acquire good knowledge.

Technology and critical thinking Taking scientific, explicit, and critical thinking as the goal for being more informed with relevant knowledge for the decision-making; what can, then, technology and algorithms do to help the progress in this process? First, taking a step backward, we could define two ways of getting more of something like critical thinking (CT); either (1) the human decision-maker to have an inner disposition to

activate an explicit, critically thinking, thought process, or (2) that technology support, and even enables it. Perhaps they are even one and the same, that at least the latter of having the situation and environment enables and activates an inner capability for an explicit thought process, that everyone has the potential to have. The option is to view CT as a skill, that can be trained, and in some sense that might also be true; but it remains a debate beyond the scope of this paper. Here we will merely accept that the two options may not be mutually excluding, and the focus is on the latter question; how technology could enable, or constrain, CT. Nonetheless, the question about how we improve CT, and scientific thinking, leads to the question how it is done with the human decision-maker. On the brighter side, it is not a new question, dating back all the way to that of Plato and the Socratic dialogues, or Aristoteles Phronesis, till this day where you often hear a cry for “more critical thinking”. On the down side, even if a lot of institutions try to achieve more critical thinking, like in school curriculums, there is no well-established, scientifically proven methodology to teach it. Part of the problem is perhaps that it is difficult to measure, and a reason for this is that critical thinking (CT) is often perceived to be field-specific. As in, to be able to CT about problem P, you have to have a knowledge base regarding problem P’s field area. The risk with not having the wider base of knowledge to stand on is that, even if you might very well make a logical and reflective conclusion, it might inevitably be made on limited data where you miss out on what is relevant to falsify your conclusion. Related to a decision-maker with an algorithm in his hands, this could mean that it is important to have experts of an area, of the specific context, and not just of a general field. Like the example mentioned in the introduction from O’Neil, it be will necessary to not just have knowledge about “teaching” as a field, but specific knowledge about the school that you are doing a predictive analysis on. A pressing question is: well how is predictive supposed to be used, then, if we do not just want to restrict and constrain certain possible action that we deem risks to promote unfairness? One way to look at it is that, there are some things that humans are better at, and some things that algorithms are better at. The instances that algorithms and humans are just as good, or bad at, are less interesting. But, where there is an interaction effect, between human and algorithms, there could be a fruitful Human-Computer Interaction to be made. For example, algorithms are good at pattern recognition; checking correlations (statistical significance), as well as relevance (statistical strength). One way to make use of this is to solely use it in instances where it is enough with rough estimations, and where there are other steps later in the process, that will fine tune alternatives of a decision to be made. In the case of recruitment and applicants for an open position, this could, perhaps, fruitfully be used as an initial first cut of applicants; a rough cut, where the rest that are left is examined more carefully. Human Resource Management (HRM) has made use of personality tests for quite some time, and there are similar problem that you run into; should we base the final decision on a tests like these, if all other things are equal? The tests are, just as they are in behavioral science, not 100% correlated to actual performance in whatever you are trying to measure; it remains an estimation. ORBIT Journal DOI:

9

However, it is not uncommon in HRM to have test scores taken into account also later in the hiring process; when deciding who of the final candidates gets the job. This is strongly adviced against, by some, saying that in that case you might as well flip a coin for who to give the job of the final candidates (Manzoo et al., 2017). Same thing, then, you could say about the use of algorithms. They are, just as psychological tests, mere estimations, and they do not hold up, for example, in a binary choice between two applicants. In those cases, the probability that the candidate with the lesser score would perform better, is too great that you might as well just flips a coin. A lesson from this could, then, be that if you use an (online) Applicant Tracking System, like many agencies and companies today do; then the score from the initial selection process should never be visible for the recruiter in the later stages of the process, since that might affect his judgement in a faulty way. While this could be a reasonable use of algorithms, it is still not without some potential problems. One concern is the question of outliers, those that differ so much from the model the algorithm is using, which then will be missed. The outliers might stem from mere statistical diversity, or the real world has changed enough that the model misses some people, and there simply might be alternative people that is missed in a generalized model. Either way, some people, just as merited for the position, will be excluded by the algorithm. Perhaps this could be a question of how much unfairness you can allow, simply deem it good enough, or where the positive effect out-weight the negative.

The demand of automation What is mentioned in the last section, and sentence, also brings up possible critique to the whole endeavor that “we need more scientific and critical thinking”. You might simply say that, fine, more CT is great, science is great, but it is slow, and we have too much information that we simply cannot be rational and reasoned in every instance of our lives. This is understandable, and a warranted critique. Much of the motivation for the use of Big Data techniques is simply this; we have too much information, and need computerized, automated, help, in analyzing, present, and interpret the massive amounts of data. A similar message can be attributed the so called Fourth Paradigm of Science (Hey, 2009), or “eScience”. It is mainly applied to vast amount of data in areas like Earth Sciences, or Hydrology, but also Health and Life Sciences, and some look towards the Humanities and Social Sciences. The proponent could simply argue; there is no turning back, we have too much information and data to siphon through, that have the potential to give lots of important insights; we simply do not have time for a rigorous, slow, traditional scientific process. Unless we can automate it, of course, as the proponent of the fourth paradigm might proclaim. It is beyond the scope of this paper to investigate if it really is possible, or if, perhaps, a traditional scientific method is not contradictory to a fourth paradigm, that tries to combine empirical observation, theory, and simulation into one.

It is likely that there will be some drawbacks, like some of the one’s mentioned in this paper. If we accept, as we might, that we need to use automated processes like algorithms, then, it would at least be wise to know the drawbacks and negative effect in the use of it. It is only then, it could be claimed, that you enter into a critical analysis of the decision to be made; to use predictive analytics or not.

Conclusions Algorithms in predictive analytics are more and more used to support, and sometimes make fully automated decisions that affect us people of society, in sometimes unsuspected ways. Predictive analytics, such as machine learning, is based on historical data. As an algorithm applies the resulting model of the training from the machine learning, it is a probabilistic estimation it can deliver; of what is likely to happen. This is similar to how we human train our motor-based skills, resulting in implicit knowledge that is fast, unconscious, and often does not even need our attention to be ready at hand, to be used. However, there are some limitations in that kind of knowledge, sometimes resulting in implicit biases, and could for example have discriminatory effects on underprivileged groups of people. Similar effects are found with algorithms and predictive analytics. What we need to make philosophically, scientifically, and critically, sound decisions, is the use of explicit knowledge processes. This is what we use to reason, evaluate, and make (scientific) judgments on what lies ahead of us, instead of (implicitly) just act upon an estimation. To make fair, reasonable, and critically sound decisions and judgments, we then need to have a focus on explicit knowledge, in combination with implicitly acquired knowledge. The solution focused on in this paper is how humans and algorithms, or ICT, could interact with ethical decision-making. What predictive analytics can produce is, arguably, mostly implicit knowledge, so what a human decision-maker could, possibly, help with is the explicit thought processes. This could be one way to conceptualize the interactive effect between humans and algorithms that could be fruitful. Presently there does not seem to be very much research regarding predictive analytics and ethical decisions, concerning this human-algorithm interaction. Rather it is often a focus on pure technological solutions, or with laws and regulation. A question often raised in the debate regarding algorithms and automated system, is where the responsibility of action should lie. In this paper, a stronger focus is on the human decision-maker, and a suggestion is to focus on how he can be supported in making ethically, and critically, sound and fair decisions. This would put the responsibility more strongly on the decision-maker, again, rather than a manufacturer of some technology, or programmer of an algorithm. A similar conclusion can be found in Gotterbarns analysis of Big Data Science (Gotterbarn, 2015). Someone might argue that the whole point of automation is to not have to involve a human decision-maker, and we might have to accept that our global society is so “datafied” that we simply need something to automatically make calculations, choose, or at least strongly affect our ORBIT Journal DOI:

11

decisions. It could very well be a conclusion we could accept, but at first we would at least want to know what the negative consequences are, along with the positive. Only then we have, at least, two alternatives we can compare, reason about, and make a judgment from. A fruitful research questions, then, could be; what is needed for decision-makers using algorithms, to make ethically reasoned decisions. This could perhaps be researched more. It also becomes a question for Human-Computer Interaction, to investigate how that exact interaction between human and algorithm, or ICT, can be achieved; and perhaps also ICT-design methodology. The nature of using Big Data, to find new correlations in vast amounts of information and data, is sometimes claimed to be something new; demanding a new paradigm of establishing knowledge. However, what we humans do, every day at every waking moment (and maybe during sleep), is siphoning vast amounts of information, classifying, and drawing conclusions of what will happen next. The main purpose of the brain is sometimes claimed to be to be able to predict the future, which, arguably, is the main purpose of algorithms within predictive analytics. There are perhaps more lessons to be learned from how we human make rational, reasoned, and ethically considered decisions, that can be applied to how the same kind of decisions can be made by, or with the help of, automated algorithms.

References Agrawal, A., & Choudhary, A. (2016). Perspective: Materials informatics and big data: Realization of the “fourth paradigm” of science in materials science. APL Materials, 4(5), 053208. https://doi.org/10.1063/1.4946894 Barocas, S., & Selbst, A. D. (2016). Big Data’s Disparate Impact. clr. https://doi.org/10.15779/Z38BG31 Barocas, S., & Selbst, A. D. (2016). Big Data’s Disparate Impact. California Law Review, 104, 671–732. https://doi.org/10.15779/Z38BG31 Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186. https://doi.org/10.1126/science.aal4230 Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. In ITCS ’12 Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (pp. 214–226). ACM Press. https://doi.org/10.1145/2090236.2090255 FiveThirtyEight. (2017, Sept 20). Nate Silver. In FiveFourtyEight. Retrieved from http://fivethirtyeight.com/features/final-election-update-theres-a-wide-range-ofoutcomes-and-most-of-them-come-up-clinton/ Gotterbarn, D. (2016). The creation of facts in the cloud: a fiction in the making. ACM SIGCAS Computers and Society, 45(3), 60–67. https://doi.org/10.1145/2874239.2874248

Gotterbarn, D. (2016). The creation of facts in the cloud: a fiction in the making. ACM SIGCAS Computers and Society, 45(3), 60–67. https://doi.org/10.1145/2874239.2874248 Gumbus, A., & Grodzinsky, F. (2016). Era of big data: danger of descrimination. ACM SIGCAS Computers and Society, 45(3), 118–125. https://doi.org/10.1145/2874239.2874256 Hey, A. J. G. (Ed.). (2009). The fourth paradigm: data-intensive scientific discovery. Redmond, Washington: Microsoft Research. International Association of Privacy Professionals. (2012). Top 10 operational impacts of the GDPR. Retrieved July 13, 2017, from https://iapp.org/resources/article/top-10-operationalimpacts-of-the-gdpr/ Jost, J. T., Rudman, L. A., Blair, I. V., Carney, D. R., Dasgupta, N., Glaser, J., & Hardin, C. D. (2009). The existence of implicit bias is beyond reasonable doubt: A refutation of ideological and methodological objections and executive summary of ten studies that no manager should ignore. Research in Organizational Behavior, 29, 39–69. https://doi.org/10.1016/j.riob.2009.10.001 Manzoo, A., Sköld, J., & Nyheter, D. (2017). Personlighetstester vid rekrytering används på fel sätt, menar experter - DN.SE. Retrieved July 13, 2017, from http://www.dn.se/ekonomi/personlighetstesterna-anvands-pa-fel-satt/ O’Neil, C. (2016). Weapons of math destruction: how big data increases inequality and threatens democracy (First edition). New York: Crown. Rosenblat, A., Kneese, T., & Boyd, D. (2014). Networked Employment Discrimination. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2543507 Scantamburlo, T. (2016). Machine learning in decisional process: a philosophical perspective. ACM SIGCAS Computers and Society, 45(3), 218–224. https://doi.org/10.1145/2874239.2874270 Schrader, S. (2016, July 3). This Is How A Formula One Driver Sees The Track. Retrieved January 31, 2017, from http://blackflag.jalopnik.com/this-is-how-a-formula-one-driversees-the-track-1782965187 Sun, R., Merrill, E., & Peterson, T. (2001). From implicit skills to explicit knowledge: a bottomup model of skill learning. Cognitive Science, 25(2), 203–244. https://doi.org/10.1207/s15516709cog2502_2 Vigen, T. (2017). Spurious Correlations. Retrieved July 13, 2017, from http://tylervigen.com/spurious-correlations

ORBIT Journal DOI:

13