Make way for the robots!

This is the authors’ own copy, free for fair use – http://en.wikipedia.org/wiki/Fair_use

Make way for the robots! Roles for autonomy in building a European Public-Private Partnership Kjetil Rommetveit Centre for the study of the sciences and humanities, University of Bergen. Niels van Dijk Law, Science, Technology and Society (LSTS) at the Vrije Universiteit Brussels. Kristrún Gunnarsdóttir Dept. of Sociology, University of Surrey, Guildford, Surrey, United Kingdom.

Abstract: This paper studies the making of ‘autonomous’ robots to address Europe’s societal problems. The paper describes how various techno-epistemic networks within industry, science and law appropriate and enact this imaginary, by working and building across the boundaries between man and machine, different disciplines and sectors. Roadmaps are the main metaphor and organising tool in this work, aligning these heterogeneous actors along a common machine-centric and future-oriented path. Focusing on a recent legislative initiative for robotics, we describe what happens as the industry-dominated project of building robot autonomy docks with public institutions in a public-private partnership. Emphasising the coproduction of robotics and politics we observe the increased speculative emphasis on ethics and on possible futures in legislation.

Keywords: Autonomy, robotics, ELSA, public-private, imaginary, innovation

2

1. Introduction: visions and politics A type of human robot, a Humanoid, is expected to work together with human partners in our living environment, and it will share the same working space and will experience the same thinking and behaviour patterns as a human being1 Robotics development has always been intimately connected with visions of autonomous machines, in particular, any human-like intelligent appearances like those found in mythology, folklore and the science fiction genre (e.g. Hephaestus' golden assistants; the Golem; Asimov, 2004; Pixar Animation Studios, 2008). Arguably, such visions have been integral to ongoing efforts towards better understanding of life through artificially recreating it (Riskin 2007). Increasingly, policy makers and industrialists are seizing upon the autonomous robot imaginary in more pro-active ways. The promise is no longer primarily about better understanding of life or its fictional representations, but to enhance life, welfare, jobs and well-being, and open new markets for European industries. Increasingly brought into the institutional organisation of innovation, the autonomous robot imaginary takes on new configurations and roles. Autonomous robots will to greater extents be capable of learning, flexibility and social interaction, than previous generations of (mainly) industrial robots. They will move from the closed-off confines of factories and into people’s daily living environments: communities, homes and public spaces. The promise is that they will be taking on new roles as providers of services to humans, and enter into intimate relations of care and companionship. Public-private conglomerates coordinate robotics development across Europe and beyond. The prime mover in these developments has been industry as demonstrated by its power to frame research and innovation policies in ways that orient to industrial production. But they do not act alone in shaping innovative and radical visions of the future of robotics. As they expand, and as robots enter into new functions, industries have to rely on other actors, including academic research communities, policy makers, lawyers, ethicists, the media and the citizens, who are expected to make use of future robots. Significantly, the making of autonomous robots has repercussions for law and regulation, that become important sites in which ongoing developments can be accessed and studied. This paper explores some key features of this ongoing story, including the making of a European Public-Private Partnership (PPP) devoted to autonomous machines, and the ways in which an imaginary of autonomous robots is appropriated, coordinated and enacted in industry, science and law. A lot has been written on the social dimensions of robots over the last thirty or so years. Significantly, philosophers and STS researchers have explored the so-called ‘soft AI’ approach to the understanding of machine agency, as opposed to ‘hard AI’, to better capture human-machine interrelations rather than assuming that machines can be fully autonomous (Dreyfus 1992, 2007, Suchman 2007). In ethics there are also a burgeoning field of roboethics (Veruggio 2006), centring on ethical codes for the robotics community, and on machine ethics in building ethical codes into machines (Wallach and Allen 2009). In the legal field, Robolaw is pondered as an area of study, possibly warranting its own methods and principles, and new legal frameworks are being developed to accommodate and guide machines on their way into society (Leenes and Luciviero 2014). Economists are also increasingly turning to the implications of automation and delegation of tasks to machines (Brynjolfsson and McAffee 2011). We build here on the ‘soft AI’ approach to human and machine agencies and their mutual interactions and mediations. For instance, Suchman’s (2007) notion of human-machine configurations is one way of shifting the focus away from machines as isolable entities, and to look instead at the relations, meanings and co-productions with which AI is interwoven. Our 1 http://www.humanoid.waseda.ac.jp/history.html


contribution to this approach, is to further emphasize the social and institutional relations around robots and roboticists which are becoming increasingly more complex (Šabanovic 2010). Human-machine configurations have been mediated for a long time through science fiction and the media, but these configurations are now also mediated through the contributions of ethicists, lawyers, policy makers and legislators (Matsuzaki and Hindemann 2015). In response to that, we focus here on what STS scholars have termed the co-production of science and society (Shapin and Schaeffer 1985, Jasanoff 2004), specifically -in sketching out the relations emerging at the intersections of robotics science, industry, law and politics. We capture some of these ongoing developments by focusing on the role of the autonomous robots imaginary (Suchman 2007, Šabanovic 2010, cf. Jasanoff and Kim 2009, Jessop 2009), and the human-machine imaginations through which emerging networks of practitioners appropriate, reconfigure and enact this imaginary. Our paper proposes a way to join together so-far separate strands of scholarship, by casting the net of analysis of human-machine configurations more widely. We study the work of co-production by which these networks cross traditional boundaries between disciplines and public and private sectors, enabled by the strategic mobilization and operationalization of the imaginary in robotics developments. This perspective provides useful correctives to fast-expanding specialized literatures on autonomous machines, by shifting the focus from the machine, towards institutional relations and the political economy in which the machines are embedded. The main organizing tool and metaphor in this co-construction work is that of the roadmap, whose main characteristics stem from the peculiar fact that that it depicts a road that has yet to be built (cf. Scott 1998), and that eventually will be the single road on the map. We observe that this tool with its metaphors is deployed to fulfil several functions, some of which we address in this article. The autonomous robot imaginary remains vague and open-ended (cf. Levidow 2013), serving to attract, mobilize and inspire different actors into innovation. Yet, the map imagines, projects and wills a road into existence, rendering the imaginary operational and concretely actionable in a series of (relatively) concrete steps. For instance, it guides robotics developments through a functional set of ‘application scenarios’ towards more concrete ‘product visions’. Closely related, the roadmap performs as a crucial aligning device. It provides a complex boundary object that serves to coordinate the clustering together of different practices (law, ethics, science, industry) in building networks along a commonly laid out path. This in turn serves to align and order contributions to robotics developments according to a common goal. Attending to this roadmapping strategy, we describe how scientific research in robotics provides new knowledge to overcome gaps by building bridges and technoscientific enablers, whereas Ethics, Law and Sociology (ELS) scholars are gathered to sweep off this road any potential ELS obstacles. Through such metaphors we observe the simultaneous strategic positioning and meaning-making to come up with credible representations that can pave the road. We describe this roadmapping work as taking place in techno-epistemic networks. The main characteristic of these networks is how they mobilize traditional sources of knowledge and authority for the purpose of innovating for some societal purpose. 2 Departing from within traditionally bounded institutions and sources of legitimacy, such as science, law, industry and politics, the techno-epistemic networks re-make their boundaries, from the inside out and outside in. Creating new meanings and concepts to accommodate robotics, new connections are created across ontological, disciplinary, and institutional boundaries. Building across the biological, physical, cognitive and social, techno-epistemic networks draw on discourses of converging technologies, cybernetics, robotics and ICTs (Dupuy 2008). Organizationally, they mobilize gaps and challenges to connect science and knowledge (including law) with societal 2 Sources of inspiration for this concept are Haas (1992) and Stengers (2005). Our approach differs from Haas’ notion of an ‘epistemic community’ by foregrounding the strong driving force of technology in the making of new relations across science and society, and across disciplinary and professional domains.

4

challenges and innovation (cf. Haas 1992, Kuhlman and Rip 2014). Even though the main referents here are ontological and institutional, we retain a focus on the ‘epistemic’, since the main currency in stabilizing cross-domain relations remains knowledge and representations. Ensuing meanings and concepts thus created, cut across previously established boundaries, ie. ‘sentient robot companions’ or ’electronic personhood’. These concepts act as indispensable (though tentative and exploratory) features of temporary stabilization and coordination of action, occurring across practices and institutions. Finally, these mechanisms resonate at, and drive developments towards higher institutional levels, since efforts to address challenges, fill gaps and overcome barriers become explicit in the pooling of resources at the European level. Here we observe, paradoxically, how drawing on traditionally distinct sources of authority in a Public-Private Partnership (PPP) in robotics, becomes itself the main strategy in establishing the partnership’s legitimacy (see Bowker, 1993). We claim that the complexity of these different elements and levels of analysis is intrinsic to the entanglements and efforts of the techno-epistemic networks themselves. It is a highly organized and structured effort simultaneously targeted at creative destruction and disruptive innovation, in which relations are dismantled, re-arranged and re-ordered through coordinated efforts and a promise of new economies of scale (Assink 2006). In the three main sections of this paper, we address (section 2) the shaping of EU robotics networks, visions and strategies, (section 3) the makings of robot autonomy as a topic for ongoing exploration, perplexity and debate by these various actors, and (section 4) the kinds of co-production enabled by these actions of emerging techno-epistemic networks. 1.1. A note on method This paper builds on research conducted within a multi-disciplinary European research project on innovation assessments.3 Our research was directed at understanding the mutual relations and interdependencies of innovation actors such as policy makers, roboticists and scientists on the one hand, and technology assessors such as lawyers, ethicists and social scientists on the other hand. These various actors seem to address the same policy problems/issues, e.g. robot autonomy (Rommetveit, et al 2015), and are increasingly drawn into collaborative networks under headings such as ‘addressing grand societal challenges’ (Kuhlmann and Rip 2014). We explored the imaginary of autonomous robots as a vision and practical achievement, including the many problems arising as such integration is sought in practice. Our research proceeded through three main stages: First, we explored the main networks involved in the making of autonomous robots in the EU, including the relevant assessment networks. Focusing on ethics, law, socio-technical analysis and vision assessment, each partner in the project examined a set of documentary data, while independently exploring a number of other data sources particularly relevant to their disciplinary background. These included participatory observations in meetings and events of some of the main networks we engaged with, ie. the RCC consortium, euRobotics and Robolaw. As for the documents, we explored green/white papers on legal issues, mission statements and other outputs from roboethicists, social-anthropological studies of AI, HCI and robotics. Secondly, based on the initial mapping and analysis, representatives of the different networks (policy, industry, ethics, law, robotics) were brought together in a two-day workshop dedicated to the discussion of ‘robot autonomy’ as public-political, ethical-legal and technological challenge (Rommetveit, et al 2015). We sought to highlight the frictions and complexities that arise in practice as collaborations are sought out across professional and institutional boundaries. Thirdly, we analysed and compared the results from our initial mapping with those from the workshop, focusing on the premises of framing and the intrinsic values and rhetoric at work in the strategic shaping of the networks. 3 (project website)


2. The constitution of a PPP The endorsement of an autonomous robot imaginary by actors from different institutional domains marked the gradual emergence of the European PPP in robotics. The imaginary projects and mobilizes a vision of a society of increasing human-machine interactions in which all sorts of human tasks are taken over by machines. Further, these developments are projected as beneficial to society, markets and technology users alike. Initially quite vague and underspecified, the imaginary has gradually evolved to take on more tangible and institutionalized forms. The imagined robot-integrated society will not come about by itself but has to be pulled and pushed into existence through coordinated actions and innovations across domains. According to the roadmap, the road for robots will be enabled through new improved technologies, guided and protected by law, and driven forward through industry and the opening of new markets. Finally, robotics will be pushed towards society and users in ways that do not invoke fear and stigma, but rather entice, prepare and seduce them into domesticating robots and embracing the vision of a robot-integrated society. Although this story builds on precedent developments4, the first decisive steps towards an institutionalization of the imaginary came through the making of the technology platform, euRobotics in 2006, in which the academic (EURON) and industry driven (EUROP) networks came together.5 At its formal launch in 2005, the commissioner for Information Society and Media, Viviane Reding, endorsed this imaginary towards societal challenges and institutionalisation: We need to achieve higher economic growth through more innovation and higher productivity, whilst creating more jobs. We need also to address many societal challenges, the ageing population, the well being of our society, and the need for security … Robotics will contribute to these challenges (Reding 2005). Reding’s speech went into some technical detail on robotics development, clearly informed by specialists working for the Information Society Technologies (IST) Programme of the European Commission, i.e., its advisory group ISTAG. This group was not an administrative body, but an expert consultancy made up of representatives from ICT-related industries and academic research, and counselled the Commission between 1999 and 2013 on investment priorities in the IST Work Programmes. ISTAG also proposed the Future and Emerging Technologies (FET) flagship initiative in 2009 and suggested that robot companions as a domain of innovation could achieve FET flagship status.

Figure1: Timeline illustrating the emergence of the robotics technology platform and PPP. Source: euRobotics.

In approaching the various ways in which different techno-epistemic networks have positioned themselves in relation to the autonomous robot imaginary, we highlight the following: Industry has increasingly appropriated the roadmapping metaphor, using it as a control tool to coordinate the actions of the other networks. The roadmap refers not so much to a single document, but to an organized strategy embedded in strategic documents (e.g. EUROP 2009, euRobotics 2014). Among key actions is forging a common language and a vision to provide 4 Research programs in rehabilitation robotics have a long history in Europe and can, at least in the cases of France, Germany, the UK and Scandinavia be traced back to the 1970s (Dallaway et al. 1995). Inside the EU structures these initiatives started to come together in the early 1990s with the establishment of the internal market. 5 The main initiating companies are EUnited Robotics, KUKA, ABB, Phillips, FhG IPA, and KTH.

6

forward guidance and to coordinate pan-European and cross-institutional activities within the robotics domain. Pluralism is recognized as necessary, yet something that must also be contained: ‘industry will be the main driver behind these targeted stimulations because its needs for innovation and strong positioning in the worldwide robotics market are the greatest’ (EUROP 2009). Although an industry body, euRobotics has largely been sustained through EU research grants. The drafting of the 2009 strategic research agenda was made possible by a coordinated action project (CARE) under the 6th framework programme, consisting of 125 partners from the European robotics industry. The technology platform has since morphed into a PPP contract with the European Commission (Bischoff, et al, 2010, euRobotics) — a form of partnership that strategically supports innovation as outlined by Reding, and establishes the mutual contractual relations between the public institutions (ie. the European Commission) and industry. As concerns science we follow one specific project proposal to the Future and Emerging Technology Flagship initiative, called Robot Companions for Citizens (RCC 2012). The project ultimately failed, and so can only be seen as illustrative.6 It did however mobilize large parts of the robotics community (with partners from 73 academic institutions), and so was a kind of a ‘project of projects’. It also mobilized the central idea in contemporary robotics research (cf. Brooks 1991, Bekey 2005), namely to mimic biology and evolution in the attempt to create biological and sentient machines. With respect to law (and ethics), we observe two key outputs of ELS activities coordinated by euRobotics with funding from the EC under Supported Coordination Action: 1. A proposal for a Green Paper on legislating robotic technologies, hereafter ‘the Green Paper’ (Leroux, et al, 2012); 2. a proposal for a White Paper on regulating robotics within the EU, hereafter ‘the White Paper’.7 Prior to these contributions seeing light, a workshop on ELS issues was arranged by euRobotics to lay out a plan of direct engagement with the legislative process. The Green Paper was meant to foreshadow a ‘White Paper on Regulating Robotics, containing guidelines and suggestions for the European Commission in the field of regulating emerging robotic technologies’8. Yet, when a White Paper finally came out, it was not worked out within the confines of the industry strategy but mainly by legal scholars from Robolaw, an EU-funded collaborative research project.9 In the end, the findings of the Robolaw project were presented to the Committee for Legal Affairs and Internal Market of the European Parliament (the JURI Committee), in a workshop, titled, ‘Upcoming issues of EU law’. The challenge of robotics had been singled out as representative of the future development and challenges of the law. The JURI Committee afterwards formed a special Working Group on the ‘Legal Questions related to the Development of Robotics and Artificial Intelligence‘ that was to hold a series of meetings with presentations by experts from different fields (starting April 2015). 10 This eventually led to the formulation of a Motion for a European Parliament Resolution on ‘Civil Law Rules on Robotics’ which incorporates elements from both the Green and White Papers. 3. The makings of autonomy In this section we focus on how industry, science and law have engaged with the makings of robot autonomy: how they perceive of the challenges and construct the visions, goals and 6 EU research is replete with robotics projects (A “robotics” search on the CORDIS project web-page renders 157 projects in FP7 alone), and so the RCC is not representative in any absolute sense. 7 Strictly speaking these documents are not Green or White Papers, since they are not official documents of the European Commission. They are thus proposals for such. 8 www.robolaw.eu/RoboLaw_files/documents/robolaw_brochure_20120322.pdf 9 http://www.robolaw.eu/ 10 http://www.europarl.europa.eu/committees/en/juri/subject--files.html?id=20150504CDT00301


strategies for its making. Our main focus is on the tools, concepts and imaginations embedded in these networks and their actions. The role of metaphors and analogies is particularly powerful in transferring meaning. In the case of industry, metaphors are taken from the assembly line of the traditional factory, and extended to the workings of society; in the case of science we observe the imaginary of the natural biological ‘machine’ which is stretched to users in need of help, and a whole range of ‘societal challenges’; finally, for law the basic metaphor remains the human being, the agent as the yardstick against which agencies can / cannot be extended to other entities. 3.1. Industry: re-making and extending the assembly line. Robots were introduced to manufacture, mainly of automobiles, in the early 1960s, but it was not until the 1980s (UN 2005, IFR 2012) that their presence became a mainstay in industrial production. The key concept was not autonomy but automation, associated with classical assembly line robots (cf. Suchman 2007). Emblematically, these would appear as robot arms in factories and assembly lines performing routine tasks such as distributing, welding, assembling, bolting and painting, one task leading deterministically to the next. ‘Degrees of machine freedom’ would occur within three-dimensional x – y – z coordinates, but this geometric field was literally and materially shackled to the shop floor. From around the mid 2000s, however, a different actor was imagined, e.g., the driverless car operating autonomously in unstructured and complex human environments. This was emblematic of the boundary crossings of autonomous robots: not just closed-off secure settings but also living social environments (EUROP 2009); not just automation but autonomy (Franklin and Graesser 1997, Floridi and Sanders 2004); not just repetitive movement, but flexibility, adaptability and learning in complex, unstructured environments (Dautenhahn 2004), and the blurring of industrial and service or assistive robotics (SRA 2014). Hence, the industry roadmap anticipates, predicts and performs that ‘With increased flexibility and ease of use, robots are at the dawn of a new era, turning them into ubiquitous helpers to improve our quality of life by delivering efficient services in our homes, offices and public spaces’ (2009, 7, cf. also SRA 2014, 15). Autonomy is mobilized together with other ‘application requirements’ such as adaptation, positioning, human-robot interaction, robot-robot interaction and dependability, to be translated into more concrete ‘product visions’ (Ibid.). Although robot autonomy is not the same as human autonomy (Haselager 2005), it shares some of the same virtual characteristics: a regulative principle or meta-property to steer actions and strategies not identifiable in any single body part or application. It is not a technical specification of machines; rather, it projects and expects an evolving relation between humans and machines. It is defined as increased interaction with humans, but is simultaneously expected to diminish human intervention with the actions of machine systems: ‘Autonomy is the system’s ability to independently perform a task, a process or system adjustment. The level of autonomy can be assessed by defining the necessary degree of human intervention…’ (EUROP 2009, 22). Yet, how do we determine what is ‘necessary’? There is no scientific specification that could settle the issue; there is rather a list of (66) technical gaps (Guhl and Zhang 2011) needing to be filled in order for the machines to develop as prescribed in strategic research agendas. This is where the scientific vision inserts itself, promoting a scientific strategy to ‘build the bridge’ to the machines of tomorrow on the assumption that the scientific principles to get there will be discovered along the way (next section). Yet, industry does not wait for scientific results, but orients towards the making and ordering of the social relations deemed necessary for expanding the assembly line:

8

The robotics market is not only composed of end user applications and robot technology suppliers but also of service and supply chains which add value. The early stage nature of the robotics market means that these are not yet fully developed (euRobotics 2014, 27). Whereas such value chains would increase the use of robots in various domains, the attention to ‘robot created IPRs’ is indicative of new imagined markets arising from harvesting information from the robots’ environments and their human users (see section 3.3). Such value chains cannot be directly accessed but must themselves be built (cf. Scott 1998, Callon 1998), hence, the need to influence legislation, regulations and standards.11 Central here is strengthening of the common language used by the robotics community (through strategic documents, roadmaps, emailing lists, etc.), and actively fostering a community. There is growing emphasis on robot competitions12, aimed to build community (as competition) and outwardly projecting promises of autonomous robots (towards investors, policymakers and ‘Society’). In the midst of gaps and technical challenges, the assembly of robot parts, properties and capabilities, the European community of sociable machines is imagined. The decisive factors hinge on the role of European industries in a globally competitive environment, especially the extent to which European robotics can develop the desired technologies internally or have to rely on others (euRobotics 2014). For instance Europe ‘has a strong foothold in drives’ but whose ‘dependencies on others with respect to gears should be decreased’ (actuation); Europe ‘is strong in motion and task planning’ but nevertheless noticing how ‘higher level mission planning in the US is more advanced due to extensive defence and space activities’ (planning). Finally, due to strong public support and a strong research community Europe is considered to be in a ‘good position to take leadership in the developing civilian markets’. In this final quote, we encounter the rationale for public-private partnerships: it is the need for ‘Europe’ to step in and take responsibility for long-term funding of risky research in the civilian domains. This includes the firm embedding of such partnerships within the ‘societal challenges’ framework of Horizon 2020, including the linkages to ‘Pre-Commercial Procurement’ and ‘Public Procurement of Innovation’ (Ibid., 22). In this sense the progression of industry, extending the assembly line into society, is inseparable from the realization of robot autonomy, and the making of public-private partnerships. Directly linked to the progression of Europe-owned and controlled technologies, the way ahead is seen as one of incremental improvements upon those same technologies. The industry strategy mobilizes and implements an ‘economic imaginary’ (Jessop 2009), providing ‘meaning and shape to the economic field’, at the same time performatively creating (Callon 2007) the technological economy (Barry and Slater 2002) of autonomous machines. This imaginary singles out new domains and value chains (European citizens’ living and working environments), for extraction and market-making. It singles out an imagined community of interest (Levidow 2013), a Europe, strongly attached to its ownership of specific technological and scientific domains of relative competitive advantage. In these ways the roadmap strategy envisions, produces and orders the regulations and technologies that will bring about economies of scale by guiding the way forward to new ‘partnerships’ of public and private enterprise, of humans and machines, lawyers, engineers and scientists. 3.2. Academic research: Making Nature’s friendly helpers? At the heart of these developments resides a conundrum as old as robotics itself: building 11 “There is a growing awareness that the safeguards and enactment of standards, norms and legislation will need to be included as a part of the systems design process that creates robotic devices and technology”, (euRobotics 2014). 12 Frequently modelled after sports events. Examples include the RoboCup, ELROB, FIRA and ICRA competitions.


artificial life to better understand life (Suchman 2007, Riskin 2007). This, one could say, is a genuinely academic and experimental endeavour, and a philosophical one, at arm-length’s distance from the industrial assembly line. The RCC project was an interesting instantiation and mirror of what happens when scientists are granted the opportunity to dream big. The FET flagships held out the promise of great funding and prestige, of revealing nature’s secrets while offering solutions to Europe’s societal and existential problems. Concerning the latter, Europe’s high standards of living were portrayed as an object of global envy: ‘democracy, advanced economies, social inclusion and quality of life’ (RCC 2012, 3). Yet, these are threatened by man-made and natural disasters, economic downturn, trade imbalances, and a dwindling industrial base. The prime challenge, though, was demographic, since ‘never before in human history have older citizens made up such a large proportion of the European populace’ (RCC 2012, 92). For this purpose, the RCC’s primary goal was the making of friendly helpers for care and companionship. The RCC identified a gap between European citizen’s expectations and their capabilities to live within their means. This gap was projected as ‘the challenge of sustainable welfare’, to be met by ‘a whole new class of machines to overcome the limitations of today’s machines, new machines based on a whole new science’ (Ibid.). Autonomous machines are thus posited as a direct response to these challenges. The roboticists seem to not address the topic of autonomy head-on, but rather work by way of (conceptual and experimental) detours13, which in the case of RCC was articulated as sentience. The building of sentient machines was the main scientific challenge, termed the robotics bottleneck. It stated how present robotics are advancing towards adaptable and learning machines capable of acting in unstructured environments (Dario et al. 2011, RCC 2012), but so far not in a position to fully deliver on this promise. This lack of knowledge and robot capabilities to think, build and act, fits well within the industry’s identification of gaps, and promise to overcome them in one paradigm-changing leap. Positively filling the gaps was branded as a challenge of enablement, to be met by building bridges (Dario 2012) or pillars (RCC 2012). The pillars of the project were five: Simplexity, Morphological Computation, Novel Fabrication Technologies, Sentience and Society. A biomimetic approach that studies and models the adaptive mechanisms developed by living beings (micro-scale, invertebrate, vertebrate and human) over millions of years, underpinned and unified the design of these bridges. Seeing nature as an engineer (cf. Bensaude-Vincent 2007), these mechanisms were turned into fundamental design principles taken to drive the evolution of bodies and brains, to be mobilized by a discovery engine. Here we can see the differences, interactions and mutual adjustments of the academic and industrial robotics networks. Industry would self-consciously map out and identify, to the greatest possible extent (ie. as 66 gaps), the knowledge gaps in need of filling, leading to a set of recommendations for gradual improvements. Science, on the other hand, would plunge itself into the unknown unknowns of scientific research, seeing the gaps as constitutive of the road-building enterprise. Industry’s incremental approach was likely to lead to ‘a gradual loss of controllability and robustness, and this will ultimately lead to a substantial cost in efficiency and safety’ (Ibid. 5).14 According to the RCC these shortcomings of autonomous robots, paralleled in the legal problem of liability, could only be met through a genuine leap, for which scientific ingenuity was crucial. This grand technoscientific task called for paradigmatic change to enable the next stage of robotics development. This challenge was accommodated within the broader roadmap strategy through the central role of concepts such as ‘bridging gaps’, and ‘overcoming limits and bottlenecks’. At its core, the vision was deeply hybrid, a socio-technical imaginary (Jasanoff and Kim 2009) of a new society, driven and assisted by robotics companions for humans through most walks of life. In line with this somewhat detached wishing and promising, the academically-driven consortium left relatively 13 Such as adaptability, intelligence, rationality (Rommetveit et al, 2015). 14 Similarly, Bekey (2005).

10

little room for input from industry and its mediating role to deliver actual products to markets. This was also paralleled in the attention to ethical, legal and societal issues: when finally the relevant expertise was taken on board, it was generously included into the discovery engine. But the careful attention to building relations, characteristic of the industry roadmap, was largely absent. Thus, the main identifier and basis for collective action of the RCC, was the fascination with building and exploring things that move, act, think and feel, without a clear pathway to innovation, i.e., to delivering societal goods and products to market. 3.3. Legal studies: Qualifying as man’s friendly helper or self-standing person? The emergence of robots capable of autonomous decisions is seen as a challenge to the legal order and perhaps resulting in a paradigmatic shift in legal thinking (De Cock Buning et al. 2012). In this section we describe the reception of advanced robotics within two different legal networks: first, a proposal for a Green Paper and related ELSi document emanating from the industry roadmap; secondly, a proposal for a White Paper coming out of the legally constituted Robolaw project. We highlight how the Green and White papers position themselves differently in relation to human-centrism in the legal framework. From that basic position other problem frames, strategies and positions follow, mainly on legislation which we describe in section 4. 3.3.1. Green Paper: from human tools to electronic persons. The Green Paper declares itself to be the first (European) effort at bringing together the robotics and legal communities, a task supported by a related ELSi document (Leroux and Labruto 2012). The roadmap metaphor is placed up front, as the general objective is ‘to act and find ways to favour the development of European robotics’. From this perspective, it has to deal with the ‘worries about the consequences of introducing robots into society’ (Ibid. 5). The main purpose is to describe the ethical, societal and ‘legal issues hindering the development of robotics in Europe’ and to provide a ‘roadmap to overcome these obstacles’. This is a crucial framing that provides the starting point for the interdisciplinary collaboration. Legal, ethical and societal issues are here presented as 'barriers' or 'obstacles' that need to be removed, preferably before they arise.15 The contributions of legal scholars, ethicists, social scientists and engineers must aim at this road-sweeping task. To this end the Green Paper sets out the goal that the communities (of engineers and lawyers) get to ‘know each other’, ‘share common language, vision and objective’ (Ibid. 8).16 This is especially so for the concept of autonomy, since it is shared by philosophy, engineering and law: ‘It is precisely that “interdisciplinary” collaboration that is the main reason for the current debate on the meaning of the word autonomy’ (Ibid. 11-12). The paper provides a basic account of the different meanings of ‘autonomy’ as seen from law, engineering and philosophy. Yet, the main authors are industry representatives and the legal problems are primarily envisioned from an engineering perspective, as emanating from ‘dependability, accountability, predictability of results and standardization/certification issues’ (ibid., 14). The Green Paper then provides an account of European legal structures. A main barrier is the sheer complexity and many-layered character of European law (Ibid. 13-14, cf. 66). The Green Paper argues the need to ‘harmonize the legislation concerning robotics in Europe. Industries are confronted to different regulation and legal constraints which represent barriers 15 This language also occurs in the task descriptions of the Topic Groups that write the Strategic Research Agenda and the multiannual Roadmap for the PPP: “they identify gaps and challenges, describe the desired paths towards solutions, milestones to be reached at specified instants in time […] and mobilize members and nonmembers to realize them, and to support their subsequent exploitation.” https://eu-robotics.net/sparc/topicgroups/index.html 16 Interestingly, this goal is also specifically enshrined as a legal requirement for creating a PPP within the context of Horizon 2020 (Article 25.3c of European Parliament & Council Regulation No 1291/2013 of 11 December 2013 on establishing Horizon 2020).


making difficult the emergence of new markets’. In order to justify a proposal for legislative change, it is thus necessary to demonstrate the extent and ubiquity of these barriers. ‘For civil law regimes, the more impacting the legal hindrances, the easier it will be to change to [sic] current legislation’ (Ibid. 66). For this reason the Green Paper takes a top-down approach to legal assessments rather than a bottom up, case-by-case approach. As we describe in section 4, this assessment strategy is translated into a legislative strategy since an analysis of the legal issues in a certain robotics sector would only demonstrate limited economic impact, ‘thus limiting the interest to change the existing legislations’ (Ibid. 8) The Green Paper makes a basic distinction between robots as physical entities (i.e. objects) and robots as a kind of agent, i.e. as quasi-subjectivities, and ventures into much more speculative and exploratory terrains with respect to the second. This tendency is clearly seen in the chapter on intellectual property rights (IPRs). Following a broad outline of existing laws on IPRs in robots as objects, the mode of analysis changes into the speculative possibility of ‘robot generated works’, whereby the machine is considered the subject, or author, of works worthy of IPR protections.17 The overarching obstacle however remains the issue of liability (De Cock Buning, 2012). No one would want to invest in robots when thereis great uncertainty about the question of who is liable for damages caused by their behaviours (Beck, 2009). Until now, robots have mainly been regarded as physical objects, but this conception might prove problematic with the advent of robots with self-adaptive and decision-making abilities (Boscarato, 2011). The Green Paper provides a gradient legal analysis in which different forms of (non-contractual) liability are tailored according to a robot’s increased level of capacity. When its behaviour is strictly determined in advance by the producer, it is not much different from a traditional machine, and subject to liability for defective products.18 The abilities of a robot capable «to move freely in the surrounding space» might be relevant for a next degree of robot capability. When a robot leaves the confines of its user or owner and causes harm or damage, it could be qualified as an ‘animal’ in the sense of article VI. 3:203 of the European Civil Code, and the custodian is liable. This qualification becomes especially relevant when robots are modelled after animals, as in the bio-mimetic approach of the RCC. The highest level of capacity is when robots possess decisional cognitive skills combined with the ability to learn new behaviours not intended by the producer or programmer. When this causes damage or harm, the robot may be qualified as a ‘child’ in the sense of article VI. 3:104 of the European Civil Code. The parents or guardians of the robot would be held liable if their supervision has been deficient. These legal concepts of animals and children can be taken as a ‘point of connection’ between the legal concept of artefact and the concept of the (human) person. According to the Green Paper, therefore, robot autonomy is a challenge to human-centric legal method and practice, stating how ‘strict differentiation between man and machine (“man-machine-dualism”) is no longer acceptable’ and that ‘man and machine should be considered simultaneously and their actions should be seen as cooperation’ (p. 58). The final chapter, exploring the possibility of ‘electronic personhood’, strengthens this impression. This is the idea that robots should be granted ‘a special legal category’ of personhood (Ibid. 61). Whereas self-admittedly speculative, it also claims that this chapter presents solutions to the problems previously outlined. Electronic personhood incorporates heterogeneous elements: first, it builds on notions of software agents in AI, and supplies this with the presumption from robotics that embodiment might make software agents more intelligent. Finally, it conceives of this hybrid entity based on an analogy with the legal personhood of corporations, and 17 Relevant, for instance, where the robot harvests information from users and environments, potentially to be stored in databases. 18 Directive 85/374/EC.

12

projects it as a solution to the problem of liability and responsibility, as a ‘bundling of capacities, material and financial responsibilities’ (Ibid. 61). 3.3.2. White Paper: putting humans and society first? The machine-centrism of the Green Paper comes more clearly into view when contrasted with the legally driven project, Robolaw. One of the main objectives of Robolaw was to investigate how emerging technologies in the field of robotics have a bearing on the content, meaning and setting of the law. Robolaw set out to research how existing legal categories and qualifications will be affected or will even have to be changed as a result of these technologies. This project starts from the observation that ‘the landscape of robolaw (in Europe and outside) is still quite fragmentary’, due to the fact that so far robots have been ‘more science fiction than actual fact’.19 This remark likens the approach quite literally to a mapping exercise of a barely chartered terrain. This recurring scenic metaphor of the landscape does not refer to robot technologies about to enter society, but to conglomerates of legal regulations dealing with robotics. The map would encompass a comparative analysis of legislation in several different legal systems and provide a comprehensive overview of legal themes that had only been studied in isolation before. The emerging robotics is presented as a-territorial and cross-boundaries phenomena, a kind of legal hybrid entities moving between the axiologies of different legal systems. They require a transnational interdisciplinary approach of ‘policy-related science’ in which the innovation processes and their regulation occur in an open space of cross-country discussions. In this way, Robolaw aims to ‘develop a specific European approach on its topic, characterized by core “European values” deriving from the main European sources of law’.20 The goal is to avoid robots becoming disruptive for societal structures and for their benefits to be fully exploited. Such a functional perspective ‘means to put rights – and fundamental rights as recognized by the European Union – first […]’. This ‘requires that the choice to adopt new regulation shall not mainly depend on the technological aspect of the application considered … Rather, the impact the single application may have on society and fundamental rights shall be guiding the choice’ (Bertolini & Palmerini 2014, p. 169).’ The Green Paper put autonomous robots centre-stage and explored how existing legal regulations can be disruptive obstacles for the extension of the assembly line into society. Contrary to this, the White Paper starts from society and explores how its existing legal safeguards like human rights will be affected by robotics. What in the Green Paper occurs as an ‘obstacle’ here occurs as a basic premise for assessing and implementing the role of machines in society. This focus on fundamental rights also puts the role of humans first, since ‘human rights are in fact an essential apparatus to deploy in order to promote and guarantee responsible advances in science and technology’. They allow the positioning of boundaries for the development of robotics by indexing the ‘intangible’ zones that cannot be infringed. They also function as ‘a test-bed for the desirability of robotic applications, since they can serve to identify the main goals and achievements expected by advancements in robotic research and industrial applications…’ (Ibid. 176-177). In this section we focussed on various mediations accomplished through the strategic and intellectual work performed by techno-epistemic networks. Whereas this work takes place in reference to an autonomous robot imaginary, each network remains dependent on its homebase or regime of practice, however, stretching its fundamental concepts towards new goals. This illustrates our analytic point (section 1), that the roadmap is used to operationalize imaginaries and as an aligning device across the networks. In addition to this analysis, we add that another crucial function of the roadmap is to systematically direct attention towards the 19 http://www.robolaw.eu/projectdetails.htm 20 http://www.robolaw.eu/RoboLaw_files/documents/Palmerini_Intro_RoboLaw_ERF.pdf


innovation object itself: it is essentially machine centric. This is already evident in the way the road is imagined, where the focus remains on the issues that are specific to future robots. In the legal fields, this came most clearly to the fore in dealing with intellectual property rights for autonomous creations, liability for harm caused by autonomous action, and the introduction of a fully new concept of electronic personhood. The spotlight is on the autonomous robot, thus shifting the meaning of agency from humans to robots. Here, the roadmap is also a level shifting tool. It is a central node that connects the object of innovation (level 1) – the autonomous robot in society - to the networks or institutions that take care of it and pave its way (level 2). This work is primarily performed by industry. The efforts of roboticists are also machine-centric but they are so mainly in a scientific sense, ignoring the institutional, regulatory and relational ordering mechanisms provided by industry, or what we could term the innovation incubator. We can clearly see this link between innovation trajectory and institutional incubator in figure 1.

Figure 2: The road-mapping process in the PPP21

This second, social-institutional level of control, implicit to but still somewhat undercommunicated in the roadmap here, is a key theme in STS perspectives on co-production, insofar as these studies typically shift attention away from the scientific or innovation object (level 1) to the social and institutional relations in which the object becomes embedded (level 2), and to the very embedding process that makes the level-shifting activities visible. 22 We will now draw upon this theme to address the co-production of science, industry, law, and politics in the making of partnerships and robot autonomy. 4. Public Realignments in co-producing the partnership When assessing the innovation object (level 1), the developments we have described could be seen as failures. The main strategy of roadmapping is to overcome the ‘gaps’ between industry and academic science, between engineering and law, and to ‘remove the obstacles’ separating robotics and society. Yet, the technical gaps persist, as reported in technical reports and communicated to us by practising roboticists. 23 Furthermore, biomimetic approaches remain riddled with shortcomings of their own, and the underlying assumptions about (‘strong’) embodied intelligence have been criticized as unrealistic by those adhering to soft AI (Suchman 2007, Dreyfus 2007). On the legal side, ambiguity and controversy persist over how to accommodate robots within human-centric legal frameworks in the service of wider society. We have learned from lawyers that law comes under pressure from industry, science and engineering (Rommetveit, et al 2015). We also learned that pre-existing epistemic differences that hinder collaboration and mutual understandings between lawyers and engineers are still deeply entrenched. Yet, when we view these developments as co-productions of science and societal order (Shapin and Schaeffer 1985), success is not merely spelled out in scientific or technical terms. The making of a Public-Private Partnership is the main measure through which the European industry can influence the societal and market relations needed to promote its innovation 21 http://sparc-robotics.net/roadmap/ 22 The material semiotics in actor-network theory has highlighted these techniques of shifting in and shifting out of situational frames of reference in studying action, or even to shift action down in delegating it to material things (Latour, 1999). 23 Perhaps the largest gap was between the political vision and technical capabilities. One roboticist told us how the European programme to address problems of ageing societies through robots is” a disaster based in erroneous ideas of what robotics can and cannot deliver” (Rommetveit et al, 2015).

14

object, hence also its own expansion. The various gaps and challenges identified in the roadmap become the means toward a different sort of end, namely to organize institutionally and partner with the public at the European governance level, where the industry has traditionally asserted its authority quite freely. By raising the issue to the European level, the industry platform however runs into new actors (layers, ethicists, social researchers) of different size and scale. It is precisely at the intersections where the public side of the partnership enters the picture, that things are further complexified, resulting in ongoing tensions and realignments of frames, directions and paths of innovation. The PPP literature mentions that an important task for public authorities is instituting policies, laws and regulations that make PPP structures possible. 24 In our case, the “‘Civil Law Rules on Robotics’ Resolution draft by the European Parliament constitutes a sector-specific bill that does not pave the way for the PPP itself, but for its innovation object: the autonomous robot in society. For the European Union, this re-routing (through robots and PPP) is part of its ongoing and perilous Leviathan-building voyage. Framing developments as of ‘European and humanistic values’, creates a direction for self-constructing the European identity: on the one hand by embracing the techno-economic imaginary of autonomous robots, on the other hand by seeking to shape the imaginary according to European value-laden logic and modes of operation. This pathway has its own barriers. Confronted with the plethora of different national legislations developed by the Member States ‘these discrepancies are expected to create obstacles for an effective development of robotics. Due to the fact that this technology has cross-border implications, the best legislative option is a European one’ (European Parliament 2016, 21). The subsidiarity principle is here raised to authorize the Union’s legislative road-sweeper for taking things to the European level. Hence, the legislators mirror and embrace the technical problem-solving strategy of the industry roadmap, at the same time re-creating it in their own image. We now describe some ways in which this entanglement is manifested. Firstly, the framing of research agendas (in expert groups) is now placed within the firm grasp of the PPP. During our workshops, robotic scientists lamented how they now have to ‘follow the money’ to pursue their research. Whereas the PPP states that its main goal is ‘to connect the science base to the marketplace, a connection that ultimately benefits society’, 25 the societal angle introduced by the Commission is slightly more ample, and must by necessity frame developments as ‘addressing societal challenges.26 The Parliament Resolution reinstates the generally beneficial role of robots for society, while also highlighting their potential for significant negative impacts (on work, privacy, human relations, etc.). Yet, if introduced in the right way, robots hold the promise of ‘virtually unbounded prosperity’ (p. 3). The negative concerns are strongly conceived in terms of ‘soft impacts’ on human rights and societal values. Importantly, these issues have to be addressed through an elaborate new ethical framework that comprises a code of conduct for engineers, a code for research ethics committees, model licences for designers and users, and ‘privacy by design’. Furthermore, this ethical approach is institutionally fortified by the suggestion of a new ethical expert body to advise EU institutions about potential issues and systematic challenges of robotics and AI. Such procedural strategies do not break free of the speculative frame of the robotic imaginary. The Resolution is noteworthy for the fact that its introductory clause cites works of science 24 See for instance: (World Bank Group, 2014), (EPEC, 2016). 25 http://sparc-robotics.net/implementation/ 26 A robotics researcher argued that “we will see all sorts of tensions arise between the different actors who are now involved in taking the research agendas forward” (Rommetveit et al, 2015). Whereas the Commission insists on the implementation of so-called Responsible Research and Innovation, industry responds by setting up its own ELS Working Group. It does so in seeming neglect of a previous roboethics roadmap that took an explicitly human-centric approach (Veruggio 2006).


fiction, and this futuring is paralleled by another novelty in the legislative universe. The Parliament’s Working Group27 commissioned the application of STOA’s newly developed Scientific Foresight in the European Parliament methodology to the field of autonomous robots.28 This was one of the first applications of the method by the Parliament, and perhaps a first step to institutionalize the legislative process.29 Compared to the roadmap methodology used by the euRobotics technology platform, this approach provides different futuring imaginations, and operationalizes differently the imaginary of autonomous robots in society (even though the imaginary itself is generally accepted). The methodology provides a different entry into the imagined ‘landscape’ sketched by its metaphorical concepts, with subsequent steps of ‘horizon scanning’, ‘360º envisioning’, ‘scenario development’ and ‘legislative back casting’. On this basis, alternative pathways are developed for reaching or avoiding scenarios that start from the current state of legislation. This then is an alternative way to articulate and make actionable robot futures, even though it shares with the roadmap the underlying narrative of economic and industrial growth. Hence, similar dynamics to those observed in the techno-epistemic networks and the roadmap are at work in the legislative universe: first, by the sheer hybridisation and complexification of the issues; second, by the increased emphasis on possible futures to stabilize meaning-making and action in the present (cf. Matsuzaki and Hindemann 2015). Concrete impacts of the legal networks on the draft proposal can also be traced. Following the Green Paper’s top-down recommendations, there was a call to elaborate criteria for an ‘“own intellectual creation” for copyrightable works produced by computers or robots’ (European Parliament 2016, 8). Furthermore, among several novel creations within this legislative universe, liability received the main attention, issued in the ‘possibility of an obligatory insurance scheme’ which would be associated with the ‘status of electronic persons with specific rights and obligations’ for the most ‘sophisticated autonomous robots’ (Ibid. 12). As for the White paper, we also observe counter articulations entering into the legislative process. The proposal supplements its top-down approach with a call for specific rules for sectoral types of robots like autonomous vehicles, care robots, medical robots, human enhancements and drones (in line with the White paper’s bottom-up approach). More importantly, the innovation trajectory becomes somewhat rerouted: ‘The road from the industrial sector to the civil society environment obliges a different approach on these technologies, […] ensuring that a set of core fundamental values is translated into every stage of contact between robots, AI and humans.’ (Ibid. 20). Fundamental values become the measure here for robotic innovation. To summarize, we can observe a public realignment of innovation goals to solve societal challenges, a reposition of the milestones along the trajectory of progressing core fundamental values rather than simply realizing a technological potential like autonomous self-learning of robots, and a remodulation of futuring according to foresight studies instead of mere roadmapping. On the one hand, the legislative process incorporates the imaginations of the techno-epistemic networks; on the other hand, we also see that this process seeks to break free by instituting its own futuring methodology. In so doing, the legislative process itself gradually incorporates more speculative approaches. The articulation of possible futures is enrolled as a means to stabilize expectations in the present (Matsuzaki and Hindemann 2015) and as a basis to inform current legislative initiatives. Simultaneously, speculative ethics are more and more institutionalized as a mechanism to evaluate increasingly far-sighted innovation projections, sometimes to the detriment of normative issues that are more nearby (Nordmann & Rip 27 http://www.europarl.europa.eu/committees/en/juri/subject-files.html?id=20150504CDT00301 28 European Parliament Science and Technology Options Assessment: http://www.europarl.europa.eu/stoa/ 29 “If adopted, the report is expected to act as a basis for future legislative activities at the EU level.” https://epthinktank.eu/2016/06/30/how-will-robots-change-our-lives-new-study-on-the-ethics-of-cyberphysical-systems-published/

16

2009). This speculative turn testifies to the inherent and increasing uncertainty, unpredictability and complexity of the innovation process. The ‘gaps’ take on new dimensions and dynamics, as the most unstable and unpredictable actor - the autonomous robot - is introduced to the community of European institutions. Whether intended or not, the PPP is an instantiation of what Pellizzoni has termed ‘governance through disorder’ since Parliament is also ‘attuned to the productive regard on indeterminacy and speculation that constitutes a trademark feature of neoliberal governmentality’ (2015, 171). Conclusions Our goal in writing this paper was to say something about the broader co-productions of ‘science’ and ‘society’ within which recent and ongoing developments in robotics occur. With regard to the expanding role of these developments we depart from the notion of humanmachine configurations (and ‘soft AI’ more generally), by including the institutional (and instituting) imaginations: e.g. concepts like ‘electronic personhood’ and ‘sentient machines’, now enacted within the new techno-epistemic networks extended to public governing institutions. Taking this perspective is important, we claim, because it draws attention to how these networks unite and build epistemic, institutional and, perhaps political legitimacy in new and more intense ways. The actors within them, implicitly or explicitly, take on an identity of being co-producers: industry has opinions on autonomy (previously almost exclusively the prerogative of philosophers and ethicists); lawyers (and ethicists) take a position not only on how to guide and steer robots into society, but also on the properties that robots should possess; scientists aim to build ethics, law and ‘society’ into their (imagined) discovery engine. Put this way, we have a broader perspective on the processes of constructing the PPP as an innovation incubator. We have highlighted the crucial role of imaginaries and futuring which is manifested, for example, in a roadmap mobilized by industry networks to order and steer the efforts of science and law (allegedly autonomous sources of legitimacy and world-making). The mobilization of this strategic tool has served to push perceptions of gaps, problems and obstacles in the making of autonomous robots further up the institutional food chain. It has triggered response by public legislative actors at the European level, who have introduced alternative human-centric and ‘societal’ perspectives. Yet the consequences of fast-tracking existing modes of problem-solving to high institutional levels of centralized organisations, also regarding the perplexity about autonomous robots, seem to introduce new levels of hyperoptic speculation and governance through disorder. The robotics imaginary is predicated on fascination and wilfulness, enabling mediations across networks and institutions. When ‘gaps’ and ‘obstacles’ become means to political and market-making ends, their character and purpose change. Hence, both scientists and lawyers complain that their professional practices are under pressure, having to bend their contributions towards ends such as ‘autonomous robots for care and companionship’ and ‘saving Europe’. From the perspective of the EU and the PPP, the mobilization of the concept of societal challenges, allows for further breakdown of the boundaries between countries, sectors and disciplines. Addressing societal challenges has its own set of challenges. 30 Large industry actors progressively attain a suffocating embrace of these hybrid processes by which practitioners of different backgrounds come together to anticipate future problems. This highlights the necessity to come back to the present to attend to more immediate challenges. There is an increasing need for establishing some breathing space for law and science in their work of co-production, so each practice can attain a certain disciplinary autonomy within the broader constellation.31 Some room for manoeuvre would allow the techno-epistemic 30 (Kuhlman and Rip 2014). 31 On this reformulation of autonomy, see (Latour, 2004, 119-120, 140).


networks to render their inner workings more faithful to the actors’ own versions and questions of what matters in the given situation, 32 their own framing of issues to be tackled, and their own proposals for how other partnering practices can contribute to finding solutions. This might well require exploring the idea of techno-epistemic checks & balances in the innovation ecology of practice.

Acknowledgements: This paper is made possible by the research project (…) co-funded by the European Union within the Seventh Framework Program (FP7) under Grant Nr. (…), and by a research mandate of the Research Foundation - Flanders (FWO) under Grant nr. (…). We further wish to express our gratitude for the valuable contributions of the participants to the expert workshop on (…) in (…) on (…), where preliminary results of this research were presented and discussed. Bibliography Asimov, I. (2004) Robot Visions (short stories and essays, 1940-1989). New York: Roc. Assink, M. (2006) Inhibitors of disruptive innovation capability: a conceptual model. European Journal of Innovation Management 9(2): 215-233. Barry, A., & Slater, D. (2002) Introduction: the technological economy. Economy and Society, 31: 175-193. Beck, S. (2009) Grundlegende Fragen zum rechtlichen Umgang mit der Robotik. JR, 6, 225230. Bekey, G. A. (2005) Autonomous Robots. From Biological Inspiration to Implementation and Control. Massachusettes. MA: The MIT Press. Bertolini, A., & Palmerini, E. (2014) Regulating Robotics: A Challenge for Europe. In: Legal Affairs Committee (Ed.), UPCOMING ISSUES OF EU LAW. Brussels: European Parliament. Bischoff, R., Guhl, T., Wendel, A., Khatami, F., Bruyninckx, H., Lafrenz, R. (2010) euRobotics– Shaping the future of European robotics. ISR/ROBOTIK: 728-735. Boscarato, C. (2011) Who is responsibe for a robot's actions? In B. van den Berg & L. Klaming (Eds.) Technologies on the Stand. Legal and Ethical Questions in Neuroscience and Robotics. Nijmegen: Wolf Legal Publishers, 383-402. Bowker, G. C. (1993) How to Be Universal: Some Cybernetic Strategies, 1943 – 1970. Social Studies of Science, 23: 107-127. Brooks, R.A. (1991) Intelligence Without Reason. Proceedings of the Twelfth International Joint Conference on Artificial Intelligence (pp. 569-595). San Mateo, CA: Morgan Kaufmann. Brynjolfsson, E. and McAffee, A. (2011) Race Against the Machine: How the Digital Revolution is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy. Lexington, MA: Digital Frontier Press. Callon M (2007) What Does It Mean to Say That Economics Is Performative? In MacKenzie D, Muniesa F & Siu L (eds.), Do Economists Make Markets? On the Performativity of Economics. Princeton: Princeton University Press, 311-357. 32 (Stengers, 2005).

18

Dallaway J.L., Jackson R.D, Timmers, P.H.A. (1995) PHA: Rehabilitation robotics in Europe, IEEE Trans Rehab Eng, 3: 35–45. Dario, P. (2012) Robot Companions for Citizens: A new and ambitious European initiative. Presentation at the First Karlsruhe Forum on Anthropomatics and Robotics, Karlsruher Institut für Technologie (KIT). Retrieved from: http://www.youtube.com/watch? v=aZHvGBjgoqY Dario, P. Dillmann, R. Christensen, H. (eds.) (2004) EURON Research Roadmaps: Key Area 1 on "Research Coordination" – KA1. Dario, P., Verschure, P. F. M. J., Prescott, T., Cheng, G., Sandini, G., Cingolani, R. and MacNeil, S. (2011) Robot companions for citizens. Procedia Computer Science 7: 47-51. Dautenhahn, K. (2007) Methodology & Themes of Human-Robot Interaction: A Growing Research Field. International Journal of Advanced Robotic Systems, 103-108. De Cock Buning, M., Belder, L., & de Bruin, R. (2012) Mapping the Legal Framework for the introduction into Society of Robots as Autonomous Intelligent Systems. In S. Muller (Ed.) The Law of the Future and the Future of Law. The Hague: HiiL. Dreyfus, H. L. (1992) What Computers Still Can't Do: A Critique of Artificial Reason. MIT Press. Dreyfus, H. L. (2007) Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian. Philosophical Psychology, 20: 247-68. Dupuy, J.-P. (2008) On the Origins of Cognitive Science. The Mechanization of the Mind. Cambridge: The MIT Press. EPEC (2016) The Guide to Guidance. How to Prepare, Procure and Deliver PPP Projects. Retrieved from: www.eib.org/epec/g2g/index.htm. euRobotics (2013) Robotics 2020. Strategic Research Agenda for Robotics in Europe.Retrieved from: https://www.kowi.de/Portaldata/2/Resources/horizon2020/coop/Robotics-2020-SRADraft.pdf EUROP (2009) Robotics Visions to 2020 and Beyond - The Strategic Research Agenda for Robotics in Europe. Coordination Action for Robotics in Europe (CARE). European Parliament (2016) Draft Report with recommendations to the Commission on Civil Law Rules on Robotics No. 2015/2103(INL). Brussels: European Parliament, Committee on Legal Affairs. Floridi, L., Sanders, J.W. (2004) On the morality of artificial agents. Minds and Machines. 14(3), 349-379. Franklin, S., Graesser, A.C. (1997) Is it an Agent, or just a Program?: A taxonomy for Autonomous Agents. In: Intelligent Agents III. Berlin: Springer. Guhl, T., Zhang, Z. (2011) euRobotics: The European Robotics Initiative. Report on identified gaps. Deliverable D1.1.1. Retrieved from: http://eurobotics-project.eu/cms/upload/euRobotics_Deliverable_D1.1.1_Report_on_gaps.pdf Haas, P. H. (1992) Knowledge, Power, and International Policy Coordination. International Organization, 46, 1-35. Haselager, W. F. G. (2005) Robotics, philosophy and the problems of autonomy. Pragmatics & Cognition 13: 515-532. http://www.roboethics.org/atelier2006/docs/ROBOETHICS%20ROADMAP%20Rel2.1.1.pdf.


ISTAG (2004) Grand Challenges in the Evolution of the Information Society. Report EU Commission. Luxemburg. Jasanoff, S. (2004) States of Knowledge. The co-production of science and social order. London and New York: Routledge. Jasanoff, S., S.-Y. Kim, (2009) Containing the atom: sociotechnical imaginaries and nuclear power in the United States and South Korea. Minerva. 47(2).: 119-146. Jessop, B. (2009) Cultural political economy and critical policy studies. Critical Policy Studies 3: 336-356. Kritikos, M. (2016) Legal and ethical reflections concerning robotics. Scientific Foresight Unit. European Parliament. Kuhlmann, S. & Rip, A. (2014) The challenge of addressing Grand Challenges. Technical Report. Retrieved from https://ec.europa.eu/research/innovation-union/pdf/expertgroups/The_challenge_of_addressing_Grand_Challenges.pdf Latour, B. (1999) Pandora’s Hope. Essays on the Reality of the Science Studies. Cambridge: Harvard University Press. Latour, B. (2004) Politics of Nature. London: Harvard University Press. Leenes, R., Lucivero, F. (2014) Laws on Robots, Laws by Robots, Laws in Robots: Regulating Robot Behaviour by Design. Law, Innovation and Technology 6: 193-220. Leroux, C., & Labruto, R. (2012) Ethical Legal and Societal issues in robotics (No. D3.2.1). euRobotics. Leroux, C., Labruto, R., Boscarato, C., et al. (2012) Suggestion for a green paper on legal issues in robotics. euRobotics. Levidow, L. (2013) EU criteria for sustainable biofuels: Accounting for carbon, depoliticising plunder. Geoforum, 44, 211–223. Matsuzaki, H., Hindemann, G. (2016) The autonomy-safety-paradox of service robotics in Europe and Japan: a comparative analysis. AI & Society 31: 501–517. Nordman, A., Rip, A. (2009) Mind the gap revisited. Nature Nanotechnology 4: 273-274. Palmerini, E., Azzarri, F., & Battaglia, F. (2014) Guidelines on Regulating Robotics (No. D6.2). Robolaw Project. Retrieved from http://www.robolaw.eu. Pellizzoni, L. (2015) Ontological Politics in a Disposable World. The New Mastery of Nature. Franham: Ashgate. RCC Consortium (2012) MANIFESTO. More than machines. Robot Companions for Citizens. Retrieved from: http://www.robotcompanions.eu. Reding, V. (2005) Advanced Robotics: why Europe needs to catch up in this strategic sector. Presentation at official launch of the Technology Platform EUROP. Retrieved from: http://europa.eu/rapid/press-release_SPEECH-05-583_en.htm?locale=en. Retrieved from http://www.euroboticsproject.eu/cms/upload/PDF/euRobotics_Deliverable_D.3.2.1_ELS_IssuesInRobotics.pdf Retrieved from https://www.unipv-lawtech.eu/files/euRobotics-legal-issues-in-roboticsDRAFT_6j6ryjyp.pdf Retrieved from: http://www.cas.kth.se/euron/euron-deliverables/ka1-3-Roadmap.pdf Riskin, J. (Ed. 2007) Genesis Redux. Essays in the History and Philosophy of Artificial Life. Chicago and London: Chicago University Press.

20

Rommetveit, K. Gunnarsdóttir, K., van Dijk, N., Smits, M. (2015) “Making robotic autonomy through science and law?” EPINET project workshop report. Available at: http://neicts.lancs.ac.uk/pdf/EPINET-WP4-EventReport.pdf. Šabanovic, S. (2010) Robots in Society, Society in Robots. Mutual Shaping of Society and Technology as a Framework for Social Robot Design. Int J Soc Robot, 2, 439–450. Scott, JC (1998) Seeing Like a State. How Certain Schemes to Improve the Human Condition Have Failed. New Haven and London: Yale University Press. Shapin, S. & S. Schaeffer (1985) Leviathan and the Air Pump. Princeton NJ: Princeton Univ. Press. Stanton, A, & Morris, J. (2008) WALL · E. Walt Disney Pictures, Pixar Animation Studios. Stengers, I. (2005) Introductory notes on an ecology of practices. Cultural Studies Review, 11, 183–196. Suchman, L. A. (2007) Human-Machine Reconfigurations: Plans and Situated Actions (2nd ed.). Cambridge: Cambridge University Press. UN Economic Commission for Europe (2005) UN World Robotics: Statistics, Market Analysis, Forecasts, Case Studies and Profitability of Robot Investment. Executive summary. Retrieved from: http://www.unece.org/fileadmin/DAM/press/pr2005/PR05_stat03e_test1.htm. Van Woensel, L. (2016) Ethical Aspects of Cyber-Physical Systems. Scientific Foresight Unit. European Parliament. Van Woensel, L., & Vrščaj, D. (2015) Towards Scientific Foresight in the European Parliament. Scientific Foresight Unit. European Parliament. Veruggio, G. (2006) The EURON Roboethics Roadmap. Retrieved from Wallach, W. and Allen, C. (2009) Moral Machines: Teaching Robots Right from Wrong. New York: Oxford University Press. World Bank Group. (2014) Public-Private Partnerships Reference Guide. Version 2.0. Retrieved from: http://documents.worldbank.org/curated/en/600511468336720455/Public-privatepartnerships-reference-guide-version-2-0.


Figure 1

Figure 2