Beyond the pilot paradox How the success conditions of pilots also hinder their up-scaling in climate governance Paper for the workshop “Beyond experiments: Understanding how climate governance innovations become embedded” (WG2) 25th – 27th April, 2016, Brighton Draft – please do not cite Arwin van Buuren*, Heleen Vreugdenhil~, Jitske Van Popering Verkerk*, Gerald Jan Ellen~, Corniel van Leeuwen*, Bas Breman^. *Erasmus University Rotterdam; ~Deltares; ^Alterra, Wageningen University Dr. Arwin van Buuren, corresponding author Associate Professor of Public Administration Erasmus University Rotterdam Room T17-13, P.O. Box 1738 3000 DR Rotterdam, the Netherlands.
[email protected] 1. Introduction In many countries, pilots are frequently used to explore and experiment novel ideas and innovative approaches of climate change adaptation (Kivimaa et al. 2015). Pilots are perceived and often acknowledged as an easy and safe manner to explore new pathways for fresh water supply, flood protection or water retention, to find out whether new solutions or working methods can help to deal with the consequences of climate change, before they are embedded in formal policies or implementation programs. Pilots are usually organized at the boundaries of the formal policy organizations, or even outside the dominant policy regime. Within transition studies pilots are often defined as policy niches: ‘fringe spaces for currently dominant technologies or alternatives to currently methods of governance’ (Kivimaa et al. 2015). They function as a safe area to develop novel ways of thinking and allow for exploring creative methods and mobilizing new sources of knowledge. This distance is necessary to give the pilot some room for innovation and exploration (Van Buuren & Loorbach, 2009). Furthermore, a pilot is often provided with additional resources for exploration in terms of time, money and expertise to explore alternative strategies and to develop new knowledge and insights. This freedom can consist of a reduced pressure to achieve certain policy targets while rather focusing on developing new insights, and a more flexible attitude towards permits and regulation. Another important characteristic of pilot projects is the relative small scale (both geographically and temporally) at which pilots are implemented. A small scale reduces risks due to limited impact and limited loss of resources. Combined with an attitude that ‘failure’ is allowed, a small scale often makes it easier for stakeholders to participate (Vreugdenhil et al. 2010). At the same time none of these characteristics 1
is written in stone, leading to a large variety in the design of pilot projects and related level of risks and willingness to participate. At the same time this distance, freedom, additional resources and small scale can hinder the broader uptake of the pilots’ results and outcomes by the existing policy regime (Vreugdenhil, 2010). The distance is sometimes designed at the start of the pilot. But the distance could also arise during the pilot, when actors within the pilot (both organizations and professionals) rapidly learn and develop new insights, while the different formal policy organizations and their practices are less connected and adjust themselves much slower or even oppose to the pilots’ outcomes. The policy freedom and a small scale may lead to non-representative or insufficient results. For example a water system may respond differently if the measure would be larger or certain effects occur only long after the piloting period. Consequently, results can be considered as insufficient proof for full implementation of the piloted approach. Moreover, the argument of limited representativeness can even be used deliberatively to prevent further implementation. The ultimate result of pilots in terms of generating regime change is due to these reasons often quite low. A deeper cause is the limited attention for the question how to anchor pilots in the existing policy regime. In addition, innovative pilots may very well contradict the ‘good virtues’ of the public policy domain (Duijn, 2009), such as reliability, stability, prudence, etc, potentially harming the functioning of formal policy organizations. This brings us to the “paradox of pilots”: the conditions that are necessary to give a pilot room to experiment and to learn (and thus come to innovative successful results), also seem to constitute the main barriers for the broader uptake and translation of its results in real policy change. The results do not fit within the policy regime and due to the distance between pilot and policy regime, the reception of the results is not prepared at all. In this paper we explore this potential paradox. We bring together a couple of climate adaptation pilots we evaluated during the last couple of years. Within these cases we analyse whether and how the paradox is apparent and thus whether the pilot conditions and design are problematic for the embedding of its results. We conclude our paper with presenting design and initiation strategies developed in the cases to deal with this paradox. This paper is based upon a secondary comparative case study. We rely upon extensive case descriptions (based upon interviews, focus groups and document study) produced by ourselves and reinterpret this material from the perspective of this paper. The reminder of the paper is structured as follows. In the next two sections we explore the conditions for successful experimentation in pilots and the necessary conditions for policy change and learning in existing policy regimes. In section four we confront these two sets of conditions to problematize the pilot paradox. Then we present the methodology of our paper. In section six we present our cases and in section 7 we analyse how the paradox is apparent in these cases. In the final section we answer the question to what extent we can actually speak of a paradox and with what strategies there can be effectively dealt with this paradox. 2. Conditions for successful pilots (“innovative outcomes”) There are many answers possible to the question: what makes a pilot successful? Success is a very subjective term. What does it mean and to whom? A first assumption is often whether the innovation 2
‘works’, and counts as ‘evidence’ (Cabinet Office 2004, Nutley 2002). In this paper we make a distinction between two aspects of success: internal success and external success. This is related to the often made distinction between output and outcome or impact. In section 2 we focus upon internal success while in section 3 external success is put central. 2.1 What counts for a successful pilot? The internal success of a pilot has to do with the extent to which the pilot successfully realize its main ambition. Often this ambition is related to testing innovative approaches or concepts and developing new insights and knowledge. Subsequently, success can also be considered as usable knowledge being developed at the basis of which new decisions can be made. Internal success of a pilot can also be more in terms of collaboration, the establishment of new/ positive cooperation between stakeholders. External success is the extent to which the knowledge or networks developed in the pilot project are being used for new initiatives, both on project and policy level. 2.2 Conditions for success So, depending on the objectives, conditions for a pilot’s success can be identified. For successful pilots a couple of conditions can be found in the literature. a. Being a “safe haven” A first condition for a successful pilot is its status as a safe heaven, at a distance from the heat of the kitchen. This distance is important because it is necessary for a pilot to have some room for experimentation and manoeuvre which is not allowed for in the regime itself. Political support is often a crucial element to safeguard this condition and to prevent for interventions from the regime within the exploring process of the pilot. Within the literature this condition is often labelled as a protective space (Smith & Raven, 2012). b. Enough resources for exploration A second crucial condition for a pilot has to do with the availability of enough resources to fulfil the experiment. These resources have to do with organizational resources, but also with sufficient expertise and means to generate knowledge, with time for undertaking experiments, monitoring and analysis, and also with attention and manpower from the participating organizations or units. c. Competent participants Pilots can be effective places for mutual learning and frame reflection. That presupposes that the actors involved in the pilot are open for new knowledge and willing to learn. Innovative processes often attract the more entrepreneurial persons, persons who are more able to span boundaries between different perspectives and ways of knowing. Such a coalition of the willing is an important success condition for pilot projects. d. Principled engagement and social learning Pilots are often collaborative efforts, especially when it comes about complex governance issues. For successful collaborative innovation it is essential that actors are able to come to frame reflection and 3
mutual learning, via a process of principled engagement (Sorensen & Torfing, 2012). Conditional for such a learning process is that actors are able to overcome their predefined boundary judgments and look beyond their organizational beliefs. Pilots need boundary spanners: people who are able to bridge different problem perceptions and who are willing to change their own normative and cognitive frames. e. Limited sale Often, a small scale is the decisive factor for a pilot to be implemented. Small scale often implies limited risks, both for the physical system and financially, makes the system boundaries of the pilot clear and allows for precise outcomes. Small scale refers to both space and time. Note that ‘small’ is a relative notion: depending on the intentions ‘small’ can refer for example to a municipal level if the intention is a national roll-out, or to a single floodplain if the idea is to be implemented at a river branch. Something similar holds for the limitation in time: limiting time gives an indication of when results can be expected that should be sufficient for understanding the functioning of the innovation. A risk however, is that the scale at which is piloted and monitored is insufficient to develop insights: some effects may only appear many years later, on for example on a river stretch level and not at a floodplain level. It is important to note that several conditions mentioned above can be strengthened during the course of a pilot process: a pilot that gains some status is often able to attract additional resources for further exploration. And within a pilot in which there is initial willingness for mutual learning, this can be reinforced by the social capital developed during the course of the explorations. 3. Conditions for learning from pilots and upscaling (“policy change”) 3.1 What counts for successful uptake? As explained above, the wider impact of a pilot can be seen as its “external success”. We can define this success as the matter to which the output of a pilot also results into more enduring change within the policy regime (by changing policies, visions, rules and standards et cetera) and whether its results are also applied in other contexts and upscaled to a larger area, time span or set of problems. This also holds for ‘negative’ results, meaning that if the pilots demonstrates that a certain solution does not work at all or under certain conditions, policies that already anticipated for the solutions are being altered, and that the solution is not being implemented elsewhere. Another form of success is if on the basis of the pilot the new questions that arise are being addressed in a follow-up pilot. 3.2 Conditions for successful uptake The successful diffusion of pilot results depends upon a variety of factors. We summarize them into four categories. a. Normative congruence: sufficient proximity between pilot and home bases For the successful uptake of pilot results, it is important that pilot participants keep in mind what the dominant values are in their home bases. Innovative ideas only succeed when they sufficiently correspond with existing insights and knowledge of actors (Van Buuren & Eshuis, 2014). There is something like an optimal cognitive distance between new ideas and standard routines. Too much distance in terms of 4
differences between the shared beliefs developed within the pilot and the fundamental values and beliefs of the regime hinders the uptake of innovations. One step further is that the ideas not only fit the organisation, but also that (some) participants act as ambassadors: they stand for the innovation, are convinced that it brings large improvement compared to current practice and actively promote it. b. Organizational appropriateness of pilot outcomes In line with the first condition, it is important that there is a sufficient degree of compatibility of the outcomes of a pilot with the standard operation procedures within the current policy regime. The regime has to be able to handle and apply the solution and to translate it into its existing routines and rules of the game. c. External representativeness of pilot composition For the wider uptake of pilot results it is important that the future users are attached to the experiment. Sufficient interactions between niche and regime are necessary to safeguard that the regime will value the pilot as a legitimate source of knowledge (Ingram et al. 2015). Such a strategy can be done by safeguarding enough interaction between pilot participants and the outer world, to engage representatives of the various home bases in the same learning process as the pilot process is in (Smink et al. 2015). d. Pilot embedding to assure a renewed logic of appropriateness Studies that focus upon the question: how to come beyond experimentation, emphasize the importance to look at how to embed pilots or experiments into its wider environment. An important component of such a strategy, has to do with assuring that the outcomes of a pilot can be easily embedded into the daily routines of the receiving organizations, their logic of doing things. That requires that the question is answered what is necessary to mainstream the outcomes of a pilot and to replicate it in normal processes of planning and decision-making (Kivimaa et al., forthc.). Another important design component has to do with organizing attention on the question how to make the pilot an instrument to achieve policy change? Consequently, is the design of the pilot such that it is in continuous interaction with the policy, stakeholders and areas where it should land? e. Sufficient system understanding and trust in results Important for uptake of the outcomes of the pilot is the perception that its results are reliable, representative and useful. Hence, this places quality criteria to the pilot project. Did the design fit the questions that had to be addressed, was the scale (both in time and space) appropriate to answer the questions, and were the ‘right’ questions addressed? All of these elements can be used to both follow up the pilot or to find a way out. A pilot can for example easily be set aside by classifying results as ‘nonrepresentative’. For further uptake of the pilot, the pilot results have to be trusted and considered to contribute sufficiently to formal policy objectives. 4. The pilot paradox Within the world of international aid and development it is a common expression: “pilots never fail, they (also) never scale”. Although pilots often succeed in testing innovative solutions and producing new 5
knowledge and often contribute to what participants learn, they are far less successful in generating enduring policy change. A successful pilot (which delivers useful knowledge and results in more knowledgeable participants) is anything but a recipe for successful diffusion and upscaling. To our mind part of the explanation for this has to do with the contradictory conditions for “internal” and “external” success. In other words: the conditions that contribute to a successful pilot are also the main barriers for a successful uptake. In table x we summarize the two sides of the coin. Element Position of the pilot Resource distribution
Conditions for successful pilots At a distance from home bases (freedom to explore novel ideas) Additional resources for the pilot to enable creativity and exploration.
Participants
Coalition of (willing) boundary spanners.
Process design
Learning environment, tailor-made collaborative process design.
Conditions for uptake Keeping connected: conscious strategy to create normative congruence. Solutions fit within the existing system of resource-distribution and contribute to organizational aims of efficiency and risk reduction. Representativeness of involved actors from all relevant disciplines and stakes of the future implementation arena Results ready for mainstreaming and broader embedding. Focus on where the results have to land. Sufficient system understanding; outcomes considered representative and of high quality
Project design
Limited scale to reduce risks and (financial) impacts, high quality (shared) monitoring and analysis Table 1. The pilot paradox
For our analysis we have conceptualized the various elements of the paradox, see table 2. Element
Conditions for successful pilots
Conditions for uptake
Position of the pilot
-
-
Resource distribution
-
Participants
-
Process design
-
Degrees of independence (with regard to content, way of working, rules) of the principal organizations More flexible applications of law and regulations (or less strict reinforcement) Additional budget; expertise; partners; time Political attention / pressure to ‘score’ with the pilot Open: 1) easy communication and 2) openness about interests etc. Act as boundary spanners Willing (able) to experiment / take risks New roles for (new) partners New forms of collaboration
-
Reporting to the own organisation on different levels (political, strategic, tactical and operational) Conscious strategy to convince home bases about fit with existing values of pilot results. Solutions help the various organizations to realize their core values Solutions are applicable within the existing resource distribution. Potential criticasters from participating organizations are included in process Future users are informed and involved Participants act as ambassadors. Possibility of application Identify potential future application areas, identify questions, ‘translate’ results
6
Project Design
-
Degrees of freedom for organizing the own process Limited scale (time and space) Well defined questions and limitations High quality monitoring
-
Connecting with potential future users, identify their questions and needs Sufficient trust in outcomes Sufficient insight in how the system which is the object of the pilot works. Representativeness of results
Table 2. Conceptualization of the conditions for success 5. Methodology For this paper we deliberately selected eleven pilots in the domain of climate adaptation we already analysed. We thus conducted a secondary analysis of existing cases. In a couple of cases we were involved as a researcher or expert and for other cases we were asked to conduct formal evaluation studies We deliberately selected our cases in relation to our main aim, to explore the paradoxical relation between success conditions for pilots and for the broader uptake of their results. We used two main criteria for our case selection: - The selected pilots have to represent the main issues regarding climate change adaptation (flood risk management, fresh water availability, soil subsidence, ground water management, water nuisance); - The available data have to allow for an analysis of internal and external success and have to provide information about whether the various conditions for success were met. The case studies are based upon empirical material collected by a variety of means: interviews, document analysis, focus groups and other methods (see appendix 1 for an overview per case). The case analysis and the comparative analysis was done within a group meeting of all authors. By motivating our scores to each other and collaboratively deciding upon the scores we realized coherence within the analysis. 6. Case studies In this section we will describe the 11 case studies. The main characteristics of the analysed pilot projects are summarized in table 1.
Pilot
Start and end
Altenheim
1988-1996
Beuningen
2004-2008
Contractvormen
2015-2020
Type of climate adaptation
Innovation
Level
Involvement of actors
7
Koopmanspolder
2011-ongoing
Landbouw op peil
2011-2014
Loosdrechtse plassen
2011-2013
Marken
2012-2015
SMART watermanagement Waalblok
2012-ongoing
Water farming Walcheren Zwolle
2010-ongoing
2007-2010
Start 2010 Now: realization Table 2. Summary of the studied pilot projects (meaning icons: see appendix 2) In the next paragraphs, we describe these pilot project and pay attention to the aim of the pilot project, the way the pilot project is organized and its results. Altenheim: Integrated Rhine Program The German State of Baden- Wuertemberg decided in 1988 to develop the so-called Integrated Rhine Program. The objective of this program was to reduce water levels of the Rhine and increase nature values of the floodplains. For this purpose thirteen floodplain areas have been indicated as inundation areas. Essentially, floodplains that have been disconnected from the river should get their original function back and so be able to serve as floodplains. Polder Altenheim was the first in 1989 and functioned as a pilot project. The pilot was organized nearby its home base. Indeed, the pilot was already included in the formal policy. Provided the political importance, there have been sufficient resources within the pilot. Results were considered positive by the policy makers and in 1996 the whole plan has been approved. However, after the pilot phase the implementation of the program did not progress well. In some of the indicated areas opposition arose from citizens living near the other planned areas, supported by their municipalities. They did not approve the results. In their view the plans were harmful for nature (e.g. drowning wildlife), reduced accessibility and more hindrance due to mosquitoes. The opposition delayed the implementation enormously (in 2013 only 4 out of 13 areas have been implemented) and in some 8
areas the concept itself was challenged. Policy makers attempt to address the concerns and reduced their ambitions (i.e. flooding the areas less frequently and less intensively). Beuningen: River maintenance based on Cyclic Floodplain Rejuvenation In the floodplains of Beuningen, Rijkswaterstaat (the implementation agency of the Dutch Ministry of Infrastructure and Environment) has identified increased water levels due to vegetation development. Hence, the discharge capacity of the river did not meet the norms anymore. Instead of simply removing all the vegetation, a nature organisation (Ark), together with Radboud University has developed a new concept called ‘Cyclic Floodplain Rejuvenation’. Free flowing rivers regularly reset vegetation and lower sandbanks as a result of erosion and so make space for the water to flow. This particularly occurs in places where the river would naturally flow. A confined river cannot do this. The idea of the concept is to imitate the natural behaviour and so to create space on one hand and increased natural value resulting from larger variety on the other hand. Moreover, the resources that become available can be used for economic activities (biomass, sediment). In 2004 the idea was launched to design the floodplain according to these principles. The idea was far off from the home base of RWS and resources were limited, particularly for implementation. Hence, it took up to 2008 before the pilot was implemented. The pilot was considered positive by the ambassadors, but at the same time could be easily set aside due to a lack of monitoring. Plans existed to extend the pilot. The ministry of environment would finance this in order to create a climate buffer. However, RWS meanwhile had reached the flood defence objectives and decided to follow the strategy of controlling and maintaining the situation as it is, meaning that there is no juridical space for the CFR approach in which vegetation shows more variety. Innovative contracts in maintaining water ways Rijkswaterstaat, as the manager of the waterways in the Netherlands, nowadays outsources the maintenance of the rivers and canals. In one area (the IJssel and the Twente canals), the contract has been extended with the clause that during the contract period (5 years), the contractor together with RWS and a knowledge institute (Deltares) should identify, test and develop innovations that ought to make river maintenance more efficient and more sustainable. Moreover, these three parties work together in a socalled learning team, where traditional client-contractor relations do not exist. Rather, the parties should work as project partners creating added value for each of the parties. Indeed, Business Cases (BCs) should be developed. So far, several interesting potential innovations have been identified (e.g. flexible groynes, mats to prevent scouring around the groyne, using herded sheep instead of machines for mowing) and pilot projects are being designed. However, clear BCs have not been developed yet. The pilot is far off from the home base of all three organisations. It requires different approaches from all of them how to deal with the other parties. Traditional roles become vaguer. The pilot receives much positive attention (it won a European prize for Innovative Public Procurement). At the moment the pilot has produced a draft text about new, national performance based contracts. This text will be included in the toolbox for Rijkswaterstaat. This success in terms of uptake is mainly the result of a personal link between the pilot and the national contract unit: the driving force behind the pilot is also a team member of this unit.
9
Waalblok The pilot Waalblok comprises the realization of a retention cellar beneath a greenhouse to deal with water nuisance because of extreme rainfall. The pilot project was part of a Knowledge and Innovation Program Water Framework Haaglanden (KIP) and was carried out as a collaborative activity of the regional Water Board, the municipality and the greenhouse owners in the area. The core of the idea was to realize a multifunctional concept for water storage and water cleaning in a cellar. Such a cellar combines public tasks (water retention) and private tasks (water storage and water cleaning). A small coalition of enthousiastic people worked together to refine the idea of the cellar. With help of additional resources of the KIP, they were able to do additional analyses and to reflect periodically upon their own process. The pilot was at a relatively large distance of the various home bases. The pilot participants maintained a direct link to their political principals and not to the rest of their organization. The pilot participants had a difficult job in convincing the people within their home bases to enable the realization of the cellar, especially in case of the Water Board. After a difficult planning process the cellar was ultimately realized. However, the solution is seen as rather costly and also too complicated due to the parallel private and public use of the cellar. The idea of large-scale water retention is exchanged for the idea to look very precisely which tailor-made solutions are necessary and possible on the very local scale, based upon a detailed calculation of the actual consequences of extreme rainfall. The internal success of the pilot was thus relatively high. It was implemented, but mainly thanks to the additional resources that were available within the regional Knowledge and Innovation Program The external success was almost absent: the solution was not replicated and also did not result in policy change. Loosdrecht: flexible water levels In the recreational area of Loosdrecht, in the Western part of the Netherlands the Water Board decided to apply flexible surface water level management. This should allow the water level to rise and fall with the precipitation and evaporation within certain preset boundaries. This is opposed to the ‘regular’ water management in the Netherlands, were water levels are set on a single level and do not fluctuate. The reason to do so was to reduce the need to let in water from the surrounding waterways with a high nutritional values in dry summer, which was harmful for the ecological goals set for the Loosdrecht area because of the Water Framework Directive (WFD). However the new water regime caused several worries among citizens about the possible effects (flooding, hindrance for water recreation) Because the pilot was a subsidized project – within the Dutch innovation programme of the WFD, the pilot was organized as an interorganizational project with many external research institutes involved in the team. There was ample funding for both the actual implementation of the pilot as well as the monitoring and insights into the effects of the pilot. The position of the pilot was well embedded within the organization of the Waterboard – but only in the ecology oriented part of the organization. The process was designed around implementation of the pilot, the monitoring and the dissemination of results. The stakeholders (municipality, marina’s, house owners) were mainly involved by means of participatory monitoring. The results of the pilot were framed as a success by the Waterboard. The Water board mainstreamed the concept of flexible surface water level management, which was also thanks to the active dissemination of the results of the pilot.
10
Koopmanspolder: a paradise for water birds The Netherlands needs new types of water management in order to be prepared for the impact of climate change. To create different functions which support nature, fish and leisure facilities the ‘inland shore’ concept, a new approach to water storage was developed. A pilot was instated in the the Koopmanspolder in the North West of the Netherlands. The first tests with the flexible water level in the Koopmanspolder are bearing fruit, the polder is partially submerged and nature is flourishing. The pilot was organised close to the home base of the primary coordinators of the project (Waterboard and Rijkswaterstaat). Furthermore the project consortium consisted of the province, the municipality, Deltares, Staatsbosbeheer and a number of local NGO’s. Funding of the pilot came from different sources but the pilot was primarily funded by the national fund for rural areas (ILG). The pilot has a very long history, which started in the ’90 by making the area part of the ecological structure of the Netherlands, but through time the focus changed from nature development to more multifunctional purposes. Which resulted in an award winning landscape design for the polder in 2011 to combine nature development with flood risk management, recreation and freshwater supply. The pilot was considered a success within the coordinating organisation, but also outside the organization. This also resulted in the start of a second pilot of the ‘inland shore’ concept by public and private parties, with a focus on agriculture, aquaculture and fishery in other parts of the province. Marken: a new approach to flood safety The water safety situation on the island of Marken in the Netherlands is not adequate to resist high water levels in the future. The dykes around the island were rejected in former test rounds. Therefore, the National water Agency developed a dike reinforcement plan in 2012. This plan could not count on the support of the inhabitants of the island because of the impact on culture history and the adjacent houses. Next to this, the plan was too expensive. At het same time, the Netherlands was developing a new water safety strategy which pays more attention to the equipment of the land and evacuation strategies in case of floods. For this reason the involved governments developed a pilot to investigate whether the new water safety strategy would be a solution for the island of Marken (technical innovation). The pilot is a cooperation between the National water Agency, the municipality, the water board and local stakeholders. The involved stakeholders work together in a project group which functions as a safe haven for the project workers. The pilot is governed by a steering committee and funded by the national government. The time horizon for the pilot is 3 years. The case Marken was successful in terms of developing an innovative water safety strategies for the island (smart development of dykes) based on new water safety principles. The results of the pilot will be used for other delicate water safety situations in other regions in the Netherlands. SMART Watermanagement: a new water management approach in the Netherlands Smart water management is water management that uses a system approach to respond in an effective and sustainable way to flooding and drought. By doing so it operates across (institutional) borders, especially between Rijkswaterstaat (responsible for the national waterways) and Water boards (responsible for the regional water management) as an effective and sustainable response to situations of flooding and drought. The concept was piloted in the south-east of the Netherlands using the term dynamic water management. Were it brought further together the two water boards and Rijkswaterstaat. 11
These partners had already been working together on watermanagement since 1994. The pilot did not have a firm connection to the homebase however, as it was mainly perceived as innovation project. The pilot was organized around three subjects: technical aspects, governance and learning experiences. Before the pilot was started it was considered what the pilot could bring to the partners, for this mainly cost reduction on potential damages and investments were considered. The pilot also brought to light challenges concerning the division of responsibilities as the pilot would cross institutional boundaries. The project did not have many resources to use, and mainly the innovation funds of Rijkswaterstaat were used to develop the pilot. However the results from the pilot were perceived as positive and were considered worth while pursueing further on a national level. This is currently being developed within a national research program on the subject of smart watermanagement in an attempt to mainstream the concept. Landbouw op Peil Climate change poses Dutch water managers for new challenges. This also accounts for the Water boards in the region Rijn-Oost. The major part of the area that these water boards manage consists of agricultural lands. Therefore one of the biggest challenges for these water boards is to develop a future-proof system of water management which leaves sufficient room for vital agricultural production systems. This requires innovative measures optimizing a sustainable water system in relation to the agricultural function. With this challenge in mind the project Landbouw op peil (“Agriculture at Water Level”) was started in 2011. The aim of the project was multi-faceted: - Gather knowledge and experience on how to maintain a vital agriculture within a changing system of water management (due to climate change); - Improve the relation between the agricultural sector and water managers; - Contribute to a ‘climate proof’ agriculture in the Eastern part of the Netherlands; - Use local knowledge from agricultural firms to improve future water management. The pilot is perceived as very successful by almost all of the participating stakeholders. One of the main factors for this internal success is the feeling amongst stakeholders that the pilot has really contributed to increased levels of trust and understanding and has helped to narrow the gap between stakeholders. Next to that, most of the measures that were designed in the pilot for dealing with the water challenges were accepted by the farmers and were continued after the pilot had finished as well. The pilot received quite a lot of (media-)exposure and was framed as (very) successful externally as well . At the end of the pilot-stage, the pilot won a prize (for innovative project) which was accompanied by a sum of money. This money was ‘reinvested’ in a follow-up of the pilot enabling the possibility to draw-up more ‘waterbedrijfsplannen’ for other farmers that were interested. These tailor-made plans at farm level were considered to be one of the crucial success factors. This has contributed to a (moderate) external success of the pilot as well. Water farming Walcheren: a pilot to improve the fresh water availability of local farmers Walcheren is a region in the Netherlands where agriculture is an important economic activity. More and more the consequences of climate change are felt (shortage of fresh water supply). This negatively influences the yield of farmers. For this reason farmers started to cooperate to improve the availability of fresh water in dry periods by conserving the water in the soil and watercourses. They started a pilot to test technical possibilities to infiltrate the water in the soil in winter and increasing the water levels in 12
water courses. The pilot is a private initiative (farmer) and is facilitated by knowledge institutes. The water board is involved for providing permits but is most of the time at a distance. Therefore, the pilot is rather far away from the home bases of the water authorities. The pilot has little resources available (only private investments and subvention of public parties). The process in the pilot can be characterized as ‘learning by doing’. No end date of the pilot is stated. Based on the effectiveness technical measures are added in the pilot. The pilot is successful in effectiveness of chosen measures, but less successful in generating enduring policy change. The pilot is local customized and based on the physical characteristics of the territory. Zwolle: climate proofing the city Zwolle is a city in the IJssel-Vecht Delta, which is a small and vulnerable delta area in the Netherlands. In this delta and especially in Zwolle, many spatial investments are planned. The regional and local governments decided to anticipate on these developments by starting a pilot project with the aim to climate proof the delta area. Based upon this regional aim, the city of Zwolle has the aim to climate proof the city by bridging the boundaries between climate adaptation and spatial developments (like housing, area (re)development and (re)construction of roads). This pilot is organized in a layered way. At the regional level intergovernmental cooperation is organized which is more at distance from the home bases. At the local level, a small group of civil servants within the municipality is responsible for the project. To stimulate the pilot, the province made a substantial budget available. Last four years, this budget was used to realize local climate adaptation projects. The pilot-status, the financial resources, and the local demarcation, makes it easier to come to rapid and successful decision-making. However, this makes it hard for other actors to understand and to participate in the pilot. The process design changes during the pilot. In the first years, the creative process at the regional level dominates and the municipality of Zwolle has to follow this process. In the later years, there is much more room for local process design. In this local process design, the municipality of Zwolle invites for each climate proofing project a variety of actors: governments, companies, NGO’s, interest groups and citizens. The pilot Zwolle resulted in various realized projects to climate proof the city. In this way, the pilot was successful. A small staff within the municipality is closely involved and takes care of the realization of this next series of projects within the pilot, but broader mainstreaming the results within local policies hardly takes place. 7. Analysis of the paradox In table 3 we summarize the way the pilot projects are organized, by using the elements of the pilot paradox (see table 1). For each case we indicate whether this element is present: very low (- -), low (-), moderate (0), high (+), and very high (+ +) .
13
Additional resources
Coalition of boundary spanners
Tailor made process design
Low risk Scale
Normative congruence
Organizational appropriateness
External representativeness
Pilot embedding
System understanding
Conditions for external success
Distance of home base
Conditions for internal success
Altenheim Beuningen Contractvormen Koopmanspolder
-+ +
++ + +
+ 0 +
0 + ++
+ + +
++ -0
+ -0
-+
+ -+
-/+ 0 +
+
++
++
+
+
+
+
++
+
0
Landbouw op peil
+
++
+
++
+
/0
-
0
-
+
++ ++ ++
++ ++
++ 0 +
+ ++ +
+ + ++
+ 0 --
-0
+ -
+ + -
++ +
+ +
++ ++
++
--
-
+
-
0
++
++
+
++
-
0
+
--
0
Loosdrecht + Marken + SMART water management Waalblok + Water farming ++ Walcheren Zwolle Table 3. Case comparison
Altenheim Beuningen
Results of the pilot projects Internal success: External success: results of the pilot project mainstreaming of results High: the initial idea was successfully High in terms of policy adoption, low in developed and implemented. terms of implementation Moderate: little monitoring, pilot Low: other requirements put in place already ‘died’ before implementation
14
Contractvormen
Koopmanspolder Landbouw op peil
Loosdrecht: Marken
Moderate: pilot results in promising innovative ideas, but they are not implemented yet. High: attempts are being made to replicate the pilot Very high. All stakeholders satisfied. Increased trust and improved relations. Moderate: the pilot was considered a success but has a distinct usergroup High: Successful development of customized water safety strategy
SMART water managment
High: results do have serious impact
Waalblok
High, the pilot was successfully implemented
Water farming Walcheren Zwolle
Very high: stakeholders typify their pilot as successful High: successful realization of various climate adaptation projects Table 4. Success of the pilots
Promising: ideas are considered to be included in the standard routines (toolbox) of the organization. High: also other locations are considered, using the same consortium Moderate: there was little ‘infrastructure’ to deal with this interest. No structural follow-up. Moderate: the measure is being considered by other public authorities Low: Newly developed water safety strategy is only slightly embedded in daily routines Moderate: the concept has found its way to the policy table, but implementation is lacking. Very low, the solution doesn’t fit into the normative convictions of the policy environment Low: solutions do not automatically fit in the policy system Low: no attention to mainstreaming of this process innovation
Explanation of the scores Altenheim The pilot was initiated at the core of policy making. The pilot started up very quickly and with large political support. Hence, resources were available and learning taking place. It was also very clear where else the innovation should land, according to the developers that included policy makers. However, when defining the policies, opposition became apparent. This came mainly from citizens and local politicians. This type of dynamics had not been taken into account in the pilot design: actor involvement was insufficient: not so much for the current pilot but for the intended future implementation areas. Beuningen The pilot was initiated by a nature organisation in cooperation with the university. Resources for developing the concept and a site were available, not directly for implementation. One of the main issues to be addressed were therefore who had to pay for it, which came down to the discussion what type of problem it was: nature or flood defence, both having different resources. Essentially, the innovation 15
addressed resources that were administratively divided in a different fashion. A second issue, determinant for the external success was that the decision-making process of the river authority (RWS) is based on static systems, whereas this concept requires dynamics. The concept could not be adopted by the river authority because system understanding was different. Contractvormen For each of the three participants, the required new way of collaboration is far off from the home base. New business models still have to be developed. On the other hand, resources have been made available to give it a serious try: financial sources, by making test locations available and managerial support. Each of the parties is well represented acting as boundary spanners. As said, it is plausible that the resulting ideas will be included in the toolbox of contract forms for Rijkswaterstaat, thanks to the linking pin role of one of the pilot members. Koopmans-polder This pilot took a very long time to find its final shape – starting in the 1990’s and developing into the pilotproces that was initiated in 2011 and continues to this day. The available resources that were brought together because the coalition of the willing was formed brought it to success. And it was actually also this coalition that created formed the basis for the successful uptake (other locations are being considered). However one can ask it will not remain a string of pilots instead of mainstreaming the concept. Landbouw op Peil One of the main factors for the (internal) succes of the Landbouw op Peil project was the room for tailormade solutions at individual farm level. In a process of intensive guidance specific watermanagement plans where discussed and developed for each of the participating firms thus targeting the specific (water)challenges at firm level with tailor-made measures and solutions. This approach resulted in a high level of enthusiasm, commitment and confidence amongst the participants. In order to be able to comeup with these tailor-made solutions, the water managers also took quite some room for manoeuvre and sometimes distanced themselves from the existing practice and protocols within their organizations. In the field /within the pilot this resulted in a very productive and fruitful dynamic, however as this dynamic was also quite far-off from the common practice it hindered the up-scaling / the follow-up of the pilot. Loosdrecht: flexible water levels The Loosdrecht pilot fits with almost all the conditions for a succesfull pilot – the only thing lacking are the boundary spanners, as the pilot was more designed top down and implemented by the waterboard – surpassing the protests of the stakeholders in the area -. Furthermore the pilot clearly established a normative congruence with its home organisation. And although the stakeholders had protested the pilot also made it possible to involve these as participants. Marken The case Marken scores high on the conditions for successful pilots. It is organized at safe distance from normal routines. Furthermore a coalition of boundary spanners is present. The pilot has sufficient 16
additional resources. The case scores relatively low on the conditions for successful mainstreaming. The developed water safety strategy is highly local customized and only slightly embedded in the daily routines of the involved organizations. SMART water management Just as with the koopmanspolder the coalition of the willing created the basis for this pilot, however placing it further from the home bases. The conditions for uptake were then also met, because the pilot did not fit with the existing system and the results are not ready for mainstreaming. But the strange aspect was that the pilot was more or less forgotten, but uptake did occur by means of bringing the concept – without the results of the pilots – to a national level. Waalblok The case Waalblok is a typical illustration of the pilot paradox. It is organized at a safe distance of the regime. It consists of a coalition of willing participants and has many additional resources for exploration, which makes it possible to realize the pilot. At the same time it encountered many difficulties in embedding itself in the responsible home bases and thus there is no upscaling of the pilot. Water Farming The case water farming Walcheren is also a typical illustration of the pilot paradox. It is organized at far distance from water authorities and has a tailor made process design. Its weakness however is its distance to the dominant values in the home bases of water authorities. The pilot embedding and organizational appropriateness scores low. At the same time there is a lack of normative congruence between the stakeholders in the pilot and the policy regime. Zwolle In the pilot project Zwolle, the innovative process design was successful because of some organizational conditions. Especially the additional resources and the involvement of willing actors were helpful to come to rapid decision-making and realization. However, this rate within the pilot makes it hard for other actors to keep the connection. Thus although the internal success, the external success is low. Generic observations It is interesting to see that in the majority of the cases we can indeed witness that a high score on the conditions for internal success goes along with low scores on conditions for wider uptake (eg. Marken, Water Farming, Zwolle, Beuningen, Waalblok, and also for Smart Watermanagement, Landbouw op peil). There are at least two mechanisms that can explain this. First of all, there is the mechanism of goal-displacement. Actors within a pilot do a really good job in making the pilot successful, because of the attention given to the pilot. But they neglect to keep the wider environment attached. They focus upon realizing the objectives of the pilot in producing an innovative outcome and thus also focus entirely upon realizing and implementing that outcome. With help of additional resources, they succeed in doing so. But they forget (or do not have the capacity) to take into
17
account what is needed to enable lesson-drawing and broader dissemination of the ideas and insights of the pilot in policies and practices. The second, related and complementary mechanism can be called regime alienation. Due to the development within the pilot (as a result of intensive learning processes at a distance of the home bases), the distance with the home bases becomes only larger instead of smaller. In that situation the outcomes of the pilot are so different from what is acceptable for the regime, that wider uptake is nearly impossible. At the same time, a couple of pilots do show far less elements of the paradox or even show good results both internally and externally (eg. Koopmanspolder, Altenheim, Flexible Waterlevels), although uptake of the results in these cases is also mainly conceptually and with regard to policies rather than substantially and in terms of implementation. We can see at least two explanations for simultaneous internal and external success. First of all, in some cases the pilot results come at the right moment. There is – in other words – a window of opportunity for the outcomes of the pilot to become mainstreamed in policies and practices of the various home bases (cf. the case Flexible Waterlevels ). Oftentimes a long incubation phase before the pilot is started, resulting in fertile ground for its results. In that case the pilot is the result of either growing regime frustration, because of the ineffectiveness of current solutions. The regime is actually ready for uptaking the results of the pilot, because they are not able to solve the problems in the traditional way, and they have come to alternative solutions. Secondly, in a couple of cases the pilot paradox is effectively prevented for by either self-correcting mechanisms within the pilot process or by making the broader uptake of the lessons of the pilot explicitly part of the aims of the pilot. Self-correcting mechanisms are visible in those pilots which are not very well embedded in their wider environment or which representativeness is low. These pilots do have fairly appropriate results that are also congruent with the dominant beliefs in the wider policy environment (Althenheim, Loosdrecht, Smart Water Management). In the case Koopmanspolder the pilot is organized in such a way that participants continuously inform their environment about what is going on and the preliminary results (cf. Koopmanspolder). This case is also the only case in which all conditions for internal and external success were met. In this case we can witness a carefully crafted strategy to organize an intelligent process for exploration in a safe environment, and an intense process of dissemination to safeguard the wider uptake of the results. 8. Conclusions: the paradox revisited Based upon this analysis we can draw a couple of conclusions. First of all, we can conclude that there is indeed a paradoxical relationship between the conditions for internal and external success. By focussing upon meeting the conditions for the first, easily at the same time a barrier is organized for external success. Especially the distance between pilots and the regime and the dedicated process to find an innovative solution in a non-standard way, forms a barrier to come to 18
appropriate outcomes and to organize sufficient embedding necessary for wider uptake in the existing regime. At the same time, the pilot paradox is far from a general causal mechanism. It is possible to prevent for the pilot, especially by carefully designing the course of events in the pilot and by formulating its main aims in terms of both internal and external success. Making broader uptake part of the main targets and organizing attention to this target from scratch is not only a very logical but also rather effective way to overcome the paradox. This brings us also to the (managerial) implications of the paradox. Designing experiments in any field can be improved when actors recognize the pilot paradox they can be confronted with. Within the design of management of pilots, there have to be attention for issues of external representativeness and boundary spanning between niche and regime (by ways of dialogue with the wider environment). This can for example be organized by integrating future users in an early phase of the pilot. Or by starting to communicate some of the (intermediate) successes and obstacles to a wider public from early on in the pilot. Finally, it would be very interesting to complete this first exploration of the pilot paradox with a more quantitative study of a large sample of pilots and to analyse these in a more quantitative way in order to get more rigorous findings about the peculiarities of the pilot paradox and the circumstances under which the paradox is present or can be prevented for.
19
References Breman, B. C., de Groot, M., Ottow, B., & Rip, W. (2014). Monitoren doe je samen. De meerwaarde van participatieve monitoring. H2O online, 2014(20-06), 1-9. Buuren, M.W. van and D. Loorbach (2009) Policy innovation in isolation? Conditions for policy renewal by transition arenas and pilot projects, Public Management Review, 11(3): 375-392. Buuren, M.W. van & Eshuis, J. (2009). Van concept naar contract: Waterberging, herstructurering en innovatie in proeftuin Waalblok. (Extern rapport). Rotterdam: Erasmus Universiteit Rotterdam. Buuren, M.W. van & Bressers, N.E.W. (2011). Waterproeven in polderland. Een terugblik op Waterkader Haaglanden. (Extern rapport). Rotterdam: Erasmus Universiteit Rotterdam. Buuren, M.W. van, G.J. Ellen, C. van Leeuwen & J. van Popering-Verkerk (2015) Die het water deert die het water keer: overstromingsrisicobeheer als maatschappelijke gebiedsopgave, opbrengsten en lessen uit de pilots meerlaagsveiligheid [in Dutch], Rotterdam: Erasmus University Rotterdam. Cash, D., Clark, W. C., Alcock, F., Dickson, N. M., Eckley, N., & Jäger, J. (2002). Salience, credibility, legitimacy and boundaries: Linking research, assessment and decision making. Chan, P.M., Havekes, H. en Hofstra, M. (2013). Governance aspecten van dynamisch waterbeheer. Analyse van de pilot Noord-Brabant (in Dutch). Water Governance Centre. Duijn, M. (2009) Embedded reflection on public policy innovation: a relativist/pragmatist inquiry into the practice of innovation and knowledge transfer in the WaterINNovation Program, Eburon Ellen, G.J. & Ottow, B. T. 2012. Maatschappelijke aspecten van flexibel peilbeheer. Deltares. 1202707-003-BGS-0015. Karstens, S. (2009). Bridging boundaries: Making scale choices in multi-actor policy analysis on water management (Vol. 4). IOS Press. Kuindersma, W. en B.C.Breman (2014). Leren van landbouw op peil. Evaluatie van een experiment met zelfsturing in het waterbeheer. Alterra-rapport 2512. ISSN 1566-7197. http://edepot.wur.nl/298586 Nutley, S., Davies, H., & Walter, I. (2002). Evidence based policy and practice: cross sector lessons from the UK. ESRC UK Centre for evidence based policy and practice: working paper, 9. Sanderson, I. (2002). Evaluation, policy learning and evidence‐based policy making. Public Administration, 80(1), 122. Sørensen, E., & Torfing, J. (2012). Introduction: Collaborative innovation in the public sector. Innovation Journal, 17(1), 1-14. Vreugdenhil, H.S.I. (2010) Pilot projects in water management: practicing change and changing practice, PhD Thesis, Delft University of Technology. Vreugdenhil, H., J. Slinger, W. Thissen, and P. Ker Rault. 2010. Pilot projects in water management . Ecology and Society 15(3): 13. [online] URL: http://www.ecologyandsociety.org/vol15/iss3/art13/ Ingram, J., Maye, D., Kirwan, J., Curry, N., & Kubinakova, K. (2015). Interactions between Niche and Regime: An Analysis of Learning and Innovation Networks for Sustainable Agriculture across Europe. The Journal of Agricultural Education and Extension, 21(1), 55-71. Smink, M., Negro, S. O., Niesten, E., & Hekkert, M. P. (2015). How mismatching institutional logics hinder niche– regime interaction and how boundary spanners intervene. Technological Forecasting and Social Change, 100, 225-237. Smith, A., & Raven, R. (2012). What is protective space? Reconsidering niches in transitions to sustainability. Research Policy, 41(6), 1025-1036. Zeeman, S. (Sascha). (2015, September 25). Van grens naar verbinding. Public Administration. Retrieved from http://hdl.handle.net/2105/32109
20
Appendix A. Data collection secondary case analysis Case
Data collection
References
Altenheim
Interviews (9), document analysis
Beuningen
Interviews (11), document analysis, observation workshops Participatory observation Document analysis and participatory observation Document Analysis + (semi)structured (Group) interviews (35 respondents) Document analysis, observations and interviews Document analysis, observation, interviews Document analysis and interviews Extensive document analysis and 12 interviews Document analysis, 5 interviews
Gewasserdirektion Sudlicher Oberrhein/ Hochrhein (1999, 2000, 2001), Siepe 2004 Freude am Fluss 2007, Peters et al. 2006, Vreugdenhil et al. 2010 www.ssrs.info https://publicwiki.deltares.nl/display/CAW/Koopmans polder Kuindersma & Breman (2014).
Contractvormen Koopmanspolder Landbouw op peil
Loosdrecht Marken SMART water management Waalblok Water farming Walcheren Zwolle
Document analysis, 5 interviews, and 1 focus group
Ellen et al, 2012 ; Breman et al 2014 Evaluation study by Van Buuren et al. 2015 Zeeman et al., 2015 Chan et al., 2013 Evaluation study by Van Buuren et al. 2009 Van Buuren et al. (forthcoming) Evaluation study by Van Buuren et al. 2015
21
Appendix 2. Meaning of the icons Type of climate adaptation
Flood safety
Fresh water supply
Water level management Nature development Urban development Innovation
Technical
Process design
Scale
Local
Regional
National
Actors involved
Few
Medium
Many
22
