Natural Computing Group Template

Experimental Optimization by Evolutionary Algorithms Thomas Bäck Natural Computing Group, Leiden University, The Netherlands http://natcomp.liacs.nl [email protected]

Ofer M. Shir Rabitz Group, Princeton University, USA

GECCO 2009, Montréal, Canada

Princeton University & Leiden University

Overview • • • • • •

What do we mean by “Experimental Optimization”? Examples of what has been done Potential Application Areas Evolutionary Algorithms that have been used Evolutionary Algorithms vs. Classical Approaches Conclusions and Open Questions


2

What do we mean by …

EXPERIMENTAL OPTIMIZATION


3

“Typical” Characteristics • • • •

Experiments are time-consuming. Experiments are expensive. Only few experiments are possible. There are exceptions as well! – –

Quantum Control: Evolution “in the loop” Thousands of experiments possible


4

1960s … … … 2000s

Examples: - Flow Plate - Bended Pipe - Nozzle - Nutrient Solutions - Coffee Formulations - Quantum Control

EXAMPLE APPLICATIONS


5

Early Experiments I: Flow Plate

• A plate with 5 controllable angle brackets • Measurable air flow drag (by a pitot tube) Figure from: I. Rechenberg, Evolutionsstrategie ´73, frommann-holzboog, Stuttgart 1973 Princeton University & Leiden University

Early Experiments I: Flow Plate Experiment 2: • Left supporting point 25% lower than right one. • Horizontal flow. • Minimize drag.

Plate drag

Experiment 1: • Left / right supporting point at same y-coordinate. • Horizontal flow. • Minimize drag.

Number of mutations and selected plate shapes

Number of mutations and selected plate shapes

Start

-30

-40

40

-30

40

Start

0

0

0

0

0

End

0

4

0

6

-6

End

16

6

2

0

-18

Figures from: I. Rechenberg, Evolutionsstrategie ´73, frommann-holzboog, Stuttgart 1973 Princeton University & Leiden University

Early Experiments II: Bended Pipe

• A flexible pipe with 6 controllable bending devices • Minimize bend losses of liquid flow • Measure drag by pitot tube Figure from: I. Rechenberg, Evolutionsstrategie ´73, frommann-holzboog, Stuttgart 1973 Princeton University & Leiden University

Pipe drag

Early Experiments II: Bended Pipe

Number of mutations and selected pipe shapes

Initial (a) and optimized (b) pipe shape

• Bend loss of final form reduced by 10% • Including drag a total reduction of 2% Figure from: I. Rechenberg, Evolutionsstrategie ´73, frommann-holzboog, Stuttgart 1973 Princeton University & Leiden University

Early Experiment III: Nozzle • What can be done if physics, (bio-) chemistry, … of process unkown? • • • •

No model or simulation program available! Idea: Optimize with the real object “Hardware in the loop” Example: Supersonic nozzle, turbulent flow, physical model not available.


10

Experimental Setup: Nozzle

• Production of differently formed conic nozzle parts (pierced plates). • Form of nozzle part is value of decision variable. choosing conic nozzle parts (by EA) clamping of conic nozzle parts (manually) steam under high pressure passed into nozzle degree of efficiency is measured! Princeton University & Leiden University

„simulator replacement“ 11

Nozzle Experiment (I) collection of conical nozzle parts

device for clamping nozzle parts


Nozzle Experiment (II)

Hans-Paul Schwefel while changing nozzle parts


Nozzle Experiment (III)

steam plant / experimental setup


Nozzle Experiment (IV)

the nozzle in operation …

… while measuring degree of efficiency


Nozzle Results (I) • Illustrative Example: Optimize Efficiency – Initial:

– Evolution:

• 32% Improvement in Efficiency !


16

Nozzle Results (II)

• • • •

250 experiments were made. 45 improvements found. Discrete ring segments, variable-dimensional optimisation Gene duplication and deletion as additional operators. J. Klockgether and H.-P. Schwefel, “Two-phase nozzle and hollow core jet experiments,” in Proceedings of the 11th Symposium on Engineering Aspects of Magneto-Hydrodynamics, Caltech, Pasadena, California, USA, 1970. Princeton University & Leiden University

Experiment: Nutrient Solution Optimization •

Fermentation Process – Maximize biomass-coefficient (BTM) – Maximize gain – Minimize cost of nutrient solution

•

Execution: 224 lab-experiments – Execute the fermentation process – Measure optical density

• Algorithm: – Standard GA – N=32, 7 generations


18

Experiment: Coffee Formulations • Optimize taste of a target coffee, 5 ingredients • Subjective evaluation by human experts • (1,5)-ES accepts deterioriations • Experts do not !


Coffee Formulations: Algorithm 1 Parent, 5 offspring coffee recipes

Mix coffees according to recipes … … and brew them …

Let the expert drink them … … and evaluate their taste

… thus selecting the closest match.


Coffee Formulations: Results Optimum taste in 11 generations (55 evaluations)

Target coffee

Result coffee


Coffee Formulations: Results • • • •

Coffee mixture differs a lot from target coffee ! Taste is identical ! Multiple realizations, but cost optimal ! Approximation of cubic polynomial: 35 evals.


EXPERIMENTAL OPTIMIZATION: FUNDAMENTALS Princeton University & Leiden University

Experimental Requirements (for an Optimizer) 1. 2.

Speed: fast convergence is required Reliability: reproducibility of results within a margin •

3. 4.

Environmental parameters often hidden (temperature, pressure, …)

Robustness: manufacturing feasibility Reference solution (recommended): pre-designed reference item, robust and stable, having a known objective function value


Convergence Speed • Experiments are typically expensive: Goal – drive the system into optimality with as few measurements as possible • Experimental systems often lack stability (short lifetime, biological environments, molecular level, etc.) • A practical solution: Derandomized Evolution Strategies


Reliability of Results • Mostly algorithm dependent • Attained results must be reproducible • Scenarios of recording experimental outliers must be avoided (elitism is tricky…) • Perceived result versus a posteriori result • Possible solutions: – Employing comma (non-elitist) strategies – Increasing sampling rate of measurements


Environmental Parameters • As many physical conditions should be recorded during the experiment • Ideally, sensitivity of the system to the environment should be assessed • Basic starting points: recording Signal/Noise, extracting power spectrum of the noise, etc.


Manufacturing Feasibility • Mostly system dependent • Realization of the prescribed decision parameters of the experiment to equivalent systems, e.g., in a manufacturing stage • Toward this end, sensitivity of the system must be assessed (electronics, for instance) • Upon obtaining reproducible results, they should be verified on equivalent systems


Noise Colors • White Noise (1 /

f

0

-noise)

• Pink Noise (flicker noise, or • Red (Brownian) Noise ( 1 /

1 / f -noise)

f

2

-noise)

Hint: Assess the stability of your system by extracting the Power Spectrum of its signal-free state. M. Roth, J. Roslund, and H. Rabitz, “Assessing and managing laser system stability for quantum control experiments”, Rev. Sci. Instrum. 77, 083107 (2006)


Signal Averaging • Experiments with high-duty cycle allow increased signal averaging. • Influence of additive noise sources is reduced (central limit theorem). • Given k single-shot measurements:

[]

fˆ = f , VAR fˆ =

ε

2 f

k


POTENTIAL APPLICATION AREAS Princeton University & Leiden University

all xpe #E nts

La rge

e rim

#E

Sm

xp eri me nt s

Evolutionary Experimental Optimization

Application Domains: QC Photonic Reagent Organic Chemistry


Application Domains: QC Fragile Systems Pharmaceuticals Automotive

Potential Application Areas • Cosmetics / Detergent Formulation Optimization • Catalyst Formulation Optimization (Cost, Effectiveness, …) • Subjective Evaluation Applications based on Human Taste or other Senses • Engineering Applications Requiring Real-World Experiments for Measurement • Concrete Formulation Optimization • Glue Formulation Optimization • Plant Startup Process • Chemical Compound Synthesis Processes (e.g., Drugs)


33

Experimental Quantum Control • • • •

High-duty cycle (kHz regime): 1 experiment = 1 second Building a system: Years, K$ Once system is up, experiments are cheap Two scenarios: 1. Stable systems will allow a large number of experiments 2. Fragile systems with a limited budget of experiments (short lifetime) • Will be presented here as a case-study.


EVOLUTIONARY ALGORITHMS USED Princeton University & Leiden University

Evolutionary Algorithms: General Evolutionary Evolutionary Algorithms Algorithms

Evolution Evolution Strategies Strategies

Genetic Genetic Algorithms Algorithms

Other Other

Main differences: Encoding, operators, adaptation


(1+1)-Evolution Strategy [Rechenberg 73’] t ←0 P(t ) ← Init() Evaluate (Pt )

while t < t max do r r x (t ) := Mutate {x (t − 1)} with stepsize σ r r Evaluate (P′(t ) := {x (t )}) : { f ( x (t ))} Select {P′(t ) ∪ P(t )} t ← t +1 if t mod n = 0 then ⎧σ (t − n ) / c if ps > 1 / 5 ⎪ σ := ⎨σ (t − n ) ⋅ c if ps < 1 / 5 ⎪σ (t − n ) if ps = 1 / 5 ⎩ else

σ (t ) := σ (t − 1)

endif endwhile Princeton University & Leiden University

r r ps = Prob { f (Mutate {x}) ≥ f ( x )}

Evolution Strategies: Mutation Schemes Drawing normally distributed mutations:

( )

r r z ~ N 0, C

Figure courtesy of Jonathan Roslund


Derandomized Evolution Strategies (DES) • •

Reducing stochasticity of mutations by accummulating information on past mutations: statistical learning Small populations sizes: logarithmic in space dimension

λ = 4 + ⎣3 ln(n )⎦ , •

⎢λ ⎥ μ=⎢ ⎥ ⎣2⎦

Various schemes, developed throughtout the years: a) CSA [Ostermeier, Galwelczyk, Hansen; 1994-1996] b) sep-CMA-ES / (μ,λ)-DR2 [Hansen et al., 1994, 2008] c) Full-blown CMA-ES [Hansen, Ostermeier; 1996, 1999-2001]


Covariance Matrix Adaptation Evolution Strategy

Learning the covariance matrix by applying principal component analysis (PCA) of the successful mutations

Two independent mechanisms: *Adapting the covariance matrix **Controlling the global step-size



CMA-ES: General Concept r r r r r x = m +σ ⋅ z z ~ N 0, C r r r m ← m + σ ⋅ z W C = RΛR t r r r 2 pc ← (1 − cc ) ⋅ pc + 1 − (1 − cc ) ⋅ z

( )

*

**

{

r rT C ← (1 − ccov ) ⋅ C + ccov ⋅ pc pc

r r pσ ← (1 − cσ ) ⋅ pσ + ⎧ ⎪ cσ σ ← σ ⋅ exp ⎨ ⎪⎩ d σ

r ⋅R ⋅ z

1 − (1 − c c ) r ⎛ pσ ⎜ r ⎜⎜ E N 0 , I ⎝


2

W

( )

W

⎞⎫ ⎪ ⎟ − 1⎟⎬ ⎟⎪ ⎠⎭

DES: Discussion • Very effective in treating high-dimensional problems with a low number of function evaluations. • Noise handling is well done by recombination. • Different mutation schemes support modularity – CSA or sep-CMA-ES (DR2) may suffice for certain problems. • May allow for landscape learning with the covariance matrix. • A long list of successful real-world simulated problems.


Case-Study:

QUANTUM CONTROL EXPERIMENTS Princeton University & Leiden University

Altering the Course of Quantum Dynamics Phenomena


Quantum Control Theory (QCT) • Quantum system is either controllable or non-controllable [Rabitz et al., Science2004]. • Given a controllable quantum system, there is always a trap free pathway up to the top of the control landscape from any location. • Assumption: no constraints whatsoever… • Gradient algorithms are basically sufficient to climb…!

max Pi → f = Pr{ψ i → ψ f } = ψ (T ) ψ f

[

2

max Tr[ρT O ] = Tr[ψ T ψ T O ] = Tr Uρ 0U O Princeton University & Leiden University

⊥

]

Quantum Control Experiments (QCE) • Yield, or success-rate, correspond to a measurement; no Hamiltonian required. • Control (field) shaped through phase (freq. vs. time) • Several levels of experimental uncertainties • From QC theoretical perspective: severely constrained landscape: limited bandwidth, limited fluence, resolution, proper basis, etc. • In practice: local traps, hard landscapes • Topology versus Local Structures


Quantum Control Experiments


The Optical Table: Shaping the Pulse





Sources of Noise and Uncertainty • Pixel Noise (input parameters): The error in realizing the prescribed parameters in the experimental setup about the shaper [most impact] • Observation Noise (1/f): Detector error (a.k.a. JohnsonNyquist noise) • System Drift: Systematic deviation in the system values over time


Single-Objective Experiments • CMA-ES was observed to perform extremely well with small population sizes • Recombination is necessary (Genetic Repair Hypothesis) [Arnold and Beyer] • Robust, reproducible, reliable solutions



QCE Systems: ES vs. GA

O. M. Shir, J. Roslund, T. Bäck, and H. Rabitz, “Performance Analysis of Derandomized Evolution Strategies in Quantum Control Experiments,” in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2008. New York, NY, USA: ACM Press, 2008.


Multi-Observable Quantum Control


Multi-Observable Quantum Control • Growing sub-field with promising implications • Simultaneous optimization of multiple quantum observables • Treatment of noisy systems is challenging (elitism) • Currently under study both in the lab and in simulations:

O. M. Shir, J. Roslund, and H. Rabitz, “Evolutionary Multi-Objective Quantum Control Experiments with the Covariance Matrix Princeton University & Leiden University Adaptation”, GECCO’09, ACM.

Extended Features: Landscape Learning • QCE and CMA-ES enjoy a happy marriage • Landscape learning by means of covariance exploitation • Recovery of experimental Hessian: (b) 5 most important Hessian eigenvectors; Physical form is indeed corroborated

(a) Retrieving the Hessian by inversion of the CMA’s covariance matrix

J. Roslund, O. M. Shir, T. Bäck, and H. Rabitz, “Accelerated Optimization and Automated Discovery with Covariance Matrix Adaptation for Experimental Quantum Control”, submitted. Princeton University & Leiden University

EAS VS. CLASSICAL APPROACHES Princeton University & Leiden University

Experimental Optimization • New product development where • Data is missing at all, or • Poor data situation

• Goal: • Fast generation of data, or • Fast optimization towards new product


DoE • Full factorial design

• With star points


Classical: Design of Experiments (DoE) No. of experiments (example) m=12 > 14

Method

1. Screening: Test for key impact factors

>m+2

2. Modeling: Investigation of interdependencies

⎛n⎞ > n + 2 + ⎜⎜ ⎟⎟ ⎝ 2⎠

> 38

Fractionalfactorial plans

3. Stepwise optimiz.: Finding optimal steps of the factors

⎛ n⎞ > 2n + 2 + ⎜⎜ ⎟⎟ ⎝ 2⎠

> 46

Central-composite plan

4. Fine optimization: Zooming in on the optimum

n up to 5n

8 - 40

Central-composite plan

Total:

n=8

>> 106


Plackett-Burman

New: Evolutionary Optimization (ES) No. of experiments (example) 1. Optimization: Goal driven experiments

k⋅ m

(1,6)-ES

> 70 Total:

• • • • • •

m=12, k=20

Method

> 70

Direct optimization without initial trial experiments Implicit fine-tuning of solutions No assumptions on model structure required No quadratic model simplification No screening of experimental space required Efficient variation of all factors at the same time Princeton University & Leiden University

Concept Visualization (2-dimensional) DoE

Evolution Strategy Start

x2

x2

x1 Optimum

x1 Optimum


Related to Experimental Optimization:

SIMULATION-BASED OPTIMIZATION Princeton University & Leiden University

Safety Optimization – Pilot Study • •

Aim: Identification of most appropriate Optimization Algorithm for realistic example! Optimizations for 3 test cases and 14 algorithms were performed (28 x 10 = 280 shots) – – –

• •

Body MDO Crash / Statics / Dynamics MCO B-Pillar MCO Shape of Engine Mount

ES performed significantly better than Monte-Carloscheme, GA, and Simulated Annealing Results confirmed by statistical hypothesis testing


MDO Crash / Statics / Dynamics • Minimization of body mass • Finite element mesh – Crash ~ 130.000 elements – NVH ~ 90.000 elements

• Independent parameters: Thickness of each unit: 109 • Constraints: 18 Algorithm

Avg. reduction (kg)

Max. reduction (kg)

Min. reduction (kg)

Best so far

-6.6

-8.3

-3.3

ES

-9.0

-13.4

-6.3


Safety Optimization – Example of use • Production Run ! • Minimization of body mass • Finite element mesh – Crash ~ 1.000.000 elements – NVH ~ 300.000 elements

• Independent parameters: – Thickness of each unit: 136

• Constraints: 47, resulting from various loading cases • 180 (10 x 18) shots ~ 12 days • No statistical evaluation due to problem complexity


Safety Optimization – Example of use NuTech’s Evolution Strategy

Mass

Initial Value

Generations

• 13,5 kg weight reduction by NuTech’s ES • Beats best so far method significantly • Typically faster convergence velocity of ES ~ 45% less time (~ 3 days saving) for comparable quality needed • Still potential of improvements after 180 shots. • Reduction of development time from 5 to 2 weeks allows for process integration Princeton University & Leiden University

MDO – Optimization Run Fitness

All Individuals

Infeasible Feasible

107,13% 105,05%

100,00%

Number of Individuals


MDO ASF® Front Optimization • Pre-optimized Space-Frame-Concept – improvement possible? • Goal: Minimization of structural weight • Degrees of freedom: – Wall thicknesses of the semi-finished products sheet & profile – Material characteristic profile

• Limitation of design space: – Semi-finished products technology – Technique for joining parts


MDO ASF® Disciplines

Damage according to insurance classification, Component Model, 2 CPUs

Global dynamic stiffness, Trimmed Body, 1 CPU

Front Crash (EURO NCAP), Complete Body 4 CPUs

Resources Resourcesper perDesign: Design:77CPUs, CPUs,approx. approx.23h 23h Princeton University & Leiden University

MDO Run Comparison Initial design, constraints violated

Optimum (exp. 924), Constraints satisfied

Mass Masse

Best so far optimizer Increased weight!

Mass

Optimum (exp. 376), Constraints satisfied Decreased weight!


CONCLUSIONS AND OPEN QUESTIONS Princeton University & Leiden University

Conclusions • Experimental Optimization is hard – but an Evolutionary approach is feasible! • EAs should be given a chance in new application areas • Fundamental research in EAs is much needed


Goals and Open Questions • Given a budget of k experiments – what strategy should be taken? • NFL holds more than ever – there will be no winner algorithm handling all experimental scenarios! • Holy Grail: A package of strategies to drive an experimental system to a reliable maximum with minimum experiments


Exciting Literature …


Leiden Institute of Advanced Computer Science (LIACS) See www.liacs.nl and http://natcomp.liacs.nl Masters in Comp. Science ICT in Business Media Technology

Elected „Best Comp. Sci. Study“ by students. Excellent job opportunities for our students. Research education with an eye on business.


LIACS Research Algorithms Prof. J.N. Kok, Prof. T. Bäck

• Novel Algorithms • • •

Technology and Innovation Management Prof. B. Katzy

Data Mining Natural Computing Applications • Drug Design • Medicine • Engineering • Logistics • Physics

Synergies & Collaboration

Imagery and Media Dr. M. Lew, Dr. F. Verbeek

• Computer Vision and Audio/Video • • • •

77

Bioimaging Multimedia Search Internet Technology Computer Graphics

Prof. H. Wijshoff, Prof. E. Deprettere

• Embedded Systems

• Coevolution of Technology and Social Structures • Entrepreneurship • Innovation Management

Core Computer Technologies • • •

Parallel / Distributed Computing Compiler Technology Data Mining

Foundations of Software Technology Prof. F. Arbab, Prof. J.N. Kok

• Software Systems • • • • •

Embedded Systems Service Composition Multicore Systems Formal Methods Coordination / Concurrency


Acknowledgements • • • •

Hans-Paul Schwefel Michael Emmerich Herschel Rabitz Jonathan Roslund
