Clever_Algorithms

Prévia do material em texto

Jason Brownlee
Clever Algorithms
Nature-Inspired Programming Recipes
ii
Jason Brownlee, PhD
Jason Brownlee studied Applied Science at Swinburne University in Melbourne,
Australia, going on to complete a Masters in Information Technology focusing on
Niching Genetic Algorithms, and a PhD in the field of Artificial Immune Systems.
Jason has worked for a number of years as a Consultant and Software Engineer
for a range of Corporate and Government organizations. When not writing books,
Jason likes to compete in Machine Learning competitions.
Cover Image
© Copyright 2011 Jason Brownlee. All Reserved.
Clever Algorithms: Nature-Inspired Programming Recipes
© Copyright 2011 Jason Brownlee. Some Rights Reserved.
First Edition. LuLu. January 2011
ISBN: 978-1-4467-8506-5
This work is licensed under a Creative Commons
Attribution-Noncommercial-Share Alike 2.5 Australia License.
The full terms of the license are located online at
http://creativecommons.org/licenses/by-nc-sa/2.5/au/legalcode
Webpage
Source code and additional resources can be downloaded from the books
companion website online at http://www.CleverAlgorithms.com
http://creativecommons.org/licenses/by-nc-sa/2.5/au/legalcode
http://www.CleverAlgorithms.com
Contents
Foreword vii
Preface ix
I Background 1
1 Introduction 3
1.1 What is AI . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Problem Domains . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3 Unconventional Optimization . . . . . . . . . . . . . . . . . 13
1.4 Book Organization . . . . . . . . . . . . . . . . . . . . . . . 16
1.5 How to Read this Book . . . . . . . . . . . . . . . . . . . . 19
1.6 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . 20
1.7 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . 21
II Algorithms 27
2 Stochastic Algorithms 29
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2 Random Search . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3 Adaptive Random Search . . . . . . . . . . . . . . . . . . . 34
2.4 Stochastic Hill Climbing . . . . . . . . . . . . . . . . . . . . 39
2.5 Iterated Local Search . . . . . . . . . . . . . . . . . . . . . . 43
2.6 Guided Local Search . . . . . . . . . . . . . . . . . . . . . . 49
2.7 Variable Neighborhood Search . . . . . . . . . . . . . . . . . 55
2.8 Greedy Randomized Adaptive Search . . . . . . . . . . . . . 60
2.9 Scatter Search . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.10 Tabu Search . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.11 Reactive Tabu Search . . . . . . . . . . . . . . . . . . . . . 79
iii
iv Contents
3 Evolutionary Algorithms 87
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.2 Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . 92
3.3 Genetic Programming . . . . . . . . . . . . . . . . . . . . . 99
3.4 Evolution Strategies . . . . . . . . . . . . . . . . . . . . . . 108
3.5 Differential Evolution . . . . . . . . . . . . . . . . . . . . . 114
3.6 Evolutionary Programming . . . . . . . . . . . . . . . . . . 120
3.7 Grammatical Evolution . . . . . . . . . . . . . . . . . . . . 126
3.8 Gene Expression Programming . . . . . . . . . . . . . . . . 134
3.9 Learning Classifier System . . . . . . . . . . . . . . . . . . . 141
3.10 Non-dominated Sorting Genetic Algorithm . . . . . . . . . . 152
3.11 Strength Pareto Evolutionary Algorithm . . . . . . . . . . . 160
4 Physical Algorithms 167
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
4.2 Simulated Annealing . . . . . . . . . . . . . . . . . . . . . . 169
4.3 Extremal Optimization . . . . . . . . . . . . . . . . . . . . . 175
4.4 Harmony Search . . . . . . . . . . . . . . . . . . . . . . . . 182
4.5 Cultural Algorithm . . . . . . . . . . . . . . . . . . . . . . . 187
4.6 Memetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . 193
5 Probabilistic Algorithms 199
5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
5.2 Population-Based Incremental Learning . . . . . . . . . . . 203
5.3 Univariate Marginal Distribution Algorithm . . . . . . . . . 208
5.4 Compact Genetic Algorithm . . . . . . . . . . . . . . . . . . 212
5.5 Bayesian Optimization Algorithm . . . . . . . . . . . . . . . 216
5.6 Cross-Entropy Method . . . . . . . . . . . . . . . . . . . . . 224
6 Swarm Algorithms 229
6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
6.2 Particle Swarm Optimization . . . . . . . . . . . . . . . . . 232
6.3 Ant System . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
6.4 Ant Colony System . . . . . . . . . . . . . . . . . . . . . . . 245
6.5 Bees Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.6 Bacterial Foraging Optimization Algorithm . . . . . . . . . 257
7 Immune Algorithms 265
7.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
7.2 Clonal Selection Algorithm . . . . . . . . . . . . . . . . . . 270
7.3 Negative Selection Algorithm . . . . . . . . . . . . . . . . . 277
7.4 Artificial Immune Recognition System . . . . . . . . . . . . 284
7.5 Immune Network Algorithm . . . . . . . . . . . . . . . . . . 292
7.6 Dendritic Cell Algorithm . . . . . . . . . . . . . . . . . . . . 299
v
8 Neural Algorithms 307
8.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
8.2 Perceptron . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
8.3 Back-propagation . . . . . . . . . . . . . . . . . . . . . . . . 316
8.4 Hopfield Network . . . . . . . . . . . . . . . . . . . . . . . . 324
8.5 Learning Vector Quantization . . . . . . . . . . . . . . . . . 330
8.6 Self-Organizing Map . . . . . . . . . . . . . . . . . . . . . . 336
III Extensions 343
9 Advanced Topics 345
9.1 Programming Paradigms . . . . . . . . . . . . . . . . . . . . 346
9.2 Devising New Algorithms . . . . . . . . . . . . . . . . . . . 356
9.3 Testing Algorithms . . . . . . . . . . . . . . . . . . . . . . . 367
9.4 Visualizing Algorithms . . . . . . . . . . . . . . . . . . . . . 374
9.5 Problem Solving Strategies . . . . . . . . . . . . . . . . . . 386
9.6 Benchmarking Algorithms . . . . . . . . . . . . . . . . . . . 400
IV Appendix 411
A Ruby: Quick-Start Guide 413
A.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
A.2 Language Basics . . . . . . . . . . . . . . . . . . . . . . . . 413
A.3 Ruby Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . 417
A.4 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . 419
Index 421
vi Contents
Foreword
I am delighted to write this foreword. This book, a reference where one
can look up the details of most any algorithm to find a clear unambiguous
description, has long been needed and here it finally is. A concise reference
that has taken many hours to write but which has the capacity to save vast
amounts of time previously spent digging out original papers.
I have known the author for several years and have had experience of his
amazing capacity for work and the sheer quality of his output, so this book
comes as no surprise to me. But I hope it will be a surprise and delight to
you, the reader for whom it has been written.
But useful as this book is, it is only a beginning. There are so many
algorithms that no one author could hope to cover them all. So if you know
of an algorithm that is not yet here, how about contributing it using the
same clear and lucid style?
Professor Tim Hendtlass
Complex Intelligent Systems Laboratory
Faculty of Information and Communication Technologies
Swinburne University of Technology
Melbourne, Australia
2010
vii
viii Foreword
Preface
About the book
The need for this project was born of frustration while working towards my
PhD. I was investigating optimization algorithms and was implementing
a large number of them for a software platform called the Optimization
Algorithm Toolkit (OAT)1. Each algorithm required considerable effort
to locate the relevant source material(from books, papers, articles, and
existing implementations), decipher and interpret the technique, and finally
attempt to piece together a working implementation.
Taking a broader perspective, I realized that the communication of
algorithmic techniques in the field of Artificial Intelligence was clearly a
difficult and outstanding open problem. Generally, algorithm descriptions
are:
� Incomplete: many techniques are ambiguously described, partially
described, or not described at all.
� Inconsistent : a given technique may be described using a variety of
formal and semi-formal methods that vary across different techniques,
limiting the transferability of background skills an audience requires
to read a technique (such as mathematics, pseudocode, program code,
and narratives). An inconsistent representation for techniques means
that the skills used to understand and internalize one technique may
not be transferable to realizing different techniques or even extensions
of the same technique.
� Distributed : the description of data structures, operations, and pa-
rameterization of a given technique may span a collection of papers,
articles, books, and source code published over a number of years, the
access to which may be restricted and difficult to obtain.
For the practitioner, a badly described algorithm may be simply frus-
trating, where the gaps in available information are filled with intuition and
1OAT located at http://optalgtoolkit.sourceforge.net
ix
http://optalgtoolkit.sourceforge.net
x Preface
‘best guess’. At the other end of the spectrum, a badly described algorithm
may be an example of bad science and the failure of the scientific method,
where the inability to understand and implement a technique may prevent
the replication of results, the application, or the investigation and extension
of a technique.
The software I produced provided a first step solution to this problem: a
set of working algorithms implemented in a (somewhat) consistent way and
downloaded from a single location (features likely provided by any library of
artificial intelligence techniques). The next logical step needed to address this
problem is to develop a methodology that anybody can follow. The strategy
to address the open problem of poor algorithm communication is to present
complete algorithm descriptions (rather than just implementations) in a
consistent manner, and in a centralized location. This book is the outcome
of developing such a strategy that not only provides a methodology for
standardized algorithm descriptions, but provides a large corpus of complete
and consistent algorithm descriptions in a single centralized location.
The algorithms described in this work are practical, interesting, and
fun, and the goal of this project was to promote these features by making
algorithms from the field more accessible, usable, and understandable.
This project was developed over a number years through a lot of writing,
discussion, and revision. This book has been released under a permissive
license that encourages the reader to explore new and creative ways of
further communicating its message and content.
I hope that this project has succeeded in some small way and that you
too can enjoy applying, learning, and playing with Clever Algorithms.
Jason Brownlee
Melbourne, Australia
2011
Acknowledgments
This book could not have been completed without the commitment, passion,
and hard work from a large group of editors and supporters.
A special thanks to Steve Dower for his incredible attention to detail
in providing technical and copy edits for large portions of this book, and
for his enthusiasm for the subject area. Also, a special thanks to Daniel
Angus for the discussions around the genesis of the project, his continued
support with the idea of an ‘algorithms atlas’ and for his attention to detail
in providing technical and copy edits for key chapters.
In no particular order, thanks to: Juan Ojeda, Martin Goddard, David
Howden, Sean Luke, David Zappia, Jeremy Wazny, and Andrew Murray.
Thanks to the hundreds of machine learning enthusiasts who voted on
potential covers and helped shape what this book became. You know who
you are!
Finally, I would like to thank my beautiful wife Ying Liu for her unre-
lenting support and patience throughout the project.
xi
xii Acknowledgments
Part I
Background
1
Chapter 1
Introduction
Welcome to Clever Algorithms! This is a handbook of recipes for com-
putational problem solving techniques from the fields of Computational
Intelligence, Biologically Inspired Computation, and Metaheuristics. Clever
Algorithms are interesting, practical, and fun to learn about and implement.
Research scientists may be interested in browsing algorithm inspirations in
search of an interesting system or process analogs to investigate. Developers
and software engineers may compare various problem solving algorithms
and technique-specific guidelines. Practitioners, students, and interested
amateurs may implement state-of-the-art algorithms to address business or
scientific needs, or simply play with the fascinating systems they represent.
This introductory chapter provides relevant background information on
Artificial Intelligence and Algorithms. The core of the book provides a large
corpus of algorithms presented in a complete and consistent manner. The
final chapter covers some advanced topics to consider once a number of
algorithms have been mastered. This book has been designed as a reference
text, where specific techniques are looked up, or where the algorithms across
whole fields of study can be browsed, rather than being read cover-to-cover.
This book is an algorithm handbook and a technique guidebook, and I hope
you find something useful.
1.1 What is AI
1.1.1 Artificial Intelligence
The field of classical Artificial Intelligence (AI) coalesced in the 1950s
drawing on an understanding of the brain from neuroscience, the new
mathematics of information theory, control theory referred to as cybernetics,
and the dawn of the digital computer. AI is a cross-disciplinary field
of research that is generally concerned with developing and investigating
3
4 Chapter 1. Introduction
systems that operate or act intelligently. It is considered a discipline in the
field of computer science given the strong focus on computation.
Russell and Norvig provide a perspective that defines Artificial Intel-
ligence in four categories: 1) systems that think like humans, 2) systems
that act like humans, 3) systems that think rationally, 4) systems that
act rationally [43]. In their definition, acting like a human suggests that
a system can do some specific things humans can do, this includes fields
such as the Turing test, natural language processing, automated reasoning,
knowledge representation, machine learning, computer vision, and robotics.
Thinking like a human suggests systems that model the cognitive informa-
tion processing properties of humans, for example a general problem solver
and systems that build internal models of their world. Thinking rationally
suggests laws of rationalism and structured thought, such as syllogisms and
formal logic. Finally, acting rationally suggests systems that do rational
things such as expected utility maximization and rational agents.
Luger and Stubblefield suggest that AI is a sub-field of computer science
concerned with the automation of intelligence, and like other sub-fields
of computer science has both theoretical concerns (how and why do the
systems work? ) and application concerns (where and when can the systems
be used? ) [34]. They suggest a strong empirical focus to research, because
although there may be a strong desire for mathematical analysis, the systems
themselves defy analysis given their complexity. The machines and software
investigated in AI are not black boxes, rather analysis proceeds by observing
the systems interactions with their environments, followed by an internal
assessment of the system to relate its structureback to its behavior.
Artificial Intelligence is therefore concerned with investigating mecha-
nisms that underlie intelligence and intelligence behavior. The traditional
approach toward designing and investigating AI (the so-called ‘good old
fashioned’ AI) has been to employ a symbolic basis for these mechanisms.
A newer approach historically referred to as scruffy artificial intelligence or
soft computing does not necessarily use a symbolic basis, instead patterning
these mechanisms after biological or natural processes. This represents a
modern paradigm shift in interest from symbolic knowledge representations,
to inference strategies for adaptation and learning, and has been referred to
as neat versus scruffy approaches to AI. The neat philosophy is concerned
with formal symbolic models of intelligence that can explain why they work,
whereas the scruffy philosophy is concerned with intelligent strategies that
explain how they work [44].
Neat AI
The traditional stream of AI concerns a top down perspective of problem
solving, generally involving symbolic representations and logic processes
that most importantly can explain why the systems work. The successes of
this prescriptive stream include a multitude of specialist approaches such
1.1. What is AI 5
as rule-based expert systems, automatic theorem provers, and operations
research techniques that underly modern planning and scheduling software.
Although traditional approaches have resulted in significant success they
have their limits, most notably scalability. Increases in problem size result in
an unmanageable increase in the complexity of such problems meaning that
although traditional techniques can guarantee an optimal, precise, or true
solution, the computational execution time or computing memory required
can be intractable.
Scruffy AI
There have been a number of thrusts in the field of AI toward less crisp
techniques that are able to locate approximate, imprecise, or partially-true
solutions to problems with a reasonable cost of resources. Such approaches
are typically descriptive rather than prescriptive, describing a process for
achieving a solution (how), but not explaining why they work (like the
neater approaches).
Scruffy AI approaches are defined as relatively simple procedures that
result in complex emergent and self-organizing behavior that can defy
traditional reductionist analyses, the effects of which can be exploited for
quickly locating approximate solutions to intractable problems. A common
characteristic of such techniques is the incorporation of randomness in
their processes resulting in robust probabilistic and stochastic decision
making contrasted to the sometimes more fragile determinism of the crisp
approaches. Another important common attribute is the adoption of an
inductive rather than deductive approach to problem solving, generalizing
solutions or decisions from sets of specific observations made by the system.
1.1.2 Natural Computation
An important perspective on scruffy Artificial Intelligence is the motivation
and inspiration for the core information processing strategy of a given
technique. Computers can only do what they are instructed, therefore a
consideration is to distill information processing from other fields of study,
such as the physical world and biology. The study of biologically motivated
computation is called Biologically Inspired Computing [16], and is one of
three related fields of Natural Computing [22, 23, 39]. Natural Computing
is an interdisciplinary field concerned with the relationship of computation
and biology, which in addition to Biologically Inspired Computing is also
comprised of Computationally Motivated Biology and Computing with
Biology [36, 40].
6 Chapter 1. Introduction
Biologically Inspired Computation
Biologically Inspired Computation is computation inspired by biological
metaphor, also referred to as Biomimicry, and Biomemetics in other engi-
neering disciplines [6, 17]. The intent of this field is to devise mathematical
and engineering tools to generate solutions to computation problems. The
field involves using procedures for finding solutions abstracted from the
natural world for addressing computationally phrased problems.
Computationally Motivated Biology
Computationally Motivated Biology involves investigating biology using
computers. The intent of this area is to use information sciences and
simulation to model biological systems in digital computers with the aim
to replicate and better understand behaviors in biological systems. The
field facilitates the ability to better understand life-as-it-is and investigate
life-as-it-could-be. Typically, work in this sub-field is not concerned with
the construction of mathematical and engineering tools, rather it is focused
on simulating natural phenomena. Common examples include Artificial
Life, Fractal Geometry (L-systems, Iterative Function Systems, Particle
Systems, Brownian motion), and Cellular Automata. A related field is that
of Computational Biology generally concerned with modeling biological
systems and the application of statistical methods such as in the sub-field
of Bioinformatics.
Computation with Biology
Computation with Biology is the investigation of substrates other than
silicon in which to implement computation [1]. Common examples include
molecular or DNA Computing and Quantum Computing.
1.1.3 Computational Intelligence
Computational Intelligence is a modern name for the sub-field of AI con-
cerned with sub-symbolic (also called messy, scruffy, and soft) techniques.
Computational Intelligence describes techniques that focus on strategy and
outcome. The field broadly covers sub-disciplines that focus on adaptive
and intelligence systems, not limited to: Evolutionary Computation, Swarm
Intelligence (Particle Swarm and Ant Colony Optimization), Fuzzy Systems,
Artificial Immune Systems, and Artificial Neural Networks [20, 41]. This
section provides a brief summary of the each of the five primary areas of
study.
1.1. What is AI 7
Evolutionary Computation
A paradigm that is concerned with the investigation of systems inspired by
the neo-Darwinian theory of evolution by means of natural selection (natural
selection theory and an understanding of genetics). Popular evolutionary
algorithms include the Genetic Algorithm, Evolution Strategy, Genetic
and Evolutionary Programming, and Differential Evolution [4, 5]. The
evolutionary process is considered an adaptive strategy and is typically
applied to search and optimization domains [26, 28].
Swarm Intelligence
A paradigm that considers collective intelligence as a behavior that emerges
through the interaction and cooperation of large numbers of lesser intelligent
agents. The paradigm consists of two dominant sub-fields 1) Ant Colony
Optimization that investigates probabilistic algorithms inspired by the
foraging behavior of ants [10, 18], and 2) Particle Swarm Optimization that
investigates probabilistic algorithms inspired by the flocking and foraging
behavior of birds and fish [30]. Like evolutionary computation, swarm
intelligence-based techniques are considered adaptive strategies and are
typically applied to search and optimization domains.
Artificial Neural Networks
Neural Networks are a paradigm that is concerned with the investigation of
architectures and learning strategies inspired by the modeling of neurons
in the brain [8]. Learning strategies are typically divided into supervised
and unsupervised which manage environmental feedback in different ways.
Neural network learning processes are considered adaptive learning and
are typically applied to function approximation and pattern recognition
domains.
Fuzzy Intelligence
Fuzzy Intelligence is a paradigm that is concerned with the investigation of
fuzzy logic, which is a form of logic that is not constrained to true and false
determinations like propositional logic, but rather functions which define
approximate truth, or degrees of truth [52]. Fuzzy logic and fuzzy systemsare a logic system used as a reasoning strategy and are typically applied to
expert system and control system domains.
Artificial Immune Systems
A collection of approaches inspired by the structure and function of the
acquired immune system of vertebrates. Popular approaches include clonal
8 Chapter 1. Introduction
selection, negative selection, the dendritic cell algorithm, and immune net-
work algorithms. The immune-inspired adaptive processes vary in strategy
and show similarities to the fields of Evolutionary Computation and Artifi-
cial Neural Networks, and are typically used for optimization and pattern
recognition domains [15].
1.1.4 Metaheuristics
Another popular name for the strategy-outcome perspective of scruffy AI is
metaheuristics. In this context, heuristic is an algorithm that locates ‘good
enough’ solutions to a problem without concern for whether the solution
can be proven to be correct or optimal [37]. Heuristic methods trade-off
concerns such as precision, quality, and accuracy in favor of computational
effort (space and time efficiency). The greedy search procedure that only
takes cost-improving steps is an example of heuristic method.
Like heuristics, metaheuristics may be considered a general algorithmic
framework that can be applied to different optimization problems with
relative few modifications to adapt them to a specific problem [25, 46]. The
difference is that metaheuristics are intended to extend the capabilities
of heuristics by combining one or more heuristic methods (referred to as
procedures) using a higher-level strategy (hence ‘meta’). A procedure in a
metaheuristic is considered black-box in that little (if any) prior knowledge
is known about it by the metaheuristic, and as such it may be replaced with
a different procedure. Procedures may be as simple as the manipulation of
a representation, or as complex as another complete metaheuristic. Some
examples of metaheuristics include iterated local search, tabu search, the
genetic algorithm, ant colony optimization, and simulated annealing.
Blum and Roli outline nine properties of metaheuristics [9], as follows:
� Metaheuristics are strategies that “guide” the search process.
� The goal is to efficiently explore the search space in order to find
(near-)optimal solutions.
� Techniques which constitute metaheuristic algorithms range from
simple local search procedures to complex learning processes.
� Metaheuristic algorithms are approximate and usually non-deterministic.
� They may incorporate mechanisms to avoid getting trapped in confined
areas of the search space.
� The basic concepts of metaheuristics permit an abstract level descrip-
tion.
� Metaheuristics are not problem-specific.
1.2. Problem Domains 9
� Metaheuristics may make use of domain-specific knowledge in the
form of heuristics that are controlled by the upper level strategy.
� Todays more advanced metaheuristics use search experience (embodied
in some form of memory) to guide the search.
Hyperheuristics are yet another extension that focuses on heuristics
that modify their parameters (online or offline) to improve the efficacy
of solution, or the efficiency of the computation. Hyperheuristics provide
high-level strategies that may employ machine learning and adapt their
search behavior by modifying the application of the sub-procedures or even
which procedures are used (operating on the space of heuristics which in
turn operate within the problem domain) [12, 13].
1.1.5 Clever Algorithms
This book is concerned with ‘clever algorithms’, which are algorithms
drawn from many sub-fields of artificial intelligence not limited to the
scruffy fields of biologically inspired computation, computational intelligence
and metaheuristics. The term ‘clever algorithms’ is intended to unify a
collection of interesting and useful computational tools under a consistent
and accessible banner. An alternative name (Inspired Algorithms) was
considered, although ultimately rejected given that not all of the algorithms
to be described in the project have an inspiration (specifically a biological or
physical inspiration) for their computational strategy. The set of algorithms
described in this book may generally be referred to as ‘unconventional
optimization algorithms’ (for example, see [14]), as optimization is the main
form of computation provided by the listed approaches. A technically more
appropriate name for these approaches is stochastic global optimization (for
example, see [49] and [35]).
Algorithms were selected in order to provide a rich and interesting
coverage of the fields of Biologically Inspired Computation, Metaheuristics
and Computational Intelligence. Rather than a coverage of just the state-of-
the-art and popular methods, the algorithms presented also include historic
and newly described methods. The final selection was designed to provoke
curiosity and encourage exploration and a wider view of the field.
1.2 Problem Domains
Algorithms from the fields of Computational Intelligence, Biologically In-
spired Computing, and Metaheuristics are applied to difficult problems, to
which more traditional approaches may not be suited. Michalewicz and
Fogel propose five reasons why problems may be difficult [37] (page 11):
� The number of possible solutions in the search space is so large as to
forbid an exhaustive search for the best answer.
10 Chapter 1. Introduction
� The problem is so complicated, that just to facilitate any answer at
all, we have to use such simplified models of the problem that any
result is essentially useless.
� The evaluation function that describes the quality of any proposed
solution is noisy or varies with time, thereby requiring not just a single
solution but an entire series of solutions.
� The possible solutions are so heavily constrained that constructing
even one feasible answer is difficult, let alone searching for an optimal
solution.
� The person solving the problem is inadequately prepared or imagines
some psychological barrier that prevents them from discovering a
solution.
This section introduces two problem formalisms that embody many of the
most difficult problems faced by Artificial and Computational Intelligence.
They are: Function Optimization and Function Approximation. Each class
of problem is described in terms of its general properties, a formalism, and
a set of specialized sub-problems. These problem classes provide a tangible
framing of the algorithmic techniques described throughout the work.
1.2.1 Function Optimization
Real-world optimization problems and generalizations thereof can be drawn
from most fields of science, engineering, and information technology (for
a sample [2, 48]). Importantly, function optimization problems have had
a long tradition in the fields of Artificial Intelligence in motivating basic
research into new problem solving techniques, and for investigating and
verifying systemic behavior against benchmark problem instances.
Problem Description
Mathematically, optimization is defined as the search for a combination of pa-
rameters commonly referred to as decision variables (x = {x1, x2, x3, . . . xn})
which minimize or maximize some ordinal quantity (c) (typically a scalar
called a score or cost) assigned by an objective function or cost function (f),
under a set of constraints (g = {g1, g2, g3, . . . gn}). For example, a general
minimization case would be as follows: f(x′) ≤ f(x), ∀xi ∈ x. Constraints
may provide boundaries on decision variables (for example in a real-value hy-
percube ℜn), or may generally define regions of feasibility and in-feasibility
in the decision variable space. In applied mathematics the field may be
referred to as Mathematical Programming. More generally the field may
be referred to as Global or Function Optimization given the focus on the
objective function. For more general information on optimization refer to
Horst et al. [29].
1.2. Problem Domains 11
Sub-Fields of Study
The study of optimization is comprised of manyspecialized sub-fields, based
on an overlapping taxonomy that focuses on the principle concerns in the
general formalism. For example, with regard to the decision variables,
one may consider univariate and multivariate optimization problems. The
type of decision variables promotes specialities for continuous, discrete,
and permutations of variables. Dependencies between decision variables
under a cost function define the fields of Linear Programming, Quadratic
Programming, and Nonlinear Programming. A large class of optimization
problems can be reduced to discrete sets and are considered in the field
of Combinatorial Optimization, to which many theoretical properties are
known, most importantly that many interesting and relevant problems
cannot be solved by an approach with polynomial time complexity (so-
called NP, for example see Papadimitriou and Steiglitz [38]).
THe evaluation of variables against a cost function, collectively may
be considered a response surface. The shape of such a response surface
may be convex, which is a class of functions to which many important
theoretical findings have been made, not limited to the fact that location of
the local optimal configuration also means the global optimal configuration
of decision variables has been located [11]. Many interesting and real-world
optimization problems produce cost surfaces that are non-convex or so called
multi-modal1 (rather than unimodal) suggesting that there are multiple
peaks and valleys. Further, many real-world optimization problems with
continuous decision variables cannot be differentiated given their complexity
or limited information availability, meaning that derivative-based gradient
decent methods (that are well understood) are not applicable, necessitating
the use of so-called ‘direct search’ (sample or pattern-based) methods [33].
Real-world objective function evaluation may be noisy, discontinuous, and/or
dynamic, and the constraints of real-world problem solving may require
an approximate solution in limited time or using resources, motivating the
need for heuristic approaches.
1.2.2 Function Approximation
Real-world Function Approximation problems are among the most computa-
tionally difficult considered in the broader field of Artificial Intelligence for
reasons including: incomplete information, high-dimensionality, noise in the
sample observations, and non-linearities in the target function. This section
considers the Function Approximation formalism and related specialization’s
as a general motivating problem to contrast and compare with Function
Optimization.
1Taken from statistics referring to the centers of mass in distributions, although in
optimization it refers to ‘regions of interest’ in the search space, in particular valleys in
minimization, and peaks in maximization cost surfaces.
12 Chapter 1. Introduction
Problem Description
Function Approximation is the problem of finding a function (f) that ap-
proximates a target function (g), where typically the approximated function
is selected based on a sample of observations (x, also referred to as the
training set) taken from the unknown target function. In machine learning,
the function approximation formalism is used to describe general problem
types commonly referred to as pattern recognition, such as classification,
clustering, and curve fitting (called a decision or discrimination function).
Such general problem types are described in terms of approximating an
unknown Probability Density Function (PDF), which underlies the relation-
ships in the problem space, and is represented in the sample data. This
perspective of such problems is commonly referred to as statistical machine
learning and/or density estimation [8, 24].
Sub-Fields of Study
The function approximation formalism can be used to phrase some of the
hardest problems faced by Computer Science, and Artificial Intelligence
in particular, such as natural language processing and computer vision.
The general process focuses on 1) the collection and preparation of the
observations from the target function, 2) the selection and/or preparation of
a model of the target function, and 3) the application and ongoing refinement
of the prepared model. Some important problem-based sub-fields include:
� Feature Selection where a feature is considered an aggregation of
one-or-more attributes, where only those features that have meaning
in the context of the target function are necessary to the modeling
function [27, 32].
� Classification where observations are inherently organized into la-
belled groups (classes) and a supervised process models an underlying
discrimination function to classify unobserved samples.
� Clustering where observations may be organized into groups based
on underlying common features, although the groups are unlabeled
requiring a process to model an underlying discrimination function
without corrective feedback.
� Curve or Surface Fitting where a model is prepared that provides a
‘best-fit’ (called a regression) for a set of observations that may be
used for interpolation over known observations and extrapolation for
observations outside what has been modeled.
The field of Function Optimization is related to Function Approxima-
tion, as many-sub-problems of Function Approximation may be defined as
optimization problems. Many of the technique paradigms used for function
1.3. Unconventional Optimization 13
approximation are differentiated based on the representation and the op-
timization process used to minimize error or maximize effectiveness on a
given approximation problem. The difficulty of Function Approximation
problems centre around 1) the nature of the unknown relationships between
attributes and features, 2) the number (dimensionality) of attributes and
features, and 3) general concerns of noise in such relationships and the
dynamic availability of samples from the target function. Additional diffi-
culties include the incorporation of prior knowledge (such as imbalance in
samples, incomplete information and the variable reliability of data), and
problems of invariant features (such as transformation, translation, rotation,
scaling, and skewing of features).
1.3 Unconventional Optimization
Not all algorithms described in this book are for optimization, although
those that are may be referred to as ‘unconventional’ to differentiate them
from the more traditional approaches. Examples of traditional approaches
include (but are not not limited) mathematical optimization algorithms
(such as Newton’s method and Gradient Descent that use derivatives to
locate a local minimum) and direct search methods (such as the Simplex
method and the Nelder-Mead method that use a search pattern to locate
optima). Unconventional optimization algorithms are designed for the
more difficult problem instances, the attributes of which were introduced in
Section 1.2.1. This section introduces some common attributes of this class
of algorithm.
1.3.1 Black Box Algorithms
Black Box optimization algorithms are those that exploit little, if any,
information from a problem domain in order to devise a solution. They are
generalized problem solving procedures that may be applied to a range of
problems with very little modification [19]. Domain specific knowledge refers
to known relationships between solution representations and the objective
cost function. Generally speaking, the less domain specific information
incorporated into a technique, the more flexible the technique, although the
less efficient it will be for a given problem. For example, ‘random search’ is
the most general black box approach and is also the most flexible requiring
only the generation of random solutions for a given problem. Random
search allows resampling of the domain which gives it a worst case behavior
that is worse than enumerating the entire search domain. In practice, the
more prior knowledge available about a problem, the more information that
can be exploited by a technique in order to efficientlylocate a solution for
the problem, heuristically or otherwise. Therefore, black box methods are
those methods suitable for those problems where little information from the
14 Chapter 1. Introduction
problem domain is available to be used by a problem solving approach.
1.3.2 No-Free-Lunch
The No-Free-Lunch Theorem of search and optimization by Wolpert and
Macready proposes that all black box optimization algorithms are the same
for searching for the extremum of a cost function when averaged over all
possible functions [50, 51]. The theorem has caused a lot of pessimism and
misunderstanding, particularly in relation to the evaluation and comparison
of Metaheuristic and Computational Intelligence algorithms.
The implication of the theorem is that searching for the ‘best’ general-
purpose black box optimization algorithm is irresponsible as no such pro-
cedure is theoretically possible. No-Free-Lunch applies to stochastic and
deterministic optimization algorithms as well as to algorithms that learn and
adjust their search strategy over time. It is independent of the performance
measure used and the representation selected. Wolpert and Macready’s
original paper was produced at a time when grandiose generalizations were
being made as to algorithm, representation, or configuration superiority.
The practical impact of the theory is to encourage practitioners to bound
claims of applicability for search and optimization algorithms. Wolpert and
Macready encouraged effort be put into devising practical problem classes
and into the matching of suitable algorithms to problem classes. Further,
they compelled practitioners to exploit domain knowledge in optimization
algorithm application, which is now an axiom in the field.
1.3.3 Stochastic Optimization
Stochastic optimization algorithms are those that use randomness to elicit
non-deterministic behaviors, contrasted to purely deterministic procedures.
Most algorithms from the fields of Computational Intelligence, Biologically
Inspired Computation, and Metaheuristics may be considered to belong the
field of Stochastic Optimization. Algorithms that exploit randomness are not
random in behavior, rather they sample a problem space in a biased manner,
focusing on areas of interest and neglecting less interesting areas [45]. A
class of techniques that focus on the stochastic sampling of a domain, called
Markov Chain Monte Carlo (MCMC) algorithms, provide good average
performance, and generally offer a low chance of the worst case performance.
Such approaches are suited to problems with many coupled degrees of
freedom, for example large, high-dimensional spaces. MCMC approaches
involve stochastically sampling from a target distribution function similar
to Monte Carlo simulation methods using a process that resembles a biased
Markov chain.
� Monte Carlo methods are used for selecting a statistical sample to
approximate a given target probability density function and are tradi-
1.3. Unconventional Optimization 15
tionally used in statistical physics. Samples are drawn sequentially
and the process may include criteria for rejecting samples and biasing
the sampling locations within high-dimensional spaces.
� Markov Chain processes provide a probabilistic model for state tran-
sitions or moves within a discrete domain called a walk or a chain of
steps. A Markov system is only dependent on the current position in
the domain in order to probabilistically determine the next step in
the walk.
MCMC techniques combine these two approaches to solve integration
and optimization problems in large dimensional spaces by generating sam-
ples while exploring the space using a Markov chain process, rather than
sequentially or independently [3]. The step generation is configured to bias
sampling in more important regions of the domain. Three examples of
MCMC techniques include the Metropolis-Hastings algorithm, Simulated
Annealing for global optimization, and the Gibbs sampler which are com-
monly employed in the fields of physics, chemistry, statistics, and economics.
1.3.4 Inductive Learning
Many unconventional optimization algorithms employ a process that includes
the iterative improvement of candidate solutions against an objective cost
function. This process of adaptation is generally a method by which the
process obtains characteristics that improve the system’s (candidate solution)
relative performance in an environment (cost function). This adaptive
behavior is commonly achieved through a ‘selectionist process’ of repetition
of the steps: generation, test, and selection. The use of non-deterministic
processes mean that the sampling of the domain (the generation step) is
typically non-parametric, although guided by past experience.
The method of acquiring information is called inductive learning or
learning from example, where the approach uses the implicit assumption
that specific examples are representative of the broader information content
of the environment, specifically with regard to anticipated need. Many
unconventional optimization approaches maintain a single candidate solution,
a population of samples, or a compression thereof that provides both an
instantaneous representation of all of the information acquired by the process,
and the basis for generating and making future decisions.
This method of simultaneously acquiring and improving information
from the domain and the optimization of decision making (where to direct
future effort) is called the k-armed bandit (two-armed and multi-armed
bandit) problem from the field of statistical decision making known as game
theory [7, 42]. This formalism considers the capability of a strategy to
allocate available resources proportional to the future payoff the strategy
is expected to receive. The classic example is the 2-armed bandit problem
16 Chapter 1. Introduction
used by Goldberg to describe the behavior of the genetic algorithm [26]. The
example involves an agent that learns which one of the two slot machines
provides more return by pulling the handle of each (sampling the domain)
and biasing future handle pulls proportional to the expected utility, based
on the probabilistic experience with the past distribution of the payoff.
The formalism may also be used to understand the properties of inductive
learning demonstrated by the adaptive behavior of most unconventional
optimization algorithms.
The stochastic iterative process of generate and test can be computation-
ally wasteful, potentially re-searching areas of the problem space already
searched, and requiring many trials or samples in order to achieve a ‘good
enough’ solution. The limited use of prior knowledge from the domain
(black box) coupled with the stochastic sampling process mean that the
adapted solutions are created without top-down insight or instruction can
sometimes be interesting, innovative, and even competitive with decades of
human expertise [31].
1.4 Book Organization
The remainder of this book is organized into two parts: Algorithms that
describes a large number of techniques in a complete and a consistent
manner presented in a rough algorithm groups, and Extensions that reviews
more advanced topics suitable for when a number of algorithms have been
mastered.
1.4.1 Algorithms
Algorithms are presented in six groups or kingdoms distilled from the broader
fields of study each in their own chapter, as follows:
� Stochastic Algorithms that focuses on the introduction of randomness
into heuristic methods (Chapter 2).
� Evolutionary Algorithms inspired by evolution by means of natural
selection (Chapter 3).
� Physical Algorithms inspired by physical and social systems (Chap-
ter 4).
� Probabilistic Algorithms that focuses on methods that build models
and estimate distributions in search domains (Chapter 5).
� Swarm Algorithms that focuses on methods that exploit the properties
of collective intelligence (Chapter 6).
� Immune Algorithms inspired by the adaptive immune system of verte-
brates (Chapter 7).
1.4. Book Organization17
� Neural Algorithms inspired by the plasticity and learning qualities of
the human nervous system (Chapter 8).
A given algorithm is more than just a procedure or code listing, each
approach is an island of research. The meta-information that define the
context of a technique is just as important to understanding and application
as abstract recipes and concrete implementations. A standardized algorithm
description is adopted to provide a consistent presentation of algorithms
with a mixture of softer narrative descriptions, programmatic descriptions
both abstract and concrete, and most importantly useful sources for finding
out more information about the technique.
The standardized algorithm description template covers the following
subjects:
� Name: The algorithm name defines the canonical name used to refer
to the technique, in addition to common aliases, abbreviations, and
acronyms. The name is used as the heading of an algorithm description.
� Taxonomy : The algorithm taxonomy defines where a technique fits
into the field, both the specific sub-fields of Computational Intelligence
and Biologically Inspired Computation as well as the broader field
of Artificial Intelligence. The taxonomy also provides a context for
determining the relationships between algorithms.
� Inspiration: (where appropriate) The inspiration describes the specific
system or process that provoked the inception of the algorithm. The
inspiring system may non-exclusively be natural, biological, physical,
or social. The description of the inspiring system may include relevant
domain specific theory, observation, nomenclature, and those salient
attributes of the system that are somehow abstractly or conceptually
manifest in the technique.
� Metaphor : (where appropriate) The metaphor is a description of the
technique in the context of the inspiring system or a different suitable
system. The features of the technique are made apparent through
an analogous description of the features of the inspiring system. The
explanation through analogy is not expected to be literal, rather the
method is used as an allegorical communication tool. The inspiring
system is not explicitly described, this is the role of the ‘inspiration’
topic, which represents a loose dependency for this topic.
� Strategy : The strategy is an abstract description of the computational
model. The strategy describes the information processing actions
a technique shall take in order to achieve an objective, providing a
logical separation between a computational realization (procedure) and
an analogous system (metaphor). A given problem solving strategy
18 Chapter 1. Introduction
may be realized as one of a number of specific algorithms or problem
solving systems.
� Procedure: The algorithmic procedure summarizes the specifics of
realizing a strategy as a systemized and parameterized computation.
It outlines how the algorithm is organized in terms of the computation,
data structures, and representations.
� Heuristics: The heuristics section describes the commonsense, best
practice, and demonstrated rules for applying and configuring a pa-
rameterized algorithm. The heuristics relate to the technical details
of the technique’s procedure and data structures for general classes
of application (neither specific implementations nor specific problem
instances).
� Code Listing : The code listing description provides a minimal but
functional version of the technique implemented with a programming
language. The code description can be typed into a computer and
provide a working execution of the technique. The technique imple-
mentation also includes a minimal problem instance to which it is
applied, and both the problem and algorithm implementations are
complete enough to demonstrate the techniques procedure. The de-
scription is presented as a programming source code listing with a
terse introductory summary.
� References: The references section includes a listing of both primary
sources of information about the technique as well as useful intro-
ductory sources for novices to gain a deeper understanding of the
theory and application of the technique. The description consists
of hand-selected reference material including books, peer reviewed
conference papers, and journal articles.
Source code examples are included in the algorithm descriptions, and
the Ruby Programming Language was selected for use throughout the
book. Ruby was selected because it supports the procedural program-
ming paradigm, adopted to ensure that examples can be easily ported to
object-oriented and other paradigms. Additionally, Ruby is an interpreted
language, meaning the code can be directly executed without an introduced
compilation step, and it is free to download and use from the Internet.2
Ruby is concise, expressive, and supports meta-programming features that
improve the readability of code examples.
The sample code provides a working version of a given technique for
demonstration purposes. Having a tinker with a technique can really
bring it to life and provide valuable insight into a method. The sample
code is a minimum implementation, providing plenty of opportunity to
2Ruby can be downloaded for free from http://www.ruby-lang.org
http://www.ruby-lang.org
1.5. How to Read this Book 19
explore, extend and optimize. All of the source code for the algorithms
presented in this book is available from the companion website, online at
http://www.CleverAlgorithms.com. All algorithm implementations were
tested with Ruby 1.8.6, 1.8.7 and 1.9.
1.4.2 Extensions
There are some some advanced topics that cannot be meaningfully considered
until one has a firm grasp of a number of algorithms, and these are discussed
at the back of the book. The Advanced Topics chapter addresses topics such
as: the use of alternative programming paradigms when implementing clever
algorithms, methodologies used when devising entirely new approaches,
strategies to consider when testing clever algorithms, visualizing the behavior
and results of algorithms, and comparing algorithms based on the results
they produce using statistical methods. Like the background information
provided in this chapter, the extensions provide a gentle introduction and
starting point into some advanced topics, and references for seeking a deeper
understanding.
1.5 How to Read this Book
This book is a reference text that provides a large compendium of algorithm
descriptions. It is a trusted handbook of practical computational recipes to
be consulted when one is confronted with difficult function optimization and
approximation problems. It is also an encompassing guidebook of modern
heuristic methods that may be browsed for inspiration, exploration, and
general interest.
The audience for this work may be interested in the fields of Computa-
tional Intelligence, Biologically Inspired Computation, and Metaheuristics
and may count themselves as belonging to one of the following broader
groups:
� Scientists: Research scientists concerned with theoretically or empir-
ically investigating algorithms, addressing questions such as: What
is the motivating system and strategy for a given technique? What
are some algorithms that may be used in a comparison within a given
subfield or across subfields?
� Engineers : Programmers and developers concerned with implementing,
applying, or maintaining algorithms, addressing questions such as:
What is the procedure for a given technique? What are the best practice
heuristics for employing a given technique?
� Students: Undergraduate and graduate students interested in learn-
ing about techniques, addressing questions such as: What are some
interesting algorithms to study? How to implement a given approach?
http://www.CleverAlgorithms.com
20 Chapter 1. Introduction
� Amateurs : Practitioners interested in knowing more about algorithms,
addressing questions such as: What classes of techniques exist and what
algorithms do they provide? How to conceptualize the computation of
a technique?
1.6Further Reading
This book is not an introduction to Artificial Intelligence or related sub-fields,
nor is it a field guide for a specific class of algorithms. This section provides
some pointers to selected books and articles for those readers seeking a
deeper understanding of the fields of study to which the Clever Algorithms
described in this book belong.
1.6.1 Artificial Intelligence
Artificial Intelligence is large field of study and many excellent texts have
been written to introduce the subject. Russell and Novig’s “Artificial
Intelligence: A Modern Approach” is an excellent introductory text providing
a broad and deep review of what the field has to offer and is useful for
students and practitioners alike [43]. Luger and Stubblefield’s “Artificial
Intelligence: Structures and Strategies for Complex Problem Solving” is also
an excellent reference text, providing a more empirical approach to the field
[34].
1.6.2 Computational Intelligence
Introductory books for the field of Computational Intelligence generally
focus on a handful of specific sub-fields and their techniques. Engelbrecht’s
“Computational Intelligence: An Introduction” provides a modern and de-
tailed introduction to the field covering classic subjects such as Evolutionary
Computation and Artificial Neural Networks, as well as more recent tech-
niques such as Swarm Intelligence and Artificial Immune Systems [20].
Pedrycz’s slightly more dated “Computational Intelligence: An Introduction”
also provides a solid coverage of the core of the field with some deeper
insights into fuzzy logic and fuzzy systems [41].
1.6.3 Biologically Inspired Computation
Computational methods inspired by natural and biologically systems repre-
sent a large portion of the algorithms described in this book. The collection
of articles published in de Castro and Von Zuben’s “Recent Developments
in Biologically Inspired Computing” provides an overview of the state of
the field, and the introductory chapter on need for such methods does an
excellent job to motivate the field of study [17]. Forbes’s “Imitation of Life:
1.7. Bibliography 21
How Biology Is Inspiring Computing” sets the scene for Natural Computing
and the interrelated disciplines, of which Biologically Inspired Computing
is but one useful example [22]. Finally, Benyus’s “Biomimicry: Innovation
Inspired by Nature” provides a good introduction into the broader related
field of a new frontier in science and technology that involves building
systems inspired by an understanding of the biological world [6].
1.6.4 Metaheuristics
The field of Metaheuristics was initially constrained to heuristics for applying
classical optimization procedures, although has expanded to encompass a
broader and diverse set of techniques. Michalewicz and Fogel’s “How to
Solve It: Modern Heuristics” provides a practical tour of heuristic methods
with a consistent set of worked examples [37]. Glover and Kochenberger’s
“Handbook of Metaheuristics” provides a solid introduction into a broad
collection of techniques and their capabilities [25].
1.6.5 The Ruby Programming Language
The Ruby Programming Language is a multi-paradigm dynamic language
that appeared in approximately 1995. Its meta-programming capabilities
coupled with concise and readable syntax have made it a popular language
of choice for web development, scripting, and application development.
The classic reference text for the language is Thomas, Fowler, and Hunt’s
“Programming Ruby: The Pragmatic Programmers’ Guide” referred to as the
‘pickaxe book’ because of the picture of the pickaxe on the cover [47]. An
updated edition is available that covers version 1.9 (compared to 1.8 in the
cited version) that will work just as well for use as a reference for the examples
in this book. Flanagan and Matsumoto’s “The Ruby Programming Language”
also provides a seminal reference text with contributions from Yukihiro
Matsumoto, the author of the language [21]. For more information on the
Ruby Programming Language, see the quick-start guide in Appendix A.
1.7 Bibliography
[1] S. Aaronson. NP-complete problems and physical reality. ACM
SIGACT News (COLUMN: Complexity theory), 36(1):30–52, 2005.
[2] M. M. Ali, C. Storey, and A Törn. Application of stochastic global
optimization algorithms to practical problems. Journal of Optimization
Theory and Applications, 95(3):545–563, 1997.
[3] C. Andrieu, N. de Freitas, A. Doucet, and M. I. Jordan. An introduction
to MCMC for machine learning. Machine Learning, 50:5–43, 2003.
22 Chapter 1. Introduction
[4] T. Bäck, D. B. Fogel, and Z. Michalewicz, editors. Evolutionary
Computation 1: Basic Algorithms and Operators. IoP, 2000.
[5] T. Bäck, D. B. Fogel, and Z. Michalewicz, editors. Evolutionary
Computation 2: Advanced Algorithms and Operations. IoP, 2000.
[6] J. M. Benyus. Biomimicry: Innovation Inspired by Nature. Quill, 1998.
[7] D. Bergemann and J. Valimaki. Bandit problems. Cowles Foundation
Discussion Papers 1551, Cowles Foundation, Yale University, January
2006.
[8] C. M. Bishop. Neural Networks for Pattern Recognition. Oxford
University Press, 1995.
[9] C. Blum and A. Roli. Metaheuristics in combinatorial optimiza-
tion: Overview and conceptual comparison. ACM Computing Surveys
(CSUR), 35(3):268–308, 2003.
[10] E. Bonabeau, M. Dorigo, and G. Theraulaz. Swarm Intelligence: From
Natural to Artificial Systems. Oxford University Press US, 1999.
[11] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge
University Press, 2004.
[12] E. K. Burke, E. Hart, G. Kendall, J. Newall, P. Ross, and S. Schulenburg.
Handbook of Metaheuristics, chapter Hyper-heuristics: An emerging
direction in modern search technology, pages 457–474. Kluwer, 2003.
[13] E. K. Burke, G. Kendall, and E. Soubeiga. A tabu-search hyper-
heuristic for timetabling and rostering. Journal of Heuristics, 9(6):451–
470, 2003.
[14] D. Corne, M. Dorigo, and F. Glover. New Ideas in Optimization.
McGraw-Hill, 1999.
[15] L. N. de Castro and J. Timmis. Artificial Immune Systems: A New
Computational Intelligence Approach. Springer, 2002.
[16] L. N. de Castro and F. J. Von Zuben. Recent developments in biologically
inspired computing, chapter From biologically inspired computing to
natural computing. Idea Group, 2005.
[17] L. N. de Castro and F. J. Von Zuben. Recent developments in biologically
inspired computing. Idea Group Inc, 2005.
[18] M. Dorigo and T. Stützle. Ant Colony Optimization. MIT Press, 2004.
[19] S. Droste, T. Jansen, and I. Wegener. Upper and lower bounds for
randomized search heuristics in black-box optimization. Theory of
Computing Systems, 39(4):525–544, 2006.
1.7. Bibliography 23
[20] A. P. Engelbrecht. Computational Intelligence: An Introduction. John
Wiley and Sons, second edition, 2007.
[21] D. Flanagan and Y. Matsumoto. The Ruby Programming Language.
O’Reilly Media, 2008.
[22] N. Forbes. Biologically inspired computing. Computing in Science and
Engineering, 2(6):83–87, 2000.
[23] N. Forbes. Imitation of Life: How Biology Is Inspiring Computing.
The MIT Press, 2005.
[24] K. Fukunaga. Introduction to Statistical Pattern Recognition. Academic
Press, 1990.
[25] F. Glover and G. A. Kochenberger. Handbook of Metaheuristics.
Springer, 2003.
[26] D. E. Goldberg. Genetic Algorithms in Search, Optimization, and
Machine Learning. Addison-Wesley, 1989.
[27] I. Guyon and A. Elisseeff. An introduction to variable and feature
selection. Journal of Machine Learning Research, 3:1157–1182, 2003.
[28] J. H. Holland. Adaptation in natural and artificial systems: An in-
troductory analysis with applications to biology, control, and artificial
intelligence. University of Michigan Press, 1975.
[29] R. Horst, P. M. Pardalos, and N. V. Thoai. Introduction to Global
Optimization. Kluwer Academic Publishers, 2nd edition, 2000.
[30] J. Kennedy, R. C. Eberhart, and Y. Shi. Swarm Intelligence. Morgan
Kaufmann, 2001.
[31] J. R. Koza, M. A. Keane, M. J. Streeter, W. Mydlowec, J. Yu, and
G. Lanza. Genetic Programming IV: RoutineHuman-Competitive
Machine Intelligence. Springer, 2003.
[32] M. Kudo and J. Sklansky. Comparison of algorithms that select features
for pattern classifiers. Pattern Recognition, 33:25–41, 2000.
[33] R. M. Lewis, V. T., and M. W. Trosset. Direct search methods: then
and now. Journal of Computational and Applied Mathematics, 124:191–
207, 2000.
[34] G. F. Luger and W. A. Stubblefield. Artificial Intelligence: Structures
and Strategies for Complex Problem Solving. Benjamin/Cummings
Pub. Co., second edition, 1993.
24 Chapter 1. Introduction
[35] S. Luke. Essentials of Metaheuristics. Lulu, 2010. available at
http://cs.gmu.edu/∼sean/book/metaheuristics/.
[36] P. Marrow. Nature-inspired computing technology and applications.
BT Technology Journal, 18(4):13–23, 2000.
[37] Z. Michalewicz and D. B. Fogel. How to Solve It: Modern Heuristics.
Springer, 2004.
[38] C. H. Papadimitriou and K. Steiglitz. Combinatorial Optimization:
Algorithms and Complexity. Courier Dover Publications, 1998.
[39] R. Paton. Computing With Biological Metaphors, chapter Introduction
to computing with biological metaphors, pages 1–8. Chapman & Hall,
1994.
[40] G. Paǔn. Bio-inspired computing paradigms (natural computing).
Unconventional Programming Paradigms, 3566:155–160, 2005.
[41] W. Pedrycz. Computational Intelligence: An Introduction. CRC Press,
1997.
[42] H. Robbins. Some aspects of the sequential design of experiments. Bull.
Amer. Math. Soc., 58:527–535, 1952.
[43] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach.
Prentice Hall, third edition, 2009.
[44] A. Sloman. Evolving Knowledge in Natural Science and Artificial
Intelligence, chapter Must intelligent systems be scruffy? Pitman,
1990.
[45] J. C. Spall. Introduction to stochastic search and optimization: estima-
tion, simulation, and control. John Wiley and Sons, 2003.
[46] E. G. Talbi. Metaheuristics: From Design to Implementation. John
Wiley and Sons, 2009.
[47] D. Thomas, C. Fowler, and A. Hunt. Programming Ruby: The Prag-
matic Programmers’ Guide. Pragmatic Bookshelf, second edition, 2004.
[48] A. Törn, M. M. Ali, and S. Viitanen. Stochastic global optimization:
Problem classes and solution techniques. Journal of Global Optimiza-
tion, 14:437–447, 1999.
[49] T. Weise. Global Optimization Algorithms - Theory and Application.
(Self Published), 2009-06-26 edition, 2007.
[50] D. H. Wolpert and W. G. Macready. No free lunch theorems for search.
Technical report, Santa Fe Institute, Sante Fe, NM, USA, 1995.
1.7. Bibliography 25
[51] D. H. Wolpert and W. G. Macready. No free lunch theorems for opti-
mization. IEEE Transactions on Evolutionary Computation, 1(67):67–
82, 1997.
[52] L. A. Zadeh, G. J. Klir, and B. Yuan. Fuzzy sets, fuzzy logic, and fuzzy
systems: selected papers. World Scientific, 1996.
26 Chapter 1. Introduction
Part II
Algorithms
27
Chapter 2
Stochastic Algorithms
2.1 Overview
This chapter describes Stochastic Algorithms.
2.1.1 Stochastic Optimization
The majority of the algorithms to be described in this book are comprised
of probabilistic and stochastic processes. What differentiates the ‘stochastic
algorithms’ in this chapter from the remaining algorithms is the specific lack
of 1) an inspiring system, and 2) a metaphorical explanation. Both ‘inspira-
tion’ and ‘metaphor’ refer to the descriptive elements in the standardized
algorithm description.
These described algorithms are predominately global optimization al-
gorithms and metaheuristics that manage the application of an embedded
neighborhood exploring (local) search procedure. As such, with the excep-
tion of ‘Stochastic Hill Climbing’ and ‘Random Search’ the algorithms may
be considered extensions of the multi-start search (also known as multi-
restart search). This set of algorithms provide various different strategies by
which ‘better’ and varied starting points can be generated and issued to a
neighborhood searching technique for refinement, a process that is repeated
with potentially improving or unexplored areas to search.
29
30 Chapter 2. Stochastic Algorithms
2.2 Random Search
Random Search, RS, Blind Random Search, Blind Search, Pure Random
Search, PRS
2.2.1 Taxonomy
Random search belongs to the fields of Stochastic Optimization and Global
Optimization. Random search is a direct search method as it does not
require derivatives to search a continuous domain. This base approach is
related to techniques that provide small improvements such as Directed
Random Search, and Adaptive Random Search (Section 2.3).
2.2.2 Strategy
The strategy of Random Search is to sample solutions from across the entire
search space using a uniform probability distribution. Each future sample
is independent of the samples that come before it.
2.2.3 Procedure
Algorithm 2.2.1 provides a pseudocode listing of the Random Search Algo-
rithm for minimizing a cost function.
Algorithm 2.2.1: Pseudocode for Random Search.
Input: NumIterations, ProblemSize, SearchSpace
Output: Best
Best ← ∅;1
foreach iteri ∈ NumIterations do2
candidatei ← RandomSolution(ProblemSize, SearchSpace);3
if Cost(candidatei) < Cost(Best) then4
Best ← candidatei;5
end6
end7
return Best;8
2.2.4 Heuristics
� Random search is minimal in that it only requires a candidate solution
construction routine and a candidate solution evaluation routine, both
of which may be calibrated using the approach.
2.2. Random Search 31
� The worst case performance for Random Search for locating the
optima is worse than an Enumeration of the search domain, given
that Random Search has no memory and can blindly resample.
� Random Search can return a reasonable approximation of the optimal
solution within a reasonable time under low problem dimensionality,
although the approach does not scale well with problem size (such as
the number of dimensions).
� Care must be taken with some problem domains to ensure that random
candidate solution construction is unbiased
� The results of a Random Search can be used to seed another search
technique, like a local search technique (such as the Hill Climbing algo-
rithm) that can be used to locate the best solution in the neighborhood
of the ‘good’ candidate solution.
2.2.5 Code Listing
Listing 2.1 provides an example of the Random Search Algorithm imple-
mented in the Ruby Programming Language. In the example, the algorithm
runs for a fixed number of iterations and returns the best candidate solution
discovered. The example problem is an instance of a continuous function
optimization that seeks min f(x) where f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and
n = 2. The optimal solution for this basin function is (v0, . . . , vn−1) = 0.0.
1 def objective_function(vector)
2 return vector.inject(0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def random_vector(minmax)
6 return Array.new(minmax.size) do |i|
7 minmax[i][0] + ((minmax[i][1] - minmax[i][0]) * rand())
8 end
9 end
10
11 def search(search_space, max_iter)
12 best = nil
13 max_iter.times do |iter|
14 candidate = {}
15 candidate[:vector] = random_vector(search_space)
16 candidate[:cost] = objective_function(candidate[:vector])
17 best = candidate if best.nil? or candidate[:cost] < best[:cost]
18 puts " > iteration=#{(iter+1)}, best=#{best[:cost]}"
19 end
20 return best
21 end
22
23 if __FILE__ == $0
24 # problem configuration
25 problem_size = 2
26 search_space = Array.new(problem_size) {|i| [-5, +5]}
32 Chapter 2. Stochastic Algorithms
27 # algorithm configuration
28 max_iter = 100
29 # execute the algorithm
30 best = search(search_space, max_iter)
31 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
32 end
Listing 2.1: Random Search in Ruby
2.2.6 References
Primary Sources
There is no seminal specification of the Random Search algorithm, rather
there are discussions of the general approach and related random search
methods from the 1950s through to the 1970s. This was around the time
that pattern and direct search methods were actively researched.Brooks is
credited with the so-called ‘pure random search’ [1]. Two seminal reviews
of ‘random search methods’ of the time include: Karnopp [2] and prhaps
Kul’chitskii [3].
Learn More
For overviews of Random Search Methods see Zhigljavsky [9], Solis and
Wets [4], and also White [7] who provide an insightful review article. Spall
provides a detailed overview of the field of Stochastic Optimization, including
the Random Search method [5] (for example, see Chapter 2). For a shorter
introduction by Spall, see [6] (specifically Section 6.2). Also see Zabinsky
for another detailed review of the broader field [8].
2.2.7 Bibliography
[1] S. H. Brooks. A discussion of random methods for seeking maxima.
Operations Research, 6(2):244–251, 1958.
[2] D. C. Karnopp. Random search techniques for optimization problems.
Automatica, 1(2–3):111–121, 1963.
[3] O. Y. Kul’chitskii. Random-search algorithm for extrema in functional
space under conditions of partial uncertainty. Cybernetics and Systems
Analysis, 12(5):794–801, 1976.
[4] F. J. Solis and J. B. Wets. Minimization by random search techniques.
Mathematics of Operations Research, 6:19–30, 1981.
[5] J. C. Spall. Introduction to stochastic search and optimization: estima-
tion, simulation, and control. John Wiley and Sons, 2003.
2.2. Random Search 33
[6] J. C. Spall. Handbook of computational statistics: concepts and methods,
chapter 6. Stochastic Optimization, pages 169–198. Springer, 2004.
[7] R. C. White. A survey of random methods for parameter optimization.
Simulation, 17(1):197–205, 1971.
[8] Z. B. Zabinsky. Stochastic adaptive search for global optimization. Kluwer
Academic Publishers, 2003.
[9] A. A. Zhigljavsky. Theory of Global Random Search. Kluwer Academic,
1991.
34 Chapter 2. Stochastic Algorithms
2.3 Adaptive Random Search
Adaptive Random Search, ARS, Adaptive Step Size Random Search, ASSRS,
Variable Step-Size Random Search.
2.3.1 Taxonomy
The Adaptive Random Search algorithm belongs to the general set of
approaches known as Stochastic Optimization and Global Optimization. It
is a direct search method in that it does not require derivatives to navigate
the search space. Adaptive Random Search is an extension of the Random
Search (Section 2.2) and Localized Random Search algorithms.
2.3.2 Strategy
The Adaptive Random Search algorithm was designed to address the lim-
itations of the fixed step size in the Localized Random Search algorithm.
The strategy for Adaptive Random Search is to continually approximate
the optimal step size required to reach the global optimum in the search
space. This is achieved by trialling and adopting smaller or larger step sizes
only if they result in an improvement in the search performance.
The Strategy of the Adaptive Step Size Random Search algorithm (the
specific technique reviewed) is to trial a larger step in each iteration and
adopt the larger step if it results in an improved result. Very large step
sizes are trialled in the same manner although with a much lower frequency.
This strategy of preferring large moves is intended to allow the technique to
escape local optima. Smaller step sizes are adopted if no improvement is
made for an extended period.
2.3.3 Procedure
Algorithm 2.3.1 provides a pseudocode listing of the Adaptive Random
Search Algorithm for minimizing a cost function based on the specification
for ‘Adaptive Step-Size Random Search’ by Schummer and Steiglitz [6].
2.3.4 Heuristics
� Adaptive Random Search was designed for continuous function opti-
mization problem domains.
� Candidates with equal cost should be considered improvements to
allow the algorithm to make progress across plateaus in the response
surface.
� Adaptive Random Search may adapt the search direction in addition
to the step size.
2.3. Adaptive Random Search 35
Algorithm 2.3.1: Pseudocode for Adaptive Random Search.
Input: Itermax, Problemsize, SearchSpace, StepSize
init
factor,
StepSizesmallfactor, StepSize
large
factor, StepSize
iter
factor,
NoChangemax
Output: S
NoChangecount ← 0;1
StepSizei ← InitializeStepSize(SearchSpace, StepSize
init
factor);2
S ← RandomSolution(Problemsize, SearchSpace);3
for i = 0 to Itermax do4
S1 ← TakeStep(SearchSpace, S, StepSizei);5
StepSizelargei ← 0;6
if i modStepSizeiterfactor then7
StepSizelargei ← StepSizei × StepSize
large
factor;8
else9
StepSizelargei ← StepSizei × StepSize
small
factor;10
end11
S2 ← TakeStep(SearchSpace, S, StepSize
large
i );12
if Cost(S1)≤Cost(S) —— Cost(S2)≤Cost(S) then13
if Cost(S2)<Cost(S1) then14
S ← S2;15
StepSizei ← StepSize
large
i ;16
else17
S ← S1;18
end19
NoChangecount ← 0;20
else21
NoChangecount ← NoChangecount + 1;22
if NoChangecount > NoChangemax then23
NoChangecount ← 0;24
StepSizei ←
StepSizei
StepSizesmallfactor
;
25
end26
end27
end28
return S;29
36 Chapter 2. Stochastic Algorithms
� The step size may be adapted for all parameters, or for each parameter
individually.
2.3.5 Code Listing
Listing 2.2 provides an example of the Adaptive Random Search Algorithm
implemented in the Ruby Programming Language, based on the specification
for ‘Adaptive Step-Size Random Search’ by Schummer and Steiglitz [6].
In the example, the algorithm runs for a fixed number of iterations and
returns the best candidate solution discovered. The example problem is an
instance of a continuous function optimization that seeks min f(x) where
f =
∑n
i=1 x
2
i , −5.0 < xi < 5.0 and n = 2. The optimal solution for this
basin function is (v0, . . . , vn−1) = 0.0.
1 def objective_function(vector)
2 return vector.inject(0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def rand_in_bounds(min, max)
6 return min + ((max-min) * rand())
7 end
8
9 def random_vector(minmax)
10 return Array.new(minmax.size) do |i|
11 rand_in_bounds(minmax[i][0], minmax[i][1])
12 end
13 end
14
15 def take_step(minmax, current, step_size)
16 position = Array.new(current.size)
17 position.size.times do |i|
18 min = [minmax[i][0], current[i]-step_size].max
19 max = [minmax[i][1], current[i]+step_size].min
20 position[i] = rand_in_bounds(min, max)
21 end
22 return position
23 end
24
25 def large_step_size(iter, step_size, s_factor, l_factor, iter_mult)
26 return step_size * l_factor if iter>0 and iter.modulo(iter_mult) == 0
27 return step_size * s_factor
28 end
29
30 def take_steps(bounds, current, step_size, big_stepsize)
31 step, big_step = {}, {}
32 step[:vector] = take_step(bounds, current[:vector], step_size)
33 step[:cost] = objective_function(step[:vector])
34 big_step[:vector] = take_step(bounds,current[:vector],big_stepsize)
35 big_step[:cost] = objective_function(big_step[:vector])
36 return step, big_step
37 end
38
39 def search(max_iter, bounds, init_factor, s_factor, l_factor, iter_mult,
max_no_impr)
2.3. Adaptive Random Search 37
40 step_size = (bounds[0][1]-bounds[0][0]) * init_factor
41 current, count = {}, 0
42 current[:vector] = random_vector(bounds)
43 current[:cost] = objective_function(current[:vector])
44 max_iter.times do |iter|
45 big_stepsize = large_step_size(iter, step_size, s_factor, l_factor,
iter_mult)
46 step, big_step = take_steps(bounds, current, step_size, big_stepsize)
47 if step[:cost] <= current[:cost] or big_step[:cost] <= current[:cost]
48 if big_step[:cost] <= step[:cost]
49 step_size, current = big_stepsize, big_step
50 else
51 current = step
52 end
53 count = 0
54 else
55 count += 1
56 count, stepSize = 0, (step_size/s_factor) if count >= max_no_impr
57 end
58 puts " > iteration #{(iter+1)}, best=#{current[:cost]}"
59 end
60 return current
61 end
62
63 if __FILE__ == $0
64 # problem configuration
65 problem_size = 2
66 bounds = Array.new(problem_size) {|i| [-5, +5]}
67 # algorithm configuration
68 max_iter = 1000
69 init_factor = 0.05
70 s_factor = 1.3
71 l_factor = 3.0
72 iter_mult = 10
73 max_no_impr = 30
74 # execute the algorithm
75 best = search(max_iter, bounds, init_factor, s_factor, l_factor,
iter_mult, max_no_impr)
76 puts "Done.Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
77 end
Listing 2.2: Adaptive Random Search in Ruby
2.3.6 References
Primary Sources
Many works in the 1960s and 1970s experimented with variable step sizes for
Random Search methods. Schummer and Steiglitz are commonly credited
the adaptive step size procedure, which they called ‘Adaptive Step-Size
Random Search’ [6]. Their approach only modifies the step size based on an
approximation of the optimal step size required to reach the global optima.
Kregting and White review adaptive random search methods and propose
38 Chapter 2. Stochastic Algorithms
an approach called ‘Adaptive Directional Random Search’ that modifies
both the algorithms step size and direction in response to the cost function
[2].
Learn More
White reviews extensions to Rastrigin’s ‘Creeping Random Search’ [4] (fixed
step size) that use probabilistic step sizes drawn stochastically from uniform
and probabilistic distributions [7]. White also reviews works that propose
dynamic control strategies for the step size, such as Karnopp [1] who proposes
increases and decreases to the step size based on performance over very
small numbers of trials. Schrack and Choit review random search methods
that modify their step size in order to approximate optimal moves while
searching, including the property of reversal [5]. Masri et al. describe an
adaptive random search strategy that alternates between periods of fixed
and variable step sizes [3].
2.3.7 Bibliography
[1] D. C. Karnopp. Random search techniques for optimization problems.
Automatica, 1(2–3):111–121, 1963.
[2] J. Kregting and R. C. White. Adaptive random search. Technical Report
TH-Report 71-E-24, Eindhoven University of Technology, Eindhoven,
Netherlands, 1971.
[3] S. F. Masri, G. A. Bekey, and F. B. Safford. Global optimization
algorithm using adaptive random search. Applied Mathematics and
Computation, 7(4):353–376, 1980.
[4] L. A. Rastrigin. The convergence of the random search method in the
extremal control of a many parameter system. Automation and Remote
Control, 24:1337–1342, 1963.
[5] G. Schrack and M. Choit. Optimized relative step size random searches.
Mathematical Programming, 10(1):230–244, 1976.
[6] M. Schumer and K. Steiglitz. Adaptive step size random search. IEEE
Transactions on Automatic Control, 13(3):270–276, 1968.
[7] R. C. White. A survey of random methods for parameter optimization.
Simulation, 17(1):197–205, 1971.
2.4. Stochastic Hill Climbing 39
2.4 Stochastic Hill Climbing
Stochastic Hill Climbing, SHC, Random Hill Climbing, RHC, Random
Mutation Hill Climbing, RMHC.
2.4.1 Taxonomy
The Stochastic Hill Climbing algorithm is a Stochastic Optimization algo-
rithm and is a Local Optimization algorithm (contrasted to Global Opti-
mization). It is a direct search technique, as it does not require derivatives
of the search space. Stochastic Hill Climbing is an extension of deterministic
hill climbing algorithms such as Simple Hill Climbing (first-best neighbor),
Steepest-Ascent Hill Climbing (best neighbor), and a parent of approaches
such as Parallel Hill Climbing and Random-Restart Hill Climbing.
2.4.2 Strategy
The strategy of the Stochastic Hill Climbing algorithm is iterate the process
of randomly selecting a neighbor for a candidate solution and only accept it
if it results in an improvement. The strategy was proposed to address the
limitations of deterministic hill climbing techniques that were likely to get
stuck in local optima due to their greedy acceptance of neighboring moves.
2.4.3 Procedure
Algorithm 2.4.1 provides a pseudocode listing of the Stochastic Hill Climbing
algorithm for minimizing a cost function, specifically the Random Mutation
Hill Climbing algorithm described by Forrest and Mitchell applied to a
maximization optimization problem [3].
Algorithm 2.4.1: Pseudocode for Stochastic Hill Climbing.
Input: Itermax, ProblemSize
Output: Current
Current ← RandomSolution(ProblemSize);1
foreach iteri ∈ Itermax do2
Candidate ← RandomNeighbor(Current);3
if Cost(Candidate) ≥ Cost(Current) then4
Current ← Candidate;5
end6
end7
return Current;8
40 Chapter 2. Stochastic Algorithms
2.4.4 Heuristics
� Stochastic Hill Climbing was designed to be used in discrete domains
with explicit neighbors such as combinatorial optimization (compared
to continuous function optimization).
� The algorithm’s strategy may be applied to continuous domains by
making use of a step-size to define candidate-solution neighbors (such
as Localized Random Search and Fixed Step-Size Random Search).
� Stochastic Hill Climbing is a local search technique (compared to
global search) and may be used to refine a result after the execution
of a global search algorithm.
� Even though the technique uses a stochastic process, it can still get
stuck in local optima.
� Neighbors with better or equal cost should be accepted, allowing the
technique to navigate across plateaus in the response surface.
� The algorithm can be restarted and repeated a number of times after
it converges to provide an improved result (called Multiple Restart
Hill Climbing).
� The procedure can be applied to multiple candidate solutions concur-
rently, allowing multiple algorithm runs to be performed at the same
time (called Parallel Hill Climbing).
2.4.5 Code Listing
Listing 2.3 provides an example of the Stochastic Hill Climbing algorithm
implemented in the Ruby Programming Language, specifically the Random
Mutation Hill Climbing algorithm described by Forrest and Mitchell [3].
The algorithm is executed for a fixed number of iterations and is applied to
a binary string optimization problem called ‘One Max’. The objective of
this maximization problem is to prepare a string of all ‘1’ bits, where the
cost function only reports the number of bits in a given string.
1 def onemax(vector)
2 return vector.inject(0.0){|sum, v| sum + ((v=="1") ? 1 : 0)}
3 end
4
5 def random_bitstring(num_bits)
6 return Array.new(num_bits){|i| (rand<0.5) ? "1" : "0"}
7 end
8
9 def random_neighbor(bitstring)
10 mutant = Array.new(bitstring)
11 pos = rand(bitstring.size)
12 mutant[pos] = (mutant[pos]=='1') ? '0' : '1'
2.4. Stochastic Hill Climbing 41
13 return mutant
14 end
15
16 def search(max_iterations, num_bits)
17 candidate = {}
18 candidate[:vector] = random_bitstring(num_bits)
19 candidate[:cost] = onemax(candidate[:vector])
20 max_iterations.times do |iter|
21 neighbor = {}
22 neighbor[:vector] = random_neighbor(candidate[:vector])
23 neighbor[:cost] = onemax(neighbor[:vector])
24 candidate = neighbor if neighbor[:cost] >= candidate[:cost]
25 puts " > iteration #{(iter+1)}, best=#{candidate[:cost]}"
26 break if candidate[:cost] == num_bits
27 end
28 return candidate
29 end
30
31 if __FILE__ == $0
32 # problem configuration
33 num_bits = 64
34 # algorithm configuration
35 max_iterations = 1000
36 # execute the algorithm
37 best = search(max_iterations, num_bits)
38 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].join}"
39 end
Listing 2.3: Stochastic Hill Climbing in Ruby
2.4.6 References
Primary Sources
Perhaps the most popular implementation of the Stochastic Hill Climbing
algorithm is by Forrest and Mitchell, who proposed the Random Muta-
tion Hill Climbing (RMHC) algorithm (with communication from Richard
Palmer) in a study that investigated the behavior of the genetic algorithm
on a deceptive class of (discrete) bit-string optimization problems called
‘royal road’ functions [3]. The RMHC was compared to two other hill
climbing algorithms in addition to the genetic algorithm, specifically: the
Steepest-Ascent Hill Climber, and the Next-Ascent Hill Climber. This study
was then followed up by Mitchell and Holland [5].
Jules and Wattenberg were also early to consider stochastic hill climbing
as an approach to compare to the genetic algorithm [4]. Skalak applied the
RMHC algorithm to a single long bit-string that represented a number of
prototype vectors for use in classification[8].
42 Chapter 2. Stochastic Algorithms
Learn More
The Stochastic Hill Climbing algorithm is related to the genetic algorithm
without crossover. Simplified version’s of the approach are investigated for
bit-string based optimization problems with the population size of the genetic
algorithm reduced to one. The general technique has been investigated
under the names Iterated Hillclimbing [6], ES(1+1,m,hc) [7], Random Bit
Climber [2], and (1+1)-Genetic Algorithm [1]. This main difference between
RMHC and ES(1+1) is that the latter uses a fixed probability of a mutation
for each discrete element of a solution (meaning the neighborhood size is
probabilistic), whereas RMHC will only stochastically modify one element.
2.4.7 Bibliography
[1] T. Bäck. Optimal mutation rates in genetic search. In Proceedings of the
Fifth International Conference on Genetic Algorithms, pages 2–9, 1993.
[2] L. Davis. Bit-climbing, representational bias, and test suite design. In
Proceedings of the fourth international conference on genetic algorithms,
pages 18–23, 1991.
[3] S. Forrest and M. Mitchell. Relative building-block fitness and the
building-block hypothesis. In Foundations of Genetic Algorithms 2,
pages 109–126. Morgan Kaufmann, 1993.
[4] A. Juels and M. Wattenberg. Stochastic hill climbing as a baseline
method for evaluating genetic algorithms. Technical report, University
of California, Berkeley, 1994.
[5] M. Mitchell and J. H. Holland. When will a genetic algorithm outperform
hill climbing? In Proceedings of the 5th International Conference on
Genetic Algorithms. Morgan Kaufmann Publishers Inc., 1993.
[6] H. Mühlenbein. Evolution in time and space - the parallel genetic
algorithm. In Foundations of Genetic Algorithms, 1991.
[7] H. Mühlenbein. How genetic algorithms really work: I. mutation and
hillclimbing. In Parallel Problem Solving from Nature 2, pages 15–26,
1992.
[8] D. B. Skalak. Prototype and feature selection by sampling and ran-
dom mutation hill climbing algorithms. In Proceedings of the eleventh
international conference on machine learning, pages 293–301. Morgan
Kaufmann, 1994.
2.5. Iterated Local Search 43
2.5 Iterated Local Search
Iterated Local Search, ILS.
2.5.1 Taxonomy
Iterated Local Search is a Metaheuristic and a Global Optimization tech-
nique. It is an extension of Mutli Start Search and may be considered a
parent of many two-phase search approaches such as the Greedy Random-
ized Adaptive Search Procedure (Section 2.8) and Variable Neighborhood
Search (Section 2.7).
2.5.2 Strategy
The objective of Iterated Local Search is to improve upon stochastic Mutli-
Restart Search by sampling in the broader neighborhood of candidate
solutions and using a Local Search technique to refine solutions to their
local optima. Iterated Local Search explores a sequence of solutions created
as perturbations of the current best solution, the result of which is refined
using an embedded heuristic.
2.5.3 Procedure
Algorithm 2.5.1 provides a pseudocode listing of the Iterated Local Search
algorithm for minimizing a cost function.
Algorithm 2.5.1: Pseudocode for Iterated Local Search.
Input:
Output: Sbest
Sbest ← ConstructInitialSolution();1
Sbest ← LocalSearch();2
SearchHistory ← Sbest;3
while ¬ StopCondition() do4
Scandidate ← Perturbation(Sbest, SearchHistory);5
Scandidate ← LocalSearch(Scandidate);6
SearchHistory ← Scandidate;7
if AcceptanceCriterion(Sbest, Scandidate, SearchHistory) then8
Sbest ← Scandidate;9
end10
end11
return Sbest;12
44 Chapter 2. Stochastic Algorithms
2.5.4 Heuristics
� Iterated Local Search was designed for and has been predominately
applied to discrete domains, such as combinatorial optimization prob-
lems.
� The perturbation of the current best solution should be in a neighbor-
hood beyond the reach of the embedded heuristic and should not be
easily undone.
� Perturbations that are too small make the algorithm too greedy,
perturbations that are too large make the algorithm too stochastic.
� The embedded heuristic is most commonly a problem-specific local
search technique.
� The starting point for the search may be a randomly constructed
candidate solution, or constructed using a problem-specific heuristic
(such as nearest neighbor).
� Perturbations can be made deterministically, although stochastic and
probabilistic (adaptive based on history) are the most common.
� The procedure may store as much or as little history as needed to
be used during perturbation and acceptance criteria. No history
represents a random walk in a larger neighborhood of the best solution
and is the most common implementation of the approach.
� The simplest and most common acceptance criteria is an improvement
in the cost of constructed candidate solutions.
2.5.5 Code Listing
Listing 2.4 provides an example of the Iterated Local Search algorithm
implemented in the Ruby Programming Language. The algorithm is applied
to the Berlin52 instance of the Traveling Salesman Problem (TSP), taken
from the TSPLIB. The problem seeks a permutation of the order to visit
cities (called a tour) that minimizes the total distance traveled. The optimal
tour distance for Berlin52 instance is 7542 units.
The Iterated Local Search runs for a fixed number of iterations. The
implementation is based on a common algorithm configuration for the TSP,
where a ‘double-bridge move’ (4-opt) is used as the perturbation technique,
and a stochastic 2-opt is used as the embedded Local Search heuristic.
The double-bridge move involves partitioning a permutation into 4 pieces
(a,b,c,d) and putting it back together in a specific and jumbled ordering
(a,d,c,b).
2.5. Iterated Local Search 45
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(permutation, cities)
6 distance =0
7 permutation.each_with_index do |c1, i|
8 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def random_permutation(cities)
15 perm = Array.new(cities.size){|i| i}
16 perm.each_index do |i|
17 r = rand(perm.size-i) + i
18 perm[r], perm[i] = perm[i], perm[r]
19 end
20 return perm
21 end
22
23 def stochastic_two_opt(permutation)
24 perm = Array.new(permutation)
25 c1, c2 = rand(perm.size), rand(perm.size)
26 exclude = [c1]
27 exclude << ((c1==0) ? perm.size-1 : c1-1)
28 exclude << ((c1==perm.size-1) ? 0 : c1+1)
29 c2 = rand(perm.size) while exclude.include?(c2)
30 c1, c2 = c2, c1 if c2 < c1
31 perm[c1...c2] = perm[c1...c2].reverse
32 return perm
33 end
34
35 def local_search(best, cities, max_no_improv)
36 count = 0
37 begin
38 candidate = {:vector=>stochastic_two_opt(best[:vector])}
39 candidate[:cost] = cost(candidate[:vector], cities)
40 count = (candidate[:cost] < best[:cost]) ? 0 : count+1
41 best = candidate if candidate[:cost] < best[:cost]
42 end until count >= max_no_improv
43 return best
44 end
45
46 def double_bridge_move(perm)
47 pos1 = 1 + rand(perm.size / 4)
48 pos2 = pos1 + 1 + rand(perm.size / 4)
49 pos3 = pos2 + 1 + rand(perm.size / 4)
50 p1 = perm[0...pos1] + perm[pos3..perm.size]
51 p2 = perm[pos2...pos3] + perm[pos1...pos2]
52 return p1 + p2
53 end
54
55 def perturbation(cities, best)
46 Chapter 2. Stochastic Algorithms
56 candidate = {}
57 candidate[:vector] = double_bridge_move(best[:vector])
58 candidate[:cost] = cost(candidate[:vector], cities)
59 return candidate
60 end
61
62 def search(cities, max_iterations, max_no_improv)
63 best = {}
64 best[:vector] = random_permutation(cities)
65 best[:cost] = cost(best[:vector], cities)
66 best = local_search(best, cities, max_no_improv)
67 max_iterations.times do |iter|
68 candidate = perturbation(cities, best)
69 candidate = local_search(candidate, cities, max_no_improv)
70 best = candidate if candidate[:cost] < best[:cost]
71 puts " > iteration #{(iter+1)}, best=#{best[:cost]}"
72 end
73 return best
74 end
75
76 if __FILE__ == $0
77 # problemconfiguration
78 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
79 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
80 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
81 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
82 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
83 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
84 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
85 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
86 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
87 # algorithm configuration
88 max_iterations = 100
89 max_no_improv = 50
90 # execute the algorithm
91 best = search(berlin52, max_iterations, max_no_improv)
92 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
93 end
Listing 2.4: Iterated Local Search in Ruby
2.5.6 References
Primary Sources
The definition and framework for Iterated Local Search was described by
Stützle in his PhD dissertation [12]. Specifically he proposed constrains on
what constitutes an Iterated Local Search algorithm as 1) a single chain of
candidate solutions, and 2) the method used to improve candidate solutions
occurs within a reduced space by a black-box heuristic. Stützle does not take
credit for the approach, instead highlighting specific instances of Iterated
Local Search from the literature, such as ‘iterated descent’ [1], ‘large-step
Markov chains’ [7], ‘iterated Lin-Kernighan’ [3], ‘chained local optimization’
2.5. Iterated Local Search 47
[6], as well as [2] that introduces the principle, and [4] that summarized it
(list taken from [8]).
Learn More
Two early technical reports by Stützle that present applications of Iterated
Local Search include a report on the Quadratic Assignment Problem [10],
and another on the permutation flow shop problem [9]. Stützle and Hoos
also published an early paper studying Iterated Local Search for to the TSP
[11]. Lourenco, Martin, and Stützle provide a concise presentation of the
technique, related techniques and the framework, much as it is presented in
Stützle’s dissertation [5]. The same author’s also preset an authoritative
summary of the approach and its applications as a book chapter [8].
2.5.7 Bibliography
[1] E. B. Baum. Towards practical “neural” computation for combinatorial
optimization problems. In AIP conference proceedings: Neural Networks
for Computing, pages 53–64, 1986.
[2] J. Baxter. Local optima avoidance in depot location. Journal of the
Operational Research Society, 32:815–819, 1981.
[3] D. S. Johnson. Local optimization and the travelling salesman problem.
In Proceedings of the 17th Colloquium on Automata, Languages, and
Programming, pages 446–461, 1990.
[4] D. S. Johnson and L. A. McGeoch. Local Search in Combinatorial
Optimization, chapter The travelling salesman problem: A case study
in local optimization, pages 215–310. John Wiley & Sons, 1997.
[5] H. R. Lourenco, O. Martin, and T. Stützle. A beginners introduction to
iterated local search. In Proceedings 4th Metaheuristics International
Conference (MIC2001), 2001.
[6] O. Martin and S. W. Otto. Combining simulated annealing with local
search heuristics. Annals of Operations Research, 63:57–75, 1996.
[7] O. Martin, S. W. Otto, and E. W. Felten. Large-step markov chains
for the traveling salesman problems. Complex Systems, 5(3):299–326,
1991.
[8] H. Ramalhinho-Lourenco, O. C. Martin, and T. Stützle. Handbook of
Metaheuristics, chapter Iterated Local Search, pages 320–353. Springer,
2003.
[9] T. Stützle. Applying iterated local search to the permutation flow shop
problem. Technical Report AIDA9804, FG Intellektik, TU Darmstadt,
1998.
48 Chapter 2. Stochastic Algorithms
[10] T. Stützle. Iterated local search for the quadratic assignment problem.
Technical Report AIDA-99-03, FG Intellektik, FB Informatik, TU
Darmstadt, 1999.
[11] T. Stützle and H. H. Hoos. Analyzing the run-time behaviour of
iterated local search for the TSP. In Proceedings III Metaheuristics
International Conference, 1999.
[12] T. G. Stützle. Local Search Algorithms for Combinatorial Problems:
Analysis, Improvements, and New Applications. PhD thesis, Darmstadt
University of Technology, Department of Computer Science, 1998.
2.6. Guided Local Search 49
2.6 Guided Local Search
Guided Local Search, GLS.
2.6.1 Taxonomy
The Guided Local Search algorithm is a Metaheuristic and a Global Op-
timization algorithm that makes use of an embedded Local Search algo-
rithm. It is an extension to Local Search algorithms such as Hill Climbing
(Section 2.4) and is similar in strategy to the Tabu Search algorithm (Sec-
tion 2.10) and the Iterated Local Search algorithm (Section 2.5).
2.6.2 Strategy
The strategy for the Guided Local Search algorithm is to use penalties to
encourage a Local Search technique to escape local optima and discover the
global optima. A Local Search algorithm is run until it gets stuck in a local
optima. The features from the local optima are evaluated and penalized,
the results of which are used in an augmented cost function employed by the
Local Search procedure. The Local Search is repeated a number of times
using the last local optima discovered and the augmented cost function that
guides exploration away from solutions with features present in discovered
local optima.
2.6.3 Procedure
Algorithm 2.6.1 provides a pseudocode listing of the Guided Local Search
algorithm for minimization. The Local Search algorithm used by the
Guided Local Search algorithm uses an augmented cost function in the form
h(s) = g(s) +λ ·
∑M
i=1 fi, where h(s) is the augmented cost function, g(s) is
the problem cost function,λ is the ‘regularization parameter’ (a coefficient
for scaling the penalties), s is a locally optimal solution of M features,
and fi is the i’th feature in locally optimal solution. The augmented cost
function is only used by the local search procedure, the Guided Local Search
algorithm uses the problem specific cost function without augmentation.
Penalties are only updated for those features in a locally optimal solution
that maximize utility, updated by adding 1 to the penalty for the future
(a counter). The utility for a feature is calculated as Ufeature =
Cfeature
1+Pfeature
,
where Ufeature is the utility for penalizing a feature (maximizing), Cfeature
is the cost of the feature, and Pfeature is the current penalty for the feature.
2.6.4 Heuristics
� The Guided Local Search procedure is independent of the Local
Search procedure embedded within it. A suitable domain-specific
50 Chapter 2. Stochastic Algorithms
Algorithm 2.6.1: Pseudocode for Guided Local Search.
Input: Itermax, λ
Output: Sbest
fpenalties ← ∅;1
Sbest ← RandomSolution();2
foreach Iteri ∈ Itermax do3
Scurr ← LocalSearch(Sbest, λ, fpenalties);4
futilities ← CalculateFeatureUtilities(Scurr, fpenalties);5
fpenalties ← UpdateFeaturePenalties(Scurr, fpenalties,6
futilities);
if Cost(Scurr) ≤ Cost(Sbest) then7
Sbest ← Scurr;8
end9
end10
return Sbest;11
search procedure should be identified and employed.
� The Guided Local Search procedure may need to be executed for
thousands to hundreds-of-thousands of iterations, each iteration of
which assumes a run of a Local Search algorithm to convergence.
� The algorithm was designed for discrete optimization problems where
a solution is comprised of independently assessable ‘features’ such as
Combinatorial Optimization, although it has been applied to continu-
ous function optimization modeled as binary strings.
� The λ parameter is a scaling factor for feature penalization that must
be in the same proportion to the candidate solution costs from the
specific problem instance to which the algorithm is being applied.
As such, the value for λ must be meaningful when used within the
augmented cost function (such as when it is added to a candidate
solution cost in minimization and subtracted from a cost in the case
of a maximization problem).
2.6.5 CodeListing
Listing 2.5 provides an example of the Guided Local Search algorithm
implemented in the Ruby Programming Language. The algorithm is applied
to the Berlin52 instance of the Traveling Salesman Problem (TSP), taken
from the TSPLIB. The problem seeks a permutation of the order to visit
cities (called a tour) that minimizes the total distance traveled. The optimal
tour distance for Berlin52 instance is 7542 units.
2.6. Guided Local Search 51
The implementation of the algorithm for the TSP was based on the
configuration specified by Voudouris in [7]. A TSP-specific local search
algorithm is used called 2-opt that selects two points in a permutation and
reconnects the tour, potentially untwisting the tour at the selected points.
The stopping condition for 2-opt was configured to be a fixed number of
non-improving moves.
The equation for setting λ for TSP instances is λ = α · cost(optima)
N
,
where N is the number of cities, cost(optima) is the cost of a local optimum
found by a local search, and α ∈ (0, 1] (around 0.3 for TSP and 2-opt).
The cost of a local optima was fixed to the approximated value of 15000
for the Berlin52 instance. The utility function for features (edges) in the
TSP is Uedge =
Dedge
1+Pedge
, where Uedge is the utility for penalizing an edge
(maximizing), Dedge is the cost of the edge (distance between cities) and
Pedge is the current penalty for the edge.
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def random_permutation(cities)
6 perm = Array.new(cities.size){|i| i}
7 perm.each_index do |i|
8 r = rand(perm.size-i) + i
9 perm[r], perm[i] = perm[i], perm[r]
10 end
11 return perm
12 end
13
14 def stochastic_two_opt(permutation)
15 perm = Array.new(permutation)
16 c1, c2 = rand(perm.size), rand(perm.size)
17 exclude = [c1]
18 exclude << ((c1==0) ? perm.size-1 : c1-1)
19 exclude << ((c1==perm.size-1) ? 0 : c1+1)
20 c2 = rand(perm.size) while exclude.include?(c2)
21 c1, c2 = c2, c1 if c2 < c1
22 perm[c1...c2] = perm[c1...c2].reverse
23 return perm
24 end
25
26 def augmented_cost(permutation, penalties, cities, lambda)
27 distance, augmented = 0, 0
28 permutation.each_with_index do |c1, i|
29 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
30 c1, c2 = c2, c1 if c2 < c1
31 d = euc_2d(cities[c1], cities[c2])
32 distance += d
33 augmented += d + (lambda * (penalties[c1][c2]))
34 end
35 return [distance, augmented]
36 end
37
38 def cost(cand, penalties, cities, lambda)
52 Chapter 2. Stochastic Algorithms
39 cost, acost = augmented_cost(cand[:vector], penalties, cities, lambda)
40 cand[:cost], cand[:aug_cost] = cost, acost
41 end
42
43 def local_search(current, cities, penalties, max_no_improv, lambda)
44 cost(current, penalties, cities, lambda)
45 count = 0
46 begin
47 candidate = {:vector=> stochastic_two_opt(current[:vector])}
48 cost(candidate, penalties, cities, lambda)
49 count = (candidate[:aug_cost] < current[:aug_cost]) ? 0 : count+1
50 current = candidate if candidate[:aug_cost] < current[:aug_cost]
51 end until count >= max_no_improv
52 return current
53 end
54
55 def calculate_feature_utilities(penal, cities, permutation)
56 utilities = Array.new(permutation.size,0)
57 permutation.each_with_index do |c1, i|
58 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
59 c1, c2 = c2, c1 if c2 < c1
60 utilities[i] = euc_2d(cities[c1], cities[c2]) / (1.0 + penal[c1][c2])
61 end
62 return utilities
63 end
64
65 def update_penalties!(penalties, cities, permutation, utilities)
66 max = utilities.max()
67 permutation.each_with_index do |c1, i|
68 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
69 c1, c2 = c2, c1 if c2 < c1
70 penalties[c1][c2] += 1 if utilities[i] == max
71 end
72 return penalties
73 end
74
75 def search(max_iterations, cities, max_no_improv, lambda)
76 current = {:vector=>random_permutation(cities)}
77 best = nil
78 penalties = Array.new(cities.size){ Array.new(cities.size, 0) }
79 max_iterations.times do |iter|
80 current=local_search(current, cities, penalties, max_no_improv, lambda)
81 utilities=calculate_feature_utilities(penalties,cities,current[:vector])
82 update_penalties!(penalties, cities, current[:vector], utilities)
83 best = current if best.nil? or current[:cost] < best[:cost]
84 puts " > iter=#{(iter+1)}, best=#{best[:cost]}, aug=#{best[:aug_cost]}"
85 end
86 return best
87 end
88
89 if __FILE__ == $0
90 # problem configuration
91 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
92 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
93 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
94 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
2.6. Guided Local Search 53
95 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
96 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
97 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
98 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
99 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
100 # algorithm configuration
101 max_iterations = 150
102 max_no_improv = 20
103 alpha = 0.3
104 local_search_optima = 12000.0
105 lambda = alpha * (local_search_optima/berlin52.size.to_f)
106 # execute the algorithm
107 best = search(max_iterations, berlin52, max_no_improv, lambda)
108 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
109 end
Listing 2.5: Guided Local Search in Ruby
2.6.6 References
Primary Sources
Guided Local Search emerged from an approach called GENET, which is
a connectionist approach to constraint satisfaction [6, 13]. Guided Local
Search was presented by Voudouris and Tsang in a series of technical re-
ports (that were later published) that described the technique and provided
example applications of it to constraint satisfaction [8], combinatorial opti-
mization [5, 10], and function optimization [9]. The seminal work on the
technique was Voudouris’ PhD dissertation [7].
Learn More
Voudouris and Tsang provide a high-level introduction to the technique [11],
and a contemporary summary of the approach in Glover and Kochenberger’s
‘Handbook of metaheuristics’ [12] that includes a review of the technique,
application areas, and demonstration applications on a diverse set of problem
instances. Mills et al. elaborated on the approach, devising an ‘Extended
Guided Local Search’ (EGLS) technique that added ‘aspiration criteria’ and
random moves to the procedure [4], work which culminated in Mills’ PhD
dissertation [3]. Lau and Tsang further extended the approach by integrating
it with a Genetic Algorithm, called the ‘Guided Genetic Algorithm’ (GGA)
[2], that also culminated in a PhD dissertation by Lau [1].
2.6.7 Bibliography
[1] L. T. Lau. Guided Genetic Algorithm. PhD thesis, Department of
Computer Science, University of Essex, 1999.
54 Chapter 2. Stochastic Algorithms
[2] T. L. Lau and E. P. K. Tsang. The guided genetic algorithm and its
application to the general assignment problems. In IEEE 10th Inter-
national Conference on Tools with Artificial Intelligence (ICTAI’98),
1998.
[3] P. Mills. Extensions to Guided Local Search. PhD thesis, Department
of Computer Science, University of Essex, 2002.
[4] P. Mills, E. Tsang, and J. Ford. Applying an extended guided local
search on the quadratic assignment problem. Annals of Operations
Research, 118:121–135, 2003.
[5] E. Tsang and C. Voudouris. Fast local search and guided local search
and their application to british telecom’s workforce scheduling prob-
lem. Technical Report CSM-246, Department of Computer Science
University of Essex Colchester CO4 3SQ, 1995.
[6] E. P. K. Tsang and C. J. Wang. A generic neural network approach for
constraint satisfaction problems. In Taylor G, editor, Neural network
applications, pages 12–22, 1992.
[7] C. Voudouris. Guided local search for combinatorial optimisation prob-
lems.PhD thesis, Department of Computer Science, University of
Essex, Colchester, UK, July 1997.
[8] C. Voudouris and E. Tsang. The tunneling algorithm for partial csps
and combinatorial optimization problems. Technical Report CSM-213,
Department of Computer Science, University of Essex, Colchester, C04
3SQ, UK, 1994.
[9] C. Voudouris and E. Tsang. Function optimization using guided local
search. Technical Report CSM-249, Department of Computer Science
University of Essex Colchester, CO4 3SQ, UK, 1995.
[10] C. Voudouris and E. Tsang. Guided local search. Technical Report CSM-
247, Department of Computer Science, University of Essex, Colchester,
C04 3SQ, UK, 1995.
[11] C. Voudouris and E. P. K. Tsang. Guided local search joins the
elite in discrete optimisation. In Proceedings, DIMACS Workshop on
Constraint Programming and Large Scale Discrete Optimisation, 1998.
[12] C. Voudouris and E. P. K. Tsang. Handbook of Metaheuristics, chapter
7: Guided Local Search, pages 185–218. Springer, 2003.
[13] C. J. Wang and E. P. K. Tsang. Solving constraint satisfaction problems
using neural networks. In Proceedings Second International Conference
on Artificial Neural Networks, pages 295–299, 1991.
2.7. Variable Neighborhood Search 55
2.7 Variable Neighborhood Search
Variable Neighborhood Search, VNS.
2.7.1 Taxonomy
Variable Neighborhood Search is a Metaheuristic and a Global Optimization
technique that manages a Local Search technique. It is related to the
Iterative Local Search algorithm (Section 2.5).
2.7.2 Strategy
The strategy for the Variable Neighborhood Search involves iterative ex-
ploration of larger and larger neighborhoods for a given local optima until
an improvement is located after which time the search across expanding
neighborhoods is repeated. The strategy is motivated by three principles:
1) a local minimum for one neighborhood structure may not be a local
minimum for a different neighborhood structure, 2) a global minimum is a
local minimum for all possible neighborhood structures, and 3) local minima
are relatively close to global minima for many problem classes.
2.7.3 Procedure
Algorithm 2.7.1 provides a pseudocode listing of the Variable Neighborhood
Search algorithm for minimizing a cost function. The Pseudocode shows
that the systematic search of expanding neighborhoods for a local optimum
is abandoned when a global improvement is achieved (shown with the Break
jump).
2.7.4 Heuristics
� Approximation methods (such as stochastic hill climbing) are suggested
for use as the Local Search procedure for large problem instances in
order to reduce the running time.
� Variable Neighborhood Search has been applied to a very wide array
of combinatorial optimization problems as well as clustering and
continuous function optimization problems.
� The embedded Local Search technique should be specialized to the
problem type and instance to which the technique is being applied.
� The Variable Neighborhood Descent (VND) can be embedded in the
Variable Neighborhood Search as a the Local Search procedure and
has been shown to be most effective.
56 Chapter 2. Stochastic Algorithms
Algorithm 2.7.1: Pseudocode for VNS.
Input: Neighborhoods
Output: Sbest
Sbest ← RandomSolution();1
while ¬ StopCondition() do2
foreach Neighborhoodi ∈ Neighborhoods do3
Neighborhoodcurr ← CalculateNeighborhood(Sbest,4
Neighborhoodi);
Scandidate ←5
RandomSolutionInNeighborhood(Neighborhoodcurr);
Scandidate ← LocalSearch(Scandidate);6
if Cost(Scandidate) < Cost(Sbest) then7
Sbest ← Scandidate;8
Break;9
end10
end11
end12
return Sbest;13
2.7.5 Code Listing
Listing 2.6 provides an example of the Variable Neighborhood Search algo-
rithm implemented in the Ruby Programming Language. The algorithm is
applied to the Berlin52 instance of the Traveling Salesman Problem (TSP),
taken from the TSPLIB. The problem seeks a permutation of the order to
visit cities (called a tour) that minimizes the total distance traveled. The
optimal tour distance for Berlin52 instance is 7542 units.
The Variable Neighborhood Search uses a stochastic 2-opt procedure as
the embedded local search. The procedure deletes two edges and reverses
the sequence in-between the deleted edges, potentially removing ‘twists’ in
the tour. The neighborhood structure used in the search is the number of
times the 2-opt procedure is performed on a permutation, between 1 and 20
times. The stopping condition for the local search procedure is a maximum
number of iterations without improvement. The same stop condition is
employed by the higher-order Variable Neighborhood Search procedure,
although with a lower boundary on the number of non-improving iterations.
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(perm, cities)
6 distance =0
7 perm.each_with_index do |c1, i|
8 c2 = (i==perm.size-1) ? perm[0] : perm[i+1]
2.7. Variable Neighborhood Search 57
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def random_permutation(cities)
15 perm = Array.new(cities.size){|i| i}
16 perm.each_index do |i|
17 r = rand(perm.size-i) + i
18 perm[r], perm[i] = perm[i], perm[r]
19 end
20 return perm
21 end
22
23 def stochastic_two_opt!(perm)
24 c1, c2 = rand(perm.size), rand(perm.size)
25 exclude = [c1]
26 exclude << ((c1==0) ? perm.size-1 : c1-1)
27 exclude << ((c1==perm.size-1) ? 0 : c1+1)
28 c2 = rand(perm.size) while exclude.include?(c2)
29 c1, c2 = c2, c1 if c2 < c1
30 perm[c1...c2] = perm[c1...c2].reverse
31 return perm
32 end
33
34 def local_search(best, cities, max_no_improv, neighborhood)
35 count = 0
36 begin
37 candidate = {}
38 candidate[:vector] = Array.new(best[:vector])
39 neighborhood.times{stochastic_two_opt!(candidate[:vector])}
40 candidate[:cost] = cost(candidate[:vector], cities)
41 if candidate[:cost] < best[:cost]
42 count, best = 0, candidate
43 else
44 count += 1
45 end
46 end until count >= max_no_improv
47 return best
48 end
49
50 def search(cities, neighborhoods, max_no_improv, max_no_improv_ls)
51 best = {}
52 best[:vector] = random_permutation(cities)
53 best[:cost] = cost(best[:vector], cities)
54 iter, count = 0, 0
55 begin
56 neighborhoods.each do |neigh|
57 candidate = {}
58 candidate[:vector] = Array.new(best[:vector])
59 neigh.times{stochastic_two_opt!(candidate[:vector])}
60 candidate[:cost] = cost(candidate[:vector], cities)
61 candidate = local_search(candidate, cities, max_no_improv_ls, neigh)
62 puts " > iteration #{(iter+1)}, neigh=#{neigh}, best=#{best[:cost]}"
63 iter += 1
64 if(candidate[:cost] < best[:cost])
58 Chapter 2. Stochastic Algorithms
65 best, count = candidate, 0
66 puts "New best, restarting neighborhood search."
67 break
68 else
69 count += 1
70 end
71 end
72 end until count >= max_no_improv
73 return best
74 end
75
76 if __FILE__ == $0
77 # problem configuration
78 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
79 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
80 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
81 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
82 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
83 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
84 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
85 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
86 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
87 # algorithm configuration
88 max_no_improv = 50
89 max_no_improv_ls = 70
90 neighborhoods = 1...20
91 # execute the algorithm
92 best = search(berlin52, neighborhoods, max_no_improv, max_no_improv_ls)
93 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
94 end
Listing 2.6: Variable Neighborhood Search in Ruby
2.7.6 References
Primary Sources
The seminal paper for describing Variable Neighborhood Search was by
Mladenovic and Hansen in 1997 [7], although an early abstract byMladenovic
is sometimes cited [6]. The approach is explained in terms of three different
variations on the general theme. Variable Neighborhood Descent (VND)
refers to the use of a Local Search procedure and the deterministic (as
opposed to stochastic or probabilistic) change of neighborhood size. Reduced
Variable Neighborhood Search (RVNS) involves performing a stochastic
random search within a neighborhood and no refinement via a local search
technique. Basic Variable Neighborhood Search is the canonical approach
described by Mladenovic and Hansen in the seminal paper.
Learn More
There are a large number of papers published on Variable Neighborhood
Search, its applications and variations. Hansen and Mladenovic provide an
2.7. Variable Neighborhood Search 59
overview of the approach that includes its recent history, extensions and a
detailed review of the numerous areas of application [4]. For some additional
useful overviews of the technique, its principles, and applications, see [1–3].
There are many extensions to Variable Neighborhood Search. Some
popular examples include: Variable Neighborhood Decomposition Search
(VNDS) that involves embedding a second heuristic or metaheuristic ap-
proach in VNS to replace the Local Search procedure [5], Skewed Variable
Neighborhood Search (SVNS) that encourages exploration of neighborhoods
far away from discovered local optima, and Parallel Variable Neighborhood
Search (PVNS) that either parallelizes the local search of a neighborhood
or parallelizes the searching of the neighborhoods themselves.
2.7.7 Bibliography
[1] P. Hansen and N. Mladenović. Meta-heuristics, Advances and trends
in local search paradigms for optimization, chapter An introduction
to Variable neighborhood search, pages 433–458. Kluwer Academic
Publishers, 1998.
[2] P. Hansen and N. Mladenović. Variable neighborhood search: Principles
and applications. European Journal of Operational Research, 130(3):449–
467, 2001.
[3] P. Hansen and N. Mladenović. Handbook of Applied Optimization, chap-
ter Variable neighbourhood search, pages 221–234. Oxford University
Press, 2002.
[4] P. Hansen and N. Mladenović. Handbook of Metaheuristics, chapter 6:
Variable Neighborhood Search, pages 145–184. Springer, 2003.
[5] P. Hansen, N. Mladenović, and D. Perez-Britos. Variable neighborhood
decomposition search. Journal of Heuristics, 7(4):1381–1231, 2001.
[6] N. Mladenović. A variable neighborhood algorithm - a new metaheuristic
for combinatorial optimization. In Abstracts of papers presented at
Optimization Days, 1995.
[7] N. Mladenović and P. Hansen. Variable neighborhood search. Computers
& Operations Research, 24(11):1097–1100, 1997.
60 Chapter 2. Stochastic Algorithms
2.8 Greedy Randomized Adaptive Search
Greedy Randomized Adaptive Search Procedure, GRASP.
2.8.1 Taxonomy
The Greedy Randomized Adaptive Search Procedure is a Metaheuristic
and Global Optimization algorithm, originally proposed for the Operations
Research practitioners. The iterative application of an embedded Local
Search technique relate the approach to Iterative Local Search (Section 2.5)
and Multi-Start techniques.
2.8.2 Strategy
The objective of the Greedy Randomized Adaptive Search Procedure is to
repeatedly sample stochastically greedy solutions, and then use a local search
procedure to refine them to a local optima. The strategy of the procedure
is centered on the stochastic and greedy step-wise construction mechanism
that constrains the selection and order-of-inclusion of the components of a
solution based on the value they are expected to provide.
2.8.3 Procedure
Algorithm 2.8.1 provides a pseudocode listing of the Greedy Randomized
Adaptive Search Procedure for minimizing a cost function.
Algorithm 2.8.1: Pseudocode for the GRASP.
Input: α
Output: Sbest
Sbest ← ConstructRandomSolution();1
while ¬ StopCondition() do2
Scandidate ← GreedyRandomizedConstruction(α);3
Scandidate ← LocalSearch(Scandidate);4
if Cost(Scandidate) < Cost(Sbest) then5
Sbest ← Scandidate;6
end7
end8
return Sbest;9
Algorithm 2.8.2 provides the pseudocode the Greedy Randomized Con-
struction function. The function involves the step-wise construction of a
candidate solution using a stochastically greedy construction process. The
function works by building a Restricted Candidate List (RCL) that con-
straints the components of a solution (features) that may be selected from
2.8. Greedy Randomized Adaptive Search 61
each cycle. The RCL may be constrained by an explicit size, or by using
a threshold (α ∈ [0, 1]) on the cost of adding each feature to the current
candidate solution.
Algorithm 2.8.2: Pseudocode the GreedyRandomizedConstruction
function.
Input: α
Output: Scandidate
Scandidate ← ∅;1
while Scandidate 6= ProblemSize do2
Featurecosts ← ∅;3
for Featurei /∈ Scandidate do4
Featurecosts ←5
CostOfAddingFeatureToSolution(Scandidate, Featurei);
end6
RCL ← ∅;7
Fcostmin ← MinCost(Featurecosts);8
Fcostmax ← MaxCost(Featurecosts);9
for Ficost ∈ Featurecosts do10
if Ficost ≤ Fcostmin + α · (Fcostmax − Fcostmin) then11
RCL ← Featurei;12
end13
end14
Scandidate ← SelectRandomFeature(RCL);15
end16
return Scandidate;17
2.8.4 Heuristics
� The α threshold defines the amount of greediness of the construction
mechanism, where values close to 0 may be too greedy, and values
close to 1 may be too generalized.
� As an alternative to using the α threshold, the RCL can be constrained
to the top n% of candidate features that may be selected from each
construction cycle.
� The technique was designed for discrete problem classes such as com-
binatorial optimization problems.
2.8.5 Code Listing
Listing 2.7 provides an example of the Greedy Randomized Adaptive Search
Procedure implemented in the Ruby Programming Language. The algorithm
62 Chapter 2. Stochastic Algorithms
is applied to the Berlin52 instance of the Traveling Salesman Problem (TSP),
taken from the TSPLIB. The problem seeks a permutation of the order to
visit cities (called a tour) that minimizes the total distance traveled. The
optimal tour distance for Berlin52 instance is 7542 units.
The stochastic and greedy step-wise construction of a tour involves
evaluating candidate cities by the the cost they contribute as being the
next city in the tour. The algorithm uses a stochastic 2-opt procedure for
the Local Search with a fixed number of non-improving iterations as the
stopping condition.
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(perm, cities)
6 distance =0
7 perm.each_with_index do |c1, i|
8 c2 = (i==perm.size-1) ? perm[0] : perm[i+1]
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def stochastic_two_opt(permutation)
15 perm = Array.new(permutation)
16 c1, c2 = rand(perm.size), rand(perm.size)
17 exclude = [c1]
18 exclude << ((c1==0) ? perm.size-1 : c1-1)
19 exclude << ((c1==perm.size-1) ? 0 : c1+1)
20 c2 = rand(perm.size) while exclude.include?(c2)
21 c1, c2 = c2, c1 if c2 < c1
22 perm[c1...c2] = perm[c1...c2].reverse
23 return perm
24 end
25
26 def local_search(best, cities, max_no_improv)
27 count = 0
28 begin
29 candidate = {:vector=>stochastic_two_opt(best[:vector])}
30 candidate[:cost] = cost(candidate[:vector], cities)
31 count = (candidate[:cost] < best[:cost]) ? 0 : count+1
32 best = candidate if candidate[:cost] < best[:cost]
33 end until count >= max_no_improv
34 return best
35 end
36
37 def construct_randomized_greedy_solution(cities, alpha)
38 candidate = {}
39 candidate[:vector] = [rand(cities.size)]
40 allCities = Array.new(cities.size) {|i| i}
41 while candidate[:vector].size < cities.size
42 candidates = allCities - candidate[:vector]
43 costs = Array.new(candidates.size) do |i|
44 euc_2d(cities[candidate[:vector].last], cities[i])
2.8. Greedy Randomized Adaptive Search 63
45 end
46 rcl, max, min = [], costs.max, costs.min
47 costs.each_with_index do |c,i|
48 rcl << candidates[i] if c <= (min +alpha*(max-min))
49 end
50 candidate[:vector] << rcl[rand(rcl.size)]
51 end
52 candidate[:cost] = cost(candidate[:vector], cities)
53 return candidate
54 end
55
56 def search(cities, max_iter, max_no_improv, alpha)
57 best = nil
58 max_iter.times do |iter|
59 candidate = construct_randomized_greedy_solution(cities, alpha);
60 candidate = local_search(candidate, cities, max_no_improv)
61 best = candidate if best.nil? or candidate[:cost] < best[:cost]
62 puts " > iteration #{(iter+1)}, best=#{best[:cost]}"
63 end
64 return best
65 end
66
67 if __FILE__ == $0
68 # problem configuration
69 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
70 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
71 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
72 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
73 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
74 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
75 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
76 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
77 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
78 # algorithm configuration
79 max_iter = 50
80 max_no_improv = 50
81 greediness_factor = 0.3
82 # execute the algorithm
83 best = search(berlin52, max_iter, max_no_improv, greediness_factor)
84 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
85 end
Listing 2.7: Greedy Randomized Adaptive Search Procedure in Ruby
2.8.6 References
Primary Sources
The seminal paper that introduces the general approach of stochastic and
greedy step-wise construction of candidate solutions is by Feo and Resende
[3]. The general approach was inspired by greedy heuristics by Hart and
Shogan [9]. The seminal review paper that is cited with the preliminary
paper is by Feo and Resende [4], and provides a coherent description
64 Chapter 2. Stochastic Algorithms
of the GRASP technique, an example, and review of early applications.
An early application was by Feo, Venkatraman and Bard for a machine
scheduling problem [7]. Other early applications to scheduling problems
include technical reports [2] (later published as [1]) and [5] (also later
published as [6]).
Learn More
There are a vast number of review, application, and extension papers for
GRASP. Pitsoulis and Resende provide an extensive contemporary overview
of the field as a review chapter [11], as does Resende and Ribeiro that
includes a clear presentation of the use of the α threshold parameter instead
of a fixed size for the RCL [13]. Festa and Resende provide an annotated
bibliography as a review chapter that provides some needed insight into large
amount of study that has gone into the approach [8]. There are numerous
extensions to GRASP, not limited to the popular Reactive GRASP for
adapting α [12], the use of long term memory to allow the technique to
learn from candidate solutions discovered in previous iterations, and parallel
implementations of the procedure such as ‘Parallel GRASP’ [10].
2.8.7 Bibliography
[1] J. F. Bard, T. A. Feo, and S. Holland. A GRASP for scheduling printed
wiring board assembly. I.I.E. Trans., 28:155–165, 1996.
[2] T. A. Feo, J. Bard, and S. Holland. A GRASP for scheduling printed
wiring board assembly. Technical Report TX 78712-1063, Operations
Research Group, Department of Mechanical Engineering, The Univer-
sity of Texas at Austin, 1993.
[3] T. A. Feo and M. G. C. Resende. A probabilistic heuristic for a
computationally difficult set covering problem. Operations Research
Letters, 8:67–71, 1989.
[4] T. A. Feo and M. G. C. Resende. Greedy randomized adaptive search
procedures. Journal of Global Optimization, 6:109–133, 1995.
[5] T. A. Feo, K. Sarathy, and J. McGahan. A GRASP for single machine
scheduling with sequence dependent setup costs and linear delay penal-
ties. Technical Report TX 78712-1063, Operations Research Group,
Department of Mechanical Engineering, The University of Texas at
Austin, 1994.
[6] T. A. Feo, K. Sarathy, and J. McGahan. A grasp for single machine
scheduling with sequence dependent setup costs and linear delay penal-
ties. Computers & Operations Research, 23(9):881–895, 1996.
2.8. Greedy Randomized Adaptive Search 65
[7] T. A. Feo, K. Venkatraman, and J. F. Bard. A GRASP for a difficult
single machine scheduling problem. Computers & Operations Research,
18:635–643, 1991.
[8] P. Festa and M. G. C. Resende. Essays and Surveys on Metaheuristics,
chapter GRASP: An annotated bibliography, pages 325–367. Kluwer
Academic Publishers, 2002.
[9] J. P. Hart and A. W. Shogan. Semi–greedy heuristics: An empirical
study. Operations Research Letters, 6:107–114, 1987.
[10] P. M. Pardalos, L. S. Pitsoulis, and M. G. C. Resende. A parallel
GRASP implementation for the quadratic assignment problems. In
Parallel Algorithms for Irregularly Structured Problems (Irregular94),
pages 111–130. Kluwer Academic Publishers, 1995.
[11] L. Pitsoulis and M. G. C. Resende. Handbook of Applied Optimization,
chapter Greedy randomized adaptive search procedures, pages 168–181.
Oxford University Press, 2002.
[12] M. Prais and C. C. Ribeiro. Reactive GRASP: An application to a
matrix decomposition problem in TDMA traffic assignment. INFORMS
Journal on Computing, 12:164–176, 2000.
[13] M. G. C. Resende and C. C. Ribeiro. Handbook of Metaheuristics,
chapter Greedy randomized adaptive search procedures, pages 219–249.
Kluwer Academic Publishers, 2003.
66 Chapter 2. Stochastic Algorithms
2.9 Scatter Search
Scatter Search, SS.
2.9.1 Taxonomy
Scatter search is a Metaheuristic and a Global Optimization algorithm. It is
also sometimes associated with the field of Evolutionary Computation given
the use of a population and recombination in the structure of the technique.
Scatter Search is a sibling of Tabu Search (Section 2.10), developed by the
same author and based on similar origins.
2.9.2 Strategy
The objective of Scatter Search is to maintain a set of diverse and high-
quality candidate solutions. The principle of the approach is that useful
information about the global optima is stored in a diverse and elite set of
solutions (the reference set) and that recombining samples from the set
can exploit this information. The strategy involves an iterative process,
where a population of diverse and high-quality candidate solutions that
are partitioned into subsets and linearly recombined to create weighted
centroids of sample-based neighborhoods. The results of recombination
are refined using an embedded heuristic and assessed in the context of the
reference set as to whether or not they are retained.
2.9.3 Procedure
Algorithm 2.9.1 provides a pseudocode listing of the Scatter Search algorithm
for minimizing a cost function. The procedure is based on the abstract form
presented by Glover as a template for the general class of technique [3], with
influences from an application of the technique to function optimization by
Glover [3].
2.9.4 Heuristics
� Scatter search is suitable for both discrete domains such as combina-
torial optimization as well as continuous domains such as non-linear
programming (continuous function optimization).
� Small set sizes are preferred for the ReferenceSet, such as 10 or 20
members.
� Subset sizes can be 2, 3, 4 or more members that are all recombined
to produce viable candidate solutions within the neighborhood of the
members of the subset.
2.9. Scatter Search 67
Algorithm 2.9.1: Pseudocode for Scatter Search.
Input: DiverseSetsize, ReferenceSetsize
Output: ReferenceSet
InitialSet ← ConstructInitialSolution(DiverseSetsize);1
RefinedSet ← ∅;2
for Si ∈ InitialSet do3
RefinedSet ← LocalSearch(Si);4
end5
ReferenceSet ← SelectInitialReferenceSet(ReferenceSetsize);6
while ¬ StopCondition() do7
Subsets ← SelectSubset(ReferenceSet);8
CandidateSet ← ∅;9
for Subseti ∈ Subsets do10
RecombinedCandidates ← RecombineMembers(Subseti);11
for Si ∈ RecombinedCandidates do12
CandidateSet← LocalSearch(Si);13
end14
end15
ReferenceSet ← Select(ReferenceSet, CandidateSet,16
ReferenceSetsize);
end17
return ReferenceSet;18
� Each subset should comprise at least one member added to the set in
the previous algorithm iteration.
� The Local Search procedure should be a problem-specific improvement
heuristic.
� The selection of members for the ReferenceSet at the end of each
iteration favors solutions with higher quality and may also promote
diversity.
� The ReferenceSet may be updated at the end of an iteration, or
dynamically as candidates are created (a so-called steady-state popu-
lation in some evolutionary computation literature).
� A lack of changes to the ReferenceSet may be used as a signal to
stop the current search, and potentially restart the search with a newly
initialized ReferenceSet.
2.9.5 Code Listing
Listing 2.8 provides an example of the Scatter Search algorithm implemented
in the Ruby Programming Language. The example problem is an instance of
68 Chapter 2. Stochastic Algorithms
a continuous function optimization that seeks min f(x) where f =
∑n
i=1 x
2
i ,
−5.0 ≤ xi ≤ 5.0 and n = 3. The optimal solution for this basin function is
(v1, . . . , vn) = 0.0.
The algorithm is an implementation of Scatter Search as described in
an application of the technique to unconstrained non-linear optimization by
Glover [6]. The seeds for initial solutions are generated as random vectors,
as opposed to stratified samples. The example was further simplified by
not including a restart strategy, and the exclusion of diversity maintenance
in the ReferenceSet. A stochastic local search algorithm is used as the
embedded heuristic that uses a stochastic step size in the range of half a
percent of the search space.
1 def objective_function(vector)
2 return vector.inject(0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def rand_in_bounds(min, max)
6 return min + ((max-min) * rand())
7 end
8
9 def random_vector(minmax)
10 return Array.new(minmax.size) do |i|
11 rand_in_bounds(minmax[i][0], minmax[i][1])
12 end
13 end
14
15 def take_step(minmax, current, step_size)
16 position = Array.new(current.size)
17 position.size.times do |i|
18 min = [minmax[i][0], current[i]-step_size].max
19 max = [minmax[i][1], current[i]+step_size].min
20 position[i] = rand_in_bounds(min, max)
21 end
22 return position
23 end
24
25 def local_search(best, bounds, max_no_improv, step_size)
26 count = 0
27 begin
28 candidate = {:vector=>take_step(bounds, best[:vector], step_size)}
29 candidate[:cost] = objective_function(candidate[:vector])
30 count = (candidate[:cost] < best[:cost]) ? 0 : count+1
31 best = candidate if candidate[:cost] < best[:cost]
32 end until count >= max_no_improv
33 return best
34 end
35
36 def construct_initial_set(bounds, set_size, max_no_improv, step_size)
37 diverse_set = []
38 begin
39 cand = {:vector=>random_vector(bounds)}
40 cand[:cost] = objective_function(cand[:vector])
41 cand = local_search(cand, bounds, max_no_improv, step_size)
42 diverse_set << cand if !diverse_set.any? {|x| x[:vector]==cand[:vector]}
2.9. Scatter Search 69
43 end until diverse_set.size == set_size
44 return diverse_set
45 end
46
47 def euclidean_distance(c1, c2)
48 sum = 0.0
49 c1.each_index {|i| sum += (c1[i]-c2[i])**2.0}
50 return Math.sqrt(sum)
51 end
52
53 def distance(v, set)
54 return set.inject(0){|s,x| s + euclidean_distance(v, x[:vector])}
55 end
56
57 def diversify(diverse_set, num_elite, ref_set_size)
58 diverse_set.sort!{|x,y| x[:cost] <=> y[:cost]}
59 ref_set = Array.new(num_elite){|i| diverse_set[i]}
60 remainder = diverse_set - ref_set
61 remainder.each{|c| c[:dist] = distance(c[:vector], ref_set)}
62 remainder.sort!{|x,y| y[:dist]<=>x[:dist]}
63 ref_set = ref_set + remainder.first(ref_set_size-ref_set.size)
64 return [ref_set, ref_set[0]]
65 end
66
67 def select_subsets(ref_set)
68 additions = ref_set.select{|c| c[:new]}
69 remainder = ref_set - additions
70 remainder = additions if remainder.nil? or remainder.empty?
71 subsets = []
72 additions.each do |a|
73 remainder.each{|r| subsets << [a,r] if a!=r && !subsets.include?([r,a])}
74 end
75 return subsets
76 end
77
78 def recombine(subset, minmax)
79 a, b = subset
80 d = rand(euclidean_distance(a[:vector], b[:vector]))/2.0
81 children = []
82 subset.each do |p|
83 step = (rand<0.5) ? +d : -d
84 child = {:vector=>Array.new(minmax.size)}
85 child[:vector].each_index do |i|
86 child[:vector][i] = p[:vector][i] + step
87 child[:vector][i]=minmax[i][0] if child[:vector][i]<minmax[i][0]
88 child[:vector][i]=minmax[i][1] if child[:vector][i]>minmax[i][1]
89 end
90 child[:cost] = objective_function(child[:vector])
91 children << child
92 end
93 return children
94 end
95
96 def explore_subsets(bounds, ref_set, max_no_improv, step_size)
97 was_change = false
98 subsets = select_subsets(ref_set)
70 Chapter 2. Stochastic Algorithms
99 ref_set.each{|c| c[:new] = false}
100 subsets.each do |subset|
101 candidates = recombine(subset, bounds)
102 improved = Array.new(candidates.size) do |i|
103 local_search(candidates[i], bounds, max_no_improv, step_size)
104 end
105 improved.each do |c|
106 if !ref_set.any? {|x| x[:vector]==c[:vector]}
107 c[:new] = true
108 ref_set.sort!{|x,y| x[:cost] <=> y[:cost]}
109 if c[:cost] < ref_set.last[:cost]
110 ref_set.delete(ref_set.last)
111 ref_set << c
112 puts " >> added, cost=#{c[:cost]}"
113 was_change = true
114 end
115 end
116 end
117 end
118 return was_change
119 end
120
121 def search(bounds, max_iter, ref_set_size, div_set_size, max_no_improv,
step_size, max_elite)
122 diverse_set = construct_initial_set(bounds, div_set_size, max_no_improv,
step_size)
123 ref_set, best = diversify(diverse_set, max_elite, ref_set_size)
124 ref_set.each{|c| c[:new] = true}
125 max_iter.times do |iter|
126 was_change = explore_subsets(bounds, ref_set, max_no_improv, step_size)
127 ref_set.sort!{|x,y| x[:cost] <=> y[:cost]}
128 best = ref_set.first if ref_set.first[:cost] < best[:cost]
129 puts " > iter=#{(iter+1)}, best=#{best[:cost]}"
130 break if !was_change
131 end
132 return best
133 end
134
135 if __FILE__ == $0
136 # problem configuration
137 problem_size = 3
138 bounds = Array.new(problem_size) {|i| [-5, +5]}
139 # algorithm configuration
140 max_iter = 100
141 step_size = (bounds[0][1]-bounds[0][0])*0.005
142 max_no_improv = 30
143 ref_set_size = 10
144 diverse_set_size = 20
145 no_elite = 5
146 # execute the algorithm
147 best = search(bounds, max_iter, ref_set_size, diverse_set_size,
max_no_improv, step_size, no_elite)
148 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
149 end
Listing 2.8: Scatter Search in Ruby
2.9. Scatter Search 71
2.9.6 References
Primary Sources
A form of the Scatter Search algorithm was proposed by Glover for integer
programming [1], based on Glover’s earlier work on surrogate constraints.
The approach remained idle until it was revisited by Glover and combined
with Tabu Search [2]. The modern canonical reference of the approach was
proposed by Glover who provides an abstract template of the procedure
that may be specialized for a given application domain [3].
Learn More
The primary reference for the approach is the book by Laguna and Mart́ı
that reviews the principles of the approach in detail and presents tutorials on
applications of the approach on standard problems using the C programming
language [7]. There are many review articles and chapters on Scatter Search
that may be used to supplement an understanding of the approach, such
as a detailed review chapter by Glover [4], a review of the fundamentals of
the approach and its relationship to an abstraction called ‘path linking’ by
Glover, Laguna, and Mart́ı [5], and a modern overview of the technique by
Mart́ı, Laguna, and Glover [8].
2.9.7 Bibliography
[1] F. Glover. Heuristics for integer programming using surrogate constraints.
Decision Sciences, 8(1):156–166,1977.
[2] F. Glover. Tabu search for nonlinear and parametric optimization (with
links to genetic algorithms). Discrete Applied Mathematics, 49:231–255,
1994.
[3] F. Glover. Artificial Evolution, chapter A Template For Scatter Search
And Path Relinking, page 13. Sprinter, 1998.
[4] F. Glover. New Ideas in Optimization, chapter Scatter search and path
relinking, pages 297–316. McGraw-Hill Ltd., 1999.
[5] F. Glover, M. Laguna, and R. Mart́ı. Fundamentals of scatter search
and path relinking. Control and Cybernetics, 39(3):653–684, 2000.
[6] F. Glover, M. Laguna, and R. Mart́ı. Advances in Evolutionary Compu-
tation: Theory and Applications, chapter Scatter Search, pages 519–537.
Springer-Verlag, 2003.
[7] M. Laguna and R. Mart́ı. Scatter search: methodology and implementa-
tions in C. Kluwer Academic Publishers, 2003.
72 Chapter 2. Stochastic Algorithms
[8] R. Mart́ı, M. Laguna, and F. Glover. Principles of scatter search.
European Journal of Operational Research, 169(1):359–372, 2006.
2.10. Tabu Search 73
2.10 Tabu Search
Tabu Search, TS, Taboo Search.
2.10.1 Taxonomy
Tabu Search is a Global Optimization algorithm and a Metaheuristic or
Meta-strategy for controlling an embedded heuristic technique. Tabu Search
is a parent for a large family of derivative approaches that introduce memory
structures in Metaheuristics, such as Reactive Tabu Search (Section 2.11)
and Parallel Tabu Search.
2.10.2 Strategy
The objective for the Tabu Search algorithm is to constrain an embedded
heuristic from returning to recently visited areas of the search space, referred
to as cycling. The strategy of the approach is to maintain a short term
memory of the specific changes of recent moves within the search space
and preventing future moves from undoing those changes. Additional
intermediate-term memory structures may be introduced to bias moves
toward promising areas of the search space, as well as longer-term memory
structures that promote a general diversity in the search across the search
space.
2.10.3 Procedure
Algorithm 2.10.1 provides a pseudocode listing of the Tabu Search algorithm
for minimizing a cost function. The listing shows the simple Tabu Search
algorithm with short term memory, without intermediate and long term
memory management.
2.10.4 Heuristics
� Tabu search was designed to manage an embedded hill climbing
heuristic, although may be adapted to manage any neighborhood
exploration heuristic.
� Tabu search was designed for, and has predominately been applied to
discrete domains such as combinatorial optimization problems.
� Candidates for neighboring moves can be generated deterministically
for the entire neighborhood or the neighborhood can be stochastically
sampled to a fixed size, trading off efficiency for accuracy.
� Intermediate-term memory structures can be introduced (complement-
ing the short-term memory) to focus the search on promising areas of
the search space (intensification), called aspiration criteria.
74 Chapter 2. Stochastic Algorithms
Algorithm 2.10.1: Pseudocode for Tabu Search.
Input: TabuListsize
Output: Sbest
Sbest ← ConstructInitialSolution();1
TabuList ← ∅;2
while ¬ StopCondition() do3
CandidateList ← ∅;4
for Scandidate ∈ Sbestneighborhood do5
if ¬ ContainsAnyFeatures(Scandidate, TabuList) then6
CandidateList ← Scandidate;7
end8
end9
Scandidate ← LocateBestCandidate(CandidateList);10
if Cost(Scandidate) ≤ Cost(Sbest) then11
Sbest ← Scandidate;12
TabuList ← FeatureDifferences(Scandidate, Sbest);13
while TabuList > TabuListsize do14
DeleteFeature(TabuList);15
end16
end17
end18
return Sbest;19
� Long-term memory structures can be introduced (complementing the
short-term memory) to encourage useful exploration of the broader
search space, called diversification. Strategies may include generating
solutions with rarely used components and biasing the generation
away from the most commonly used solution components.
2.10.5 Code Listing
Listing 2.9 provides an example of the Tabu Search algorithm implemented
in the Ruby Programming Language. The algorithm is applied to the
Berlin52 instance of the Traveling Salesman Problem (TSP), taken from
the TSPLIB. The problem seeks a permutation of the order to visit cities
(called a tour) that minimizes the total distance traveled. The optimal tour
distance for Berli52 instance is 7542 units.
The algorithm is an implementation of the simple Tabu Search with a
short term memory structure that executes for a fixed number of iterations.
The starting point for the search is prepared using a random permutation
that is refined using a stochastic 2-opt Local Search procedure. The stochas-
tic 2-opt procedure is used as the embedded hill climbing heuristic with
a fixed sized candidate list. The two edges that are deleted in each 2-opt
2.10. Tabu Search 75
move are stored on the tabu list. This general approach is similar to that
used by Knox in his work on Tabu Search for symmetrical TSP [12] and
Fiechter for the Parallel Tabu Search for the TSP [2].
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(perm, cities)
6 distance = 0
7 perm.each_with_index do |c1, i|
8 c2 = (i==perm.size-1) ? perm[0] : perm[i+1]
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def random_permutation(cities)
15 perm = Array.new(cities.size){|i| i}
16 perm.each_index do |i|
17 r = rand(perm.size-i) + i
18 perm[r], perm[i] = perm[i], perm[r]
19 end
20 return perm
21 end
22
23 def stochastic_two_opt(parent)
24 perm = Array.new(parent)
25 c1, c2 = rand(perm.size), rand(perm.size)
26 exclude = [c1]
27 exclude << ((c1==0) ? perm.size-1 : c1-1)
28 exclude << ((c1==perm.size-1) ? 0 : c1+1)
29 c2 = rand(perm.size) while exclude.include?(c2)
30 c1, c2 = c2, c1 if c2 < c1
31 perm[c1...c2] = perm[c1...c2].reverse
32 return perm, [[parent[c1-1], parent[c1]], [parent[c2-1], parent[c2]]]
33 end
34
35 def is_tabu?(permutation, tabu_list)
36 permutation.each_with_index do |c1, i|
37 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
38 tabu_list.each do |forbidden_edge|
39 return true if forbidden_edge == [c1, c2]
40 end
41 end
42 return false
43 end
44
45 def generate_candidate(best, tabu_list, cities)
46 perm, edges = nil, nil
47 begin
48 perm, edges = stochastic_two_opt(best[:vector])
49 end while is_tabu?(perm, tabu_list)
50 candidate = {:vector=>perm}
51 candidate[:cost] = cost(candidate[:vector], cities)
52 return candidate, edges
76 Chapter 2. Stochastic Algorithms
53 end
54
55 def search(cities, tabu_list_size, candidate_list_size, max_iter)
56 current = {:vector=>random_permutation(cities)}
57 current[:cost] = cost(current[:vector], cities)
58 best = current
59 tabu_list = Array.new(tabu_list_size)
60 max_iter.times do |iter|
61 candidates = Array.new(candidate_list_size) do |i|
62 generate_candidate(current, tabu_list, cities)
63 end
64 candidates.sort! {|x,y| x.first[:cost] <=> y.first[:cost]}
65 best_candidate = candidates.first[0]
66 best_candidate_edges = candidates.first[1]
67 if best_candidate[:cost] < current[:cost]
68 current = best_candidate
69 best = best_candidate if best_candidate[:cost] < best[:cost]
70 best_candidate_edges.each {|edge| tabu_list.push(edge)}
71 tabu_list.pop while tabu_list.size > tabu_list_size
72 end
73 puts " > iteration #{(iter+1)}, best=#{best[:cost]}"
74 end
75 return best
76 end
77
78 if __FILE__ == $0
79 # problem configuration
80 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
81 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
82 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
83 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
84 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
85 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
86 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
87 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],88 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
89 # algorithm configuration
90 max_iter = 100
91 tabu_list_size = 15
92 max_candidates = 50
93 # execute the algorithm
94 best = search(berlin52, tabu_list_size, max_candidates, max_iter)
95 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
96 end
Listing 2.9: Tabu Search in Ruby
2.10.6 References
Primary Sources
Tabu Search was introduced by Glover applied to scheduling employees to
duty rosters [9] and a more general overview in the context of the TSP [5],
based on his previous work on surrogate constraints on integer programming
2.10. Tabu Search 77
problems [4]. Glover provided a seminal overview of the algorithm in a
two-part journal article, the first part of which introduced the algorithm
and reviewed then-recent applications [6], and the second which focused on
advanced topics and open areas of research [7].
Learn More
Glover provides a high-level introduction to Tabu Search in the form of a
practical tutorial [8], as does Glover and Taillard in a user guide format
[10]. The best source of information for Tabu Search is the book dedicated
to the approach by Glover and Laguna that covers the principles of the
technique in detail as well as an in-depth review of applications [11]. The
approach appeared in Science, that considered a modification for its appli-
cation to continuous function optimization problems [1]. Finally, Gendreau
provides an excellent contemporary review of the algorithm, highlighting
best practices and application heuristics collected from across the field of
study [3].
2.10.7 Bibliography
[1] D. Cvijovic and J. Klinowski. Taboo search: An approach to the
multiple minima problem. Science, 267:664–666, 1995.
[2] C-N. Fiechter. A parallel tabu search algorithm for large traveling
salesman problems. Discrete Applied Mathematics, 3(6):243–267, 1994.
[3] M. Gendreau. Handbook of Metaheuristics, chapter 2: An Introduction
to Tabu Search, pages 37–54. Springer, 2003.
[4] F. Glover. Heuristics for integer programming using surrogate con-
straints. Decision Sciences, 8(1):156–166, 1977.
[5] F. Glover. Future paths for integer programming and links to artificial
intelligence. Computers and Operations Research, 13(5):533–549, 1986.
[6] F. Glover. Tabu search – Part I. ORSA Journal on Computing,
1(3):190–206, 1989.
[7] F. Glover. Tabu search – Part II. ORSA Journal on Computing,
2(1):4–32, 1990.
[8] F. Glover. Tabu search: A tutorial. Interfaces, 4:74–94, 1990.
[9] F. Glover and C. McMillan. The general employee scheduling problem:
an integration of MS and AI. Computers and Operations Research,
13(5):536–573, 1986.
[10] F. Glover and E. Taillard. A user’s guide to tabu search. Annals of
Operations Research, 41(1):1–28, 1993.
78 Chapter 2. Stochastic Algorithms
[11] F. W. Glover and M. Laguna. Tabu Search. Springer, 1998.
[12] J. Knox. Tabu search performance on the symmetric traveling salesman
problem. Computers & Operations Research, 21(8):867–876, 1994.
2.11. Reactive Tabu Search 79
2.11 Reactive Tabu Search
Reactive Tabu Search, RTS, R-TABU, Reactive Taboo Search.
2.11.1 Taxonomy
Reactive Tabu Search is a Metaheuristic and a Global Optimization algo-
rithm. It is an extension of Tabu Search (Section 2.10) and the basis for a
field of reactive techniques called Reactive Local Search and more broadly
the field of Reactive Search Optimization.
2.11.2 Strategy
The objective of Tabu Search is to avoid cycles while applying a local search
technique. The Reactive Tabu Search addresses this objective by explicitly
monitoring the search and reacting to the occurrence of cycles and their
repetition by adapting the tabu tenure (tabu list size). The strategy of the
broader field of Reactive Search Optimization is to automate the process by
which a practitioner configures a search procedure by monitoring its online
behavior and to use machine learning techniques to adapt a techniques
configuration.
2.11.3 Procedure
Algorithm 2.11.1 provides a pseudocode listing of the Reactive Tabu Search
algorithm for minimizing a cost function. The Pseudocode is based on the
version of the Reactive Tabu Search described by Battiti and Tecchiolli in [9]
with supplements like the IsTabu function from [7]. The procedure has been
modified for brevity to exude the diversification procedure (escape move).
Algorithm 2.11.2 describes the memory based reaction that manipulates
the size of the ProhibitionPeriod in response to identified cycles in the
ongoing search. Algorithm 2.11.3 describes the selection of the best move
from a list of candidate moves in the neighborhood of a given solution. The
function permits prohibited moves in the case where a prohibited move is
better than the best know solution and the selected admissible move (called
aspiration). Algorithm 2.11.4 determines whether a given neighborhood
move is tabu based on the current ProhibitionPeriod, and is employed
by sub-functions of the Algorithm 2.11.3 function.
2.11.4 Heuristics
� Reactive Tabu Search is an extension of Tabu Search and as such
should exploit the best practices used for the parent algorithm.
80 Chapter 2. Stochastic Algorithms
Algorithm 2.11.1: Pseudocode for Reactive Tabu Search.
Input: Iterationmax, Increase, Decrease, ProblemSize
Output: Sbest
Scurr ← ConstructInitialSolution();1
Sbest ← Scurr;2
TabuList ← ∅;3
ProhibitionPeriod ← 1;4
foreach Iterationi ∈ Iterationmax do5
MemoryBasedReaction(Increase, Decrease, ProblemSize);6
CandidateList ← GenerateCandidateNeighborhood(Scurr);7
Scurr ← BestMove(CandidateList);8
TabuList ← Scurrfeature;9
if Cost(Scurr) ≤ Cost(Sbest) then10
Sbest ← Scurr;11
end12
end13
return Sbest;14
Algorithm 2.11.2: Pseudocode for the MemoryBasedReaction func-
tion.
Input: Increase, Decrease, ProblemSize
Output:
if HaveVisitedSolutionBefore(Scurr, VisitedSolutions) then1
Scurrt ← RetrieveLastTimeVisited(VisitedSolutions, Scurr);2
RepetitionInterval ← Iterationi − Scurrt;3
Scurrt ← Iterationi;4
if RepetitionInterval < 2 × ProblemSize then5
RepetitionIntervalavg ← 0.1 × RepetitionInterval + 0.9 ×6
RepetitionIntervalavg;
ProhibitionPeriod ← ProhibitionPeriod × Increase;7
ProhibitionPeriodt ← Iterationi;8
end9
else10
VisitedSolutions ← Scurr;11
Scurrt ← Iterationi;12
end13
if Iterationi − ProhibitionPeriodt > RepetitionIntervalavg then14
ProhibitionPeriod ← Max(1, ProhibitionPeriod × Decrease);15
ProhibitionPeriodt ← Iterationi;16
end17
2.11. Reactive Tabu Search 81
Algorithm 2.11.3: Pseudocode for the BestMove function.
Input: ProblemSize
Output: Scurr
CandidateListadmissible ← GetAdmissibleMoves(CandidateList);1
CandidateListtabu ← CandidateList − CandidateListadmissible;2
if Size(CandidateListadmissible) < 2 then3
ProhibitionPeriod ← ProblemSize − 2;4
ProhibitionPeriodt ← Iterationi;5
end6
Scurr ← GetBest(CandidateListadmissible);7
Sbesttabu ← GetBest(CandidateListtabu);8
if Cost(Sbesttabu) < Cost(Sbest) ∧ Cost(Sbesttabu) < Cost(Scurr)9
then
Scurr ← Sbesttabu;10
end11
return Scurr;12
Algorithm 2.11.4: Pseudocode for the IsTabu function.
Input:
Output: Tabu
Tabu ← FALSE;1
Scurrtfeature ← RetrieveTimeFeatureLastUsed(Scurrfeature);2
if Scurrtfeature ≥ Iterationcurr − ProhibitionPeriod then3
Tabu ← TRUE;4
end5
return Tabu;6
� Reactive Tabu Search was designed for discrete domains such as
combinatorial optimization, although has been applied to continuous
function optimization.
� Reactive Tabu Search was proposed to use efficient memory data
structures such as hash tables.
� Reactive Tabu Search was proposed to use an long-term memory to
diversify the search after a threshold of cycle repetitions has been
reached.
� The increase parameter should be greater than one (such as 1.1 or
1.3) and the decrease parameter should be less than one (such as 0.9
or 0.8).
82 Chapter 2. Stochastic Algorithms
2.11.5 Code Listing
Listing 2.10 provides an example of the Reactive Tabu Search algorithm
implemented in the Ruby Programming Language. The algorithm is appliedto the Berlin52 instance of the Traveling Salesman Problem (TSP), taken
from the TSPLIB. The problem seeks a permutation of the order to visit
cities (called a tour) that minimizes the total distance traveled. The optimal
tour distance for Berlin52 instance is 7542 units.
The procedure is based on the code listing described by Battiti and
Tecchiolli in [9] with supplements like the IsTabu function from [7]. The
implementation does not use efficient memory data structures such as hash
tables. The algorithm is initialized with a stochastic 2-opt local search,
and the neighborhood is generated as a fixed candidate list of stochastic
2-opt moves. The edges selected for changing in the 2-opt move are stored
as features in the tabu list. The example does not implement the escape
procedure for search diversification.
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(perm, cities)
6 distance = 0
7 perm.each_with_index do |c1, i|
8 c2 = (i==perm.size-1) ? perm[0] : perm[i+1]
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def random_permutation(cities)
15 perm = Array.new(cities.size){|i| i}
16 perm.each_index do |i|
17 r = rand(perm.size-i) + i
18 perm[r], perm[i] = perm[i], perm[r]
19 end
20 return perm
21 end
22
23 def stochastic_two_opt(parent)
24 perm = Array.new(parent)
25 c1, c2 = rand(perm.size), rand(perm.size)
26 exclude = [c1]
27 exclude << ((c1==0) ? perm.size-1 : c1-1)
28 exclude << ((c1==perm.size-1) ? 0 : c1+1)
29 c2 = rand(perm.size) while exclude.include?(c2)
30 c1, c2 = c2, c1 if c2 < c1
31 perm[c1...c2] = perm[c1...c2].reverse
32 return perm, [[parent[c1-1], parent[c1]], [parent[c2-1], parent[c2]]]
33 end
34
35 def is_tabu?(edge, tabu_list, iter, prohib_period)
36 tabu_list.each do |entry|
2.11. Reactive Tabu Search 83
37 if entry[:edge] == edge
38 return true if entry[:iter] >= iter-prohib_period
39 return false
40 end
41 end
42 return false
43 end
44
45 def make_tabu(tabu_list, edge, iter)
46 tabu_list.each do |entry|
47 if entry[:edge] == edge
48 entry[:iter] = iter
49 return entry
50 end
51 end
52 entry = {:edge=>edge, :iter=>iter}
53 tabu_list.push(entry)
54 return entry
55 end
56
57 def to_edge_list(perm)
58 list = []
59 perm.each_with_index do |c1, i|
60 c2 = (i==perm.size-1) ? perm[0] : perm[i+1]
61 c1, c2 = c2, c1 if c1 > c2
62 list << [c1, c2]
63 end
64 return list
65 end
66
67 def equivalent?(el1, el2)
68 el1.each {|e| return false if !el2.include?(e) }
69 return true
70 end
71
72 def generate_candidate(best, cities)
73 candidate = {}
74 candidate[:vector], edges = stochastic_two_opt(best[:vector])
75 candidate[:cost] = cost(candidate[:vector], cities)
76 return candidate, edges
77 end
78
79 def get_candidate_entry(visited_list, permutation)
80 edgeList = to_edge_list(permutation)
81 visited_list.each do |entry|
82 return entry if equivalent?(edgeList, entry[:edgelist])
83 end
84 return nil
85 end
86
87 def store_permutation(visited_list, permutation, iteration)
88 entry = {}
89 entry[:edgelist] = to_edge_list(permutation)
90 entry[:iter] = iteration
91 entry[:visits] = 1
92 visited_list.push(entry)
84 Chapter 2. Stochastic Algorithms
93 return entry
94 end
95
96 def sort_neighborhood(candidates, tabu_list, prohib_period, iteration)
97 tabu, admissable = [], []
98 candidates.each do |a|
99 if is_tabu?(a[1][0], tabu_list, iteration, prohib_period) or
100 is_tabu?(a[1][1], tabu_list, iteration, prohib_period)
101 tabu << a
102 else
103 admissable << a
104 end
105 end
106 return [tabu, admissable]
107 end
108
109 def search(cities, max_cand, max_iter, increase, decrease)
110 current = {:vector=>random_permutation(cities)}
111 current[:cost] = cost(current[:vector], cities)
112 best = current
113 tabu_list, prohib_period = [], 1
114 visited_list, avg_size, last_change = [], 1, 0
115 max_iter.times do |iter|
116 candidate_entry = get_candidate_entry(visited_list, current[:vector])
117 if !candidate_entry.nil?
118 repetition_interval = iter - candidate_entry[:iter]
119 candidate_entry[:iter] = iter
120 candidate_entry[:visits] += 1
121 if repetition_interval < 2*(cities.size-1)
122 avg_size = 0.1*(iter-candidate_entry[:iter]) + 0.9*avg_size
123 prohib_period = (prohib_period.to_f * increase)
124 last_change = iter
125 end
126 else
127 store_permutation(visited_list, current[:vector], iter)
128 end
129 if iter-last_change > avg_size
130 prohib_period = [prohib_period*decrease,1].max
131 last_change = iter
132 end
133 candidates = Array.new(max_cand) do |i|
134 generate_candidate(current, cities)
135 end
136 candidates.sort! {|x,y| x.first[:cost] <=> y.first[:cost]}
137 tabu,admis = sort_neighborhood(candidates,tabu_list,prohib_period,iter)
138 if admis.size < 2
139 prohib_period = cities.size-2
140 last_change = iter
141 end
142 current,best_move_edges = (admis.empty?) ? tabu.first : admis.first
143 if !tabu.empty?
144 tf = tabu.first[0]
145 if tf[:cost]<best[:cost] and tf[:cost]<current[:cost]
146 current, best_move_edges = tabu.first
147 end
148 end
2.11. Reactive Tabu Search 85
149 best_move_edges.each {|edge| make_tabu(tabu_list, edge, iter)}
150 best = candidates.first[0] if candidates.first[0][:cost] < best[:cost]
151 puts " > it=#{iter}, tenure=#{prohib_period.round}, best=#{best[:cost]}"
152 end
153 return best
154 end
155
156 if __FILE__ == $0
157 # problem configuration
158 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
159 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
160 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
161 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
162 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
163 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
164 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
165 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
166 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
167 # algorithm configuration
168 max_iter = 100
169 max_candidates = 50
170 increase = 1.3
171 decrease = 0.9
172 # execute the algorithm
173 best = search(berlin52, max_candidates, max_iter, increase, decrease)
174 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
175 end
Listing 2.10: Reactive Tabu Search in Ruby
2.11.6 References
Primary Sources
Reactive Tabu Search was proposed by Battiti and Tecchiolli as an extension
to Tabu Search that included an adaptive tabu list size in addition to a
diversification mechanism [7]. The technique also used efficient memory
structures that were based on an earlier work by Battiti and Tecchiolli that
considered a parallel tabu search [6]. Some early application papers by
Battiti and Tecchiolli include a comparison to Simulated Annealing applied
to the Quadratic Assignment Problem [8], benchmarked on instances of
the knapsack problem and N-K models and compared with Repeated Local
Minima Search, Simulated Annealing, and Genetic Algorithms [9], and
training neural networks on an array of problem instances [10].
Learn More
Reactive Tabu Search was abstracted to a form called Reactive Local
Search that considers adaptive methods that learn suitable parameters for
heuristics that manage an embedded local search technique [4, 5]. Under
this abstraction, the Reactive Tabu Search algorithm is a single example
86 Chapter 2. Stochastic Algorithms
of the Reactive Local Search principle applied to the Tabu Search. This
framework was further extended to the use of any adaptive machine learning
techniques to adapt the parameters of an algorithm by reacting to algorithm
outcomes online while solving a problem, called Reactive Search [1]. The
best reference for this general framework is the book on Reactive Search
Optimization by Battiti, Brunato, and Mascia [3]. Additionally, the review
chapter by Battitiand Brunato provides a contemporary description [2].
2.11.7 Bibliography
[1] R. Battiti. Machine learning methods for parameter tuning in heuristics.
In 5th DIMACS Challenge Workshop: Experimental Methodology Day,
1996.
[2] R. Battiti and M. Brunato. Handbook of Metaheuristics, chapter
Reactive Search Optimization: Learning while Optimizing. Springer
Verlag, 2nd edition, 2009.
[3] R. Battiti, M. Brunato, and F. Mascia. Reactive Search and Intelligent
Optimization. Springer, 2008.
[4] R. Battiti and M. Protasi. Reactive local search for the maximum
clique problem. Technical Report TR-95-052, International Computer
Science Institute, Berkeley, CA, 1995.
[5] R. Battiti and M. Protasi. Reactive local search for the maximum
clique problem. Algorithmica, 29(4):610–637, 2001.
[6] R. Battiti and G. Tecchiolli. Parallel biased search for combinato-
rial optimization: genetic algorithms and tabu. Microprocessors and
Microsystems, 16(7):351–367, 1992.
[7] R. Battiti and G. Tecchiolli. The reactive tabu search. ORSA Journal
on Computing, 6(2):126–140, 1994.
[8] R. Battiti and G. Tecchiolli. Simulated annealing and tabu search in
the long run: a comparison on qap tasks. Computer and Mathematics
with Applications, 28(6):1–8, 1994.
[9] R. Battiti and G. Tecchiolli. Local search with memory: Benchmarking
RTS. Operations Research Spektrum, 17(2/3):67–86, 1995.
[10] R. Battiti and G. Tecchiolli. Training neural nets with the reactive
tabu search. IEEE Transactions on Neural Networks, 6(5):1185–1200,
1995.
Chapter 3
Evolutionary Algorithms
3.1 Overview
This chapter describes Evolutionary Algorithms.
3.1.1 Evolution
Evolutionary Algorithms belong to the Evolutionary Computation field of
study concerned with computational methods inspired by the process and
mechanisms of biological evolution. The process of evolution by means
of natural selection (descent with modification) was proposed by Darwin
to account for the variety of life and its suitability (adaptive fit) for its
environment. The mechanisms of evolution describe how evolution actually
takes place through the modification and propagation of genetic material
(proteins). Evolutionary Algorithms are concerned with investigating com-
putational systems that resemble simplified versions of the processes and
mechanisms of evolution toward achieving the effects of these processes
and mechanisms, namely the development of adaptive systems. Additional
subject areas that fall within the realm of Evolutionary Computation are
algorithms that seek to exploit the properties from the related fields of
Population Genetics, Population Ecology, Coevolutionary Biology, and
Developmental Biology.
3.1.2 References
Evolutionary Algorithms share properties of adaptation through an iterative
process that accumulates and amplifies beneficial variation through trial
and error. Candidate solutions represent members of a virtual population
striving to survive in an environment defined by a problem specific objective
function. In each case, the evolutionary process refines the adaptive fit of
the population of candidate solutions in the environment, typically using
87
88 Chapter 3. Evolutionary Algorithms
surrogates for the mechanisms of evolution such as genetic recombination
and mutation.
There are many excellent texts on the theory of evolution, although
Darwin’s original source can be an interesting and surprisingly enjoyable
read [5]. Huxley’s book defined the modern synthesis in evolutionary biology
that combined Darwin’s natural selection with Mendel’s genetic mechanisms
[25], although any good textbook on evolution will suffice (such as Futuyma’s
“Evolution” [13]). Popular science books on evolution are an easy place to
start, such as Dawkins’ “The Selfish Gene” that presents a gene-centric
perspective on evolution [6], and Dennett’s “Darwin’s Dangerous Idea” that
considers the algorithmic properties of the process [8].
Goldberg’s classic text is still a valuable resource for the Genetic Algo-
rithm [14], and Holland’s text is interesting for those looking to learn about
the research into adaptive systems that became the Genetic Algorithm
[23]. Additionally, the seminal work by Koza should be considered for
those interested in Genetic Programming [30], and Schwefel’s seminal work
should be considered for those with an interest in Evolution Strategies [34].
For an in-depth review of the history of research into the use of simulated
evolutionary processed for problem solving, see Fogel [12] For a rounded and
modern review of the field of Evolutionary Computation, Bäck, Fogel, and
Michalewicz’s two volumes of “Evolutionary Computation” are an excellent
resource covering the major techniques, theory, and application specific
concerns [2, 3]. For some additional modern books on the unified field of
Evolutionary Computation and Evolutionary Algorithms, see De Jong [26],
a recent edition of Fogel [11], and Eiben and Smith [9].
3.1.3 Extensions
There are many other algorithms and classes of algorithm that were not
described from the field of Evolutionary Computation, not limited to:
� Distributed Evolutionary Computation: that are designed to
partition a population across computer networks or computational
units such as the Distributed or ‘Island Population’ Genetic Algorithm
[4, 35] and Diffusion Genetic Algorithms (also known as Cellular
Genetic Algorithms) [1].
� Niching Genetic Algorithms: that form groups or sub-populations
automatically within a population such as the Deterministic Crowding
Genetic Algorithm [31, 32], Restricted Tournament Selection [20, 21],
and Fitness Sharing Genetic Algorithm [7, 19].
� Evolutionary Multiple Objective Optimization Algorithms:
such as Vector-Evaluated Genetic Algorithm (VEGA) [33], Pareto
Archived Evolution Strategy (PAES) [28, 29], and the Niched Pareto
Genetic Algorithm (NPGA) [24].
3.1. Overview 89
� Classical Techniques: such as GENITOR [36], and the CHC Ge-
netic Algorithm [10].
� Competent Genetic Algorithms: (so-called [15]) such as the
Messy Genetic Algorithm [17, 18], Fast Messy Genetic Algorithm
[16], Gene Expression Messy Genetic Algorithm [27], and the Linkage-
Learning Genetic Algorithm [22].
3.1.4 Bibliography
[1] E. Alba and B. Dorronsoro. Cellular Genetic Algorithms. Springer,
2008.
[2] T. Bäck, D. B. Fogel, and Z. Michalewicz, editors. Evolutionary
Computation 1: Basic Algorithms and Operators. IoP, 2000.
[3] T. Bäck, D. B. Fogel, and Z. Michalewicz, editors. Evolutionary
Computation 2: Advanced Algorithms and Operations. IoP, 2000.
[4] E. Cantú-Paz. Efficient and Accurate Parallel Genetic Algorithms.
Kluwer Academic Publishers (Springer), 2000.
[5] C. Darwin. On the Origin of Species by Means of Natural Selection,
or the Preservation of Favoured Races in the Struggle for Life. John
Murray, 1859.
[6] R. Dawkins. The selfish gene. Oxford University Press, 1976.
[7] K. Deb and D. E. Goldberg. An investigation of niche and species
formation in genetic function optimization. In Proceedings of the Second
International Conference on Genetic Algorithms, 1989.
[8] D. C. Dennett. Darwin’s Dangerous Idea. Simon & Schuster, 1995.
[9] A. E. Eiben and J. E. Smith. Introduction to evolutionary computing.
Springer, 2003.
[10] L. J. Eshelman. The CHC adaptive search algorithm: How to do
safe search when engaging in nontraditional genetic recombination. In
Proceedings Foundations of Genetic Algorithms Conf., pages 265–283,
1991.
[11] D. B. Fogel. Evolutionary computation: Toward a new philosophy of
machine intelligence. IEEE Press, 1995.
[12] D. B. Fogel. Evolutionary Computation: The Fossil Record. Wiley-IEEE
Press, 1998.
[13] D. Futuyma. Evolution. Sinauer Associates Inc., 2nd edition, 2009.
90 Chapter 3. Evolutionary Algorithms
[14] D. E. Goldberg. Genetic Algorithms in Search, Optimization, and
Machine Learning. Addison-Wesley, 1989.
[15] D. E. Goldberg. The design of innovation: Lessons from and for
competent genetic algorithms. Springer, 2002.
[16] D. E. Goldberg, K. Deb, H.Kargupta, and G. Harik. Rapid, accurate
optimization of difficult problems using fast messy genetic algorithms.
In Proceedings of the Fifth International Conference on Genetic Algo-
rithms, 1993.
[17] D. E. Goldberg, K. Deb, and B. Korb. Messy genetic algorithms
revisited: studies in mixed size and scale. Complex Systems, 4:415–444,
1990.
[18] D. E. Goldberg, B. Korb, and K. Deb. Messy genetic algorithms:
Motivation, analysis, and first results. Complex Systems, 3:493–530,
1989.
[19] D. E. Goldberg and J. Richardson. Genetic algorithms with shar-
ing for multimodal function optimization. In Proceedings of the 2nd
Internaltional Conference on Genetic Algorithms, 1987.
[20] G. Harik. Finding multiple solutions in problems of bounded difficulty.
Technical Report IlliGAL Report No. 94002, University of Illinois at
Urbana–Champaign, 1994.
[21] G. Harik. Finding multimodal solutions using restricted tournament
selection. In Proceedings of the Sixth International Conference on
Genetic Algorithms, pages 24–31, 1995.
[22] G. R. Harik and D. E. Goldberg. Learning linkage. In Foundations of
Genetic Algorithms 4, pages 247–262, 1996.
[23] J. H. Holland. Adaptation in natural and artificial systems: An in-
troductory analysis with applications to biology, control, and artificial
intelligence. University of Michigan Press, 1975.
[24] J. Horn, N. Nafpliotis, and D. E. Goldberg. A niched pareto genetic
algorithm for multiobjective optimization. In Proceedings of the First
IEEE Conference on Evolutionary Computation, IEEE World Congress
on Computational Intelligence, volume 1, pages 82–87, 1994.
[25] J. Huxley. Evolution: The Modern Synthesis. Allen & Unwin, 1942.
[26] K. A. De Jong. Evolutionary computation: A unified approach. MIT
Press, 2006.
3.1. Overview 91
[27] H. Kargupta. The gene expression messy genetic algorithm. In Pro-
ceedings of the IEEE International Conference on Evolutionary Com-
putation, pages 814–819, 1996.
[28] J. D. Knowles and D. W. Corne. Local search, multiobjective optimiza-
tion and the pareto archived evolution strategy. In Proceedings of the
Third Australia–Japan Joint Workshop on Intelligent and Evolutionary
Systems, pages 209–216, 1999.
[29] J. D. Knowles and D. W. Corne. The pareto archived evolution strategy
: A new baseline algorithm for pareto multiobjective optimisation. In
Proceedings of the 1999 Congress on Evolutionary Computation, pages
98–105, 1999.
[30] J. R. Koza. Genetic programming: On the programming of computers
by means of natural selection. MIT Press, 1992.
[31] S. W. Mahfoud. Crowding and preselection revised. In Parallel Problem
Solving from Nature 2, pages 27–36, 1992.
[32] S. W. Mahfoud. Niching Methods for Genetic Algorithms. PhD thesis,
University of Illinois at Urbana–Champaign, 1995.
[33] D. J. Schaffer. Some experiments in machine learning using vector eval-
uated genetic algorithms. PhD thesis, Vanderbilt University, Tennessee,
1984.
[34] H-P. Schwefel. Numerical Optimization of Computer Models. John
Wiley & Sons, 1981.
[35] R. Tanese. Distributed genetic algorithms. In Proceedings of the third
international conference on Genetic algorithms, pages 434–439. Morgan
Kaufmann Publishers Inc., 1989.
[36] D. Whitley. The GENITOR algorithm and selective pressure: Why
rank-based allocation of reproductive trials is best. In D. Schaffer,
editor, Proceedings of the 3rd International Conference on Genetic
Algorithms, pages 116–121. Morgan Kaufmann, 1989.
92 Chapter 3. Evolutionary Algorithms
3.2 Genetic Algorithm
Genetic Algorithm, GA, Simple Genetic Algorithm, SGA, Canonical Genetic
Algorithm, CGA.
3.2.1 Taxonomy
The Genetic Algorithm is an Adaptive Strategy and a Global Optimization
technique. It is an Evolutionary Algorithm and belongs to the broader
study of Evolutionary Computation. The Genetic Algorithm is a sibling of
other Evolutionary Algorithms such as Genetic Programming (Section 3.3),
Evolution Strategies (Section 3.4), Evolutionary Programming (Section 3.6),
and Learning Classifier Systems (Section 3.9). The Genetic Algorithm is a
parent of a large number of variant techniques and sub-fields too numerous
to list.
3.2.2 Inspiration
The Genetic Algorithm is inspired by population genetics (including heredity
and gene frequencies), and evolution at the population level, as well as the
Mendelian understanding of the structure (such as chromosomes, genes,
alleles) and mechanisms (such as recombination and mutation). This is the
so-called new or modern synthesis of evolutionary biology.
3.2.3 Metaphor
Individuals of a population contribute their genetic material (called the
genotype) proportional to their suitability of their expressed genome (called
their phenotype) to their environment, in the form of offspring. The next
generation is created through a process of mating that involves recombination
of two individuals genomes in the population with the introduction of random
copying errors (called mutation). This iterative process may result in an
improved adaptive-fit between the phenotypes of individuals in a population
and the environment.
3.2.4 Strategy
The objective of the Genetic Algorithm is to maximize the payoff of candidate
solutions in the population against a cost function from the problem domain.
The strategy for the Genetic Algorithm is to repeatedly employ surrogates
for the recombination and mutation genetic mechanisms on the population
of candidate solutions, where the cost function (also known as objective or
fitness function) applied to a decoded representation of a candidate governs
the probabilistic contributions a given candidate solution can make to the
subsequent generation of candidate solutions.
3.2. Genetic Algorithm 93
3.2.5 Procedure
Algorithm 3.2.1 provides a pseudocode listing of the Genetic Algorithm for
minimizing a cost function.
Algorithm 3.2.1: Pseudocode for the Genetic Algorithm.
Input: Populationsize, Problemsize, Pcrossover, Pmutation
Output: Sbest
Population ← InitializePopulation(Populationsize,1
Problemsize);
EvaluatePopulation(Population);2
Sbest ← GetBestSolution(Population);3
while ¬StopCondition() do4
Parents ← SelectParents(Population, Populationsize);5
Children ← ∅;6
foreach Parent1, Parent2 ∈ Parents do7
Child1, Child2 ← Crossover(Parent1, Parent2, Pcrossover);8
Children ← Mutate(Child1, Pmutation);9
Children ← Mutate(Child2, Pmutation);10
end11
EvaluatePopulation(Children);12
Sbest ← GetBestSolution(Children);13
Population ← Replace(Population, Children);14
end15
return Sbest;16
3.2.6 Heuristics
� Binary strings (referred to as ‘bitstrings’) are the classical represen-
tation as they can be decoded to almost any desired representation.
Real-valued and integer variables can be decoded using the binary
coded decimal method, one’s or two’s complement methods, or the
gray code method, the latter of which is generally preferred.
� Problem specific representations and customized genetic operators
should be adopted, incorporating as much prior information about
the problem domain as possible.
� The size of the population must be large enough to provide sufficient
coverage of the domain and mixing of the useful sub-components of
the solution [7].
� The Genetic Algorithm is classically configured with a high probability
of recombination (such as 95%-99% of the selected population) and
94 Chapter 3. Evolutionary Algorithms
a low probability of mutation (such as 1
L
where L is the number of
components in a solution) [1, 18].
� The fitness-proportionate selection of candidate solutions to contribute
to the next generation should be neither too greedy (to avoid the
takeover of fitter candidate solutions) nor too random.
3.2.7 Code Listing
Listing 3.1 provides an example of the Genetic Algorithm implemented in the
Ruby Programming Language. The demonstration problem is a maximizing
binary optimization problem called OneMax that seeks a binary string of
unity (all ‘1’ bits). The objective function provides only an indication of
the number of correct bits in acandidate string, not the positions of the
correct bits.
The Genetic Algorithm is implemented with a conservative configuration
including binary tournament selection for the selection operator, one-point
crossover for the recombination operator, and point mutations for the
mutation operator.
1 def onemax(bitstring)
2 sum = 0
3 bitstring.size.times {|i| sum+=1 if bitstring[i].chr=='1'}
4 return sum
5 end
6
7 def random_bitstring(num_bits)
8 return (0...num_bits).inject(""){|s,i| s<<((rand<0.5) ? "1" : "0")}
9 end
10
11 def binary_tournament(pop)
12 i, j = rand(pop.size), rand(pop.size)
13 j = rand(pop.size) while j==i
14 return (pop[i][:fitness] > pop[j][:fitness]) ? pop[i] : pop[j]
15 end
16
17 def point_mutation(bitstring, rate=1.0/bitstring.size)
18 child = ""
19 bitstring.size.times do |i|
20 bit = bitstring[i].chr
21 child << ((rand()<rate) ? ((bit=='1') ? "0" : "1") : bit)
22 end
23 return child
24 end
25
26 def crossover(parent1, parent2, rate)
27 return ""+parent1 if rand()>=rate
28 point = 1 + rand(parent1.size-2)
29 return parent1[0...point]+parent2[point...(parent1.size)]
30 end
31
32 def reproduce(selected, pop_size, p_cross, p_mutation)
3.2. Genetic Algorithm 95
33 children = []
34 selected.each_with_index do |p1, i|
35 p2 = (i.modulo(2)==0) ? selected[i+1] : selected[i-1]
36 p2 = selected[0] if i == selected.size-1
37 child = {}
38 child[:bitstring] = crossover(p1[:bitstring], p2[:bitstring], p_cross)
39 child[:bitstring] = point_mutation(child[:bitstring], p_mutation)
40 children << child
41 break if children.size >= pop_size
42 end
43 return children
44 end
45
46 def search(max_gens, num_bits, pop_size, p_crossover, p_mutation)
47 population = Array.new(pop_size) do |i|
48 {:bitstring=>random_bitstring(num_bits)}
49 end
50 population.each{|c| c[:fitness] = onemax(c[:bitstring])}
51 best = population.sort{|x,y| y[:fitness] <=> x[:fitness]}.first
52 max_gens.times do |gen|
53 selected = Array.new(pop_size){|i| binary_tournament(population)}
54 children = reproduce(selected, pop_size, p_crossover, p_mutation)
55 children.each{|c| c[:fitness] = onemax(c[:bitstring])}
56 children.sort!{|x,y| y[:fitness] <=> x[:fitness]}
57 best = children.first if children.first[:fitness] >= best[:fitness]
58 population = children
59 puts " > gen #{gen}, best: #{best[:fitness]}, #{best[:bitstring]}"
60 break if best[:fitness] == num_bits
61 end
62 return best
63 end
64
65 if __FILE__ == $0
66 # problem configuration
67 num_bits = 64
68 # algorithm configuration
69 max_gens = 100
70 pop_size = 100
71 p_crossover = 0.98
72 p_mutation = 1.0/num_bits
73 # execute the algorithm
74 best = search(max_gens, num_bits, pop_size, p_crossover, p_mutation)
75 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:bitstring]}"
76 end
Listing 3.1: Genetic Algorithm in Ruby
3.2.8 References
Primary Sources
Holland is the grandfather of the field that became Genetic Algorithms.
Holland investigated adaptive systems in the late 1960s proposing an adap-
tive system formalism and adaptive strategies referred to as ‘adaptive plans’
96 Chapter 3. Evolutionary Algorithms
[8–10]. Holland’s theoretical framework was investigated and elaborated
by his Ph.D. students at the University of Michigan. Rosenberg investi-
gated a chemical and molecular model of a biological inspired adaptive plan
[19]. Bagley investigated meta-environments and a genetic adaptive plan
referred to as a genetic algorithm applied to a simple game called hexapawn
[2]. Cavicchio further elaborated the genetic adaptive plan by proposing
numerous variations, referring to some as ‘reproductive plans’ [15].
Other important contributions were made by Frantz who investigated
what were referred to as genetic algorithms for search [3], and Hollstien who
investigated genetic plans for adaptive control and function optimization [12].
De Jong performed a seminal investigation of the genetic adaptive model
(genetic plans) applied to continuous function optimization and his suite of
test problems adopted are still commonly used [13]. Holland wrote the the
seminal book on his research focusing on the proposed adaptive systems
formalism, the reproductive and genetic adaptive plans, and provided a
theoretical framework for the mechanisms used and explanation for the
capabilities of what would become genetic algorithms [11].
Learn More
The field of genetic algorithms is very large, resulting in large numbers of
variations on the canonical technique. Goldberg provides a classical overview
of the field in a review article [5], as does Mitchell [16]. Whitley describes
a classical tutorial for the Genetic Algorithm covering both practical and
theoretical concerns [20].
The algorithm is highly-modular and a sub-field exists to study each sub-
process, specifically: selection, recombination, mutation, and representation.
The Genetic Algorithm is most commonly used as an optimization technique,
although it should also be considered a general adaptive strategy [14]. The
schema theorem is a classical explanation for the power of the Genetic
Algorithm proposed by Holland [11], and investigated by Goldberg under
the name of the building block hypothesis [4].
The classical book on genetic algorithms as an optimization and machine
learning technique was written by Goldberg and provides an in-depth review
and practical study of the approach [4]. Mitchell provides a contemporary
reference text introducing the technique and the field [17]. Finally, Goldberg
provides a modern study of the field, the lessons learned, and reviews the
broader toolset of optimization algorithms that the field has produced [6].
3.2.9 Bibliography
[1] T. Bäck. Optimal mutation rates in genetic search. In Proceedings of
the Fifth International Conference on Genetic Algorithms, pages 2–9,
1993.
3.2. Genetic Algorithm 97
[2] J. D. Bagley. The behavior of adaptive systems which employ genetic
and correlation algorithms. PhD thesis, University of Michigan, 1967.
[3] D. R. Frantz. Non-linearities in genetic adaptive search. PhD thesis,
University of Michigan, 1972.
[4] D. E. Goldberg. Genetic Algorithms in Search, Optimization, and
Machine Learning. Addison-Wesley, 1989.
[5] D. E. Goldberg. Genetic and evolutionary algorithms come of age.
Communications of the ACM, 37(3):113–119, 1994.
[6] D. E. Goldberg. The design of innovation: Lessons from and for
competent genetic algorithms. Springer, 2002.
[7] D. E. Goldberg, K. Deb, and J. H. Clark. Genetic algorithms, noise,
and the sizing of populations. Complex Systems, 6:333–362, 1992.
[8] J. H. Holland. Information processing in adaptive systems. In Processing
of Information in the Nervous System, pages 330–338, 1962.
[9] J. H. Holland. Outline for a logical theory of adaptive systems. Journal
of the ACM (JACM), 9(3):297–314, 1962.
[10] J. H. Holland. Adaptive plans optimal for payoff-only environments.
In Proceedings of the Second Hawaii Conference on Systems Sciences,
1969.
[11] J. H. Holland. Adaptation in natural and artificial systems: An in-
troductory analysis with applications to biology, control, and artificial
intelligence. University of Michigan Press, 1975.
[12] R. B. Hollstien. Artificial genetic adaptation in computer control
systems. PhD thesis, The University of Michigan, 1971.
[13] K. A. De Jong. An analysis of the behavior of a class of genetic adaptive
systems. PhD thesis, University of Michigan Ann Arbor, MI, USA,
1975.
[14] K. A. De Jong. Genetic algorithms are NOT function optimizers.
In Proceedings of the Second Workshop on Foundations of Genetic
Algorithms, pages 5–17. Morgan Kaufmann, 1992.
[15] D. J. Cavicchio Jr. Adaptive Search Using Simulated Evolution. PhD
thesis, The University of Michigan, 1970.
[16] M. Mitchell. Genetic algorithms: An overview. Complexity, 1(1):31–39,
1995.
[17] M. Mitchell. An Introduction to Genetic Algorithms. MIT Press, 1998.
98 Chapter 3. Evolutionary Algorithms
[18] H. Mühlenbein. How genetic algorithmsreally work: I. mutation and
hillclimbing. In Parallel Problem Solving from Nature 2, pages 15–26,
1992.
[19] R. Rosenberg. Simulation of genetic populations with biochemical
properties. PhD thesis, University of Michigan, 1967.
[20] D. Whitley. A genetic algorithm tutorial. Statistics and Computing,
4:65–85, 1994.
3.3. Genetic Programming 99
3.3 Genetic Programming
Genetic Programming, GP.
3.3.1 Taxonomy
The Genetic Programming algorithm is an example of an Evolutionary
Algorithm and belongs to the field of Evolutionary Computation and more
broadly Computational Intelligence and Biologically Inspired Computation.
The Genetic Programming algorithm is a sibling to other Evolutionary
Algorithms such as the Genetic Algorithm (Section 3.2), Evolution Strate-
gies (Section 3.4), Evolutionary Programming (Section 3.6), and Learning
Classifier Systems (Section 3.9). Technically, the Genetic Programming
algorithm is an extension of the Genetic Algorithm. The Genetic Algorithm
is a parent to a host of variations and extensions.
3.3.2 Inspiration
The Genetic Programming algorithm is inspired by population genetics
(including heredity and gene frequencies), and evolution at the population
level, as well as the Mendelian understanding of the structure (such as
chromosomes, genes, alleles) and mechanisms (such as recombination and
mutation). This is the so-called new or modern synthesis of evolutionary
biology.
3.3.3 Metaphor
Individuals of a population contribute their genetic material (called the
genotype) proportional to their suitability of their expressed genome (called
their phenotype) to their environment. The next generation is created
through a process of mating that involves genetic operators such as recom-
bination of two individuals genomes in the population and the introduction
of random copying errors (called mutation). This iterative process may
result in an improved adaptive-fit between the phenotypes of individuals in
a population and the environment.
Programs may be evolved and used in a secondary adaptive process,
where an assessment of candidates at the end of that secondary adaptive
process is used for differential reproductive success in the first evolution-
ary process. This system may be understood as the inter-dependencies
experienced in evolutionary development where evolution operates upon
an embryo that in turn develops into an individual in an environment that
eventually may reproduce.
100 Chapter 3. Evolutionary Algorithms
3.3.4 Strategy
The objective of the Genetic Programming algorithm is to use induction to
devise a computer program. This is achieved by using evolutionary operators
on candidate programs with a tree structure to improve the adaptive fit
between the population of candidate programs and an objective function.
An assessment of a candidate solution involves its execution.
3.3.5 Procedure
Algorithm 3.3.1 provides a pseudocode listing of the Genetic Programming
algorithm for minimizing a cost function, based on Koza and Poli’s tutorial
[9].
The Genetic Program uses LISP-like symbolic expressions called S-
expressions that represent the graph of a program with function nodes and
terminal nodes. While the algorithm is running, the programs are treated
like data, and when they are evaluated they are executed. The traversal of
a program graph is always depth first, and functions must always return a
value.
3.3.6 Heuristics
� The Genetic Programming algorithm was designed for inductive auto-
matic programming and is well suited to symbolic regression, controller
design, and machine learning tasks under the broader name of function
approximation.
� Traditionally Lisp symbolic expressions are evolved and evaluated
in a virtual machine, although the approach has been applied with
compiled programming languages.
� The evaluation (fitness assignment) of a candidate solution typically
takes the structure of the program into account, rewarding parsimony.
� The selection process should be balanced between random selection and
greedy selection to bias the search towards fitter candidate solutions
(exploitation), whilst promoting useful diversity into the population
(exploration).
� A program may respond to zero or more input values and may produce
one or more outputs.
� All functions used in the function node set must return a usable result.
For example, the division function must return a sensible value (such
as zero or one) when a division by zero occurs.
� All genetic operations ensure (or should ensure) that syntactically valid
and executable programs are produced as a result of their application.
3.3. Genetic Programming 101
Algorithm 3.3.1: Pseudocode for Genetic Programming.
Input: Populationsize, nodesfunc, nodesterm, Pcrossover, Pmutation,
Preproduction, Palteration
Output: Sbest
Population ← InitializePopulation(Populationsize, nodesfunc,1
nodesterm);
EvaluatePopulation(Population);2
Sbest ← GetBestSolution(Population);3
while ¬StopCondition() do4
Children ← ∅;5
while Size(Children) < Populationsize do6
Operator ← SelectGeneticOperator(Pcrossover, Pmutation,7
Preproduction, Palteration);
if Operator ≡ CrossoverOperator then8
Parent1, Parent2 ← SelectParents(Population,9
Populationsize);
Child1, Child2 ← Crossover(Parent1, Parent2);10
Children ← Child1;11
Children ← Child2;12
else if Operator ≡ MutationOperator then13
Parent1 ← SelectParents(Population, Populationsize);14
Child1 ← Mutate(Parent1);15
Children ← Child1;16
else if Operator ≡ ReproductionOperator then17
Parent1 ← SelectParents(Population, Populationsize);18
Child1 ← Reproduce(Parent1);19
Children ← Child1;20
else if Operator ≡ AlterationOperator then21
Parent1 ← SelectParents(Population, Populationsize);22
Child1 ← AlterArchitecture(Parent1);23
Children ← Child1;24
end25
end26
EvaluatePopulation(Children);27
Sbest ← GetBestSolution(Children, Sbest);28
Population ← Children;29
end30
return Sbest;31
102 Chapter 3. Evolutionary Algorithms
� The Genetic Programming algorithm is commonly configured with a
high-probability of crossover (≥ 90%) and a low-probability of muta-
tion (≤ 1%). Other operators such as reproduction and architecture
alterations are used with moderate-level probabilities and fill in the
probabilistic gap.
� Architecture altering operations are not limited to the duplication
and deletion of sub-structures of a given program.
� The crossover genetic operator in the algorithm is commonly configured
to select a function as a the cross-point with a high-probability (≥ 90%)
and low-probability of selecting a terminal as a cross-point (≤ 10%).
� The function set may also include control structures such as conditional
statements and loop constructs.
� The Genetic Programing algorithm can be realized as a stack-based
virtual machine as opposed to a call graph [11].
� The Genetic Programming algorithm can make use of Automatically
Defined Functions (ADFs) that are sub-graphs and are promoted to
the status of functions for reuse and are co-evolved with the programs.
� The genetic operators employed during reproduction in the algorithm
may be considered transformation programs for candidate solutions
and may themselves be co-evolved in the algorithm [1].
3.3.7 Code Listing
Listing 3.2 provides an example of the Genetic Programming algorithm
implemented in the Ruby Programming Language based on Koza and Poli’s
tutorial [9].
The demonstration problem is an instance of a symbolic regression, where
a function must be devised to match a set of observations. In this case the
target function is a quadratic polynomial x2 + x+ 1 where x ∈ [−1, 1]. The
observations are generated directly from the target function without noise
for the purposes of this example. In practical problems, if one knew and
had access to the target function then the genetic program would not be
required.
The algorithm is configured to search for a program with the function
set {+,−,×,÷} and the terminal set {X,R}, where X is the input value,
and R is a static random variable generatedfor a program X ∈ [−5, 5]. A
division by zero returns a value of one. The fitness of a candidate solution is
calculated by evaluating the program on range of random input values and
calculating the Root Mean Squared Error (RMSE). The algorithm is config-
ured with a 90% probability of crossover, 8% probability of reproduction
(copying), and a 2% probability of mutation. For brevity, the algorithm
3.3. Genetic Programming 103
does not implement the architecture altering genetic operation and does not
bias crossover points towards functions over terminals.
1 def rand_in_bounds(min, max)
2 return min + (max-min)*rand()
3 end
4
5 def print_program(node)
6 return node if !node.kind_of?(Array)
7 return "(#{node[0]} #{print_program(node[1])} #{print_program(node[2])})"
8 end
9
10 def eval_program(node, map)
11 if !node.kind_of?(Array)
12 return map[node].to_f if !map[node].nil?
13 return node.to_f
14 end
15 arg1, arg2 = eval_program(node[1], map), eval_program(node[2], map)
16 return 0 if node[0] === :/ and arg2 == 0.0
17 return arg1.__send__(node[0], arg2)
18 end
19
20 def generate_random_program(max, funcs, terms, depth=0)
21 if depth==max-1 or (depth>1 and rand()<0.1)
22 t = terms[rand(terms.size)]
23 return ((t=='R') ? rand_in_bounds(-5.0, +5.0) : t)
24 end
25 depth += 1
26 arg1 = generate_random_program(max, funcs, terms, depth)
27 arg2 = generate_random_program(max, funcs, terms, depth)
28 return [funcs[rand(funcs.size)], arg1, arg2]
29 end
30
31 def count_nodes(node)
32 return 1 if !node.kind_of?(Array)
33 a1 = count_nodes(node[1])
34 a2 = count_nodes(node[2])
35 return a1+a2+1
36 end
37
38 def target_function(input)
39 return input**2 + input + 1
40 end
41
42 def fitness(program, num_trials=20)
43 sum_error = 0.0
44 num_trials.times do |i|
45 input = rand_in_bounds(-1.0, 1.0)
46 error = eval_program(program, {'X'=>input}) - target_function(input)
47 sum_error += error.abs
48 end
49 return sum_error / num_trials.to_f
50 end
51
52 def tournament_selection(pop, bouts)
53 selected = Array.new(bouts){pop[rand(pop.size)]}
104 Chapter 3. Evolutionary Algorithms
54 selected.sort!{|x,y| x[:fitness]<=>y[:fitness]}
55 return selected.first
56 end
57
58 def replace_node(node, replacement, node_num, cur_node=0)
59 return [replacement,(cur_node+1)] if cur_node == node_num
60 cur_node += 1
61 return [node,cur_node] if !node.kind_of?(Array)
62 a1, cur_node = replace_node(node[1], replacement, node_num, cur_node)
63 a2, cur_node = replace_node(node[2], replacement, node_num, cur_node)
64 return [[node[0], a1, a2], cur_node]
65 end
66
67 def copy_program(node)
68 return node if !node.kind_of?(Array)
69 return [node[0], copy_program(node[1]), copy_program(node[2])]
70 end
71
72 def get_node(node, node_num, current_node=0)
73 return node,(current_node+1) if current_node == node_num
74 current_node += 1
75 return nil,current_node if !node.kind_of?(Array)
76 a1, current_node = get_node(node[1], node_num, current_node)
77 return a1,current_node if !a1.nil?
78 a2, current_node = get_node(node[2], node_num, current_node)
79 return a2,current_node if !a2.nil?
80 return nil,current_node
81 end
82
83 def prune(node, max_depth, terms, depth=0)
84 if depth == max_depth-1
85 t = terms[rand(terms.size)]
86 return ((t=='R') ? rand_in_bounds(-5.0, +5.0) : t)
87 end
88 depth += 1
89 return node if !node.kind_of?(Array)
90 a1 = prune(node[1], max_depth, terms, depth)
91 a2 = prune(node[2], max_depth, terms, depth)
92 return [node[0], a1, a2]
93 end
94
95 def crossover(parent1, parent2, max_depth, terms)
96 pt1, pt2 = rand(count_nodes(parent1)-2)+1, rand(count_nodes(parent2)-2)+1
97 tree1, c1 = get_node(parent1, pt1)
98 tree2, c2 = get_node(parent2, pt2)
99 child1, c1 = replace_node(parent1, copy_program(tree2), pt1)
100 child1 = prune(child1, max_depth, terms)
101 child2, c2 = replace_node(parent2, copy_program(tree1), pt2)
102 child2 = prune(child2, max_depth, terms)
103 return [child1, child2]
104 end
105
106 def mutation(parent, max_depth, functs, terms)
107 random_tree = generate_random_program(max_depth/2, functs, terms)
108 point = rand(count_nodes(parent))
109 child, count = replace_node(parent, random_tree, point)
3.3. Genetic Programming 105
110 child = prune(child, max_depth, terms)
111 return child
112 end
113
114 def search(max_gens, pop_size, max_depth, bouts, p_repro, p_cross, p_mut,
functs, terms)
115 population = Array.new(pop_size) do |i|
116 {:prog=>generate_random_program(max_depth, functs, terms)}
117 end
118 population.each{|c| c[:fitness] = fitness(c[:prog])}
119 best = population.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
120 max_gens.times do |gen|
121 children = []
122 while children.size < pop_size
123 operation = rand()
124 p1 = tournament_selection(population, bouts)
125 c1 = {}
126 if operation < p_repro
127 c1[:prog] = copy_program(p1[:prog])
128 elsif operation < p_repro+p_cross
129 p2 = tournament_selection(population, bouts)
130 c2 = {}
131 c1[:prog],c2[:prog] = crossover(p1[:prog], p2[:prog], max_depth,
terms)
132 children << c2
133 elsif operation < p_repro+p_cross+p_mut
134 c1[:prog] = mutation(p1[:prog], max_depth, functs, terms)
135 end
136 children << c1 if children.size < pop_size
137 end
138 children.each{|c| c[:fitness] = fitness(c[:prog])}
139 population = children
140 population.sort!{|x,y| x[:fitness] <=> y[:fitness]}
141 best = population.first if population.first[:fitness] <= best[:fitness]
142 puts " > gen #{gen}, fitness=#{best[:fitness]}"
143 break if best[:fitness] == 0
144 end
145 return best
146 end
147
148 if __FILE__ == $0
149 # problem configuration
150 terms = ['X', 'R']
151 functs = [:+, :-, :*, :/]
152 # algorithm configuration
153 max_gens = 100
154 max_depth = 7
155 pop_size = 100
156 bouts = 5
157 p_repro = 0.08
158 p_cross = 0.90
159 p_mut = 0.02
160 # execute the algorithm
161 best = search(max_gens, pop_size, max_depth, bouts, p_repro, p_cross,
p_mut, functs, terms)
162 puts "done! Solution: f=#{best[:fitness]}, #{print_program(best[:prog])}"
106 Chapter 3. Evolutionary Algorithms
163 end
Listing 3.2: Genetic Programming in Ruby
3.3.8 References
Primary Sources
An early work by Cramer involved the study of a Genetic Algorithm using an
expression tree structure for representing computer programs for primitive
mathematical operations [3]. Koza is credited with the development of
the field of Genetic Programming. An early paper by Koza referred to
his hierarchical genetic algorithms as an extension to the simple genetic
algorithm that use symbolic expressions (S-expressions) as a representation
and were applied to a range of induction-style problems [4]. The seminal
reference for the field is Koza’s 1992 book on Genetic Programming [5].
Learn More
The field of Genetic Programming is vast, including many books, dedicated
conferences and thousands of publications. Koza is generally credited with
the development and popularizing of the field, publishing a large number of
books and papers himself. Koza provides a practical introduction to the
field as a tutorial and provides recent overview of the broader field and
usage of the technique [9].
In addition his the seminal 1992 book, Koza has released three more
volumes in the series including volume II on Automatically Defined Functions
(ADFs) [6], volume III that considered the Genetic Programming Problem
Solver (GPPS) for automatically defining the function set and program
structure for a given problem [7], and volume IV that focuses on the human
competitive results the technique is able to achieve in a routine manner
[8]. All books are rich with targeted and practical demonstration problem
instances.
Some additional excellent books include a text by Banzhaf et al. that
provides an introduction to the field [2], Langdon and Poli’s detailed look
at the technique [10], and Poli, Langdon, and McPhee’s contemporary and
practical field guideto Genetic Programming [12].
3.3.9 Bibliography
[1] P. J. Angeline. Two self-adaptive crossover operators for genetic pro-
gramming. In Peter J. Angeline and K. E. Kinnear, Jr., editors,
Advances in Genetic Programming 2, pages 89–110. MIT Press, 1996.
3.3. Genetic Programming 107
[2] W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone. Genetic Pro-
gramming – An Introduction; On the Automatic Evolution of Computer
Programs and its Applications. Morgan Kaufmann, 1998.
[3] N. L. Cramer. A representation for the adaptive generation of simple
sequential programs. In J. J. Grefenstette, editor, Proceedings of the
1st International Conference on Genetic Algorithms, pages 183–187,
1985.
[4] J. R. Koza. Hierarchical genetic algorithms operating on populations
of computer programs. In N. S. Sridharan, editor, Proceedings of
the Eleventh International Joint Conference on Artificial Intelligence
IJCAI-89, volume 1, pages 768–774, 1989.
[5] J. R. Koza. Genetic programming: On the programming of computers
by means of natural selection. MIT Press, 1992.
[6] J. R. Koza. Genetic programming II: Automatic discovery of reusable
programs. MIT Press, 1994.
[7] J. R. Koza, F. H. Bennett III, D. Andre, and M. A. Keane. Genetic
programming III: Darwinian invention and problem solving. Morgan
Kaufmann, 1999.
[8] J. R. Koza, M. A. Keane, M. J. Streeter, W. Mydlowec, J. Yu, and
G. Lanza. Genetic Programming IV: Routine Human-Competitive
Machine Intelligence. Springer, 2003.
[9] J. R. Koza and R. Poli. Search methodologies: Introductory tutorials
in optimization and decision support techniques, chapter 5: Genetic
Programming, pages 127–164. Springer, 2005.
[10] W. B. Langdon and R. Poli. Foundations of Genetic Programming.
Springer-Verlag, 2002.
[11] T. Perkis. Stack-based genetic programming. In Proc IEEE Congress
on Computational Intelligence, 1994.
[12] R. Poli, W. B. Langdon, and N. F. McPhee. A Field Programmers
Guide to Genetic Programming. Lulu Enterprises, 2008.
108 Chapter 3. Evolutionary Algorithms
3.4 Evolution Strategies
Evolution Strategies, Evolution Strategy, Evolutionary Strategies, ES.
3.4.1 Taxonomy
Evolution Strategies is a global optimization algorithm and is an instance
of an Evolutionary Algorithm from the field of Evolutionary Computa-
tion. Evolution Strategies is a sibling technique to other Evolutionary
Algorithms such as Genetic Algorithms (Section 3.2), Genetic Programming
(Section 3.3), Learning Classifier Systems (Section 3.9), and Evolutionary
Programming (Section 3.6). A popular descendant of the Evolution Strate-
gies algorithm is the Covariance Matrix Adaptation Evolution Strategies
(CMA-ES).
3.4.2 Inspiration
Evolution Strategies is inspired by the theory of evolution by means of
natural selection. Specifically, the technique is inspired by macro-level
or the species-level process of evolution (phenotype, hereditary, variation)
and is not concerned with the genetic mechanisms of evolution (genome,
chromosomes, genes, alleles).
3.4.3 Strategy
The objective of the Evolution Strategies algorithm is to maximize the
suitability of collection of candidate solutions in the context of an ob-
jective function from a domain. The objective was classically achieved
through the adoption of dynamic variation, a surrogate for descent with
modification, where the amount of variation was adapted dynamically with
performance-based heuristics. Contemporary approaches co-adapt param-
eters that control the amount and bias of variation with the candidate
solutions.
3.4.4 Procedure
Instances of Evolution Strategies algorithms may be concisely described with
a custom terminology in the form (µ, λ)−ES, where µ is number of candidate
solutions in the parent generation, and λ is the number of candidate solutions
generated from the parent generation. In this configuration, the best µ are
kept if λ > µ, where λ must be great or equal to µ. In addition to the
so-called comma-selection Evolution Strategies algorithm, a plus-selection
variation may be defined (µ+λ)−ES, where the best members of the union
of the µ and λ generations compete based on objective fitness for a position
in the next generation. The simplest configuration is the (1 + 1) − ES,
3.4. Evolution Strategies 109
which is a type of greedy hill climbing algorithm. Algorithm 3.4.1 provides
a pseudocode listing of the (µ, λ) − ES algorithm for minimizing a cost
function. The algorithm shows the adaptation of candidate solutions that co-
adapt their own strategy parameters that influence the amount of mutation
applied to a candidate solutions descendants.
Algorithm 3.4.1: Pseudocode for (µ, λ) Evolution Strategies.
Input: µ, λ, ProblemSize
Output: Sbest
Population ← InitializePopulation(µ, ProblemSize);1
EvaluatePopulation(Population);2
Sbest ← GetBest(Population, 1);3
while ¬StopCondition() do4
Children ← ∅;5
for i = 0 to λ do6
Parenti ← GetParent(Population, i);7
Si ← ∅;8
Siproblem ← Mutate(Piproblem, Pistrategy);9
Sistrategy ← Mutate(Pistrategy);10
Children ← Si;11
end12
EvaluatePopulation(Children);13
Sbest ← GetBest(Children + Sbest, 1);14
Population ← SelectBest(Population, Children, µ);15
end16
return Sbest;17
3.4.5 Heuristics
� Evolution Strategies uses problem specific representations, such as
real values for continuous function optimization.
� The algorithm is commonly configured such that 1 ≤ µ ≤ λ.
� The ratio of µ to λ influences the amount of selection pressure (greed-
iness) exerted by the algorithm.
� A contemporary update to the algorithms notation includes a ρ as
(µ/ρ, λ)−ES that specifies the number of parents that will contribute
to each new candidate solution using a recombination operator.
� A classical rule used to govern the amount of mutation (standard
deviation used in mutation for continuous function optimization) was
the 15 -rule, where the ratio of successful mutations should be
1
5 of all
110 Chapter 3. Evolutionary Algorithms
mutations. If it is greater the variance is increased, otherwise if the
ratio is is less, the variance is decreased.
� The comma-selection variation of the algorithm can be good for dy-
namic problem instances given its capability for continued exploration
of the search space, whereas the plus-selection variation can be good
for refinement and convergence.
3.4.6 Code Listing
Listing 3.3 provides an example of the Evolution Strategies algorithm
implemented in the Ruby Programming Language. The demonstration
problem is an instance of a continuous function optimization that seeks
min f(x) where f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and n = 2. The optimal
solution for this basin function is (v0, . . . , vn−1) = 0.0. The algorithm is a
implementation of Evolution Strategies based on simple version described
by Bäck and Schwefel [2], which was also used as the basis of a detailed
empirical study [11]. The algorithm is an (30+20)−ES that adapts both the
problem and strategy (standard deviations) variables. More contemporary
implementations may modify the strategy variables differently, and include
an additional set of adapted strategy parameters to influence the direction
of mutation (see [7] for a concise description).
1 def objective_function(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def random_vector(minmax)
6 return Array.new(minmax.size) do |i|
7 minmax[i][0] + ((minmax[i][1] - minmax[i][0]) * rand())
8 end
9 end
10
11 def random_gaussian(mean=0.0, stdev=1.0)
12 u1 = u2 = w = 0
13 begin
14 u1 = 2 * rand() - 1
15 u2 = 2 * rand() - 1
16 w = u1 * u1 + u2 * u2
17 end while w >= 1
18 w = Math.sqrt((-2.0 * Math.log(w)) / w)
19 return mean + (u2 * w) * stdev
20 end
21
22 def mutate_problem(vector, stdevs, search_space)
23 child = Array(vector.size)
24 vector.each_with_index do |v, i|
25 child[i] = v + stdevs[i] * random_gaussian()
26 child[i] = search_space[i][0] if child[i] < search_space[i][0]
27 child[i] = search_space[i][1] if child[i] > search_space[i][1]
28 end
29 return child
3.4.Evolution Strategies 111
30 end
31
32 def mutate_strategy(stdevs)
33 tau = Math.sqrt(2.0*stdevs.size.to_f)**-1.0
34 tau_p = Math.sqrt(2.0*Math.sqrt(stdevs.size.to_f))**-1.0
35 child = Array.new(stdevs.size) do |i|
36 stdevs[i] * Math.exp(tau_p*random_gaussian() + tau*random_gaussian())
37 end
38 return child
39 end
40
41 def mutate(par, minmax)
42 child = {}
43 child[:vector] = mutate_problem(par[:vector], par[:strategy], minmax)
44 child[:strategy] = mutate_strategy(par[:strategy])
45 return child
46 end
47
48 def init_population(minmax, pop_size)
49 strategy = Array.new(minmax.size) do |i|
50 [0, (minmax[i][1]-minmax[i][0]) * 0.05]
51 end
52 pop = Array.new(pop_size, {})
53 pop.each_index do |i|
54 pop[i][:vector] = random_vector(minmax)
55 pop[i][:strategy] = random_vector(strategy)
56 end
57 pop.each{|c| c[:fitness] = objective_function(c[:vector])}
58 return pop
59 end
60
61 def search(max_gens, search_space, pop_size, num_children)
62 population = init_population(search_space, pop_size)
63 best = population.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
64 max_gens.times do |gen|
65 children = Array.new(num_children) do |i|
66 mutate(population[i], search_space)
67 end
68 children.each{|c| c[:fitness] = objective_function(c[:vector])}
69 union = children+population
70 union.sort!{|x,y| x[:fitness] <=> y[:fitness]}
71 best = union.first if union.first[:fitness] < best[:fitness]
72 population = union.first(pop_size)
73 puts " > gen #{gen}, fitness=#{best[:fitness]}"
74 end
75 return best
76 end
77
78 if __FILE__ == $0
79 # problem configuration
80 problem_size = 2
81 search_space = Array.new(problem_size) {|i| [-5, +5]}
82 # algorithm configuration
83 max_gens = 100
84 pop_size = 30
85 num_children = 20
112 Chapter 3. Evolutionary Algorithms
86 # execute the algorithm
87 best = search(max_gens, search_space, pop_size, num_children)
88 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:vector].inspect}"
89 end
Listing 3.3: Evolution Strategies in Ruby
3.4.7 References
Primary Sources
Evolution Strategies was developed by three students (Bienert, Rechenberg,
Schwefel) at the Technical University in Berlin in 1964 in an effort to
robotically optimize an aerodynamics design problem. The seminal work
in Evolution Strategies was Rechenberg’s PhD thesis [5] that was later
published as a book [6], both in German. Many technical reports and
papers were published by Schwefel and Rechenberg, although the seminal
paper published in English was by Klockgether and Schwefel on the two–
phase nozzle design problem [4].
Learn More
Schwefel published his PhD dissertation [8] not long after Rechenberg, which
was also published as a book [9], both in German. Schwefel’s book was
later translated into English and represents a classical reference for the
technique [10]. Bäck et al. provide a classical introduction to the technique,
covering the history, development of the algorithm, and the steps that lead
it to where it was in 1991 [1]. Beyer and Schwefel provide a contemporary
introduction to the field that includes a detailed history of the approach,
the developments and improvements since its inception, and an overview of
the theoretical findings that have been made [3].
3.4.8 Bibliography
[1] T. Bäck, F. Hoffmeister, and H-P. Schwefel. A survey of evolution
strategies. In Proceedings of the Fourth International Conference on
Genetic Algorithms, pages 2–9, 1991.
[2] T. Bäck and H-P. Schwefel. An overview of evolutionary algorithms for
parameter optimization. Evolutionary Computation, 1(1):1–23, 1993.
[3] H-G. Beyer and H-P. Schwefel. Evolution strategies: A comprehensive
introduction. Natural Computing: an international journal, 1(1):3–52,
2002.
[4] J. Klockgether and H-P. Schwefel. Two–phase nozzle and hollow core
jet experiments. In Proceedings of the Eleventh Symp. Engineering
3.4. Evolution Strategies 113
Aspects of Magnetohydrodynamics, pages 141–148. California Institute
of Technology, 1970.
[5] I. Rechenberg. Evolutionsstrategie: Optimierung technischer Systeme
nach Prinzipien der biologischen Evolution. PhD thesis, Technical
University of Berlin, Department of Process Engineering, 1971.
[6] I. Rechenberg. Evolutionsstrategie: Optimierung technischer Systeme
nach Prinzipien der biologischen Evolution. Frommann-Holzboog Ver-
lag, 1973.
[7] G. Rudolph. Evolutionary Computation 1: Basic Algorithms and
Operations, chapter 9: Evolution Strategies, pages 81–88. IoP Press,
2000.
[8] H-P. Schwefel. Evolutionsstrategie und numerische Optimierung. PhD
thesis, Technical University of Berlin, Department of Process Engineer-
ing, 1975.
[9] H-P. Schwefel. Numerische Optimierung von Computer – Modellen
mittels der Evolutionsstrategie. Birkhaeuser, 1977.
[10] H-P. Schwefel. Numerical Optimization of Computer Models. John
Wiley & Sons, 1981.
[11] X. Yao and Y. Liu. Fast evolution strategies. In Proceedings of the
6th International Conference on Evolutionary Programming VI, pages
151–162, 1997.
114 Chapter 3. Evolutionary Algorithms
3.5 Differential Evolution
Differential Evolution, DE.
3.5.1 Taxonomy
Differential Evolution is a Stochastic Direct Search and Global Optimiza-
tion algorithm, and is an instance of an Evolutionary Algorithm from the
field of Evolutionary Computation. It is related to sibling Evolutionary
Algorithms such as the Genetic Algorithm (Section 3.2), Evolutionary Pro-
gramming (Section 3.6), and Evolution Strategies (Section 3.4), and has
some similarities with Particle Swarm Optimization (Section 6.2).
3.5.2 Strategy
The Differential Evolution algorithm involves maintaining a population of
candidate solutions subjected to iterations of recombination, evaluation,
and selection. The recombination approach involves the creation of new
candidate solution components based on the weighted difference between
two randomly selected population members added to a third population
member. This perturbs population members relative to the spread of the
broader population. In conjunction with selection, the perturbation effect
self-organizes the sampling of the problem space, bounding it to known
areas of interest.
3.5.3 Procedure
Differential Evolution has a specialized nomenclature that describes the
adopted configuration. This takes the form of DE/x/y/z, where x represents
the solution to be perturbed (such a random or best). The y signifies the
number of difference vectors used in the perturbation of x, where a difference
vectors is the difference between two randomly selected although distinct
members of the population. Finally, z signifies the recombination operator
performed such as bin for binomial and exp for exponential.
Algorithm 3.5.1 provides a pseudocode listing of the Differential Evo-
lution algorithm for minimizing a cost function, specifically a DE/rand/-
1/bin configuration. Algorithm 3.5.2 provides a pseudocode listing of the
NewSample function from the Differential Evolution algorithm.
3.5.4 Heuristics
� Differential evolution was designed for nonlinear, non-differentiable
continuous function optimization.
� The weighting factor F ∈ [0, 2] controls the amplification of differential
variation, a value of 0.8 is suggested.
3.5. Differential Evolution 115
Algorithm 3.5.1: Pseudocode for Differential Evolution.
Input: Populationsize, Problemsize, Weightingfactor,
Crossoverrate
Output: Sbest
Population ← InitializePopulation(Populationsize,1
Problemsize);
EvaluatePopulation(Population);2
Sbest ← GetBestSolution(Population);3
while ¬ StopCondition() do4
NewPopulation ← ∅;5
foreach Pi ∈ Population do6
Si ← NewSample(Pi, Population, Problemsize,7
Weightingfactor, Crossoverrate);
if Cost(Si) ≤ Cost(Pi) then8
NewPopulation ← Si;9
else10
NewPopulation ← Pi;11
end12
end13
Population ← NewPopulation;14
EvaluatePopulation(Population);15
Sbest ← GetBestSolution(Population);16
end17
return Sbest;18
� the crossover weight CR ∈ [0, 1] probabilistically controls the amount
of recombination, avalue of 0.9 is suggested.
� The initial population of candidate solutions should be randomly
generated from within the space of valid solutions.
� The popular configurations are DE/rand/1/* and DE/best/2/*.
3.5.5 Code Listing
Listing 3.4 provides an example of the Differential Evolution algorithm
implemented in the Ruby Programming Language. The demonstration
problem is an instance of a continuous function optimization that seeks
min f(x) where f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and n = 3. The optimal
solution for this basin function is (v0, . . . , vn−1) = 0.0. The algorithm is an
implementation of Differential Evolution with the DE/rand/1/bin configu-
ration proposed by Storn and Price [9].
116 Chapter 3. Evolutionary Algorithms
Algorithm 3.5.2: Pseudocode for the NewSample function.
Input: P0, Population, NP, F, CR
Output: S
repeat1
P1 ← RandomMember(Population);2
until P1 6= P0 ;3
repeat4
P2 ← RandomMember(Population);5
until P2 6= P0 ∨ P2 6= P1 ;6
repeat7
P3 ← RandomMember(Population);8
until P3 6= P0 ∨ P3 6= P1 ∨ P3 6= P2 ;9
CutPoint ← RandomPosition(NP);10
S ← 0;11
for i to NP do12
if i ≡ CutPoint ∧ Rand() < CR then13
Si ← P3i + F × (P1i - P2i);14
else15
Si ← P0i ;16
end17
end18
return S;19
1 def objective_function(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def random_vector(minmax)
6 return Array.new(minmax.size) do |i|
7 minmax[i][0] + ((minmax[i][1] - minmax[i][0]) * rand())
8 end
9 end
10
11 def de_rand_1_bin(p0, p1, p2, p3, f, cr, search_space)
12 sample = {:vector=>Array.new(p0[:vector].size)}
13 cut = rand(sample[:vector].size-1) + 1
14 sample[:vector].each_index do |i|
15 sample[:vector][i] = p0[:vector][i]
16 if (i==cut or rand() < cr)
17 v = p3[:vector][i] + f * (p1[:vector][i] - p2[:vector][i])
18 v = search_space[i][0] if v < search_space[i][0]
19 v = search_space[i][1] if v > search_space[i][1]
20 sample[:vector][i] = v
21 end
22 end
23 return sample
24 end
3.5. Differential Evolution 117
25
26 def select_parents(pop, current)
27 p1, p2, p3 = rand(pop.size), rand(pop.size), rand(pop.size)
28 p1 = rand(pop.size) until p1 != current
29 p2 = rand(pop.size) until p2 != current and p2 != p1
30 p3 = rand(pop.size) until p3 != current and p3 != p1 and p3 != p2
31 return [p1,p2,p3]
32 end
33
34 def create_children(pop, minmax, f, cr)
35 children = []
36 pop.each_with_index do |p0, i|
37 p1, p2, p3 = select_parents(pop, i)
38 children << de_rand_1_bin(p0, pop[p1], pop[p2], pop[p3], f, cr, minmax)
39 end
40 return children
41 end
42
43 def select_population(parents, children)
44 return Array.new(parents.size) do |i|
45 (children[i][:cost]<=parents[i][:cost]) ? children[i] : parents[i]
46 end
47 end
48
49 def search(max_gens, search_space, pop_size, f, cr)
50 pop = Array.new(pop_size) {|i| {:vector=>random_vector(search_space)}}
51 pop.each{|c| c[:cost] = objective_function(c[:vector])}
52 best = pop.sort{|x,y| x[:cost] <=> y[:cost]}.first
53 max_gens.times do |gen|
54 children = create_children(pop, search_space, f, cr)
55 children.each{|c| c[:cost] = objective_function(c[:vector])}
56 pop = select_population(pop, children)
57 pop.sort!{|x,y| x[:cost] <=> y[:cost]}
58 best = pop.first if pop.first[:cost] < best[:cost]
59 puts " > gen #{gen+1}, fitness=#{best[:cost]}"
60 end
61 return best
62 end
63
64 if __FILE__ == $0
65 # problem configuration
66 problem_size = 3
67 search_space = Array.new(problem_size) {|i| [-5, +5]}
68 # algorithm configuration
69 max_gens = 200
70 pop_size = 10*problem_size
71 weightf = 0.8
72 crossf = 0.9
73 # execute the algorithm
74 best = search(max_gens, search_space, pop_size, weightf, crossf)
75 puts "done! Solution: f=#{best[:cost]}, s=#{best[:vector].inspect}"
76 end
Listing 3.4: Differential Evolution in Ruby
118 Chapter 3. Evolutionary Algorithms
3.5.6 References
Primary Sources
The Differential Evolution algorithm was presented by Storn and Price in
a technical report that considered DE1 and DE2 variants of the approach
applied to a suite of continuous function optimization problems [7]. An early
paper by Storn applied the approach to the optimization of an IIR-filter
(Infinite Impulse Response) [5]. A second early paper applied the approach to
a second suite of benchmark problem instances, adopting the contemporary
nomenclature for describing the approach, including the DE/rand/1/* and
DE/best/2/* variations [8]. The early work including technical reports and
conference papers by Storn and Price culminated in a seminal journal article
[9].
Learn More
A classical overview of Differential Evolution was presented by Price and
Storn [2], and terse introduction to the approach for function optimization
is presented by Storn [6]. A seminal extended description of the algorithm
with sample applications was presented by Storn and Price as a book chapter
[3]. Price, Storn, and Lampinen released a contemporary book dedicated
to Differential Evolution including theory, benchmarks, sample code, and
numerous application demonstrations [4]. Chakraborty also released a book
considering extensions to address complexities such as rotation invariance
and stopping criteria [1].
3.5.7 Bibliography
[1] U. K. Chakraborty. Advances in Differential Evolution. Springer, 2008.
[2] K. Price and R. Storn. Differential evolution: Numerical optimization
made easy. Dr. Dobb’s Journal, 78:18–24, 1997.
[3] K. V. Price. New Ideas in Optimization, chapter An introduction to
differential evolution, pages 79–108. McGraw-Hill Ltd., UK, 1999.
[4] K. V. Price, R. M. Storn, and J. A. Lampinen. Differential evolution: A
practical approach to global optimization. Springer, 2005.
[5] R. Storn. Differential evolution design of an IIR-filter. In Proceedings
IEEE Conference Evolutionary Computation, pages 268–273. IEEE,
1996.
[6] R. Storn. On the usage of differential evolution for function optimization.
In Proceedings Fuzzy Information Processing Society, 1996 Biennial
Conference of the North American, pages 519–523, 1996.
3.5. Differential Evolution 119
[7] R. Storn and K. Price. Differential evolution: A simple and efficient adap-
tive scheme for global optimization over continuous spaces. Technical
Report TR-95-012, International Computer Science Institute, Berkeley,
CA, 1995.
[8] R. Storn and K. Price. Minimizing the real functions of the ICEC’96
contest by differential evolution. In Proceedings of IEEE International
Conference on Evolutionary Computation, pages 842–844. IEEE, 1996.
[9] R. Storn and K. Price. Differential evolution: A simple and efficient
heuristic for global optimization over continuous spaces. Journal of
Global Optimization, 11:341–359, 1997.
120 Chapter 3. Evolutionary Algorithms
3.6 Evolutionary Programming
Evolutionary Programming, EP.
3.6.1 Taxonomy
Evolutionary Programming is a Global Optimization algorithm and is
an instance of an Evolutionary Algorithm from the field of Evolutionary
Computation. The approach is a sibling of other Evolutionary Algorithms
such as the Genetic Algorithm (Section 3.2), and Learning Classifier Systems
(Section 3.9). It is sometimes confused with Genetic Programming given
the similarity in name (Section 3.3), and more recently it shows a strong
functional similarity to Evolution Strategies (Section 3.4).
3.6.2 Inspiration
Evolutionary Programming is inspired by the theory of evolution by means
of natural selection. Specifically, the technique is inspired by macro-level
or the species-level process of evolution (phenotype, hereditary, variation)
and is not concerned with the genetic mechanisms of evolution (genome,
chromosomes, genes, alleles).
3.6.3 Metaphor
A population of a species reproduce, creating progeny with small pheno-
typical variation. The progeny and the parents compete based on their
suitability to the environment, where the generally more fit members con-
stitute the subsequent generation and are provided with the opportunity
to reproduce themselves. This processrepeats, improving the adaptive fit
between the species and the environment.
3.6.4 Strategy
The objective of the Evolutionary Programming algorithm is to maximize the
suitability of a collection of candidate solutions in the context of an objective
function from the domain. This objective is pursued by using an adaptive
model with surrogates for the processes of evolution, specifically hereditary
(reproduction with variation) under competition. The representation used
for candidate solutions is directly assessable by a cost or objective function
from the domain.
3.6.5 Procedure
Algorithm 3.6.1 provides a pseudocode listing of the Evolutionary Program-
ming algorithm for minimizing a cost function.
3.6. Evolutionary Programming 121
Algorithm 3.6.1: Pseudocode for Evolutionary Programming.
Input: Populationsize, ProblemSize, BoutSize
Output: Sbest
Population ← InitializePopulation(Populationsize, ProblemSize);1
EvaluatePopulation(Population);2
Sbest ← GetBestSolution(Population);3
while ¬StopCondition() do4
Children ← ∅;5
foreach Parenti ∈ Population do6
Childi ← Mutate(Parenti);7
Children ← Childi;8
end9
EvaluatePopulation(Children);10
Sbest ← GetBestSolution(Children, Sbest);11
Union ← Population + Children;12
foreach Si ∈ Union do13
for 1 to BoutSize do14
Sj ← RandomSelection(Union);15
if Cost(Si) < Cost(Sj) then16
Siwins ← Siwins + 1;17
end18
end19
end20
Population ← SelectBestByWins(Union, Populationsize);21
end22
return Sbest;23
3.6.6 Heuristics
� The representation for candidate solutions should be domain specific,
such as real numbers for continuous function optimization.
� The sample size (bout size) for tournament selection during competi-
tion is commonly between 5% and 10% of the population size.
� Evolutionary Programming traditionally only uses the mutation opera-
tor to create new candidate solutions from existing candidate solutions.
The crossover operator that is used in some other Evolutionary Algo-
rithms is not employed in Evolutionary Programming.
� Evolutionary Programming is concerned with the linkage between par-
ent and child candidate solutions and is not concerned with surrogates
for genetic mechanisms.
122 Chapter 3. Evolutionary Algorithms
� Continuous function optimization is a popular application for the
approach, where real-valued representations are used with a Gaussian-
based mutation operator.
� The mutation-specific parameters used in the application of the algo-
rithm to continuous function optimization can be adapted in concert
with the candidate solutions [4].
3.6.7 Code Listing
Listing 3.5 provides an example of the Evolutionary Programming algorithm
implemented in the Ruby Programming Language. The demonstration
problem is an instance of a continuous function optimization that seeks
min f(x) where f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and n = 2. The optimal
solution for this basin function is (v0, . . . , vn−1) = 0.0. The algorithm is
an implementation of Evolutionary Programming based on the classical
implementation for continuous function optimization by Fogel et al. [4] with
per-variable adaptive variance based on Fogel’s description for a self-adaptive
variation on page 160 of his 1995 book [3].
1 def objective_function(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def random_vector(minmax)
6 return Array.new(minmax.size) do |i|
7 minmax[i][0] + ((minmax[i][1] - minmax[i][0]) * rand())
8 end
9 end
10
11 def random_gaussian(mean=0.0, stdev=1.0)
12 u1 = u2 = w = 0
13 begin
14 u1 = 2 * rand() - 1
15 u2 = 2 * rand() - 1
16 w = u1 * u1 + u2 * u2
17 end while w >= 1
18 w = Math.sqrt((-2.0 * Math.log(w)) / w)
19 return mean + (u2 * w) * stdev
20 end
21
22 def mutate(candidate, search_space)
23 child = {:vector=>[], :strategy=>[]}
24 candidate[:vector].each_with_index do |v_old, i|
25 s_old = candidate[:strategy][i]
26 v = v_old + s_old * random_gaussian()
27 v = search_space[i][0] if v < search_space[i][0]
28 v = search_space[i][1] if v > search_space[i][1]
29 child[:vector] << v
30 child[:strategy] << s_old + random_gaussian() * s_old.abs**0.5
31 end
32 return child
3.6. Evolutionary Programming 123
33 end
34
35 def tournament(candidate, population, bout_size)
36 candidate[:wins] = 0
37 bout_size.times do |i|
38 other = population[rand(population.size)]
39 candidate[:wins] += 1 if candidate[:fitness] < other[:fitness]
40 end
41 end
42
43 def init_population(minmax, pop_size)
44 strategy = Array.new(minmax.size) do |i|
45 [0, (minmax[i][1]-minmax[i][0]) * 0.05]
46 end
47 pop = Array.new(pop_size, {})
48 pop.each_index do |i|
49 pop[i][:vector] = random_vector(minmax)
50 pop[i][:strategy] = random_vector(strategy)
51 end
52 pop.each{|c| c[:fitness] = objective_function(c[:vector])}
53 return pop
54 end
55
56 def search(max_gens, search_space, pop_size, bout_size)
57 population = init_population(search_space, pop_size)
58 population.each{|c| c[:fitness] = objective_function(c[:vector])}
59 best = population.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
60 max_gens.times do |gen|
61 children = Array.new(pop_size) {|i| mutate(population[i], search_space)}
62 children.each{|c| c[:fitness] = objective_function(c[:vector])}
63 children.sort!{|x,y| x[:fitness] <=> y[:fitness]}
64 best = children.first if children.first[:fitness] < best[:fitness]
65 union = children+population
66 union.each{|c| tournament(c, union, bout_size)}
67 union.sort!{|x,y| y[:wins] <=> x[:wins]}
68 population = union.first(pop_size)
69 puts " > gen #{gen}, fitness=#{best[:fitness]}"
70 end
71 return best
72 end
73
74 if __FILE__ == $0
75 # problem configuration
76 problem_size = 2
77 search_space = Array.new(problem_size) {|i| [-5, +5]}
78 # algorithm configuration
79 max_gens = 200
80 pop_size = 100
81 bout_size = 5
82 # execute the algorithm
83 best = search(max_gens, search_space, pop_size, bout_size)
84 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:vector].inspect}"
85 end
Listing 3.5: Evolutionary Programming in Ruby
124 Chapter 3. Evolutionary Algorithms
3.6.8 References
Primary Sources
Evolutionary Programming was developed by Lawrence Fogel, outlined in
early papers (such as [5]) and later became the focus of his PhD dissertation
[6]. Fogel focused on the use of an evolutionary process for the development
of control systems using Finite State Machine (FSM) representations. Fogel’s
early work on Evolutionary Programming culminated in a book (co-authored
with Owens and Walsh) that elaborated the approach, focusing on the
evolution of state machines for the prediction of symbols in time series data
[9].
Learn More
The field of Evolutionary Programming lay relatively dormant for 30 years
until it was revived by Fogel’s son, David. Early works considered the
application of Evolutionary Programming to control systems [11], and
later function optimization (system identification) culminating in a book
on the approach [1], and David Fogel’s PhD dissertation [2]. Lawrence
Fogel collaborated in the revival of the technique, including reviews [7, 8]
and extensions on what became the focus of the approach on function
optimization [4].
Yao et al. provide a seminal study of Evolutionary Programming propos-
ing an extension and racing it against the classical approach on a large
number of test problems [12]. Finally, Porto provides an excellent contem-
porary overview of the field and the technique [10].
3.6.9 Bibliography
[1] D. B. Fogel. System Identification Through Simulated Evolution: A
Machine Learning Approach to Modeling. Needham Heights, 1991.
[2] D. B. Fogel. Evolving artificial intelligence. PhD thesis, University of
California, San Diego, CA, USA, 1992.
[3] D. B. Fogel. Evolutionary computation: Toward a new philosophy of
machine intelligence. IEEE Press, 1995.
[4] D. B. Fogel, L. J. Fogel, and J. W. Atmar. Meta-evolutionary pro-
gramming. In Proceedings 25th Asilomar Conf. Signals, Systems, and
Computers, pages 540–545, 1991.[5] L. J. Fogel. Autonomous automata. Industrial Research, 4:14–19, 1962.
[6] L. J. Fogel. On the Organization of Intellect. PhD thesis, UCLA, 1964.
3.6. Evolutionary Programming 125
[7] L. J. Fogel. The future of evolutionary programming. In Proceedings
of the Conference on Signals, Systems and Computers, 1990.
[8] L. J. Fogel. Computational Intelligence: Imitating Life, chapter Evo-
lutionary Programming in Perspective: the Top-down View, pages
135–146. IEEE Press, 1994.
[9] L. J. Fogel, A. J. Owens, and M. J. Walsh. Artificial Intelligence
Through Simulated Evolution. Wiley, 1966.
[10] V. W. Porto. Evolutionary Computation 1: Basic Algorithms and
Operations, chapter 10: Evolutionary Programming, pages 89–102. IoP
Press, 2000.
[11] A. V. Sebald and D. B. Fogel. Design of SLAYR neural networks
using evolutionary programming. In Proceedings of the 24th Asilomar
Conference on Signals, Systems and Computers, pages 1020–1024, 1990.
[12] X. Yao, Y. Liu, and G. Lin. Evolutionary programming made faster.
IEEE Transactions on Evolutionary Computation, 3(2):82–102, 1999.
126 Chapter 3. Evolutionary Algorithms
3.7 Grammatical Evolution
Grammatical Evolution, GE.
3.7.1 Taxonomy
Grammatical Evolution is a Global Optimization technique and an instance
of an Evolutionary Algorithm from the field of Evolutionary Computation.
It may also be considered an algorithm for Automatic Programming. Gram-
matical Evolution is related to other Evolutionary Algorithms for evolving
programs such as Genetic Programming (Section 3.3) and Gene Expression
Programming (Section 3.8), as well as the classical Genetic Algorithm that
uses binary strings (Section 3.2).
3.7.2 Inspiration
The Grammatical Evolution algorithm is inspired by the biological process
used for generating a protein from genetic material as well as the broader
genetic evolutionary process. The genome is comprised of DNA as a string
of building blocks that are transcribed to RNA. RNA codons are in turn
translated into sequences of amino acids and used in the protein. The
resulting protein in its environment is the phenotype.
3.7.3 Metaphor
The phenotype is a computer program that is created from a binary string-
based genome. The genome is decoded into a sequence of integers that
are in turn mapped onto pre-defined rules that makeup the program. The
mapping from genotype to the phenotype is a one-to-many process that
uses a wrapping feature. This is like the biological process observed in many
bacteria, viruses, and mitochondria, where the same genetic material is used
in the expression of different genes. The mapping adds robustness to the
process both in the ability to adopt structure-agnostic genetic operators used
during the evolutionary process on the sub-symbolic representation and the
transcription of well-formed executable programs from the representation.
3.7.4 Strategy
The objective of Grammatical Evolution is to adapt an executable program
to a problem specific objective function. This is achieved through an iterative
process with surrogates of evolutionary mechanisms such as descent with
variation, genetic mutation and recombination, and genetic transcription
and gene expression. A population of programs are evolved in a sub-
symbolic form as variable length binary strings and mapped to a symbolic
and well-structured form as a context free grammar for execution.
3.7. Grammatical Evolution 127
3.7.5 Procedure
A grammar is defined in Backus Normal Form (BNF), which is a context free
grammar expressed as a series of production rules comprised of terminals
and non-terminals. A variable-length binary string representation is used
for the optimization process. Bits are read from the a candidate solutions
genome in blocks of 8 called a codon, and decoded to an integer (in the
range between 0 and 28 − 1). If the end of the binary string is reached
when reading integers, the reading process loops back to the start of the
string, effectively creating a circular genome. The integers are mapped to
expressions from the BNF until a complete syntactically correct expression
is formed. This may not use a solutions entire genome, or use the decoded
genome more than once given it’s circular nature. Algorithm 3.7.1 provides
a pseudocode listing of the Grammatical Evolution algorithm for minimizing
a cost function.
3.7.6 Heuristics
� Grammatical Evolution was designed to optimize programs (such as
mathematical equations) to specific cost functions.
� Classical genetic operators used by the Genetic Algorithm may be
used in the Grammatical Evolution algorithm, such as point mutations
and one-point crossover.
� Codons (groups of bits mapped to an integer) are commonly fixed at
8 bits, proving a range of integers ∈ [0, 28 − 1] that is scaled to the
range of rules using a modulo function.
� Additional genetic operators may be used with variable-length rep-
resentations such as codon segments, duplication (add to the end),
number of codons selected at random, and deletion.
3.7.7 Code Listing
Listing 3.6 provides an example of the Grammatical Evolution algorithm
implemented in the Ruby Programming Language based on the version
described by O’Neill and Ryan [5]. The demonstration problem is an
instance of symbolic regression f(x) = x4 + x3 + x2 + x, where x ∈ [1, 10].
The grammar used in this problem is:
� Non-terminals: N = {expr, op, pre op}
� Terminals: T = {+,−,÷,×, x, 1.0}
� Expression (program): S =<expr>
The production rules for the grammar in BNF are:
128 Chapter 3. Evolutionary Algorithms
Algorithm 3.7.1: Pseudocode for Grammatical Evolution.
Input: Grammar, Codonnumbits, Populationsize, Pcrossover,
Pmutation, Pdelete, Pduplicate
Output: Sbest
Population ← InitializePopulation(Populationsize,1
Codonnumbits);
foreach Si ∈ Population do2
Siintegers ← Decode(Sibitstring, Codonnumbits);3
Siprogram ← Map(Siintegers, Grammar);4
Sicost ← Execute(Siprogram);5
end6
Sbest ← GetBestSolution(Population);7
while ¬StopCondition() do8
Parents ← SelectParents(Population, Populationsize);9
Children ← ∅;10
foreach Parenti, Parentj ∈ Parents do11
Si ← Crossover(Parenti, Parentj, Pcrossover);12
Sibitstring ← CodonDeletion(Sibitstring, Pdelete);13
Sibitstring ← CodonDuplication(Sibitstring, Pduplicate);14
Sibitstring ← Mutate(Sibitstring, Pmutation);15
Children ← Si;16
end17
foreach Si ∈ Children do18
Siintegers ← Decode(Sibitstring, Codonnumbits);19
Siprogram ← Map(Siintegers, Grammar);20
Sicost ← Execute(Siprogram);21
end22
Sbest ← GetBestSolution(Children);23
Population ← Replace(Population, Children);24
end25
return Sbest;26
� <expr> ::= <expr><op><expr> , (<expr><op><expr>), <pre op>(<expr>),
<var>
� <op> ::= +,−,÷,×
� <var> ::= x, 1.0
The algorithm uses point mutation and a codon-respecting one-point
crossover operator. Binary tournament selection is used to determine
the parent population’s contribution to the subsequent generation. Binary
strings are decoded to integers using an unsigned binary. Candidate solutions
are then mapped directly into executable Ruby code and executed. A given
3.7. Grammatical Evolution 129
candidate solution is evaluated by comparing its output against the target
function and taking the sum of the absolute errors over a number of trials.
The probabilities of point mutation, codon deletion, and codon duplication
are hard coded as relative probabilities to each solution, although should
be parameters of the algorithm. In this case they are heuristically defined
as 1.0
L
, 0.5
NC
and 1.0
NC
respectively, where L is the total number of bits, and
NC is the number of codons in a given candidate solution.
Solutions are evaluated by generating a number of random samples from
the domain and calculating the mean error of the program to the expected
outcome. Programs that contain a single term or those that return an
invalid (NaN) or infinite result are penalized with an enormous error value.
The implementation uses a maximum depth in the expression tree, whereas
traditionally such deep expressiontrees are marked as invalid. Programs
that resolve to a single expression that returns the output are penalized.
1 def binary_tournament(pop)
2 i, j = rand(pop.size), rand(pop.size)
3 j = rand(pop.size) while j==i
4 return (pop[i][:fitness] < pop[j][:fitness]) ? pop[i] : pop[j]
5 end
6
7 def point_mutation(bitstring, rate=1.0/bitstring.size.to_f)
8 child = ""
9 bitstring.size.times do |i|
10 bit = bitstring[i].chr
11 child << ((rand()<rate) ? ((bit=='1') ? "0" : "1") : bit)
12 end
13 return child
14 end
15
16 def one_point_crossover(parent1, parent2, codon_bits, p_cross=0.30)
17 return ""+parent1[:bitstring] if rand()>=p_cross
18 cut = rand([parent1.size, parent2.size].min/codon_bits)
19 cut *= codon_bits
20 p2size = parent2[:bitstring].size
21 return parent1[:bitstring][0...cut]+parent2[:bitstring][cut...p2size]
22 end
23
24 def codon_duplication(bitstring, codon_bits, rate=1.0/codon_bits.to_f)
25 return bitstring if rand() >= rate
26 codons = bitstring.size/codon_bits
27 return bitstring + bitstring[rand(codons)*codon_bits, codon_bits]
28 end
29
30 def codon_deletion(bitstring, codon_bits, rate=0.5/codon_bits.to_f)
31 return bitstring if rand() >= rate
32 codons = bitstring.size/codon_bits
33 off = rand(codons)*codon_bits
34 return bitstring[0...off] + bitstring[off+codon_bits...bitstring.size]
35 end
36
37 def reproduce(selected, pop_size, p_cross, codon_bits)
38 children = []
130 Chapter 3. Evolutionary Algorithms
39 selected.each_with_index do |p1, i|
40 p2 = (i.modulo(2)==0) ? selected[i+1] : selected[i-1]
41 p2 = selected[0] if i == selected.size-1
42 child = {}
43 child[:bitstring] = one_point_crossover(p1, p2, codon_bits, p_cross)
44 child[:bitstring] = codon_deletion(child[:bitstring], codon_bits)
45 child[:bitstring] = codon_duplication(child[:bitstring], codon_bits)
46 child[:bitstring] = point_mutation(child[:bitstring])
47 children << child
48 break if children.size == pop_size
49 end
50 return children
51 end
52
53 def random_bitstring(num_bits)
54 return (0...num_bits).inject(""){|s,i| s<<((rand<0.5) ? "1" : "0")}
55 end
56
57 def decode_integers(bitstring, codon_bits)
58 ints = []
59 (bitstring.size/codon_bits).times do |off|
60 codon = bitstring[off*codon_bits, codon_bits]
61 sum = 0
62 codon.size.times do |i|
63 sum += ((codon[i].chr=='1') ? 1 : 0) * (2 ** i);
64 end
65 ints << sum
66 end
67 return ints
68 end
69
70 def map(grammar, integers, max_depth)
71 done, offset, depth = false, 0, 0
72 symbolic_string = grammar["S"]
73 begin
74 done = true
75 grammar.keys.each do |key|
76 symbolic_string = symbolic_string.gsub(key) do |k|
77 done = false
78 set = (k=="EXP" && depth>=max_depth-1) ? grammar["VAR"] : grammar[k]
79 integer = integers[offset].modulo(set.size)
80 offset = (offset==integers.size-1) ? 0 : offset+1
81 set[integer]
82 end
83 end
84 depth += 1
85 end until done
86 return symbolic_string
87 end
88
89 def target_function(x)
90 return x**4.0 + x**3.0 + x**2.0 + x
91 end
92
93 def sample_from_bounds(bounds)
94 return bounds[0] + ((bounds[1] - bounds[0]) * rand())
3.7. Grammatical Evolution 131
95 end
96
97 def cost(program, bounds, num_trials=30)
98 return 9999999 if program.strip == "INPUT"
99 sum_error = 0.0
100 num_trials.times do
101 x = sample_from_bounds(bounds)
102 expression = program.gsub("INPUT", x.to_s)
103 begin score = eval(expression) rescue score = 0.0/0.0 end
104 return 9999999 if score.nan? or score.infinite?
105 sum_error += (score - target_function(x)).abs
106 end
107 return sum_error / num_trials.to_f
108 end
109
110 def evaluate(candidate, codon_bits, grammar, max_depth, bounds)
111 candidate[:integers] = decode_integers(candidate[:bitstring], codon_bits)
112 candidate[:program] = map(grammar, candidate[:integers], max_depth)
113 candidate[:fitness] = cost(candidate[:program], bounds)
114 end
115
116 def search(max_gens, pop_size, codon_bits, num_bits, p_cross, grammar,
max_depth, bounds)
117 pop = Array.new(pop_size) {|i| {:bitstring=>random_bitstring(num_bits)}}
118 pop.each{|c| evaluate(c,codon_bits, grammar, max_depth, bounds)}
119 best = pop.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
120 max_gens.times do |gen|
121 selected = Array.new(pop_size){|i| binary_tournament(pop)}
122 children = reproduce(selected, pop_size, p_cross,codon_bits)
123 children.each{|c| evaluate(c, codon_bits, grammar, max_depth, bounds)}
124 children.sort!{|x,y| x[:fitness] <=> y[:fitness]}
125 best = children.first if children.first[:fitness] <= best[:fitness]
126 pop=(children+pop).sort{|x,y| x[:fitness]<=>y[:fitness]}.first(pop_size)
127 puts " > gen=#{gen}, f=#{best[:fitness]}, s=#{best[:bitstring]}"
128 break if best[:fitness] == 0.0
129 end
130 return best
131 end
132
133 if __FILE__ == $0
134 # problem configuration
135 grammar = {"S"=>"EXP",
136 "EXP"=>[" EXP BINARY EXP ", " (EXP BINARY EXP) ", " VAR "],
137 "BINARY"=>["+", "-", "/", "*" ],
138 "VAR"=>["INPUT", "1.0"]}
139 bounds = [1, 10]
140 # algorithm configuration
141 max_depth = 7
142 max_gens = 50
143 pop_size = 100
144 codon_bits = 4
145 num_bits = 10*codon_bits
146 p_cross = 0.30
147 # execute the algorithm
148 best = search(max_gens, pop_size, codon_bits, num_bits, p_cross, grammar,
max_depth, bounds)
132 Chapter 3. Evolutionary Algorithms
149 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:program]}"
150 end
Listing 3.6: Grammatical Evolution in Ruby
3.7.8 References
Primary Sources
Grammatical Evolution was proposed by Ryan, Collins and O’Neill in a
seminal conference paper that applied the approach to a symbolic regression
problem [7]. The approach was born out of the desire for syntax preservation
while evolving programs using the Genetic Programming algorithm. This
seminal work was followed by application papers for a symbolic integration
problem [2, 3] and solving trigonometric identities [8].
Learn More
O’Neill and Ryan provide a high-level introduction to Grammatical Evolu-
tion and early demonstration applications [4]. The same authors provide
a thorough introduction to the technique and overview of the state of the
field [5]. O’Neill and Ryan present a seminal reference for Grammatical
Evolution in their book [6]. A second more recent book considers extensions
to the approach improving its capability on dynamic problems [1].
3.7.9 Bibliography
[1] I. Dempsey, M. O’Neill, and A. Brabazon. Foundations in Grammatical
Evolution for Dynamic Environments. Springer, 2009.
[2] M. O’Neill and C. Ryan. Grammatical evolution: A steady state ap-
proach. In Proceedings of the Second International Workshop on Fron-
tiers in Evolutionary Algorithms, pages 419–423, 1998.
[3] M. O’Neill and C. Ryan. Grammatical evolution: A steady state ap-
proach. In Late Breaking Papers at the Genetic Programming 1998
Conference, 1998.
[4] M. O’Neill and C. Ryan. Under the hood of grammatical evolution. In
Proceedings of the Genetic and Evolutionary Computation Conference,
1999.
[5] M. O’Neill and C. Ryan. Grammatical evolution. IEEE Transactions
on Evolutionary Computation, 5(4):349–358, 2001.
[6] M. O’Neill and C. Ryan. Grammatical Evolution: Evolutionary Auto-
matic Programming in an Arbitrary Language. Springer, 2003.
3.7. Grammatical Evolution 133
[7] C. Ryan, J. J. Collins, and M. O’Neill. Grammatical evolution: Evolving
programs for an arbitrary language. In Lecture Notes in Computer
Science 1391. First European Workshop on Genetic Programming, 1998.
[8] C. Ryan, J. J. Collins, and M. O’Neill. Grammatical evolution: Solving
trigonometric identities. In Proceedings of Mendel 1998: 4th Interna-
tional Mendel Conference on Genetic Algorithms, Optimisation Problems,
Fuzzy Logic, Neural Networks, Rough Sets., pages 111–119, 1998.
134 Chapter 3. Evolutionary Algorithms
3.8 Gene Expression Programming
Gene Expression Programming, GEP.
3.8.1 Taxonomy
Gene Expression Programming is a Global Optimization algorithm and an
Automatic Programmingtechnique, and it is an instance of an Evolution-
ary Algorithm from the field of Evolutionary Computation. It is a sibling
of other Evolutionary Algorithms such as a the Genetic Algorithm (Sec-
tion 3.2) as well as other Evolutionary Automatic Programming techniques
such as Genetic Programming (Section 3.3) and Grammatical Evolution
(Section 3.7).
3.8.2 Inspiration
Gene Expression Programming is inspired by the replication and expression
of the DNA molecule, specifically at the gene level. The expression of a
gene involves the transcription of its DNA to RNA which in turn forms
amino acids that make up proteins in the phenotype of an organism. The
DNA building blocks are subjected to mechanisms of variation (mutations
such as coping errors) as well as recombination during sexual reproduction.
3.8.3 Metaphor
Gene Expression Programming uses a linear genome as the basis for genetic
operators such as mutation, recombination, inversion, and transposition.
The genome is comprised of chromosomes and each chromosome is comprised
of genes that are translated into an expression tree to solve a given problem.
The robust gene definition means that genetic operators can be applied to
the sub-symbolic representation without concern for the structure of the
resultant gene expression, providing separation of genotype and phenotype.
3.8.4 Strategy
The objective of the Gene Expression Programming algorithm is to im-
prove the adaptive fit of an expressed program in the context of a problem
specific cost function. This is achieved through the use of an evolutionary
process that operates on a sub-symbolic representation of candidate solu-
tions using surrogates for the processes (descent with modification) and
mechanisms (genetic recombination, mutation, inversion, transposition, and
gene expression) of evolution.
3.8. Gene Expression Programming 135
3.8.5 Procedure
A candidate solution is represented as a linear string of symbols called
Karva notation or a K-expression, where each symbol maps to a function or
terminal node. The linear representation is mapped to an expression tree in
a breadth-first manner. A K-expression has fixed length and is comprised
of one or more sub-expressions (genes), which are also defined with a fixed
length. A gene is comprised of two sections, a head which may contain
any function or terminal symbols, and a tail section that may only contain
terminal symbols. Each gene will always translate to a syntactically correct
expression tree, where the tail portion of the gene provides a genetic buffer
which ensures closure of the expression.
Algorithm 3.8.1 provides a pseudocode listing of the Gene Expression
Programming algorithm for minimizing a cost function.
Algorithm 3.8.1: Pseudocode for GEP.
Input: Grammar, Populationsize, Headlength, Taillength, Pcrossover,
Pmutation
Output: Sbest
Population ← InitializePopulation(Populationsize, Grammar,1
Headlength, Taillength);
foreach Si ∈ Population do2
Siprogram ← DecodeBreadthFirst(Sigenome, Grammar);3
Sicost ← Execute(Siprogram);4
end5
Sbest ← GetBestSolution(Population);6
while ¬StopCondition() do7
Parents ← SelectParents(Population, Populationsize);8
Children ← ∅;9
foreach Parent1, Parent2 ∈ Parents do10
Sigenome ← Crossover(Parent1, Parent2, Pcrossover);11
Sigenome ← Mutate(Sigenome, Pmutation);12
Children ← Si;13
end14
foreach Si ∈ Children do15
Siprogram ← DecodeBreadthFirst(Sigenome, Grammar);16
Sicost ← Execute(Siprogram);17
end18
Population ← Replace(Population, Children);19
Sbest ← GetBestSolution(Children);20
end21
return Sbest;22
136 Chapter 3. Evolutionary Algorithms
3.8.6 Heuristics
� The length of a chromosome is defined by the number of genes, where
a gene length is defined by h+ t. The h is a user defined parameter
(such as 10), and t is defined as t = h(n−1)+1, where the n represents
the maximum arity of functional nodes in the expression (such as 2 if
the arithmetic functions ×,÷,−,+ are used).
� The mutation operator substitutes expressions along the genome,
although must respect the gene rules such that function and terminal
nodes are mutated in the head of genes, whereas only terminal nodes
are substituted in the tail of genes.
� Crossover occurs between two selected parents from the population
and can occur based on a one-point cross, two point cross, or a gene-
based approach where genes are selected from the parents with uniform
probability.
� An inversion operator may be used with a low probability that reverses
a small sequence of symbols (1-3) within a section of a gene (tail or
head).
� A transposition operator may be used that has a number of different
modes, including: duplicate a small sequences (1-3) from somewhere
on a gene to the head, small sequences on a gene to the root of the
gene, and moving of entire genes in the chromosome. In the case
of intra-gene transpositions, the sequence in the head of the gene is
moved down to accommodate the copied sequence and the length of
the head is truncated to maintain consistent gene sizes.
� A ‘?’ may be included in the terminal set that represents a numeric
constant from an array that is evolved on the end of the genome. The
constants are read from the end of the genome and are substituted for
‘?’ as the expression tree is created (in breadth first order). Finally the
numeric constants are used as array indices in yet another chromosome
of numerical values which are substituted into the expression tree.
� Mutation is low (such as 1
L
), selection can be any of the classical
approaches (such as roulette wheel or tournament), and crossover
rates are typically high (0.7 of offspring)
� Use multiple sub-expressions linked together on hard problems when
one gene is not sufficient to address the problem. The sub-expressions
are linked using link expressions which are function nodes that are
either statically defined (such as a conjunction) or evolved on the
genome with the genes.
3.8. Gene Expression Programming 137
3.8.7 Code Listing
Listing 3.7 provides an example of the Gene Expression Programming
algorithm implemented in the Ruby Programming Language based on the
seminal version proposed by Ferreira [1]. The demonstration problem is an
instance of symbolic regression f(x) = x4 + x3 + x2 + x, where x ∈ [1, 10].
The grammar used in this problem is: Functions: F = {+,−,÷,×, } and
Terminals: T = {x}.
The algorithm uses binary tournament selection, uniform crossover
and point mutations. The K-expression is decoded to an expression tree
in a breadth-first manner, which is then parsed depth first as a Ruby
expression string for display and direct evaluation. Solutions are evaluated
by generating a number of random samples from the domain and calculating
the mean error of the program to the expected outcome. Programs that
contain a single term or those that return an invalid (NaN) or infinite result
are penalized with an enormous error value.
1 def binary_tournament(pop)
2 i, j = rand(pop.size), rand(pop.size)
3 return (pop[i][:fitness] < pop[j][:fitness]) ? pop[i] : pop[j]
4 end
5
6 def point_mutation(grammar, genome, head_length, rate=1.0/genome.size.to_f)
7 child =""
8 genome.size.times do |i|
9 bit = genome[i].chr
10 if rand() < rate
11 if i < head_length
12 selection = (rand() < 0.5) ? grammar["FUNC"]: grammar["TERM"]
13 bit = selection[rand(selection.size)]
14 else
15 bit = grammar["TERM"][rand(grammar["TERM"].size)]
16 end
17 end
18 child << bit
19 end
20 return child
21 end
22
23 def crossover(parent1, parent2, rate)
24 return ""+parent1 if rand()>=rate
25 child = ""
26 parent1.size.times do |i|
27 child << ((rand()<0.5) ? parent1[i] : parent2[i])
28 end
29 return child
30 end
31
32 def reproduce(grammar, selected, pop_size, p_crossover, head_length)
33 children = []
34 selected.each_with_index do |p1, i|
35 p2 = (i.modulo(2)==0) ? selected[i+1] : selected[i-1]
36 p2 = selected[0] if i == selected.size-1
138 Chapter 3. Evolutionary Algorithms
37 child = {}
38 child[:genome] = crossover(p1[:genome],p2[:genome], p_crossover)
39 child[:genome] = point_mutation(grammar, child[:genome], head_length)
40 children << child
41 end
42 return children
43 end
44
45 def random_genome(grammar, head_length, tail_length)
46 s = ""
47 head_length.times do
48 selection = (rand() < 0.5) ? grammar["FUNC"]: grammar["TERM"]
49 s << selection[rand(selection.size)]
50 end
51 tail_length.times { s << grammar["TERM"][rand(grammar["TERM"].size)]}
52 return s
53 end
54
55 def target_function(x)
56 return x**4.0 + x**3.0 + x**2.0 + x
57 end
58
59 def sample_from_bounds(bounds)
60 return bounds[0] + ((bounds[1] - bounds[0]) * rand())
61 end
62
63 def cost(program, bounds, num_trials=30)
64 errors = 0.0
65 num_trials.times do
66 x = sample_from_bounds(bounds)
67 expression, score = program.gsub("x", x.to_s), 0.0
68 begin score = eval(expression) rescue score = 0.0/0.0 end
69 return 9999999 if score.nan? or score.infinite?
70 errors += (score - target_function(x)).abs
71 end
72 return errors / num_trials.to_f
73 end
74
75 def mapping(genome, grammar)
76 off, queue = 0, []
77 root = {}
78 root[:node] = genome[off].chr; off+=1
79 queue.push(root)
80 while !queue.empty? do
81 current = queue.shift
82 if grammar["FUNC"].include?(current[:node])
83 current[:left] = {}
84 current[:left][:node] = genome[off].chr; off+=1
85 queue.push(current[:left])
86 current[:right] = {}
87 current[:right][:node] = genome[off].chr; off+=1
88 queue.push(current[:right])
89 end
90 end
91 return root
92 end
3.8. Gene Expression Programming 139
93
94 def tree_to_string(exp)
95 return exp[:node] if (exp[:left].nil? or exp[:right].nil?)
96 left = tree_to_string(exp[:left])
97 right = tree_to_string(exp[:right])
98 return "(#{left} #{exp[:node]} #{right})"
99 end
100
101 def evaluate(candidate, grammar, bounds)
102 candidate[:expression] = mapping(candidate[:genome], grammar)
103 candidate[:program] = tree_to_string(candidate[:expression])
104 candidate[:fitness] = cost(candidate[:program], bounds)
105 end
106
107 def search(grammar, bounds, h_length, t_length, max_gens, pop_size, p_cross)
108 pop = Array.new(pop_size) do
109 {:genome=>random_genome(grammar, h_length, t_length)}
110 end
111 pop.each{|c| evaluate(c, grammar, bounds)}
112 best = pop.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
113 max_gens.times do |gen|
114 selected = Array.new(pop){|i| binary_tournament(pop)}
115 children = reproduce(grammar, selected, pop_size, p_cross, h_length)
116 children.each{|c| evaluate(c, grammar, bounds)}
117 children.sort!{|x,y| x[:fitness] <=> y[:fitness]}
118 best = children.first if children.first[:fitness] <= best[:fitness]
119 pop = (children+pop).first(pop_size)
120 puts " > gen=#{gen}, f=#{best[:fitness]}, g=#{best[:genome]}"
121 end
122 return best
123 end
124
125 if __FILE__ == $0
126 # problem configuration
127 grammar = {"FUNC"=>["+","-","*","/"], "TERM"=>["x"]}
128 bounds = [1.0, 10.0]
129 # algorithm configuration
130 h_length = 20
131 t_length = h_length * (2-1) + 1
132 max_gens = 150
133 pop_size = 80
134 p_cross = 0.85
135 # execute the algorithm
136 best = search(grammar, bounds, h_length, t_length, max_gens, pop_size,
p_cross)
137 puts "done! Solution: f=#{best[:fitness]}, program=#{best[:program]}"
138 end
Listing 3.7: Gene Expression Programming in Ruby
140 Chapter 3. Evolutionary Algorithms
3.8.8 References
Primary Sources
The Gene Expression Programming algorithm was proposed by Ferreira in
a paper that detailed the approach, provided a careful walkthrough of the
process and operators, and demonstrated the the algorithm on a number of
benchmark problem instances including symbolic regression [1].
Learn More
Ferreira provided an early and detailed introduction and overview of the
approach as book chapter, providing a step-by-step walkthrough of the
procedure and sample applications [2]. A more contemporary and detailed
introduction is provided in a later book chapter [3]. Ferreira published a
book on the approach in 2002 covering background, the algorithm, and
demonstration applications which is now in its second edition [4].
3.8.9 Bibliography
[1] C. Ferreira. Gene expression programming: A new adaptive algorithm
for solving problems. Complex Systems, 13(2):87–129, 2001.
[2] C. Ferreira. Soft Computing and Industry: Recent Applications, chapter
Gene Expression Programming in Problem Solving, pages 635–654.
Springer-Verlag, 2002.
[3] C. Ferreira. Recent Developments in Biologically Inspired Computing,
chapter Gene Expression Programming and the Evolution of computer
programs, pages 82–103. Idea Group Publishing, 2005.
[4] C. Ferreira. Gene expression programming: Mathematical modeling by
an artificial intelligence. Springer-Verlag, second edition, 2006.
3.9. Learning Classifier System 141
3.9 Learning Classifier System
Learning Classifier System, LCS.
3.9.1 Taxonomy
The Learning Classifier System algorithm is both an instance of an Evo-
lutionary Algorithm from the field of Evolutionary Computation and an
instance of a Reinforcement Learning algorithm from Machine Learning.
Internally, Learning Classifier Systems make use of a Genetic Algorithm
(Section 3.2). The Learning Classifier System is a theoretical system with a
number of implementations. The two main approaches to implementing and
investigating the system empirically are the Pittsburgh-style that seeks to
optimize the whole classifier, and the Michigan-style that optimize respon-
sive rulesets. The Michigan-style Learning Classifier is the most common
and is comprised of two versions: the ZCS (zeroth-level classifier system)
and the XCS (accuracy-based classifier system).
3.9.2 Strategy
The objective of the Learning Classifier System algorithm is to optimize
payoff based on exposure to stimuli from a problem-specific environment.
This is achieved by managing credit assignment for those rules that prove
useful and searching for new rules and new variations on existing rules using
an evolutionary process.
3.9.3 Procedure
The actors of the system include detectors, messages, effectors, feedback,
and classifiers. Detectors are used by the system to perceive the state of the
environment. Messages are the discrete information packets passed from the
detectors into the system. The system performs information processing on
messages, and messages may directly result in actions in the environment.
Effectors control the actions of the system on and within the environment.
In addition to the system actively perceiving via its detections, it may
also receive directed feedback from the environment (payoff). Classifiers
are condition-action rules that provide a filter for messages. If a message
satisfies the conditional part of the classifier, the action of the classifier
triggers. Rules act as message processors. Message a fixed length bitstring.
A classifier is defined as a ternary string with an alphabet ∈ {1, 0,#}, where
the # represents do not care (matching either 1 or 0).
The processing loop for the Learning Classifier system is as follows:
1. Messages from the environment are placed on the message list.
142 Chapter 3. Evolutionary Algorithms
2. The conditions of each classifier are checked to see if they are satisfied
by at least one message in the message list.
3. All classifiers that are satisfied participate in a competition, those
that win post their action to the message list.
4. All messages directed to the effectors are executed (causing actions in
the environment).
5. All messages on the message list from the previous cycle are deleted
(messages persist for a single cycle).
The algorithm may be described in terms of the main processing loop and
two sub-algorithms: a reinforcement learning algorithm such as the bucket
brigade algorithm or Q-learning, and a genetic algorithm for optimization of
the system. Algorithm 3.9.1 provides a pseudocode listing of the high-level
processing loop of the Learning Classifier System, specifically the XCSas
described by Butz and Wilson [3].
3.9.4 Heuristics
The majority of the heuristics in this section are specific to the XCS Learning
Classifier System as described by Butz and Wilson [3].
� Learning Classifier Systems are suited for problems with the following
characteristics: perpetually novel events with significant noise, contin-
ual real-time requirements for action, implicitly or inexactly defined
goals, and sparse payoff or reinforcement obtainable only through long
sequences of tasks.
� The learning rate β for a classifier’s expected payoff, error, and fitness
are typically in the range [0.1, 0.2].
� The frequency of running the genetic algorithm θGA should be in the
range [25, 50].
� The discount factor used in multi-step programs γ are typically in the
around 0.71.
� The minimum error whereby classifiers are considered to have equal
accuracy ǫ0 is typically 10% of the maximum reward.
� The probability of crossover in the genetic algorithm χ is typically in
the range [0.5, 1.0].
� The probability of mutating a single position in a classifier in the
genetic algorithm µ is typically in the range [0.01, 0.05].
3.9. Learning Classifier System 143
Algorithm 3.9.1: Pseudocode for the LCS.
Input: EnvironmentDetails
Output: Population
env ← InitializeEnvironment(EnvironmentDetails);1
Population ← InitializePopulation();2
ActionSett−1 ← ∅;3
Inputt−1 ← ∅;4
Rewardt−1 ← ∅;5
while ¬StopCondition() do6
Inputt ← env;7
Matchset ← GenerateMatchSet(Population, Inputt);8
Prediction ← GeneratePrediction(Matchset);9
Action ← SelectionAction(Prediction);10
ActionSett ← GenerateActionSet(Action, Matchset);11
Rewardt ← ExecuteAction(Action, env);12
if ActionSett−1 6= ∅ then13
Payofft ← CalculatePayoff(Rewardt−1, Prediction);14
PerformLearning(ActionSett−1, Payofft, Population);15
RunGeneticAlgorithm(ActionSett−1, Inputt−1, Population);16
end17
if LastStepOfTask(env, Action) then18
Payofft ← Rewardt;19
PerformLearning(ActionSett, Payofft, Population);20
RunGeneticAlgorithm(ActionSett, Inputt, Population);21
ActionSett−1 ← ∅;22
else23
ActionSett−1 ← ActionSett;24
Inputt−1 ← Inputt;25
Rewardt−1 ← Rewardt;26
end27
end28
� The experience threshold during classifier deletion θdel is typically
about 20.
� The experience threshold for a classifier during subsumption θsub is
typically around 20.
� The initial values for a classifier’s expected payoff p1, error ǫ1, and
fitness f1 are typically small and close to zero.
� The probability of selecting a random action for the purposes of
exploration pexp is typically close to 0.5.
144 Chapter 3. Evolutionary Algorithms
� The minimum number of different actions that must be specified in a
match set θmna is usually the total number of possible actions in the
environment for the input.
� Subsumption should be used on problem domains that are known
contain well defined rules for mapping inputs to outputs.
3.9.5 Code Listing
Listing 3.8 provides an example of the Learning Classifier System algorithm
implemented in the Ruby Programming Language. The problem is an
instance of a Boolean multiplexer called the 6-multiplexer. It can be
described as a classification problem, where each of the 26 patterns of bits
is associated with a boolean class ∈ {1, 0}. For this problem instance, the
first two bits may be decoded as an address into the remaining four bits
that specify the class (for example in 100011, ‘10’ decode to the index of
‘2’ in the remaining 4 bits making the class ‘1’). In propositional logic this
problem instance may be described as F = (¬x0)(¬x1)x2 + (¬x0)x1x3 +
x0(¬x1)x4 + x0x1x5. The algorithm is an instance of XCS based on the
description provided by Butz and Wilson [3] with the parameters based
on the application of XCS to Boolean multiplexer problems by Wilson
[14, 15]. The population is grown as needed, and subsumption which would
be appropriate for the Boolean multiplexer problem was not used for brevity.
The multiplexer problem is a single step problem, so the complexities of
delayed payoff are not required. A number of parameters were hard coded to
recommended values, specifically: α = 0.1, v = −0.5, δ = 0.1 and P# =
1
3 .
1 def neg(bit)
2 return (bit==1) ? 0 : 1
3 end
4
5 def target_function(s)
6 ints = Array.new(6){|i| s[i].chr.to_i}
7 x0,x1,x2,x3,x4,x5 = ints
8 return neg(x0)*neg(x1)*x2 + neg(x0)*x1*x3 + x0*neg(x1)*x4 + x0*x1*x5
9 end
10
11 def new_classifier(condition, action, gen, p1=10.0, e1=0.0, f1=10.0)
12 other = {}
13 other[:condition],other[:action],other[:lasttime] = condition, action, gen
14 other[:pred], other[:error], other[:fitness] = p1, e1, f1
15 other[:exp], other[:setsize], other[:num] = 0.0, 1.0, 1.0
16 return other
17 end
18
19 def copy_classifier(parent)
20 copy = {}
21 parent.keys.each do |k|
22 copy[k] = (parent[k].kind_of? String) ? ""+parent[k] : parent[k]
23 end
3.9. Learning Classifier System 145
24 copy[:num],copy[:exp] = 1.0, 0.0
25 return copy
26 end
27
28 def random_bitstring(size=6)
29 return (0...size).inject(""){|s,i| s+((rand<0.5) ? "1" : "0")}
30 end
31
32 def calculate_deletion_vote(classifier, pop, del_thresh, f_thresh=0.1)
33 vote = classifier[:setsize] * classifier[:num]
34 total = pop.inject(0.0){|s,c| s+c[:num]}
35 avg_fitness = pop.inject(0.0){|s,c| s + (c[:fitness]/total)}
36 derated = classifier[:fitness] / classifier[:num].to_f
37 if classifier[:exp]>del_thresh and derated<(f_thresh*avg_fitness)
38 return vote * (avg_fitness / derated)
39 end
40 return vote
41 end
42
43 def delete_from_pop(pop, pop_size, del_thresh=20.0)
44 total = pop.inject(0) {|s,c| s+c[:num]}
45 return if total <= pop_size
46 pop.each {|c| c[:dvote] = calculate_deletion_vote(c, pop, del_thresh)}
47 vote_sum = pop.inject(0.0) {|s,c| s+c[:dvote]}
48 point = rand() * vote_sum
49 vote_sum, index = 0.0, 0
50 pop.each_with_index do |c,i|
51 vote_sum += c[:dvote]
52 if vote_sum >= point
53 index = i
54 break
55 end
56 end
57 if pop[index][:num] > 1
58 pop[index][:num] -= 1
59 else
60 pop.delete_at(index)
61 end
62 end
63
64 def generate_random_classifier(input, actions, gen, rate=1.0/3.0)
65 condition = ""
66 input.size.times {|i| condition << ((rand<rate) ? '#' : input[i].chr)}
67 action = actions[rand(actions.size)]
68 return new_classifier(condition, action, gen)
69 end
70
71 def does_match?(input, condition)
72 input.size.times do |i|
73 return false if condition[i].chr!='#' and input[i].chr!=condition[i].chr
74 end
75 return true
76 end
77
78 def get_actions(pop)
79 actions = []
146 Chapter 3. Evolutionary Algorithms
80 pop.each do |c|
81 actions << c[:action] if !actions.include?(c[:action])
82 end
83 return actions
84 end
85
86 def generate_match_set(input, pop, all_actions, gen, pop_size)
87 match_set = pop.select{|c| does_match?(input, c[:condition])}
88 actions = get_actions(match_set)
89 while actions.size < all_actions.size do
90 remaining = all_actions - actions
91 classifier = generate_random_classifier(input, remaining, gen)
92 pop << classifier
93 match_set << classifier
94 delete_from_pop(pop, pop_size)
95 actions << classifier[:action]
96 end
97 return match_set
98 end
99
100 def generate_prediction(match_set)
101 pred = {}
102 match_set.each do |classifier|
103 key = classifier[:action]
104 pred[key] = {:sum=>0.0,:count=>0.0,:weight=>0.0} if pred[key].nil?
105 pred[key][:sum] += classifier[:pred]*classifier[:fitness]
106 pred[key][:count] += classifier[:fitness]
107 end
108 pred.keys.each do |key|
109 pred[key][:weight] = 0.0
110 if pred[key][:count] > 0
111 pred[key][:weight] = pred[key][:sum]/pred[key][:count]
112 end
113 end
114 return pred
115 end
116
117 def select_action(predictions, p_explore=false)
118 keys = Array.new(predictions.keys)
119 return keys[rand(keys.size)] if p_explore
120 keys.sort!{|x,y| predictions[y][:weight]<=>predictions[x][:weight]}
121 return keys.first
122 end
123
124 def update_set(action_set, reward, beta=0.2)
125 sum = action_set.inject(0.0){|s,other| s+other[:num]}
126 action_set.each do |c|
127 c[:exp] += 1.0
128 if c[:exp] < 1.0/beta
129 c[:error] = (c[:error]*(c[:exp]-1.0)+(reward-c[:pred]).abs)/c[:exp]
130 c[:pred] = (c[:pred] * (c[:exp]-1.0) + reward) / c[:exp]
131 c[:setsize] = (c[:setsize]*(c[:exp]-1.0)+sum) / c[:exp]
132 else
133 c[:error] += beta * ((reward-c[:pred]).abs - c[:error])
134 c[:pred] += beta * (reward-c[:pred])
135 c[:setsize] += beta * (sum - c[:setsize])
3.9. Learning Classifier System 147
136 end
137 end
138 end
139
140 def update_fitness(action_set, min_error=10, l_rate=0.2, alpha=0.1, v=-5.0)
141 sum = 0.0
142 acc = Array.new(action_set.size)
143 action_set.each_with_index do |c,i|
144 acc[i] = (c[:error]<min_error) ? 1.0 : alpha*(c[:error]/min_error)**v
145 sum += acc[i] * c[:num].to_f
146 end
147 action_set.each_with_index do |c,i|
148 c[:fitness] += l_rate * ((acc[i] * c[:num].to_f) / sum - c[:fitness])
149 end
150 end
151
152 def can_run_genetic_algorithm(action_set, gen, ga_freq)
153 return false if action_set.size <= 2
154 total = action_set.inject(0.0) {|s,c| s+c[:lasttime]*c[:num]}
155 sum = action_set.inject(0.0) {|s,c| s+c[:num]}
156 return true if gen - (total/sum) > ga_freq
157 return false
158 end
159
160 def binary_tournament(pop)
161 i, j = rand(pop.size), rand(pop.size)
162 j = rand(pop.size) while j==i
163 return (pop[i][:fitness] > pop[j][:fitness]) ? pop[i] : pop[j]
164 end
165
166 def mutation(cl, action_set, input, rate=0.04)
167 cl[:condition].size.times do |i|
168 if rand() < rate
169 cl[:condition][i] = (cl[:condition][i].chr=='#') ? input[i] : '#'
170 end
171 end
172 if rand() < rate
173 subset = action_set - [cl[:action]]
174 cl[:action] = subset[rand(subset.size)]
175 end
176 end
177
178 def uniform_crossover(parent1, parent2)
179 child = ""
180 parent1.size.times do |i|
181 child << ((rand()<0.5) ? parent1[i].chr : parent2[i].chr)
182 end
183 return child
184 end
185
186 def insert_in_pop(cla, pop)
187 pop.each do |c|
188 if cla[:condition]==c[:condition] and cla[:action]==c[:action]
189 c[:num] += 1
190 return
191 end
148 Chapter 3. Evolutionary Algorithms
192 end
193 pop << cla
194 end
195
196 def crossover(c1, c2, p1, p2)
197 c1[:condition] = uniform_crossover(p1[:condition], p2[:condition])
198 c2[:condition] = uniform_crossover(p1[:condition], p2[:condition])
199 c2[:pred] = c1[:pred] = (p1[:pred]+p2[:pred])/2.0
200 c2[:error] = c1[:error] = 0.25*(p1[:error]+p2[:error])/2.0
201 c2[:fitness] = c1[:fitness] = 0.1*(p1[:fitness]+p2[:fitness])/2.0
202 end
203
204 def run_ga(actions, pop, action_set, input, gen, pop_size, crate=0.8)
205 p1, p2 = binary_tournament(action_set), binary_tournament(action_set)
206 c1, c2 = copy_classifier(p1), copy_classifier(p2)
207 crossover(c1, c2, p1, p2) if rand() < crate
208 [c1,c2].each do |c|
209 mutation(c, actions, input)
210 insert_in_pop(c, pop)
211 end
212 while pop.inject(0) {|s,c| s+c[:num]} > pop_size
213 delete_from_pop(pop, pop_size)
214 end
215 end
216
217 def train_model(pop_size, max_gens, actions, ga_freq)
218 pop, perf = [], []
219 max_gens.times do |gen|
220 explore = gen.modulo(2)==0
221 input = random_bitstring()
222 match_set = generate_match_set(input, pop, actions, gen, pop_size)
223 pred_array = generate_prediction(match_set)
224 action = select_action(pred_array, explore)
225 reward = (target_function(input)==action.to_i) ? 1000.0 : 0.0
226 if explore
227 action_set = match_set.select{|c| c[:action]==action}
228 update_set(action_set, reward)
229 update_fitness(action_set)
230 if can_run_genetic_algorithm(action_set, gen, ga_freq)
231 action_set.each {|c| c[:lasttime] = gen}
232 run_ga(actions, pop, action_set, input, gen, pop_size)
233 end
234 else
235 e,a = (pred_array[action][:weight]-reward).abs, ((reward==1000.0)?1:0)
236 perf << {:error=>e,:correct=>a}
237 if perf.size >= 50
238 err = (perf.inject(0){|s,x|s+x[:error]}/perf.size).round
239 acc = perf.inject(0.0){|s,x|s+x[:correct]}/perf.size
240 puts " >iter=#{gen+1} size=#{pop.size}, error=#{err}, acc=#{acc}"
241 perf = []
242 end
243 end
244 end
245 return pop
246 end
247
3.9. Learning Classifier System 149
248 def test_model(system, num_trials=50)
249 correct = 0
250 num_trials.times do
251 input = random_bitstring()
252 match_set = system.select{|c| does_match?(input, c[:condition])}
253 pred_array = generate_prediction(match_set)
254 action = select_action(pred_array, false)
255 correct += 1 if target_function(input) == action.to_i
256 end
257 puts "Done! classified correctly=#{correct}/#{num_trials}"
258 return correct
259 end
260
261 def execute(pop_size, max_gens, actions, ga_freq)
262 system = train_model(pop_size, max_gens, actions, ga_freq)
263 test_model(system)
264 return system
265 end
266
267 if __FILE__ == $0
268 # problem configuration
269 all_actions = ['0', '1']
270 # algorithm configuration
271 max_gens, pop_size = 5000, 200
272 ga_freq = 25
273 # execute the algorithm
274 execute(pop_size, max_gens, all_actions, ga_freq)
275 end
Listing 3.8: Learning Classifier System in Ruby
3.9.6 References
Primary Sources
Early ideas on the theory of Learning Classifier Systems were proposed
by Holland [4, 7], culminating in a standardized presentation a few years
later [5]. A number of implementations of the theoretical system were
investigated, although a taxonomy of the two main streams was proposed by
De Jong [9]: 1) Pittsburgh-style proposed by Smith [11, 12] and 2) Holland-
style or Michigan-style Learning classifiers that are further comprised of the
Zeroth-level classifier (ZCS) [13] and the accuracy-based classifier (XCS)
[14].
Learn More
Booker, Goldberg, and Holland provide a classical introduction to Learning
Classifier Systems including an overview of the state of the field and the
algorithm in detail [1]. Wilson and Goldberg also provide an introduction
and review of the approach, taking a more critical stance [16]. Holmes et al.
provide a contemporary review of the field focusing both on a description of
150 Chapter 3. Evolutionary Algorithms
the method and application areas to which the approach has been demon-
strated successfully [8]. Lanzi, Stolzmann, and Wilson provide a seminal
book in the field as a collection of papers covering the basics, advanced
topics, and demonstration applications; a particular highlight from this book
is the first section that provides a concise description of Learning Classifier
Systems by many leaders and major contributors to the field [6], providing
rare insight. Another paper from Lanzi and Riolo’s book provides a detailed
review of the development of the approach as it matured throughout the
1990s [10]. Bull and Kovacs provide a second book introductory book to the
field focusing on the theory of the approach and its practical application [2].
3.9.7 Bibliography
[1] L. B. Booker, D. E. Goldberg, and J. H. Holland. Classifier systems
and genetic algorithms. Artificial Intelligence, 40:235–282, 1989.
[2] L. Bull and T. Kovacs. Foundations of learning classifier systems.
Springer, 2005.
[3] M. V. Butz and S. W. Wilson. An algorithmic description of XCS.
Journal of Soft Computing, 6(3–4):144–153, 2002.
[4] J. H. Holland. Progress in Theoretical Biology IV, chapter Adaptation,
pages 263–293. Academic Press, 1976.
[5] J. H. Holland. Adaptive algorithms for discovering and using general
patterns in growing knowledge-bases. International Journal of Policy
Analysis and Information Systems, 4:217–240, 1980.
[6] J. H. Holland, L. B. Booker, M. Colombetti, M. Dorigo, D. E. Goldberg,
S. Forrest, R. L. Riolo, R. E. Smith, P. L. Lanzi, W. Stolzmann,
and S. W. Wilson. Learning classifier systems: from foundations to
applications, chapter What is a learning classifier system?, pages 3–32.
Springer, 2000.
[7] J. H. Holland and J. S. Reitman. Cognitive systems based on adaptive
algorithms. ACM SIGART Bulletin, 63:49, 1977.
[8] J. H. Holmes, P. L. Lanzi,W. Stolzmann, and S. W. Wilson. Learning
classifier systems: New models, successful applications. Information
Processing Letters, 82:23–30, 2002.
[9] K. De Jong. Learning with genetic algorithms: An overview. Machine
Learning, 3:121–138, 1988.
[10] P. L. Lanzi and R. L. Riolo. Learning classifier systems: from foun-
dations to applications, chapter A Roadmap to the Last Decade of
Learning Classifier System Research, pages 33–62. Springer, 2000.
3.9. Learning Classifier System 151
[11] S. Smith. Flexible learning of problem solving heuristics through
adaptive search. In Proceedings 8th International Joint Conference on
Artificial Intelligence, pages 422–425, 1983.
[12] S. F. Smith. A learning system based on genetic adaptive algorithms.
PhD thesis, Department of Computer Science, University of Pittsburgh,
1980.
[13] S. W. Wilson. ZCS: A zeroth level classifier systems. Evolutionary
Computation, 2:1–18, 1994.
[14] S. W. Wilson. Classifier fitness based on accuracy. Evolutionary
Computation, 3:149–175, 1995.
[15] S. W. Wilson. Generalization in the XCS classifier systems. In Genetic
Programming 1998: Proceedings of the Third Annual Conference, pages
665–674. Morgan Kaufmann, 1998.
[16] S. W. Wilson and D. E. Goldberg. A critical review of classifier
systems. In Proceedings of the third international conference on Genetic
algorithms, pages 244–255, 1989.
152 Chapter 3. Evolutionary Algorithms
3.10 Non-dominated Sorting Genetic Algorithm
Non-dominated Sorting Genetic Algorithm, Nondominated Sorting Genetic
Algorithm, Fast Elitist Non-dominated Sorting Genetic Algorithm, NSGA,
NSGA-II, NSGAII.
3.10.1 Taxonomy
The Non-dominated Sorting Genetic Algorithm is a Multiple Objective Opti-
mization (MOO) algorithm and is an instance of an Evolutionary Algorithm
from the field of Evolutionary Computation. Refer to Section 9.5.3 for more
information and references on Multiple Objective Optimization. NSGA
is an extension of the Genetic Algorithm for multiple objective function
optimization (Section 3.2). It is related to other Evolutionary Multiple
Objective Optimization Algorithms (EMOO) (or Multiple Objective Evolu-
tionary Algorithms MOEA) such as the Vector-Evaluated Genetic Algorithm
(VEGA), Strength Pareto Evolutionary Algorithm (SPEA) (Section 3.11),
and Pareto Archived Evolution Strategy (PAES). There are two versions of
the algorithm, the classical NSGA and the updated and currently canonical
form NSGA-II.
3.10.2 Strategy
The objective of the NSGA algorithm is to improve the adaptive fit of a
population of candidate solutions to a Pareto front constrained by a set
of objective functions. The algorithm uses an evolutionary process with
surrogates for evolutionary operators including selection, genetic crossover,
and genetic mutation. The population is sorted into a hierarchy of sub-
populations based on the ordering of Pareto dominance. Similarity between
members of each sub-group is evaluated on the Pareto front, and the
resulting groups and similarity measures are used to promote a diverse front
of non-dominated solutions.
3.10.3 Procedure
Algorithm 3.10.1 provides a pseudocode listing of the Non-dominated Sort-
ing Genetic Algorithm II (NSGA-II) for minimizing a cost function. The
SortByRankAndDistance function orders the population into a hierarchy
of non-dominated Pareto fronts. The CrowdingDistanceAssignment cal-
culates the average distance between members of each front on the front
itself. Refer to Deb et al. for a clear presentation of the Pseudocode
and explanation of these functions [4]. The CrossoverAndMutation func-
tion performs the classical crossover and mutation genetic operators of
the Genetic Algorithm. Both the SelectParentsByRankAndDistance and
3.10. Non-dominated Sorting Genetic Algorithm 153
SortByRankAndDistance functions discriminate members of the popula-
tion first by rank (order of dominated precedence of the front to which
the solution belongs) and then distance within the front (calculated by
CrowdingDistanceAssignment).
Algorithm 3.10.1: Pseudocode for NSGAII.
Input: Populationsize, ProblemSize, Pcrossover, Pmutation
Output: Children
Population ← InitializePopulation(Populationsize, ProblemSize);1
EvaluateAgainstObjectiveFunctions(Population);2
FastNondominatedSort(Population);3
Selected ← SelectParentsByRank(Population, Populationsize);4
Children ← CrossoverAndMutation(Selected, Pcrossover, Pmutation);5
while ¬StopCondition() do6
EvaluateAgainstObjectiveFunctions(Children);7
Union ← Merge(Population, Children);8
Fronts ← FastNondominatedSort(Union);9
Parents ← ∅;10
FrontL ← ∅;11
foreach Fronti ∈ Fronts do12
CrowdingDistanceAssignment(Fronti);13
if Size(Parents)+Size(Fronti) > Populationsize then14
FrontL ← i;15
Break();16
else17
Parents ← Merge(Parents, Fronti);18
end19
end20
if Size(Parents)<Populationsize then21
FrontL ← SortByRankAndDistance(FrontL);22
for P1 to PPopulationsize−Size(FrontL ) do23
Parents ← Pi;24
end25
end26
Selected ← SelectParentsByRankAndDistance(Parents,27
Populationsize);
Population ← Children;28
Children ← CrossoverAndMutation(Selected, Pcrossover,29
Pmutation);
end30
return Children;31
154 Chapter 3. Evolutionary Algorithms
3.10.4 Heuristics
� NSGA was designed for and is suited to continuous function multiple
objective optimization problem instances.
� A binary representation can be used in conjunction with classical
genetic operators such as one-point crossover and point mutation.
� A real-valued representation is recommended for continuous function
optimization problems, in turn requiring representation specific genetic
operators such as Simulated Binary Crossover (SBX) and polynomial
mutation [2].
3.10.5 Code Listing
Listing 3.9 provides an example of the Non-dominated Sorting Genetic
Algorithm II (NSGA-II) implemented in the Ruby Programming Language.
The demonstration problem is an instance of continuous multiple objective
function optimization called SCH (problem one in [4]). The problem seeks
the minimum of two functions: f1 =
∑n
i=1 x
2
i and f2 =
∑n
i=1(xi − 2)
2,
−10 ≤ xi ≤ 10 and n = 1. The optimal solution for this function are
x ∈ [0, 2]. The algorithm is an implementation of NSGA-II based on
the presentation by Deb et al. [4]. The algorithm uses a binary string
representation (16 bits per objective function parameter) that is decoded
and rescaled to the function domain. The implementation uses a uniform
crossover operator and point mutations with a fixed mutation rate of 1
L
,
where L is the number of bits in a solution’s binary string.
1 def objective1(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x**2.0)}
3 end
4
5 def objective2(vector)
6 return vector.inject(0.0) {|sum, x| sum + ((x-2.0)**2.0)}
7 end
8
9 def decode(bitstring, search_space, bits_per_param)
10 vector = []
11 search_space.each_with_index do |bounds, i|
12 off, sum = i*bits_per_param, 0.0
13 param = bitstring[off...(off+bits_per_param)].reverse
14 param.size.times do |j|
15 sum += ((param[j].chr=='1') ? 1.0 : 0.0) * (2.0 ** j.to_f)
16 end
17 min, max = bounds
18 vector << min + ((max-min)/((2.0**bits_per_param.to_f)-1.0)) * sum
19 end
20 return vector
21 end
22
23 def random_bitstring(num_bits)
3.10. Non-dominated Sorting Genetic Algorithm 155
24 return (0...num_bits).inject(""){|s,i| s<<((rand<0.5) ? "1" : "0")}
25 end
26
27 def point_mutation(bitstring, rate=1.0/bitstring.size)
28 child = ""
29 bitstring.size.times do |i|
30 bit = bitstring[i].chr
31 child << ((rand()<rate) ? ((bit=='1') ? "0" : "1") : bit)
32 end
33 return child
34 end
35
36 def crossover(parent1, parent2, rate)
37 return ""+parent1 if rand()>=rate
38 child = ""
39 parent1.size.times do |i|
40 child << ((rand()<0.5) ? parent1[i].chr : parent2[i].chr)
41 end
42 return child
43 end
44
45 def reproduce(selected, pop_size, p_cross)
46 children = []
47 selected.each_with_index do |p1, i|
48 p2 = (i.modulo(2)==0) ? selected[i+1] : selected[i-1]
49 p2 = selected[0] if i == selected.size-150 child = {}
51 child[:bitstring] = crossover(p1[:bitstring], p2[:bitstring], p_cross)
52 child[:bitstring] = point_mutation(child[:bitstring])
53 children << child
54 break if children.size >= pop_size
55 end
56 return children
57 end
58
59 def calculate_objectives(pop, search_space, bits_per_param)
60 pop.each do |p|
61 p[:vector] = decode(p[:bitstring], search_space, bits_per_param)
62 p[:objectives] = [objective1(p[:vector]), objective2(p[:vector])]
63 end
64 end
65
66 def dominates(p1, p2)
67 p1[:objectives].each_index do |i|
68 return false if p1[:objectives][i] > p2[:objectives][i]
69 end
70 return true
71 end
72
73 def fast_nondominated_sort(pop)
74 fronts = Array.new(1){[]}
75 pop.each do |p1|
76 p1[:dom_count], p1[:dom_set] = 0, []
77 pop.each do |p2|
78 if dominates(p1, p2)
79 p1[:dom_set] << p2
156 Chapter 3. Evolutionary Algorithms
80 elsif dominates(p2, p1)
81 p1[:dom_count] += 1
82 end
83 end
84 if p1[:dom_count] == 0
85 p1[:rank] = 0
86 fronts.first << p1
87 end
88 end
89 curr = 0
90 begin
91 next_front = []
92 fronts[curr].each do |p1|
93 p1[:dom_set].each do |p2|
94 p2[:dom_count] -= 1
95 if p2[:dom_count] == 0
96 p2[:rank] = (curr+1)
97 next_front << p2
98 end
99 end
100 end
101 curr += 1
102 fronts << next_front if !next_front.empty?
103 end while curr < fronts.size
104 return fronts
105 end
106
107 def calculate_crowding_distance(pop)
108 pop.each {|p| p[:dist] = 0.0}
109 num_obs = pop.first[:objectives].size
110 num_obs.times do |i|
111 min = pop.min{|x,y| x[:objectives][i]<=>y[:objectives][i]}
112 max = pop.max{|x,y| x[:objectives][i]<=>y[:objectives][i]}
113 rge = max[:objectives][i] - min[:objectives][i]
114 pop.first[:dist], pop.last[:dist] = 1.0/0.0, 1.0/0.0
115 next if rge == 0.0
116 (1...(pop.size-1)).each do |j|
117 pop[j][:dist]+=(pop[j+1][:objectives][i]-pop[j-1][:objectives][i])/rge
118 end
119 end
120 end
121
122 def crowded_comparison_operator(x,y)
123 return y[:dist]<=>x[:dist] if x[:rank] == y[:rank]
124 return x[:rank]<=>y[:rank]
125 end
126
127 def better(x,y)
128 if !x[:dist].nil? and x[:rank] == y[:rank]
129 return (x[:dist]>y[:dist]) ? x : y
130 end
131 return (x[:rank]<y[:rank]) ? x : y
132 end
133
134 def select_parents(fronts, pop_size)
135 fronts.each {|f| calculate_crowding_distance(f)}
3.10. Non-dominated Sorting Genetic Algorithm 157
136 offspring, last_front = [], 0
137 fronts.each do |front|
138 break if (offspring.size+front.size) > pop_size
139 front.each {|p| offspring << p}
140 last_front += 1
141 end
142 if (remaining = pop_size-offspring.size) > 0
143 fronts[last_front].sort! {|x,y| crowded_comparison_operator(x,y)}
144 offspring += fronts[last_front][0...remaining]
145 end
146 return offspring
147 end
148
149 def weighted_sum(x)
150 return x[:objectives].inject(0.0) {|sum, x| sum+x}
151 end
152
153 def search(search_space, max_gens, pop_size, p_cross, bits_per_param=16)
154 pop = Array.new(pop_size) do |i|
155 {:bitstring=>random_bitstring(search_space.size*bits_per_param)}
156 end
157 calculate_objectives(pop, search_space, bits_per_param)
158 fast_nondominated_sort(pop)
159 selected = Array.new(pop_size) do
160 better(pop[rand(pop_size)], pop[rand(pop_size)])
161 end
162 children = reproduce(selected, pop_size, p_cross)
163 calculate_objectives(children, search_space, bits_per_param)
164 max_gens.times do |gen|
165 union = pop + children
166 fronts = fast_nondominated_sort(union)
167 parents = select_parents(fronts, pop_size)
168 selected = Array.new(pop_size) do
169 better(parents[rand(pop_size)], parents[rand(pop_size)])
170 end
171 pop = children
172 children = reproduce(selected, pop_size, p_cross)
173 calculate_objectives(children, search_space, bits_per_param)
174 best = parents.sort!{|x,y| weighted_sum(x)<=>weighted_sum(y)}.first
175 best_s = "[x=#{best[:vector]}, objs=#{best[:objectives].join(', ')}]"
176 puts " > gen=#{gen+1}, fronts=#{fronts.size}, best=#{best_s}"
177 end
178 union = pop + children
179 fronts = fast_nondominated_sort(union)
180 parents = select_parents(fronts, pop_size)
181 return parents
182 end
183
184 if __FILE__ == $0
185 # problem configuration
186 problem_size = 1
187 search_space = Array.new(problem_size) {|i| [-10, 10]}
188 # algorithm configuration
189 max_gens = 50
190 pop_size = 100
191 p_cross = 0.98
158 Chapter 3. Evolutionary Algorithms
192 # execute the algorithm
193 pop = search(search_space, max_gens, pop_size, p_cross)
194 puts "done!"
195 end
Listing 3.9: NSGA-II in Ruby
3.10.6 References
Primary Sources
Srinivas and Deb proposed the NSGA inspired by Goldberg’s notion of a
non-dominated sorting procedure [6]. Goldberg proposed a non-dominated
sorting procedure in his book in considering the biases in the Pareto optimal
solutions provided by VEGA [5]. Srinivas and Deb’s NSGA used the sorting
procedure as a ranking selection method, and a fitness sharing niching
method to maintain stable sub-populations across the Pareto front. Deb
et al. later extended NSGA to address three criticism of the approach: the
O(mN3) time complexity, the lack of elitism, and the need for a sharing
parameter for the fitness sharing niching method [3, 4].
Learn More
Deb provides in depth coverage of Evolutionary Multiple Objective Op-
timization algorithms in his book, including a detailed description of the
NSGA in Chapter 5 [1].
3.10.7 Bibliography
[1] K. Deb. Multi-Objective Optimization Using Evolutionary Algorithms.
John Wiley and Sons, 2001.
[2] K. Deb and R. B. Agrawal. Simulated binary crossover for continuous
search space. Complex Systems, 9:115–148, 1995.
[3] K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan. A fast elitist non–
dominated sorting genetic algorithm for multi–objective optimization:
NSGA–II. Parallel Problem Solving from Nature PPSN VI, 1917:849–858,
2000.
[4] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist
multiobjective genetic algorithm: NSGA–II. IEEE Transactions on
Evolutionary Computation, 6(2):182–197, 2002.
[5] D. E. Goldberg. Genetic Algorithms in Search, Optimization, and
Machine Learning. Addison-Wesley, 1989.
3.10. Non-dominated Sorting Genetic Algorithm 159
[6] N. Srinivas and K. Deb. Muiltiobjective optimization using nondomi-
nated sorting in genetic algorithms. Evolutionary Computation, 2(3):221–
248, 1994.
160 Chapter 3. Evolutionary Algorithms
3.11 Strength Pareto Evolutionary Algorithm
Strength Pareto Evolutionary Algorithm, SPEA, SPEA2.
3.11.1 Taxonomy
Strength Pareto Evolutionary Algorithm is a Multiple Objective Optimiza-
tion (MOO) algorithm and an Evolutionary Algorithm from the field of
Evolutionary Computation. It belongs to the field of Evolutionary Multiple
Objective (EMO) algorithms. Refer to Section 9.5.3 for more information
and references on Multiple Objective Optimization. Strength Pareto Evo-
lutionary Algorithm is an extension of the Genetic Algorithm for multiple
objective optimization problems (Section 3.2). It is related to sibling Evo-
lutionary Algorithms such as Non-dominated Sorting Genetic Algorithm
(NSGA) (Section 3.10), Vector-Evaluated Genetic Algorithm (VEGA), and
Pareto Archived Evolution Strategy (PAES). There are two versions of
SPEA, the original SPEA algorithm and the extension SPEA2. Additional
extensions include SPEA+ and iSPEA.
3.11.2 Strategy
The objective of the algorithm is to locate and and maintain a front of
non-dominated solutions, ideally a set of Pareto optimal solutions. This is
achieved by using an evolutionary process (with surrogate procedures for
genetic recombination and mutation) to explore the search space, and a
selection process that uses a combination of the degree to which a candi-
date solution is dominated (strength) and an estimation of density of the
Pareto front as an assigned fitness. An archive of the non-dominated set is
maintained separate from the population of candidate solutions used in the
evolutionary process,providing a form of elitism.
3.11.3 Procedure
Algorithm 3.11.1 provides a pseudocode listing of the Strength Pareto
Evolutionary Algorithm 2 (SPEA2) for minimizing a cost function. The
CalculateRawFitness function calculates the raw fitness as the sum of the
strength values of the solutions that dominate a given candidate, where
strength is the number of solutions that a give solution dominate. The
CandidateDensity function estimates the density of an area of the Pareto
front as 1.0
σk+2
where σk is the Euclidean distance of the objective values
between a given solution the kth nearest neighbor of the solution, and k is
the square root of the size of the population and archive combined. The
PopulateWithRemainingBest function iteratively fills the archive with the
remaining candidate solutions in order of fitness. The RemoveMostSimilar
function truncates the archive population removing those members with the
3.11. Strength Pareto Evolutionary Algorithm 161
smallest σk values as calculated against the archive. The SelectParents
function selects parents from a population using a Genetic Algorithm se-
lection method such as binary tournament selection. The CrossoverAnd-
Mutation function performs the crossover and mutation genetic operators
from the Genetic Algorithm.
Algorithm 3.11.1: Pseudocode for SPEA2.
Input: Populationsize, Archivesize, ProblemSize, Pcrossover,
Pmutation
Output: Archive
Population ← InitializePopulation(Populationsize, ProblemSize);1
Archive ← ∅;2
while ¬StopCondition() do3
for Si ∈ Population do4
Siobjectives ← CalculateObjectives(Si);5
end6
Union ← Population + Archive;7
for Si ∈ Union do8
Siraw ← CalculateRawFitness(Si, Union);9
Sidensity ← CalculateSolutionDensity(Si, Union);10
Sifitness ← Siraw + Sidensity;11
end12
Archive ← GetNonDominated(Union);13
if Size(Archive) < Archivesize then14
PopulateWithRemainingBest(Union, Archive, Archivesize);15
else if Size(Archive) > Archivesize then16
RemoveMostSimilar(Archive, Archivesize);17
end18
Selected ← SelectParents(Archive, Populationsize);19
Population ← CrossoverAndMutation(Selected, Pcrossover,20
Pmutation);
end21
return GetNonDominatedArchive;22
3.11.4 Heuristics
� SPEA was designed for and is suited to combinatorial and continuous
function multiple objective optimization problem instances.
� A binary representation can be used for continuous function optimiza-
tion problems in conjunction with classical genetic operators such as
one-point crossover and point mutation.
162 Chapter 3. Evolutionary Algorithms
� A k value of 1 may be used for efficiency whilst still providing useful
results.
� The size of the archive is commonly smaller than the size of the
population.
� There is a lot of room for implementation optimization in density and
Pareto dominance calculations.
3.11.5 Code Listing
Listing 3.10 provides an example of the Strength Pareto Evolutionary
Algorithm 2 (SPEA2) implemented in the Ruby Programming Language.
The demonstration problem is an instance of continuous multiple objective
function optimization called SCH (problem one in [1]). The problem seeks
the minimum of two functions: f1 =
∑n
i=1 x
2
i and f2 =
∑n
i=1(xi − 2)
2,
−10 ≤ xi ≤ 10 and n = 1. The optimal solutions for this function are
x ∈ [0, 2]. The algorithm is an implementation of SPEA2 based on the
presentation by Zitzler, Laumanns, and Thiele [5]. The algorithm uses a
binary string representation (16 bits per objective function parameter) that
is decoded and rescaled to the function domain. The implementation uses a
uniform crossover operator and point mutations with a fixed mutation rate
of 1
L
, where L is the number of bits in a solution’s binary string.
1 def objective1(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x**2.0)}
3 end
4
5 def objective2(vector)
6 return vector.inject(0.0) {|sum, x| sum + ((x-2.0)**2.0)}
7 end
8
9 def decode(bitstring, search_space, bits_per_param)
10 vector = []
11 search_space.each_with_index do |bounds, i|
12 off, sum = i*bits_per_param, 0.0
13 param = bitstring[off...(off+bits_per_param)].reverse
14 param.size.times do |j|
15 sum += ((param[j].chr=='1') ? 1.0 : 0.0) * (2.0 ** j.to_f)
16 end
17 min, max = bounds
18 vector << min + ((max-min)/((2.0**bits_per_param.to_f)-1.0)) * sum
19 end
20 return vector
21 end
22
23 def point_mutation(bitstring, rate=1.0/bitstring.size)
24 child = ""
25 bitstring.size.times do |i|
26 bit = bitstring[i].chr
27 child << ((rand()<rate) ? ((bit=='1') ? "0" : "1") : bit)
28 end
3.11. Strength Pareto Evolutionary Algorithm 163
29 return child
30 end
31
32 def binary_tournament(pop)
33 i, j = rand(pop.size), rand(pop.size)
34 j = rand(pop.size) while j==i
35 return (pop[i][:fitness] < pop[j][:fitness]) ? pop[i] : pop[j]
36 end
37
38 def crossover(parent1, parent2, rate)
39 return ""+parent1 if rand()>=rate
40 child = ""
41 parent1.size.times do |i|
42 child << ((rand()<0.5) ? parent1[i].chr : parent2[i].chr)
43 end
44 return child
45 end
46
47 def reproduce(selected, pop_size, p_cross)
48 children = []
49 selected.each_with_index do |p1, i|
50 p2 = (i.modulo(2)==0) ? selected[i+1] : selected[i-1]
51 p2 = selected[0] if i == selected.size-1
52 child = {}
53 child[:bitstring] = crossover(p1[:bitstring], p2[:bitstring], p_cross)
54 child[:bitstring] = point_mutation(child[:bitstring])
55 children << child
56 break if children.size >= pop_size
57 end
58 return children
59 end
60
61 def random_bitstring(num_bits)
62 return (0...num_bits).inject(""){|s,i| s<<((rand<0.5) ? "1" : "0")}
63 end
64
65 def calculate_objectives(pop, search_space, bits_per_param)
66 pop.each do |p|
67 p[:vector] = decode(p[:bitstring], search_space, bits_per_param)
68 p[:objectives] = []
69 p[:objectives] << objective1(p[:vector])
70 p[:objectives] << objective2(p[:vector])
71 end
72 end
73
74 def dominates?(p1, p2)
75 p1[:objectives].each_index do |i|
76 return false if p1[:objectives][i] > p2[:objectives][i]
77 end
78 return true
79 end
80
81 def weighted_sum(x)
82 return x[:objectives].inject(0.0) {|sum, x| sum+x}
83 end
84
164 Chapter 3. Evolutionary Algorithms
85 def euclidean_distance(c1, c2)
86 sum = 0.0
87 c1.each_index {|i| sum += (c1[i]-c2[i])**2.0}
88 return Math.sqrt(sum)
89 end
90
91 def calculate_dominated(pop)
92 pop.each do |p1|
93 p1[:dom_set] = pop.select {|p2| p1!=p2 and dominates?(p1, p2) }
94 end
95 end
96
97 def calculate_raw_fitness(p1, pop)
98 return pop.inject(0.0) do |sum, p2|
99 (dominates?(p2, p1)) ? sum + p2[:dom_set].size.to_f : sum
100 end
101 end
102
103 def calculate_density(p1, pop)
104 pop.each do |p2|
105 p2[:dist] = euclidean_distance(p1[:objectives], p2[:objectives])
106 end
107 list = pop.sort{|x,y| x[:dist]<=>y[:dist]}
108 k = Math.sqrt(pop.size).to_i
109 return 1.0 / (list[k][:dist] + 2.0)
110 end
111
112 def calculate_fitness(pop, archive, search_space, bits_per_param)
113 calculate_objectives(pop, search_space, bits_per_param)
114 union = archive + pop
115 calculate_dominated(union)
116 union.each do |p|
117 p[:raw_fitness] = calculate_raw_fitness(p, union)
118 p[:density] = calculate_density(p, union)
119 p[:fitness] = p[:raw_fitness] + p[:density]
120 end
121 end
122
123 def environmental_selection(pop, archive, archive_size)
124 union = archive + pop
125 environment = union.select {|p| p[:fitness]<1.0}
126 if environment.size < archive_size
127 union.sort!{|x,y| x[:fitness]<=>y[:fitness]}
128 union.each do |p|
129 environment << p if p[:fitness] >= 1.0
130 break if environment.size >= archive_size
131 end
132 elsif environment.size > archive_size
133 begin
134 k = Math.sqrt(environment.size).to_i
135 environment.each do |p1|
136 environment.each do |p2|
137 p2[:dist] = euclidean_distance(p1[:objectives], p2[:objectives])
138 end
139 list = environment.sort{|x,y| x[:dist]<=>y[:dist]}
140 p1[:density] = list[k][:dist]
3.11. Strength Pareto Evolutionary Algorithm 165
141 end
142 environment.sort!{|x,y|x[:density]<=>y[:density]}
143 environment.shift
144 end until environment.size <= archive_size
145 end
146 return environment
147 end
148
149 def search(search_space, max_gens, pop_size, archive_size, p_cross,
bits_per_param=16)
150 pop = Array.new(pop_size) do |i|
151 {:bitstring=>random_bitstring(search_space.size*bits_per_param)}
152 end
153 gen, archive = 0, []
154 begin
155 calculate_fitness(pop, archive, search_space, bits_per_param)
156 archive = environmental_selection(pop, archive, archive_size)
157 best = archive.sort{|x,y| weighted_sum(x)<=>weighted_sum(y)}.first
158 puts ">gen=#{gen}, objs=#{best[:objectives].join(', ')}"
159 break if gen >= max_gens
160 selected = Array.new(pop_size){binary_tournament(archive)}
161 pop = reproduce(selected, pop_size, p_cross)
162 gen += 1
163 end while true
164 return archive
165 end
166
167 if __FILE__ == $0
168 # problem configuration
169 problem_size = 1
170 search_space = Array.new(problem_size) {|i| [-10, 10]}
171 # algorithm configuration
172 max_gens = 50
173 pop_size = 80
174 archive_size = 40
175 p_cross = 0.90
176 # execute the algorithm
177 pop = search(search_space, max_gens, pop_size, archive_size, p_cross)
178 puts "done!"
179 end
Listing 3.10: SPEA2 in Ruby
3.11.6 References
Primary Sources
Zitzler and Thiele introduced the Strength Pareto Evolutionary Algorithm
as a technical report on a multiple objective optimization algorithm with
elitism and clustering along the Pareto front [6]. The technical report
was later published [7]. The Strength Pareto Evolutionary Algorithm was
developed as a part of Zitzler’s PhD thesis [2]. Zitzler, Laumanns, and
Thiele later extended SPEA to address some inefficiencies of the approach,
166 Chapter 3. Evolutionary Algorithms
the algorithm was called SPEA2 and was released as a technical report [4]
and later published [5]. SPEA2 provides fine-grained fitness assignment,
density estimation of the Pareto front, and an archive truncation operator.
Learn More
Zitzler, Laumanns, and Bleuler provide a tutorial on SPEA2 as a book
chapter that considers the basics of multiple objective optimization, and the
differences from SPEA and the other related Multiple Objective Evolutionary
Algorithms [3].
3.11.7 Bibliography
[1] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist
multiobjective genetic algorithm: NSGA–II. IEEE Transactions on
Evolutionary Computation, 6(2):182–197, 2002.
[2] E. Zitzler. Evolutionary Algorithms for Multiobjective Optimization:
Methods and Applications. PhD thesis, Shaker Verlag, Aachen, Germany,
1999.
[3] E. Zitzler, M. Laumanns, and S. Bleuler. Metaheuristics for Multiobjec-
tive Optimisation, chapter A Tutorial on Evolutionary Multiobjective
Optimization, pages 3–37. Springer, 2004.
[4] E. Zitzler, M. Laumanns, and L. Thiele. SPEA2: Improving the strength
pareto evolutionary algorithm. Technical Report 103, Computer En-
gineering and Networks Laboratory (TIK), Swiss Federal Institute of
Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich, Switzer-
land, May 2001.
[5] E. Zitzler, M. Laumanns, and L. Thiele. SPEA2: Improving the strength
pareto evolutionary algorithm for multiobjective optimization. In Evolu-
tionary Methods for Design, Optimisation and Control with Application
to Industrial Problems (EUROGEN 2001), pages 95–100, 2002.
[6] E. Zitzler and L. Thiele. An evolutionary algorithm for multiobjective
optimization: The strength pareto approach. Technical Report 43,
Computer Engineering and Networks Laboratory (TIK), Swiss Federal
Institute of Technology (ETH) Zurich, Gloriastrasse 35, CH-8092 Zurich,
Switzerland, May 1998.
[7] E. Zitzler and L. Thiele. Multiobjective evolutionary algorithms: A
comparative case study and the strength pareto approach. IEEE Trans-
actions on Evolutionary Computation, 3(4):257–271, 1999.
Chapter 4
Physical Algorithms
4.1 Overview
This chapter describes Physical Algorithms.
4.1.1 Physical Properties
Physical algorithms are those algorithms inspired by a physical process. The
described physical algorithm generally belong to the fields of Metaheustics
and Computational Intelligence, although do not fit neatly into the existing
categories of the biological inspired techniques (such as Swarm, Immune,
Neural, and Evolution). In this vein, they could just as easily be referred to
as nature inspired algorithms.
The inspiring physical systems range from metallurgy, music, the inter-
play between culture and evolution, and complex dynamic systems such as
avalanches. They are generally stochastic optimization algorithms with a
mixtures of local (neighborhood-based) and global search techniques.
4.1.2 Extensions
There are many other algorithms and classes of algorithm that were not
described inspired by natural systems, not limited to:
� More Annealing: Extensions to the classical Simulated Annealing
algorithm, such as Adaptive Simulated Annealing (formally Very Fast
Simulated Re-annealing) [3, 4], and Quantum Annealing [1, 2].
� Stochastic tunneling: based on the physical idea of a particle
tunneling through structures [5].
167
168 Chapter 4. Physical Algorithms
4.1.3 Bibliography
[1] B. Apolloni, C. Caravalho, and D. De Falco. Quantum stochastic
optimization. Stochastic Processes and their Applications, 33:233–244,
1989.
[2] A. Das and B. K. Chakrabarti. Quantum annealing and related opti-
mization methods. Springer, 2005.
[3] L. Ingber. Very fast simulated re-annealing. Mathematical and Computer
Modelling, 12(8):967–973, 1989.
[4] L. Ingber. Adaptive simulated annealing (ASA): Lessons learned. Control
and Cybernetics, 25(1):33–54, 1996.
[5] W. Wenzel and K. Hamacher. A stochastic tunneling approach for global
minimization of complex potential energy landscapes. Phys. Rev. Lett.,
82(15):3003–3007, 1999.
4.2. Simulated Annealing 169
4.2 Simulated Annealing
Simulated Annealing, SA.
4.2.1 Taxonomy
Simulated Annealing is a global optimization algorithm that belongs to the
field of Stochastic Optimization and Metaheuristics. Simulated Annealing
is an adaptation of the Metropolis-Hastings Monte Carlo algorithm and is
used in function optimization. Like the Genetic Algorithm (Section 3.2), it
provides a basis for a large variety of extensions and specialization’s of the
general method not limited to Parallel Simulated Annealing, Fast Simulated
Annealing, and Adaptive Simulated Annealing.
4.2.2 Inspiration
Simulated Annealing is inspired by the process of annealing in metallurgy. In
this natural process a material is heated and slowly cooled under controlled
conditions to increase the size of the crystals in the material and reduce their
defects. This has the effect of improving the strength and durability of the
material. The heat increases the energy of the atoms allowing them to move
freely, and the slow cooling schedule allows a new low-energy configuration
to be discovered and exploited.
4.2.3 Metaphor
Each configuration of a solution in the search space represents a different
internal energy of the system. Heating the system results in a relaxation of
the acceptance criteria of the samples taken from the search space. As the
system is cooled, the acceptance criteria of samples is narrowed to focus on
improving movements. Once the system has cooled, the configuration will
represent a sample at or close to a global optimum.
4.2.4 Strategy
The information processing objective of the technique is to locate the
minimum cost configuration in the search space. The algorithms plan
of action is to probabilistically re-sample the problem space where the
acceptance of new samples into the currently held sample is managed by a
probabilistic function that becomes more discerning of the cost of samples it
accepts over the execution time of the algorithm. This probabilistic decision
is based on the Metropolis-Hastings algorithm for simulating samples from
a thermodynamic system.
170 Chapter 4. Physical Algorithms
4.2.5 Procedure
Algorithm 4.2.1 provides apseudocode listing of the main Simulated An-
nealing algorithm for minimizing a cost function.
Algorithm 4.2.1: Pseudocode for Simulated Annealing.
Input: ProblemSize, iterationsmax, tempmax
Output: Sbest
Scurrent ← CreateInitialSolution(ProblemSize);1
Sbest ← Scurrent;2
for i = 1 to iterationsmax do3
Si ← CreateNeighborSolution(Scurrent);4
tempcurr ← CalculateTemperature(i, tempmax);5
if Cost(Si) ≤ Cost(Scurrent) then6
Scurrent ← Si;7
if Cost(Si) ≤ Cost(Sbest) then8
Sbest ← Si;9
end10
else if Exp( Cost(Scurrent )−Cost(Si )
tempcurr
) > Rand() then11
Scurrent ← Si;12
end13
end14
return Sbest;15
4.2.6 Heuristics
� Simulated Annealing was designed for use with combinatorial optimiza-
tion problems, although it has been adapted for continuous function
optimization problems.
� The convergence proof suggests that with a long enough cooling period,
the system will always converge to the global optimum. The downside
of this theoretical finding is that the number of samples taken for
optimum convergence to occur on some problems may be more than
a complete enumeration of the search space.
� Performance improvements can be given with the selection of a can-
didate move generation scheme (neighborhood) that is less likely to
generate candidates of significantly higher cost.
� Restarting the cooling schedule using the best found solution so far
can lead to an improved outcome on some problems.
� A common acceptance method is to always accept improving solu-
tions and accept worse solutions with a probability of P (accept) ←
4.2. Simulated Annealing 171
exp( e−e
′
T
), where T is the current temperature, e is the energy (or cost)
of the current solution and e′ is the energy of a candidate solution
being considered.
� The size of the neighborhood considered in generating candidate
solutions may also change over time or be influenced by the tempera-
ture, starting initially broad and narrowing with the execution of the
algorithm.
� A problem specific heuristic method can be used to provide the starting
point for the search.
4.2.7 Code Listing
Listing 4.1 provides an example of the Simulated Annealing algorithm
implemented in the Ruby Programming Language. The algorithm is applied
to the Berlin52 instance of the Traveling Salesman Problem (TSP), taken
from the TSPLIB. The problem seeks a permutation of the order to visit
cities (called a tour) that minimizes the total distance traveled. The optimal
tour distance for Berlin52 instance is 7542 units.
The algorithm implementation uses a two-opt procedure for the neigh-
borhood function and the classical P (accept)← exp( e−e
′
T
) as the acceptance
function. A simple linear cooling regime is used with a large initial temper-
ature which is decreased each iteration.
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(permutation, cities)
6 distance =0
7 permutation.each_with_index do |c1, i|
8 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def random_permutation(cities)
15 perm = Array.new(cities.size){|i| i}
16 perm.each_index do |i|
17 r = rand(perm.size-i) + i
18 perm[r], perm[i] = perm[i], perm[r]
19 end
20 return perm
21 end
22
23 def stochastic_two_opt!(perm)
24 c1, c2 = rand(perm.size), rand(perm.size)
25 exclude = [c1]
26 exclude << ((c1==0) ? perm.size-1 : c1-1)
172 Chapter 4. Physical Algorithms
27 exclude << ((c1==perm.size-1) ? 0 : c1+1)
28 c2 = rand(perm.size) while exclude.include?(c2)
29 c1, c2 = c2, c1 if c2 < c1
30 perm[c1...c2] = perm[c1...c2].reverse
31 return perm
32 end
33
34 def create_neighbor(current, cities)
35 candidate = {}
36 candidate[:vector] = Array.new(current[:vector])
37 stochastic_two_opt!(candidate[:vector])
38 candidate[:cost] = cost(candidate[:vector], cities)
39 return candidate
40 end
41
42 def should_accept?(candidate, current, temp)
43 return true if candidate[:cost] <= current[:cost]
44 return Math.exp((current[:cost] - candidate[:cost]) / temp) > rand()
45 end
46
47 def search(cities, max_iter, max_temp, temp_change)
48 current = {:vector=>random_permutation(cities)}
49 current[:cost] = cost(current[:vector], cities)
50 temp, best = max_temp, current
51 max_iter.times do |iter|
52 candidate = create_neighbor(current, cities)
53 temp = temp * temp_change
54 current = candidate if should_accept?(candidate, current, temp)
55 best = candidate if candidate[:cost] < best[:cost]
56 if (iter+1).modulo(10) == 0
57 puts " > iteration #{(iter+1)}, temp=#{temp}, best=#{best[:cost]}"
58 end
59 end
60 return best
61 end
62
63 if __FILE__ == $0
64 # problem configuration
65 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
66 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
67 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
68 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
69 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
70 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
71 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
72 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
73 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
74 # algorithm configuration
75 max_iterations = 2000
76 max_temp = 100000.0
77 temp_change = 0.98
78 # execute the algorithm
79 best = search(berlin52, max_iterations, max_temp, temp_change)
80 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
81 end
4.2. Simulated Annealing 173
Listing 4.1: Simulated Annealing in Ruby
4.2.8 References
Primary Sources
Simulated Annealing is credited to Kirkpatrick, Gelatt, and Vecchi in 1983
[5]. Granville, Krivanek, and Rasson provided the proof for convergence
for Simulated Annealing in 1994 [2]. There were a number of early studies
and application papers such as Kirkpatrick’s investigation into the TSP
and minimum cut problems [4], and a study by Vecchi and Kirkpatrick on
Simulated Annealing applied to the global wiring problem [7].
Learn More
There are many excellent reviews of Simulated Annealing, not limited to
the review by Ingber that describes improved methods such as Adaptive
Simulated Annealing, Simulated Quenching, and hybrid methods [3]. There
are books dedicated to Simulated Annealing, applications and variations.
Two examples of good texts include “Simulated Annealing: Theory and
Applications” by Laarhoven and Aarts [6] that provides an introduction
to the technique and applications, and “Simulated Annealing: Paralleliza-
tion Techniques” by Robert Azencott [1] that focuses on the theory and
applications of parallel methods for Simulated Annealing.
4.2.9 Bibliography
[1] R. Azencott. Simulated annealing: parallelization techniques. Wiley,
1992.
[2] V. Granville, M. Krivanek, and J-P. Rasson. Simulated annealing: A
proof of convergence. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 16(6):652–656, 1994.
[3] L. Ingber. Simulated annealing: Practice versus theory. Math. Comput.
Modelling, 18:29–57, 1993.
[4] S. Kirkpatrick. Optimization by simulated annealing: Quantitative
studies. Journal of Statistical Physics, 34:975–986, 1983.
[5] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simu-
lated annealing. Science, 220(4598):671–680, 1983.
[6] P. J. M. van Laarhoven and E. H. L. Aarts. Simulated Annealing: Theory
and Applications. Springer, 1988.
174 Chapter 4. Physical Algorithms
[7] M. P. Vecchi and S. Kirkpatrick. Global wiring by simulated annealing.
IEEE Transactions on Computer-Aided Design of Integrated Circuits
and Systems, 2(4):215–222, 1983.
4.3. Extremal Optimization 175
4.3 Extremal Optimization
Extremal Optimization, EO.
4.3.1 Taxonomy
Extremal Optimization is a stochastic search technique that has the prop-
erties of being a local and global search method. It is generally related
to hill-climbing algorithmsand provides the basis for extensions such as
Generalized Extremal Optimization.
4.3.2 Inspiration
Extremal Optimization is inspired by the Bak-Sneppen self-organized crit-
icality model of co-evolution from the field of statistical physics. The
self-organized criticality model suggests that some dynamical systems have
a critical point as an attractor, whereby the systems exhibit periods of
slow movement or accumulation followed by short periods of avalanche or
instability. Examples of such systems include land formation, earthquakes,
and the dynamics of sand piles. The Bak-Sneppen model considers these
dynamics in co-evolutionary systems and in the punctuated equilibrium
model, which is described as long periods of status followed by short periods
of extinction and large evolutionary change.
4.3.3 Metaphor
The dynamics of the system result in the steady improvement of a candidate
solution with sudden and large crashes in the quality of the candidate
solution. These dynamics allow two main phases of activity in the system:
1) to exploit higher quality solutions in a local search like manner, and 2)
escape possible local optima with a population crash and explore the search
space for a new area of high quality solutions.
4.3.4 Strategy
The objective of the information processing strategy is to iteratively identify
the worst performing components of a given solution and replace or swap
them with other components. This is achieved through the allocation of cost
to the components of the solution based on their contribution to the overall
cost of the solution in the problem domain. Once components are assessed
they can be ranked and the weaker components replaced or switched with a
randomly selected component.
176 Chapter 4. Physical Algorithms
4.3.5 Procedure
Algorithm 4.3.1 provides a pseudocode listing of the Extremal Optimization
algorithm for minimizing a cost function. The deterministic selection of the
worst component in the SelectWeakComponent function and replacement
in the SelectReplacementComponent function is classical EO. If these
decisions are probabilistic making use of τ parameter, this is referred to as
τ -Extremal Optimization.
Algorithm 4.3.1: Pseudocode for Extremal Optimization.
Input: ProblemSize, iterationsmax, τ
Output: Sbest
Scurrent ← CreateInitialSolution(ProblemSize);1
Sbest ← Scurrent;2
for i = 1 to iterationsmax do3
foreach Componenti ∈ Scurrent do4
Componentcosti ← Cost(Componenti, Scurrent);5
end6
RankedComponents ← Rank(Sicomponents)7
Componenti ← SelectWeakComponent(RankedComponents,8
Componenti, τ);
Componentj ←9
SelectReplacementComponent(RankedComponents, τ);
Scandidate ← Replace(Scurrent, Componenti, Componentj);10
if Cost(Scandidate) ≤ Cost(Sbest) then11
Sbest ← Scandidate;12
end13
end14
return Sbest;15
4.3.6 Heuristics
� Extremal Optimization was designed for combinatorial optimization
problems, although variations have been applied to continuous function
optimization.
� The selection of the worst component and the replacement component
each iteration can be deterministic or probabilistic, the latter of
which is referred to as τ -Extremal Optimization given the use of a τ
parameter.
� The selection of an appropriate scoring function of the components of
a solution is the most difficult part in the application of the technique.
4.3. Extremal Optimization 177
� For τ -Extremal Optimization, low τ values are used (such as τ ∈
[1.2, 1.6]) have been found to be effective for the TSP.
4.3.7 Code Listing
Listing 4.2 provides an example of the Extremal Optimization algorithm
implemented in the Ruby Programming Language. The algorithm is applied
to the Berlin52 instance of the Traveling Salesman Problem (TSP), taken
from the TSPLIB. The problem seeks a permutation of the order to visit
cities (called a tour) that minimizes the total distance traveled. The optimal
tour distance for Berlin52 instance is 7542 units.
The algorithm implementation is based on the seminal work by Boettcher
and Percus [5]. A solution is comprised of a permutation of city components.
Each city can potentially form a connection to any other city, and the
connections to other cities ordered by distance may be considered its neigh-
borhood. For a given candidate solution, the city components of a solution
are scored based on the neighborhood rank of the cities to which they are
connected: fitnessk ←
3
ri+rj
, where ri and rj are the neighborhood ranks
of cities i and j against city k. A city is selected for modification probabilis-
tically where the probability of selecting a given city is proportional to n−τi ,
where n is the rank of city i. The longest connection is broken, and the
city is connected with another neighboring city that is also probabilistically
selected.
1 def euc_2d(c1, c2)
2 Math.sqrt((c1[0] - c2[0])**2.0 + (c1[1] - c2[1])**2.0).round
3 end
4
5 def cost(permutation, cities)
6 distance =0
7 permutation.each_with_index do |c1, i|
8 c2 = (i==permutation.size-1) ? permutation[0] : permutation[i+1]
9 distance += euc_2d(cities[c1], cities[c2])
10 end
11 return distance
12 end
13
14 def random_permutation(cities)
15 perm = Array.new(cities.size){|i| i}
16 perm.each_index do |i|
17 r = rand(perm.size-i) + i
18 perm[r], perm[i] = perm[i], perm[r]
19 end
20 return perm
21 end
22
23 def calculate_neighbor_rank(city_number, cities, ignore=[])
24 neighbors = []
25 cities.each_with_index do |city, i|
26 next if i==city_number or ignore.include?(i)
27 neighbor = {:number=>i}
178 Chapter 4. Physical Algorithms
28 neighbor[:distance] = euc_2d(cities[city_number], city)
29 neighbors << neighbor
30 end
31 return neighbors.sort!{|x,y| x[:distance] <=> y[:distance]}
32 end
33
34 def get_edges_for_city(city_number, permutation)
35 c1, c2 = nil, nil
36 permutation.each_with_index do |c, i|
37 if c == city_number
38 c1 = (i==0) ? permutation.last : permutation[i-1]
39 c2 = (i==permutation.size-1) ? permutation.first : permutation[i+1]
40 break
41 end
42 end
43 return [c1, c2]
44 end
45
46 def calculate_city_fitness(permutation, city_number, cities)
47 c1, c2 = get_edges_for_city(city_number, permutation)
48 neighbors = calculate_neighbor_rank(city_number, cities)
49 n1, n2 = -1, -1
50 neighbors.each_with_index do |neighbor,i|
51 n1 = i+1 if neighbor[:number] == c1
52 n2 = i+1 if neighbor[:number] == c2
53 break if n1!=-1 and n2!=-1
54 end
55 return 3.0 / (n1.to_f + n2.to_f)
56 end
57
58 def calculate_city_fitnesses(cities, permutation)
59 city_fitnesses = []
60 cities.each_with_index do |city, i|
61 city_fitness = {:number=>i}
62 city_fitness[:fitness] = calculate_city_fitness(permutation, i, cities)
63 city_fitnesses << city_fitness
64 end
65 return city_fitnesses.sort!{|x,y| y[:fitness] <=> x[:fitness]}
66 end
67
68 def calculate_component_probabilities(ordered_components, tau)
69 sum = 0.0
70 ordered_components.each_with_index do |component, i|
71 component[:prob] = (i+1.0)**(-tau)
72 sum += component[:prob]
73 end
74 return sum
75 end
76
77 def make_selection(components, sum_probability)
78 selection = rand()
79 components.each_with_index do |component, i|
80 selection -= (component[:prob] / sum_probability)
81 return component[:number] if selection <= 0.0
82 end
83 return components.last[:number]
4.3. Extremal Optimization 179
84 end
85
86 def probabilistic_selection(ordered_components, tau, exclude=[])
87 sum = calculate_component_probabilities(ordered_components, tau)
88 selected_city = nil
89 begin
90 selected_city = make_selection(ordered_components, sum)
91 end while exclude.include?(selected_city)
92 return selected_city
93 end
94
95 def vary_permutation(permutation, selected, new, long_edge)
96 perm = Array.new(permutation)
97 c1, c2 = perm.rindex(selected), perm.rindex(new)
98 p1,p2 = (c1<c2) ? [c1,c2] : [c2,c1]
99 right = (c1==perm.size-1) ? 0 : c1+1
100 if perm[right] == long_edge
101 perm[p1+1..p2] = perm[p1+1..p2].reverse
102 else
103 perm[p1...p2]= perm[p1...p2].reverse
104 end
105 return perm
106 end
107
108 def get_long_edge(edges, neighbor_distances)
109 n1 = neighbor_distances.find {|x| x[:number]==edges[0]}
110 n2 = neighbor_distances.find {|x| x[:number]==edges[1]}
111 return (n1[:distance] > n2[:distance]) ? n1[:number] : n2[:number]
112 end
113
114 def create_new_perm(cities, tau, perm)
115 city_fitnesses = calculate_city_fitnesses(cities, perm)
116 selected_city = probabilistic_selection(city_fitnesses.reverse, tau)
117 edges = get_edges_for_city(selected_city, perm)
118 neighbors = calculate_neighbor_rank(selected_city, cities)
119 new_neighbor = probabilistic_selection(neighbors, tau, edges)
120 long_edge = get_long_edge(edges, neighbors)
121 return vary_permutation(perm, selected_city, new_neighbor, long_edge)
122 end
123
124 def search(cities, max_iterations, tau)
125 current = {:vector=>random_permutation(cities)}
126 current[:cost] = cost(current[:vector], cities)
127 best = current
128 max_iterations.times do |iter|
129 candidate = {}
130 candidate[:vector] = create_new_perm(cities, tau, current[:vector])
131 candidate[:cost] = cost(candidate[:vector], cities)
132 current = candidate
133 best = candidate if candidate[:cost] < best[:cost]
134 puts " > iter #{(iter+1)}, curr=#{current[:cost]}, best=#{best[:cost]}"
135 end
136 return best
137 end
138
139 if __FILE__ == $0
180 Chapter 4. Physical Algorithms
140 # problem configuration
141 berlin52 = [[565,575],[25,185],[345,750],[945,685],[845,655],
142 [880,660],[25,230],[525,1000],[580,1175],[650,1130],[1605,620],
143 [1220,580],[1465,200],[1530,5],[845,680],[725,370],[145,665],
144 [415,635],[510,875],[560,365],[300,465],[520,585],[480,415],
145 [835,625],[975,580],[1215,245],[1320,315],[1250,400],[660,180],
146 [410,250],[420,555],[575,665],[1150,1160],[700,580],[685,595],
147 [685,610],[770,610],[795,645],[720,635],[760,650],[475,960],
148 [95,260],[875,920],[700,500],[555,815],[830,485],[1170,65],
149 [830,610],[605,625],[595,360],[1340,725],[1740,245]]
150 # algorithm configuration
151 max_iterations = 250
152 tau = 1.8
153 # execute the algorithm
154 best = search(berlin52, max_iterations, tau)
155 puts "Done. Best Solution: c=#{best[:cost]}, v=#{best[:vector].inspect}"
156 end
Listing 4.2: Extremal Optimization in Ruby
4.3.8 References
Primary Sources
Extremal Optimization was proposed as an optimization heuristic by Boettcher
and Percus applied to graph partitioning and the Traveling Salesman Prob-
lem [5]. The approach was inspired by the Bak-Sneppen self-organized
criticality model of co-evolution [1, 2].
Learn More
A number of detailed reviews of Extremal Optimization have been presented,
including a review and studies by Boettcher and Percus [4], an accessible
review by Boettcher [3], and a focused study on the Spin Glass problem by
Boettcher and Percus [6].
4.3.9 Bibliography
[1] P. Bak and K. Sneppen. Punctuated equilibrium and criticality in a
simple model of evolution. Physical Review Letters, 71:4083–4086, 1993.
[2] P. Bak, C. Tang, and K. Wiesenfeld. Self-organized criticality: An
explanation of the 1/f noise. Physical Review Letters, 59:381–384, 1987.
[3] S. Boettcher. Extremal optimization: heuristics via coevolutionary
avalanches. Computing in Science & Engineering, 2(6):75–82, 2000.
[4] S. Boettcher and A. Percus. Natures way of optimizing. Artificial
Intelligence, 119(1-2):275–286, 2000.
4.3. Extremal Optimization 181
[5] S. Boettcher and A. G. Percus. Extremal optimization: Methods derived
from co-evolution. In Proceedings of the Genetic and Evolutionary
Computation Conference, 1999.
[6] S. Boettcher and A. G. Percus. Optimization with extremal dynamics.
Phys. Rev. Lett., 86:5211–5214, 2001.
182 Chapter 4. Physical Algorithms
4.4 Harmony Search
Harmony Search, HS.
4.4.1 Taxonomy
Harmony Search belongs to the fields of Computational Intelligence and
Metaheuristics.
4.4.2 Inspiration
Harmony Search was inspired by the improvisation of Jazz musicians. Specif-
ically, the process by which the musicians (who may have never played
together before) rapidly refine their individual improvisation through varia-
tion resulting in an aesthetic harmony.
4.4.3 Metaphor
Each musician corresponds to an attribute in a candidate solution from a
problem domain, and each instrument’s pitch and range corresponds to the
bounds and constraints on the decision variable. The harmony between the
musicians is taken as a complete candidate solution at a given time, and
the audiences aesthetic appreciation of the harmony represent the problem
specific cost function. The musicians seek harmony over time through small
variations and improvisations, which results in an improvement against the
cost function.
4.4.4 Strategy
The information processing objective of the technique is to use good candi-
date solutions already discovered to influence the creation of new candidate
solutions toward locating the problems optima. This is achieved by stochas-
tically creating candidate solutions in a step-wise manner, where each
component is either drawn randomly from a memory of high-quality so-
lutions, adjusted from the memory of high-quality solutions, or assigned
randomly within the bounds of the problem. The memory of candidate
solutions is initially random, and a greedy acceptance criteria is used to
admit new candidate solutions only if they have an improved objective value,
replacing an existing member.
4.4.5 Procedure
Algorithm 4.4.1 provides a pseudocode listing of the Harmony Search algo-
rithm for minimizing a cost function. The adjustment of a pitch selected
4.4. Harmony Search 183
from the harmony memory is typically linear, for example for continuous
function optimization:
x′ ← x+ range× ǫ (4.1)
where range is a the user parameter (pitch bandwidth) to control the size
of the changes, and ǫ is a uniformly random number ∈ [−1, 1].
Algorithm 4.4.1: Pseudocode for Harmony Search.
Input: Pitchnum, Pitchbounds, Memorysize, Consolidationrate,
PitchAdjustrate, Improvisationmax
Output: Harmonybest
Harmonies ← InitializeHarmonyMemory(Pitchnum, Pitchbounds,1
Memorysize);
EvaluateHarmonies(Harmonies);2
for i to Improvisationmax do3
Harmony ← ∅;4
foreach Pitchi ∈ Pitchnum do5
if Rand() ≤ Consolidationrate then6
RandomHarmonyipitch ←7
SelectRandomHarmonyPitch(Harmonies, Pitchi);
if Rand() ≤ PitchAdjustrate then8
Harmonyipitch ←9
AdjustPitch(RandomHarmonyipitch);
else10
Harmonyipitch ← RandomHarmony
i
pitch;11
end12
else13
Harmonyipitch ← RandomPitch(Pitchbounds);14
end15
end16
EvaluateHarmonies(Harmony);17
if Cost(Harmony) ≤ Cost(Worst(Harmonies)) then18
Worst(Harmonies) ← Harmony;19
end20
end21
return Harmonybest;22
4.4.6 Heuristics
� Harmony Search was designed as a generalized optimization method
for continuous, discrete, and constrained optimization and has been
applied to numerous types of optimization problems.
184 Chapter 4. Physical Algorithms
� The harmony memory considering rate (HMCR) ∈ [0, 1] controls the
use of information from the harmony memory or the generation of
a random pitch. As such, it controls the rate of convergence of the
algorithm and is typically configured ∈ [0.7, 0.95].
� The pitch adjustment rate (PAR) ∈ [0, 1] controls the frequency of
adjustment of pitches selected from harmony memory, typically config-
ured ∈ [0.1, 0.5]. High values can result in the premature convergence
of the search.
� The pitch adjustment rate and the adjustment method (amount of
adjustment or fret width) are typically fixed, having a linear effect
through time. Non-linear methods have been considered, for example
refer to Geem [4].
� When creating a new harmony, aggregations of pitches can be taken
from across musicians in the harmony memory.
� The harmony memory update is typically a greedy process, although
other considerations such as diversity may be used where the most
similar harmony is replaced.
4.4.7 Code Listing
Listing 4.3 provides an example of the HarmonySearch algorithm imple-
mented in the Ruby Programming Language. The demonstration problem is
an instance of a continuous function optimization that seeks minf(x) where
f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and n = 3. The optimal solution for this
basin function is (v0, . . . , vn−1) = 0.0. The algorithm implementation and
parameterization are based on the description by Yang [7], with refinement
from Geem [4].
1 def objective_function(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def rand_in_bounds(min, max)
6 return min + ((max-min) * rand())
7 end
8
9 def random_vector(search_space)
10 return Array.new(search_space.size) do |i|
11 rand_in_bounds(search_space[i][0], search_space[i][1])
12 end
13 end
14
15 def create_random_harmony(search_space)
16 harmony = {}
17 harmony[:vector] = random_vector(search_space)
18 harmony[:fitness] = objective_function(harmony[:vector])
4.4. Harmony Search 185
19 return harmony
20 end
21
22 def initialize_harmony_memory(search_space, mem_size, factor=3)
23 memory = Array.new(mem_size*factor){create_random_harmony(search_space)}
24 memory.sort!{|x,y| x[:fitness]<=>y[:fitness]}
25 return memory.first(mem_size)
26 end
27
28 def create_harmony(search_space, memory, consid_rate, adjust_rate, range)
29 vector = Array.new(search_space.size)
30 search_space.size.times do |i|
31 if rand() < consid_rate
32 value = memory[rand(memory.size)][:vector][i]
33 value = value + range*rand_in_bounds(-1.0, 1.0) if rand()<adjust_rate
34 value = search_space[i][0] if value < search_space[i][0]
35 value = search_space[i][1] if value > search_space[i][1]
36 vector[i] = value
37 else
38 vector[i] = rand_in_bounds(search_space[i][0], search_space[i][1])
39 end
40 end
41 return {:vector=>vector}
42 end
43
44 def search(bounds, max_iter, mem_size, consid_rate, adjust_rate, range)
45 memory = initialize_harmony_memory(bounds, mem_size)
46 best = memory.first
47 max_iter.times do |iter|
48 harm = create_harmony(bounds, memory, consid_rate, adjust_rate, range)
49 harm[:fitness] = objective_function(harm[:vector])
50 best = harm if harm[:fitness] < best[:fitness]
51 memory << harm
52 memory.sort!{|x,y| x[:fitness]<=>y[:fitness]}
53 memory.delete_at(memory.size-1)
54 puts " > iteration=#{iter}, fitness=#{best[:fitness]}"
55 end
56 return best
57 end
58
59 if __FILE__ == $0
60 # problem configuration
61 problem_size = 3
62 bounds = Array.new(problem_size) {|i| [-5, 5]}
63 # algorithm configuration
64 mem_size = 20
65 consid_rate = 0.95
66 adjust_rate = 0.7
67 range = 0.05
68 max_iter = 500
69 # execute the algorithm
70 best = search(bounds, max_iter, mem_size, consid_rate, adjust_rate, range)
71 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:vector].inspect}"
72 end
Listing 4.3: Harmony Search in Ruby
186 Chapter 4. Physical Algorithms
4.4.8 References
Primary Sources
Geem et al. proposed the Harmony Search algorithm in 2001, which was
applied to a range of optimization problems including a constraint optimiza-
tion, the Traveling Salesman problem, and the design of a water supply
network [6].
Learn More
A book on Harmony Search, edited by Geem provides a collection of papers
on the technique and its applications [2], chapter 1 provides a useful summary
of the method heuristics for its configuration [7]. Similarly a second edited
volume by Geem focuses on studies that provide more advanced applications
of the approach [5], and chapter 1 provides a detailed walkthrough of the
technique itself [4]. Geem also provides a treatment of Harmony Search
applied to the optimal design of water distribution networks [3] and edits
yet a third volume on papers related to the application of the technique to
structural design optimization problems [1].
4.4.9 Bibliography
[1] Z. W. Geem, editor. Harmony Search Algorithms for Structural Design
Optimization. Springer, 2009.
[2] Z. W. Geem, editor. Music-Inspired Harmony Search Algorithm: Theory
and Applications. Springer, 2009.
[3] Z. W. Geem. Optimal Design of Water Distribution Networks Using
Harmony Search. Lap Lambert Academic Publishing, 2009.
[4] Z. W. Geem. Recent Advances In Harmony Search Algorithms, chapter
State-of-the-Art in the Structure of Harmony Search Algorithm, pages
1–10. Springer, 2010.
[5] Z. W. Geem, editor. Recent Advances in Harmony Search Algorithms.
Springer, 2010.
[6] Z. W. Geem, J. H. Kim, and G. V. Loganathan. A new heuristic
optimization algorithm: Harmony search. Simulation, 76:60–68, 2001.
[7] X-S. Yang. Music-Inspired Harmony Search Algorithm: Theory and
Applications, chapter Harmony Search as a Metaheuristic, pages 1–14.
Springer, 2009.
4.5. Cultural Algorithm 187
4.5 Cultural Algorithm
Cultural Algorithm, CA.
4.5.1 Taxonomy
The Cultural Algorithm is an extension to the field of Evolutionary Computa-
tion and may be considered a Meta-Evolutionary Algorithm. It more broadly
belongs to the field of Computational Intelligence and Metaheuristics. It is
related to other high-order extensions of Evolutionary Computation such as
the Memetic Algorithm (Section 4.6).
4.5.2 Inspiration
The Cultural Algorithm is inspired by the principle of cultural evolution.
Culture includes the habits, knowledge, beliefs, customs, and morals of a
member of society. Culture does not exist independent of the environment,
and can interact with the environment via positive or negative feedback
cycles. The study of the interaction of culture in the environment is referred
to as Cultural Ecology.
4.5.3 Metaphor
The Cultural Algorithm may be explained in the context of the inspiring
system. As the evolutionary process unfolds, individuals accumulate infor-
mation about the world which is communicated to other individuals in the
population. Collectively this corpus of information is a knowledge base that
members of the population may tap-into and exploit. Positive feedback
mechanisms can occur where cultural knowledge indicates useful areas of
the environment, information which is passed down between generations,
exploited, refined, and adapted as situations change. Additionally, areas of
potential hazard may also be communicated through the cultural knowledge
base.
4.5.4 Strategy
The information processing objective of the algorithm is to improve the
learning or convergence of an embedded search technique (typically an
evolutionary algorithm) using a higher-order cultural evolution. The algo-
rithm operates at two levels: a population level and a cultural level. The
population level is like an evolutionary search, where individuals repre-
sent candidate solutions, are mostly distinct and their characteristics are
translated into an objective or cost function in the problem domain. The
second level is the knowledge or believe space where information acquired
by generations is stored, and which is accessible to the current generation.
188 Chapter 4. Physical Algorithms
A communication protocol is used to allow the two spaces to interact and
the types of information that can be exchanged.
4.5.5 Procedure
The focus of the algorithm is the KnowledgeBase data structure that records
different knowledge types based on the nature of the problem. For example,
the structure may be used to record the best candidate solution found as well
as generalized information about areas of the search space that are expected
to payoff (result in good candidate solutions). This cultural knowledge is
discovered by the population-based evolutionary search, and is in turn used
to influence subsequent generations. The acceptance function constrain the
communication of knowledge from the population to the knowledge base.
Algorithm 4.5.1 provides a pseudocode listing of the Cultural Algorithm.
The algorithm is abstract, providing flexibility in the interpretation of
the processes such as the acceptance of information, the structure of the
knowledge base, and the specific embedded evolutionary algorithm.
Algorithm 4.5.1: Pseudocode for the CulturalAlgorithm.
Input: Problemsize, Populationnum
Output: KnowledgeBase
Population ← InitializePopulation(Problemsize,1
Populationnum);
KnowledgeBase ← InitializeKnowledgebase(Problemsize,2
Populationnum);
while ¬StopCondition() do3
Evaluate(Population);4
SituationalKnowledgecandidate ←5
AcceptSituationalKnowledge(Population);
UpdateSituationalKnowledge(KnowledgeBase,6
SituationalKnowledgecandidate);
Children ← ReproduceWithInfluence(Population,7
KnowledgeBase);
Population ← Select(Children, Population);8
NormativeKnowledgecandidate ←9
AcceptNormativeKnowledge(Population);
UpdateNormativeKnowledge(KnowledgeBase,10
NormativeKnowledgecandidate);
end11
return KnowledgeBase;12
4.5. Cultural Algorithm 189
4.5.6 Heuristics
� The Cultural Algorithm was initially used as a simulation tool to
investigate Cultural Ecology. It has been adapted for use as an
optimization algorithm for a wide variety of domains not-limited to
constraint optimization, combinatorial optimization, and continuous
function optimization.
� The knowledge base structure provides a mechanism for incorporating
problem-specific information into the execution of an evolutionary
search.
� The acceptance functions that control the flow of information into
the knowledge base are typically greedy, only including the best
information from the current generation, and not replacing existing
knowledge unless it is an improvement.
� Acceptance functions are traditionally deterministic, although proba-
bilistic and fuzzy acceptance functions have been investigated.
4.5.7 Code Listing
Listing 4.4 provides an example of the Cultural Algorithm implemented
in the Ruby Programming Language. The demonstration problem is an
instance of a continuous function optimization that seeks min f(x) where
f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and n = 2. The optimal solution for this
basin function is (v0, . . . , vn−1) = 0.0.
The Cultural Algorithm was implemented based on the description of the
Cultural Algorithm Evolutionary Program (CAEP) presented by Reynolds
[4]. A real-valued Genetic Algorithm was used as the embedded evolutionary
algorithm. The overall best solution is taken as the ‘situational’ cultural
knowledge, whereas the bounds of the top 20% of the best solutions each
generation are taken as the ‘normative’ cultural knowledge. The situational
knowledge is returned as the result of the search, whereas the normative
knowledge is used to influence the evolutionary process. Specifically, vector
bounds in the normative knowledge are used to define a subspace from which
new candidate solutions are uniformly sampled during the reproduction
step of the evolutionary algorithm’s variation mechanism. A real-valued
representation and a binary tournament selection strategy are used by the
evolutionary algorithm.
1 def objective_function(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def rand_in_bounds(min, max)
6 return min + ((max-min) * rand())
7 end
190 Chapter 4. Physical Algorithms
8
9 def random_vector(minmax)
10 return Array.new(minmax.size) do |i|
11 rand_in_bounds(minmax[i][0], minmax[i][1])
12 end
13 end
14
15 def mutate_with_inf(candidate, beliefs, minmax)
16 v = Array.new(candidate[:vector].size)
17 candidate[:vector].each_with_index do |c,i|
18 v[i]=rand_in_bounds(beliefs[:normative][i][0],beliefs[:normative][i][1])
19 v[i] = minmax[i][0] if v[i] < minmax[i][0]
20 v[i] = minmax[i][1] if v[i] > minmax[i][1]
21 end
22 return {:vector=>v}
23 end
24
25 def binary_tournament(pop)
26 i, j = rand(pop.size), rand(pop.size)
27 j = rand(pop.size) while j==i
28 return (pop[i][:fitness] < pop[j][:fitness]) ? pop[i] : pop[j]
29 end
30
31 def initialize_beliefspace(search_space)
32 belief_space = {}
33 belief_space[:situational] = nil
34 belief_space[:normative] = Array.new(search_space.size) do |i|
35 Array.new(search_space[i])
36 end
37 return belief_space
38 end
39
40 def update_beliefspace_situational!(belief_space, best)
41 curr_best = belief_space[:situational]
42 if curr_best.nil? or best[:fitness] < curr_best[:fitness]
43 belief_space[:situational] = best
44 end
45 end
46
47 def update_beliefspace_normative!(belief_space, acc)
48 belief_space[:normative].each_with_index do |bounds,i|
49 bounds[0] = acc.min{|x,y| x[:vector][i]<=>y[:vector][i]}[:vector][i]
50 bounds[1] = acc.max{|x,y| x[:vector][i]<=>y[:vector][i]}[:vector][i]
51 end
52 end
53
54 def search(max_gens, search_space, pop_size, num_accepted)
55 # initialize
56 pop = Array.new(pop_size) { {:vector=>random_vector(search_space)} }
57 belief_space = initialize_beliefspace(search_space)
58 # evaluate
59 pop.each{|c| c[:fitness] = objective_function(c[:vector])}
60 best = pop.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
61 # update situational knowledge
62 update_beliefspace_situational!(belief_space, best)
63 max_gens.times do |gen|
4.5. Cultural Algorithm 191
64 # create next generation
65 children = Array.new(pop_size) do |i|
66 mutate_with_inf(pop[i], belief_space, search_space)
67 end
68 # evaluate
69 children.each{|c| c[:fitness] = objective_function(c[:vector])}
70 best = children.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
71 # update situational knowledge
72 update_beliefspace_situational!(belief_space, best)
73 # select next generation
74 pop = Array.new(pop_size) { binary_tournament(children + pop) }
75 # update normative knowledge
76 pop.sort!{|x,y| x[:fitness] <=> y[:fitness]}
77 acccepted = pop[0...num_accepted]
78 update_beliefspace_normative!(belief_space, acccepted)
79 # user feedback
80 puts " > generation=#{gen}, f=#{belief_space[:situational][:fitness]}"
81 end
82 return belief_space[:situational]
83 end
84
85 if __FILE__ == $0
86 # problem configuration
87 problem_size = 2
88 search_space = Array.new(problem_size) {|i| [-5, +5]}
89 # algorithm configuration
90 max_gens = 200
91 pop_size = 100
92 num_accepted = (pop_size*0.20).round
93 # execute the algorithm
94 best = search(max_gens, search_space, pop_size, num_accepted)
95 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:vector].inspect}"
96 end
Listing 4.4: Cultural Algorithm in Ruby
4.5.8 References
Primary Sources
The Cultural Algorithm was proposed by Reynolds in 1994 that combined
the method with the Version Space Algorithm (a binary string based Genetic
Algorithm), where generalizations of individual solutions were communicated
as cultural knowledge in the form of schema patterns (strings of 1’s, 0’s and
#’s, where ‘#’ represents a wildcard) [3].
Learn More
Chung and Reynolds provide a study of the Cultural Algorithm on a
testbed of constraint satisfaction problems [1]. Reynolds provides a detailed
overview of the history of the technique as a book chapter that presents
the state of the art and summaries of application areas including concept
192 Chapter 4. Physical Algorithms
learning and continuous function optimization [4]. Coello Coello and Becerra
proposed a variation of the Cultural Algorithm that uses Evolutionary
Programming as the embedded weak search method, for use with Multi-
Objective Optimization problems [2].
4.5.9 Bibliography
[1] C.-J. Chung and R. G. Reynolds. A testbed for solving optimization
problems using cultural algorithms. In L. J. Fogel, P. J. Angeline, and
T. Bäck, editors, Evolutionary Programming V: Proceedings of the Fifth
Annual Conference on Evolutionary Programming, pages 225–236, 1996.
[2] C. A. Coello Coello and R. L. Becerra. Evolutionary multiobjective
optimization using a cultural algorithm. In Proceedings of the 2003
IEEE Swarm Intelligence Symposium, pages 6–13. IEEE Press, 2003.
[3] R. G. Reynolds. An introduction to cultural algorithms. In Proceedings
of the 3rd Annual Conference on Evolutionary Programming, pages
131–139. World Scienfific Publishing, 1994.
[4] R. G. Reynolds. New Ideas in Optimization, chapter Cultural Algorithms:
Theory and Applications, pages 367–378. McGraw-Hill Ltd., 1999.
4.6. Memetic Algorithm193
4.6 Memetic Algorithm
Memetic Algorithm, MA.
4.6.1 Taxonomy
Memetic Algorithms have elements of Metaheuristics and Computational
Intelligence. Although they have principles of Evolutionary Algorithms, they
may not strictly be considered an Evolutionary Technique. Memetic Algo-
rithms have functional similarities to Baldwinian Evolutionary Algorithms,
Lamarckian Evolutionary Algorithms, Hybrid Evolutionary Algorithms, and
Cultural Algorithms (Section 4.5). Using ideas of memes and Memetic
Algorithms in optimization may be referred to as Memetic Computing.
4.6.2 Inspiration
Memetic Algorithms are inspired by the interplay of genetic evolution and
memetic evolution. Universal Darwinism is the generalization of genes
beyond biological-based systems to any system where discrete units of
information can be inherited and be subjected to evolutionary forces of
selection and variation. The term ‘meme’ is used to refer to a piece of
discrete cultural information, suggesting at the interplay of genetic and
cultural evolution.
4.6.3 Metaphor
The genotype is evolved based on the interaction the phenotype has with
the environment. This interaction is metered by cultural phenomena that
influence the selection mechanisms, and even the pairing and recombination
mechanisms. Cultural information is shared between individuals, spreading
through the population as memes relative to their fitness or fitness the memes
impart to the individuals. Collectively, the interplay of the geneotype and
the memeotype strengthen the fitness of population in the environment.
4.6.4 Strategy
The objective of the information processing strategy is to exploit a popu-
lation based global search technique to broadly locate good areas of the
search space, combined with the repeated usage of a local search heuristic
by individual solutions to locate local optimum. Ideally, memetic algo-
rithms embrace the duality of genetic and cultural evolution, allowing the
transmission, selection, inheritance, and variation of memes as well as genes.
194 Chapter 4. Physical Algorithms
4.6.5 Procedure
Algorithm 4.6.1 provides a pseudocode listing of the Memetic Algorithm for
minimizing a cost function. The procedure describes a simple or first order
Memetic Algorithm that shows the improvement of individual solutions
separate from a global search, although does not show the independent
evolution of memes.
Algorithm 4.6.1: Pseudocode for the Memetic Algorithm.
Input: ProblemSize, Popsize, MemePopsize
Output: Sbest
Population ← InitializePopulation(ProblemSize, Popsize);1
while ¬StopCondition() do2
foreach Si ∈ Population do3
Sicost ← Cost(Si);4
end5
Sbest ← GetBestSolution(Population);6
Population ← StochasticGlobalSearch(Population);7
MemeticPopulation ← SelectMemeticPopulation(Population,8
MemePopsize);
foreach Si ∈ MemeticPopulation do9
Si ← LocalSearch(Si);10
end11
end12
return Sbest;13
4.6.6 Heuristics
� The global search provides the broad exploration mechanism, whereas
the individual solution improvement via local search provides an
exploitation mechanism.
� Balance is needed between the local and global mechanisms to ensure
the system does not prematurely converge to a local optimum and
does not consume unnecessary computational resources.
� The local search should be problem and representation specific, where
as the global search may be generic and non-specific (black-box).
� Memetic Algorithms have been applied to a range of constraint, com-
binatorial, and continuous function optimization problem domains.
4.6. Memetic Algorithm 195
4.6.7 Code Listing
Listing 4.5 provides an example of the Memetic Algorithm implemented
in the Ruby Programming Language. The demonstration problem is an
instance of a continuous function optimization that seeks min f(x) where
f =
∑n
i=1 x
2
i , −5.0 ≤ xi ≤ 5.0 and n = 3. The optimal solution for this
basin function is (v0, . . . , vn−1) = 0.0. The Memetic Algorithm uses a
canonical Genetic Algorithm as the global search technique that operates
on binary strings, uses tournament selection, point mutations, uniform
crossover and a binary coded decimal decoding of bits to real values. A
bit climber local search is used that performs probabilistic bit flips (point
mutations) and only accepts solutions with the same or improving fitness.
1 def objective_function(vector)
2 return vector.inject(0.0) {|sum, x| sum + (x ** 2.0)}
3 end
4
5 def random_bitstring(num_bits)
6 return (0...num_bits).inject(""){|s,i| s<<((rand<0.5) ? "1" : "0")}
7 end
8
9 def decode(bitstring, search_space, bits_per_param)
10 vector = []
11 search_space.each_with_index do |bounds, i|
12 off, sum = i*bits_per_param, 0.0
13 param = bitstring[off...(off+bits_per_param)].reverse
14 param.size.times do |j|
15 sum += ((param[j].chr=='1') ? 1.0 : 0.0) * (2.0 ** j.to_f)
16 end
17 min, max = bounds
18 vector << min + ((max-min)/((2.0**bits_per_param.to_f)-1.0)) * sum
19 end
20 return vector
21 end
22
23 def fitness(candidate, search_space, param_bits)
24 candidate[:vector]=decode(candidate[:bitstring], search_space, param_bits)
25 candidate[:fitness] = objective_function(candidate[:vector])
26 end
27
28 def binary_tournament(pop)
29 i, j = rand(pop.size), rand(pop.size)
30 j = rand(pop.size) while j==i
31 return (pop[i][:fitness] < pop[j][:fitness]) ? pop[i] : pop[j]
32 end
33
34 def point_mutation(bitstring, rate=1.0/bitstring.size)
35 child = ""
36 bitstring.size.times do |i|
37 bit = bitstring[i].chr
38 child << ((rand()<rate) ? ((bit=='1') ? "0" : "1") : bit)
39 end
40 return child
41 end
196 Chapter 4. Physical Algorithms
42
43 def crossover(parent1, parent2, rate)
44 return ""+parent1 if rand()>=rate
45 child = ""
46 parent1.size.times do |i|
47 child << ((rand()<0.5) ? parent1[i].chr : parent2[i].chr)
48 end
49 return child
50 end
51
52 def reproduce(selected, pop_size, p_cross, p_mut)
53 children = []
54 selected.each_with_index do |p1, i|
55 p2 = (i.modulo(2)==0) ? selected[i+1] : selected[i-1]
56 p2 = selected[0] if i == selected.size-1
57 child = {}
58 child[:bitstring] = crossover(p1[:bitstring], p2[:bitstring], p_cross)
59 child[:bitstring] = point_mutation(child[:bitstring], p_mut)
60 children << child
61 break if children.size >= pop_size
62 end
63 return children
64 end
65
66 def bitclimber(child, search_space, p_mut, max_local_gens, bits_per_param)
67 current = child
68 max_local_gens.times do
69 candidate = {}
70 candidate[:bitstring] = point_mutation(current[:bitstring], p_mut)
71 fitness(candidate, search_space, bits_per_param)
72 current = candidate if candidate[:fitness] <= current[:fitness]
73 end
74 return current
75 end
76
77 def search(max_gens, search_space, pop_size, p_cross, p_mut,
max_local_gens, p_local, bits_per_param=16)
78 pop = Array.new(pop_size) do |i|
79 {:bitstring=>random_bitstring(search_space.size*bits_per_param)}
80 end
81 pop.each{|candidate| fitness(candidate, search_space, bits_per_param) }
82 gen, best = 0, pop.sort{|x,y| x[:fitness] <=> y[:fitness]}.first
83 max_gens.times do |gen|
84 selected = Array.new(pop_size){|i| binary_tournament(pop)}
85 children = reproduce(selected, pop_size, p_cross, p_mut)
86 children.each{|cand| fitness(cand, search_space, bits_per_param)}
87 pop = []
88 children.each do |child|
89 if rand() < p_local
90 child = bitclimber(child, search_space, p_mut, max_local_gens,
bits_per_param)
91 end
92 pop << child
93 end
94 pop.sort!{|x,y| x[:fitness] <=> y[:fitness]}
95 best = pop.first if pop.first[:fitness] <= best[:fitness]
4.6. Memetic Algorithm 197
96 puts ">gen=#{gen}, f=#{best[:fitness]}, b=#{best[:bitstring]}"
97 end
98 return best
99 end
100
101 if __FILE__ == $0
102 # problem configuration
103 problem_size = 3
104 search_space = Array.new(problem_size) {|i| [-5, +5]}
105 # algorithm configuration
106 max_gens = 100
107 pop_size = 100
108 p_cross = 0.98
109 p_mut = 1.0/(problem_size*16).to_f
110 max_local_gens = 20
111 p_local= 0.5
112 # execute the algorithm
113 best = search(max_gens, search_space, pop_size, p_cross, p_mut,
max_local_gens, p_local)
114 puts "done! Solution: f=#{best[:fitness]}, b=#{best[:bitstring]},
v=#{best[:vector].inspect}"
115 end
Listing 4.5: Memetic Algorithm in Ruby
4.6.8 References
Primary Sources
The concept of a Memetic Algorithm is credited to Moscato [5], who was
inspired by the description of meme’s in Dawkins’ “The Selfish Gene” [1].
Moscato proposed Memetic Algorithms as the marriage between population
based global search and heuristic local search made by each individual with-
out the constraints of a genetic representation and investigated variations
on the Traveling Salesman Problem.
Learn More
Moscato and Cotta provide a gentle introduction to the field of Memetic
Algorithms as a book chapter that covers formal descriptions of the approach,
a summary of the fields of application, and the state of the art [6]. An
overview and classification of the types of Memetic Algorithms is presented
by Ong et al. who describe a class of adaptive Memetic Algorithms [7].
Krasnogor and Smith also provide a taxonomy of Memetic Algorithms,
focusing on the properties needed to design ‘competent’ implementations
of the approach with examples on a number of combinatorial optimization
problems [4]. Work by Krasnogor and Gustafson investigate what they refer
to as ‘self-generating’ Memetic Algorithms that use the memetic principle to
co-evolve the local search applied by individual solutions [3]. For a broader
198 Chapter 4. Physical Algorithms
overview of the field, see the 2005 book “Recent Advances in Memetic
Algorithms” that provides an overview and a number of studies [2].
4.6.9 Bibliography
[1] R. Dawkins. The selfish gene. Oxford University Press, 1976.
[2] W. E. Hart, N. Krasnogor, and J. E. Smith. Recent Advances in Memetic
Algorithms. Springer, 2005.
[3] N. Krasnogor and S. Gustafson. A study on the use of “self-generation”
in memetic algorithms. Natural Computing, 3(1):53–76, 2004.
[4] N. Krasnogor and J. Smith. A tutorial for competent memetic algorithms:
Model, taxonomy and design issues. IEEE Transactions on Evolutionary
Computation, 9(5):474–488, 2005.
[5] P. Moscato. On evolution, search, optimization, genetic algorithms and
martial arts: Towards memetic algorithms. Technical report, California
Institute of Technology, 1989.
[6] P. Moscato and C. Cotta. Handbook of Metaheuristics, chapter A gentle
introduction to memetic algorithms, pages 105–144. Kluwer Academic
Publishers, 2003.
[7] Y-S. Ong, M-H. Lim, N. Zhu, and K-W. Wong. Classification of adaptive
memetic algorithms: A comparative study. IEEE Transactions on
Systems, Man, and Cybernetics-Part B: Cybernetics, 36(1):141–152,
2006.
Chapter 5
Probabilistic Algorithms
5.1 Overview
This chapter describes Probabilistic Algorithms
5.1.1 Probabilistic Models
Probabilistic Algorithms are those algorithms that model a problem or
search a problem space using an probabilistic model of candidate solutions.
Many Metaheuristics and Computational Intelligence algorithms may be
considered probabilistic, although the difference with algorithms is the
explicit (rather than implicit) use of the tools of probability in problem
solving. The majority of the algorithms described in this Chapter are
referred to as Estimation of Distribution Algorithms.
5.1.2 Estimation of Distribution Algorithms
Estimation of Distribution Algorithms (EDA) also called Probabilistic
Model-Building Genetic Algorithms (PMBGA) are an extension of the
field of Evolutionary Computation that model a population of candidate
solutions as a probabilistic model. They generally involve iterations that
alternate between creating candidate solutions in the problem space from
a probabilistic model, and reducing a collection of generated candidate
solutions into a probabilistic model.
The model at the heart of an EDA typically provides the probabilistic
expectation of a component or component configuration comprising part
of an optimal solution. This estimation is typically based on the observed
frequency of use of the component in better than average candidate solutions.
The probabilistic model is used to generate candidate solutions in the
problem space, typically in a component-wise or step-wise manner using a
domain specific construction method to ensure validity.
199
200 Chapter 5. Probabilistic Algorithms
Pelikan et al. provide a comprehensive summary of the field of prob-
abilistic optimization algorithms, summarizing the core approaches and
their differences [10]. The edited volume by Pelikan, Sastry, and Cantu-Paz
provides a collection of studies on the popular Estimation of Distribution
algorithms as well as methodology for designing algorithms and applica-
tion demonstration studies [13]. An edited volume on studies of EDAs by
Larranaga and Lozano [4] and the follow-up volume by Lozano et al. [5]
provide an applied foundation for the field.
5.1.3 Extensions
There are many other algorithms and classes of algorithm that were not
described from the field of Estimation of Distribution Algorithm, not limited
to:
� Extensions to UMDA: Extensions to the Univariate Marginal Dis-
tribution Algorithm such as the Bivariate Marginal Distribution Al-
gorithm (BMDA) [11, 12] and the Factorized Distribution Algorithm
(FDA) [7].
� Extensions to cGA: Extensions to the Compact Genetic Algorithm
such as the Extended Compact Genetic Algorithm (ECGA) [2, 3].
� Extensions to BOA: Extensions to the Bayesian Optimization Al-
gorithm such as the Hierarchal Bayesian Optimization Algorithm
(hBOA) [8, 9] and the Incremental Bayesian Optimization Algorithm
(iBOA) [14].
� Bayesian Network Algorithms: Other Bayesian network algo-
rithms such as The Estimation of Bayesian Network Algorithm [1],
and the Learning Factorized Distribution Algorithm (LFDA) [6].
� PIPE: The Probabilistic Incremental Program Evolution that uses
EDA methods for constructing programs [16].
� SHCLVND: The Stochastic Hill-Climbing with Learning by Vectors
of Normal Distributions algorithm [15].
5.1.4 Bibliography
[1] R. Etxeberria and P. Larranaga. Global optimization using bayesian
networks. In Proceedings of the Second Symposium on Artificial Intelli-
gence (CIMAF-99), pages 151–173, 1999.
[2] G. R. Harik. Linkage learning via probabilistic modeling in the extended
compact genetic algorithm (ECGA). Technical Report 99010, Illinois
Genetic Algorithms Laboratory, Department of General Engineering,
University of Illinois, 1999.
5.1. Overview 201
[3] G. R. Harik, F. G. Lobo, and K. Sastry. Scalable Optimization via
Probabilistic Modeling, chapter Linkage Learning via Probabilistic Mod-
eling in the Extended Compact Genetic Algorithm (ECGA), pages
39–61. Springer, 2006.
[4] P. Larranaga and J. A. Lozano. Estimation of distribution algorithms:
A new tool for evolutionary computation. Springer, 2002.
[5] J. A. Lozano, P. Larranaga, I. Inza, and E. Bengoetxea. Towards a
new evolutionary computation. Advances in estimation of distribution
algorithms. Springer, 2006.
[6] H. Mühlenbein and T. Mahnig. FDA–a scalable evolutionary algo-
rithm for the optimization of additively decomposed discrete functions.
Evolutionary Compilation, 7(4):353–376, 1999.
[7] H. Mühlenbein, T. Mahnig, and A. O. Rodriguez. Schemata, distribu-
tions and graphical models in evolutionary optimization. Journal of
Heuristics, 5(2):215–247, 1999.
[8] M. Pelikan and D. E. Goldberg. Hierarchical problem solving and
the bayesian optimization algorithms. In Genetic and Evolutionary
Computation Conference 2000 (GECCO-2000), pages 267–274, 2000.
[9] M. Pelikan and D. E. Goldberg. Escaping hierarchical traps with
competent genetic algorithms. In Proceedings of the Genetic and
Evolutionary Computation Conference (GECCO-2001), number 511–
518, 2001.
[10] M. Pelikan, D. E. Goldberg, and F. G. Lobo. A survey of optimization
by building and using probabilistic models. Computational Optimization
and Applications, 21:5–20, 2002.[11] M. Pelikan and H. Mühlenbein. Marginal distributions in evolutionary
algorithms. In Proceedings of the International Conference on Genetic
Algorithms Mendel, 1998.
[12] M. Pelikan and H. Mühlenbein. Advances in Soft Computing: Engi-
neering Design and Manufacturing, chapter The Bivariate Marginal
Distribution Algorithm, pages 521–535. Springer, 1999.
[13] M. Pelikan, K. Sastry, and E. Cantú-Paz, editors. Scalable Optimization
via Probabilistic Modeling: From Algorithms to Applications. Springer,
2006.
[14] M. Pelikan, K. Sastry, and D. E. Goldberg. iBOA: The incremental
bayesian optimization algorithms. In Proceedings of the Genetic and
Evolutionary Computation Conference (GECCO-2008), pages 455–462,
2008.
202 Chapter 5. Probabilistic Algorithms
[15] S. Rudlof and M. Koppen. Stochastic hill climbing with learning by
vectors of normal distributions. In First On-line Workshop on Soft
Computing, Nagoya, Japan, 1996.
[16] R. Salustowicz and J. Schmidhuber. Probabilistic incremental program
evolution: Stochastic search through program space. In Proceedings
of the 9th European Conference on Machine Learning Prague, pages
213–220, 1997.
5.2. Population-Based Incremental Learning 203
5.2 Population-Based Incremental Learning
Population-Based Incremental Learning, PBIL.
5.2.1 Taxonomy
Population-Based Incremental Learning is an Estimation of Distribution
Algorithm (EDA), also referred to as Population Model-Building Genetic
Algorithms (PMBGA) an extension to the field of Evolutionary Computation.
PBIL is related to other EDAs such as the Compact Genetic Algorithm
(Section 5.4), the Probabilistic Incremental Programing Evolution Algorithm,
and the Bayesian Optimization Algorithm (Section 5.5). The fact the the
algorithm maintains a single prototype vector that is updated competitively
shows some relationship to the Learning Vector Quantization algorithm
(Section 8.5).
5.2.2 Inspiration
Population-Based Incremental Learning is a population-based technique
without an inspiration. It is related to the Genetic Algorithm and other Evo-
lutionary Algorithms that are inspired by the biological theory of evolution
by means of natural selection.
5.2.3 Strategy
The information processing objective of the PBIL algorithm is to reduce the
memory required by the genetic algorithm. This is done by reducing the
population of a candidate solutions to a single prototype vector of attributes
from which candidate solutions can be generated and assessed. Updates
and mutation operators are also performed to the prototype vector, rather
than the generated candidate solutions.
5.2.4 Procedure
The Population-Based Incremental Learning algorithm maintains a real-
valued prototype vector that represents the probability of each component
being expressed in a candidate solution. Algorithm 5.2.1 provides a pseu-
docode listing of the Population-Based Incremental Learning algorithm for
maximizing a cost function.
5.2.5 Heuristics
� PBIL was designed to optimize the probability of components from
low cardinality sets, such as bit’s in a binary string.
204 Chapter 5. Probabilistic Algorithms
Algorithm 5.2.1: Pseudocode for PBIL.
Input: Bitsnum, Samplesnum, Learnrate, Pmutation, Mutationfactor
Output: Sbest
V ← InitializeVector(Bitsnum);1
Sbest ← ∅;2
while ¬StopCondition() do3
Scurrent ← ∅;4
for i to Samplesnum do5
Si ← GenerateSamples(V );6
if Cost(Si) ≤ Cost(Scurrent) then7
Scurrent ← Si;8
if Cost(Si) ≤ Cost(Sbest) then9
Sbest ← Si;10
end11
end12
end13
foreach Sibit ∈ Scurrent do14
V ibit ← V
i
bit × (1.0 − Learnrate) + S
i
bit × Learnrate;15
if Rand() < Pmutation then16
V ibit ← V
i
bit × (1.0 − Mutationfactor) + Rand() ×17
Mutationfactor;
end18
end19
end20
return Sbest;21
� The algorithm has a very small memory footprint (compared to some
population-based evolutionary algorithms) given the compression of
information into a single prototype vector.
� Extensions to PBIL have been proposed that extend the representation
beyond sets to real-valued vectors.
� Variants of PBIL that were proposed in the original paper include up-
dating the prototype vector with more than one competitive candidate
solution (such as an average of top candidate solutions), and mov-
ing the prototype vector away from the least competitive candidate
solution each iteration.
� Low learning rates are preferred, such as 0.1.
5.2. Population-Based Incremental Learning 205
5.2.6 Code Listing
Listing 5.1 provides an example of the Population-Based Incremental Learn-
ing algorithm implemented in the Ruby Programming Language. The
demonstration problem is a maximizing binary optimization problem called
OneMax that seeks a binary string of unity (all ‘1’ bits). The objective
function only provides an indication of the number of correct bits in a
candidate string, not the positions of the correct bits. The algorithm is an
implementation of the simple PBIL algorithm that updates the prototype
vector based on the best candidate solution generated each iteration.
1 def onemax(vector)
2 return vector.inject(0){|sum, value| sum + value}
3 end
4
5 def generate_candidate(vector)
6 candidate = {}
7 candidate[:bitstring] = Array.new(vector.size)
8 vector.each_with_index do |p, i|
9 candidate[:bitstring][i] = (rand()<p) ? 1 : 0
10 end
11 return candidate
12 end
13
14 def update_vector(vector, current, lrate)
15 vector.each_with_index do |p, i|
16 vector[i] = p*(1.0-lrate) + current[:bitstring][i]*lrate
17 end
18 end
19
20 def mutate_vector(vector, current, coefficient, rate)
21 vector.each_with_index do |p, i|
22 if rand() < rate
23 vector[i] = p*(1.0-coefficient) + rand()*coefficient
24 end
25 end
26 end
27
28 def search(num_bits, max_iter, num_samples, p_mutate, mut_factor, l_rate)
29 vector = Array.new(num_bits){0.5}
30 best = nil
31 max_iter.times do |iter|
32 current = nil
33 num_samples.times do
34 candidate = generate_candidate(vector)
35 candidate[:cost] = onemax(candidate[:bitstring])
36 current = candidate if current.nil? or candidate[:cost]>current[:cost]
37 best = candidate if best.nil? or candidate[:cost]>best[:cost]
38 end
39 update_vector(vector, current, l_rate)
40 mutate_vector(vector, current, mut_factor, p_mutate)
41 puts " >iteration=#{iter}, f=#{best[:cost]}, s=#{best[:bitstring]}"
42 break if best[:cost] == num_bits
43 end
44 return best
206 Chapter 5. Probabilistic Algorithms
45 end
46
47 if __FILE__ == $0
48 # problem configuration
49 num_bits = 64
50 # algorithm configuration
51 max_iter = 100
52 num_samples = 100
53 p_mutate = 1.0/num_bits
54 mut_factor = 0.05
55 l_rate = 0.1
56 # execute the algorithm
57 best=search(num_bits, max_iter, num_samples, p_mutate, mut_factor, l_rate)
58 puts "done! Solution: f=#{best[:cost]}/#{num_bits}, s=#{best[:bitstring]}"
59 end
Listing 5.1: Population-Based Incremental Learning in Ruby
5.2.7 References
Primary Sources
The Population-Based Incremental Learning algorithm was proposed by
Baluja in a technical report that proposed the base algorithm as well as a
number of variants inspired by the Learning Vector Quantization algorithm
[1].
Learn More
Baluja and Caruana provide an excellent overview of PBIL and compare
it to the standard Genetic Algorithm, released as a technical report [3]
and later published [4]. Baluja provides a detailed comparison between
the Genetic algorithm and PBIL on a range of problems and scales in
another technical report [2]. Greene provided an excellent account on the
applicability of PBIL as a practical optimization algorithm [5]. Höhfeld and
Rudolph provide the first theoretical analysis of the technique and provide
a convergence proof [6].
5.2.8 Bibliography
[1] S. Baluja. Population-based incremental learning: A method for in-
tegrating genetic search based function optimization and competitive
learning. Technical Report CMU-CS-94-163, School of Computer Sci-
ence, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, June
1994.
[2] S. Baluja. An empiricalcomparison of seven iterative and evolutionary
function optimization heuristics. Technical Report CMU-CS-95-193,
5.2. Population-Based Incremental Learning 207
School of Computer Science Carnegie Mellon University, Pittsburgh,
Pennsylvania 15213, September 1995.
[3] S. Baluja and R. Caruana. Removing the genetics from the standard ge-
netic algorithm. Technical Report CMU-CS-95-141, School of Computer
Science Carnegie Mellon University, Pittsburgh, Pennsylvania 15213,
May 1995.
[4] S. Baluja and R. Caruana. Removing the genetics from the standard
genetic algorithm. In Proceedings of the International Conference on
Machine Learning, pages 36–46. Morgan Kaufmann, 1995.
[5] J. R. Greene. Population-based incremental learning as a simple versatile
tool for engineering optimization. In Proceedings of the First Interna-
tional Conference on Evolutionary Computation and Its Applications,
pages 258–269, 1996.
[6] M. Höhfeld and G. Rudolph. Towards a theory of population based incre-
mental learning. In Proceedings of the IEEE Conference on Evolutionary
Computation, pages 1–5. IEEE Press, 1997.
208 Chapter 5. Probabilistic Algorithms
5.3 Univariate Marginal Distribution Algorithm
Univariate Marginal Distribution Algorithm, UMDA, Univariate Marginal
Distribution, UMD.
5.3.1 Taxonomy
The Univariate Marginal Distribution Algorithm belongs to the field of Es-
timation of Distribution Algorithms (EDA), also referred to as Population
Model-Building Genetic Algorithms (PMBGA), an extension to the field of
Evolutionary Computation. UMDA is closely related to the Factorized Dis-
tribution Algorithm (FDA) and an extension called the Bivariate Marginal
Distribution Algorithm (BMDA). UMDA is related to other EDAs such as
the Compact Genetic Algorithm (Section 5.4), the Population-Based Incre-
mental Learning algorithm (Section 5.2), and the Bayesian Optimization
Algorithm (Section 5.5).
5.3.2 Inspiration
Univariate Marginal Distribution Algorithm is a population technique-
based without an inspiration. It is related to the Genetic Algorithm and
other Evolutionary Algorithms that are inspired by the biological theory of
evolution by means of natural selection.
5.3.3 Strategy
The information processing strategy of the algorithm is to use the frequency
of the components in a population of candidate solutions in the construction
of new candidate solutions. This is achieved by first measuring the frequency
of each component in the population (the univariate marginal probabil-
ity) and using the probabilities to influence the probabilistic selection of
components in the component-wise construction of new candidate solutions.
5.3.4 Procedure
Algorithm 5.3.1 provides a pseudocode listing of the Univariate Marginal
Distribution Algorithm for minimizing a cost function.
5.3.5 Heuristics
� UMDA was designed for problems where the components of a solution
are independent (linearly separable).
� A selection method is needed to identify the subset of good solutions
from which to calculate the univariate marginal probabilities. Many
5.3. Univariate Marginal Distribution Algorithm 209
Algorithm 5.3.1: Pseudocode for the UMDA.
Input: Bitsnum, Populationsize, Selectionsize
Output: Sbest
Population ← InitializePopulation(Bitsnum, Populationsize);1
EvaluatePopulation(Population);2
Sbest ← GetBestSolution(Population);3
while ¬StopCondition() do4
Selected ← SelectFitSolutions(Population, Selectionsize);5
V ← CalculateFrequencyOfComponents(Selected);6
Offspring ← ∅;7
for i to Populationsize do8
Offspring ← ProbabilisticallyConstructSolution(V );9
end10
EvaluatePopulation(Offspring);11
Sbest ← GetBestSolution(Offspring);12
Population ← Offspring;13
end14
return Sbest;15
selection methods from the field of Evolutionary Computation may
be used.
5.3.6 Code Listing
Listing 5.2 provides an example of the Univariate Marginal Distribution Algo-
rithm implemented in the Ruby Programming Language. The demonstration
problem is a maximizing binary optimization problem called OneMax that
seeks a binary string of unity (all ‘1’ bits). The objective function provides
only an indication of the number of correct bits in a candidate string, not
the positions of the correct bits.
The algorithm is an implementation of UMDA that uses the integers
1 and 0 to represent bits in a binary string representation. A binary
tournament selection strategy is used and the whole population is replaced
each iteration. The mechanisms from Evolutionary Computation such as
elitism and more elaborate selection methods may be implemented as an
extension.
1 def onemax(vector)
2 return vector.inject(0){|sum, value| sum + value}
3 end
4
5 def random_bitstring(size)
6 return Array.new(size){ ((rand()<0.5) ? 1 : 0) }
7 end
8
210 Chapter 5. Probabilistic Algorithms
9 def binary_tournament(pop)
10 i, j = rand(pop.size), rand(pop.size)
11 j = rand(pop.size) while j==i
12 return (pop[i][:fitness] > pop[j][:fitness]) ? pop[i] : pop[j]
13 end
14
15 def calculate_bit_probabilities(pop)
16 vector = Array.new(pop.first[:bitstring].length, 0.0)
17 pop.each do |member|
18 member[:bitstring].each_with_index {|v, i| vector[i] += v}
19 end
20 vector.each_with_index {|f, i| vector[i] = (f.to_f/pop.size.to_f)}
21 return vector
22 end
23
24 def generate_candidate(vector)
25 candidate = {}
26 candidate[:bitstring] = Array.new(vector.size)
27 vector.each_with_index do |p, i|
28 candidate[:bitstring][i] = (rand()<p) ? 1 : 0
29 end
30 return candidate
31 end
32
33 def search(num_bits, max_iter, pop_size, select_size)
34 pop = Array.new(pop_size) do
35 {:bitstring=>random_bitstring(num_bits)}
36 end
37 pop.each{|c| c[:fitness] = onemax(c[:bitstring])}
38 best = pop.sort{|x,y| y[:fitness] <=> x[:fitness]}.first
39 max_iter.times do |iter|
40 selected = Array.new(select_size) { binary_tournament(pop) }
41 vector = calculate_bit_probabilities(selected)
42 samples = Array.new(pop_size) { generate_candidate(vector) }
43 samples.each{|c| c[:fitness] = onemax(c[:bitstring])}
44 samples.sort!{|x,y| y[:fitness] <=> x[:fitness]}
45 best = samples.first if samples.first[:fitness] > best[:fitness]
46 pop = samples
47 puts " >iteration=#{iter}, f=#{best[:fitness]}, s=#{best[:bitstring]}"
48 end
49 return best
50 end
51
52 if __FILE__ == $0
53 # problem configuration
54 num_bits = 64
55 # algorithm configuration
56 max_iter = 100
57 pop_size = 50
58 select_size = 30
59 # execute the algorithm
60 best = search(num_bits, max_iter, pop_size, select_size)
61 puts "done! Solution: f=#{best[:fitness]}, s=#{best[:bitstring]}"
62 end
Listing 5.2: Univariate Marginal Distribution Algorithm in Ruby
5.3. Univariate Marginal Distribution Algorithm 211
5.3.7 References
Primary Sources
The Univariate Marginal Distribution Algorithm was described by Mühlenbein
in 1997 in which a theoretical foundation is provided (for the field of in-
vestigation in general and the algorithm specifically) [2]. Mühlenbein also
describes an incremental version of UMDA (IUMDA) that is described as
being equivalent to Baluja’s Population-Based Incremental Learning (PBIL)
algorithm [1].
Learn More
Pelikan and Mühlenbein extended the approach to cover problems that
have dependencies between the components (specifically pair-dependencies),
referring to the technique as the Bivariate Marginal Distribution Algorithm
(BMDA) [3, 4].
5.3.8 Bibliography
[1] S. Baluja. Population-based incremental learning: A method for in-
tegrating genetic search based function optimization and competitive
learning. Technical Report CMU-CS-94-163, School of Computer Sci-
ence, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213, June
1994.
[2] H. Mühlenbein. The equation for response to selection and its use for
prediction. Evolutionary Computation, 5(3):303–346, 1997.
[3] M. Pelikan and H. Mühlenbein. Marginal distributions in evolutionary
algorithms. In Proceedings of the International Conference on Genetic
Algorithms Mendel, 1998.
[4] M. Pelikanand H. Mühlenbein. Advances in Soft Computing: Engi-
neering Design and Manufacturing, chapter The Bivariate Marginal
Distribution Algorithm, pages 521–535. Springer, 1999.
212 Chapter 5. Probabilistic Algorithms
5.4 Compact Genetic Algorithm
Compact Genetic Algorithm, CGA, cGA.
5.4.1 Taxonomy
The Compact Genetic Algorithm is an Estimation of Distribution Algorithm
(EDA), also referred to as Population Model-Building Genetic Algorithms
(PMBGA), an extension to the field of Evolutionary Computation. The
Compact Genetic Algorithm is the basis for extensions such as the Extended
Compact Genetic Algorithm (ECGA). It is related to other EDAs such as the
Univariate Marginal Probability Algorithm (Section 5.3), the Population-
Based Incremental Learning algorithm (Section 5.2), and the Bayesian
Optimization Algorithm (Section 5.5).
5.4.2 Inspiration
The Compact Genetic Algorithm is a probabilistic technique without an
inspiration. It is related to the Genetic Algorithm and other Evolutionary
Algorithms that are inspired by the biological theory of evolution by means
of natural selection.
5.4.3 Strategy
The information processing objective of the algorithm is to simulate the
behavior of a Genetic Algorithm with a much smaller memory footprint
(without requiring a population to be maintained). This is achieved by
maintaining a vector that specifies the probability of including each com-
ponent in a solution in new candidate solutions. Candidate solutions are
probabilistically generated from the vector and the components in the better
solution are used to make small changes to the probabilities in the vector.
5.4.4 Procedure
The Compact Genetic Algorithm maintains a real-valued prototype vector
that represents the probability of each component being expressed in a
candidate solution. Algorithm 5.4.1 provides a pseudocode listing of the
Compact Genetic Algorithm for maximizing a cost function. The parameter
n indicates the amount to update probabilities for conflicting bits in each
algorithm iteration.
5.4.5 Heuristics
� The vector update parameter (n) influences the amount that the
probabilities are updated each algorithm iteration.
5.4. Compact Genetic Algorithm 213
Algorithm 5.4.1: Pseudocode for the cGA.
Input: Bitsnum, n
Output: Sbest
V ← InitializeVector(Bitsnum, 0.5);1
Sbest ← ∅;2
while ¬StopCondition() do3
S1 ← GenerateSamples(V );4
S2 ← GenerateSamples(V );5
Swinner, Sloser ← SelectWinnerAndLoser(S1, S2);6
if Cost(Swinner) ≤ Cost(Sbest) then7
Sbest ← Swinner;8
end9
for i to Bitsnum do10
if Siwinner 6= S
i
loser then11
if Siwinner ≡ 1 then12
V ii ← V
i
i +
1
n
;13
else14
V ii ← V
i
i −
1
n
;15
end16
end17
end18
end19
return Sbest;20
� The vector update parameter (n) may be considered to be comparable
to the population size parameter in the Genetic Algorithm.
� Early results demonstrate that the cGA may be comparable to a
standard Genetic Algorithm on classical binary string optimization
problems (such as OneMax).
� The algorithm may be considered to have converged if the vector
probabilities are all either 0 or 1.
5.4.6 Code Listing
Listing 5.3 provides an example of the Compact Genetic Algorithm imple-
mented in the Ruby Programming Language. The demonstration problem
is a maximizing binary optimization problem called OneMax that seeks a
binary string of unity (all ‘1’ bits). The objective function only provides an
indication of the number of correct bits in a candidate string, not the posi-
tions of the correct bits. The algorithm is an implementation of Compact
Genetic Algorithm that uses integer values to represent 1 and 0 bits in a
binary string representation.
214 Chapter 5. Probabilistic Algorithms
1 def onemax(vector)
2 return vector.inject(0){|sum, value| sum + value}
3 end
4
5 def generate_candidate(vector)
6 candidate = {}
7 candidate[:bitstring] = Array.new(vector.size)
8 vector.each_with_index do |p, i|
9 candidate[:bitstring][i] = (rand()<p) ? 1 : 0
10 end
11 candidate[:cost] = onemax(candidate[:bitstring])
12 return candidate
13 end
14
15 def update_vector(vector, winner, loser, pop_size)
16 vector.size.times do |i|
17 if winner[:bitstring][i] != loser[:bitstring][i]
18 if winner[:bitstring][i] == 1
19 vector[i] += 1.0/pop_size.to_f
20 else
21 vector[i] -= 1.0/pop_size.to_f
22 end
23 end
24 end
25 end
26
27 def search(num_bits, max_iterations, pop_size)
28 vector = Array.new(num_bits){0.5}
29 best = nil
30 max_iterations.times do |iter|
31 c1 = generate_candidate(vector)
32 c2 = generate_candidate(vector)
33 winner, loser = (c1[:cost] > c2[:cost] ? [c1,c2] : [c2,c1])
34 best = winner if best.nil? or winner[:cost]>best[:cost]
35 update_vector(vector, winner, loser, pop_size)
36 puts " >iteration=#{iter}, f=#{best[:cost]}, s=#{best[:bitstring]}"
37 break if best[:cost] == num_bits
38 end
39 return best
40 end
41
42 if __FILE__ == $0
43 # problem configuration
44 num_bits = 32
45 # algorithm configuration
46 max_iterations = 200
47 pop_size = 20
48 # execute the algorithm
49 best = search(num_bits, max_iterations, pop_size)
50 puts "done! Solution: f=#{best[:cost]}/#{num_bits}, s=#{best[:bitstring]}"
51 end
Listing 5.3: Compact Genetic Algorithm in Ruby
5.4. Compact Genetic Algorithm 215
5.4.7 References
Primary Sources
The Compact Genetic Algorithm was proposed by Harik, Lobo, and Gold-
berg in 1999 [3], based on a random walk model previously introduced by
Harik et al. [2]. In the introductory paper, the cGA is demonstrated to be
comparable to the Genetic Algorithm on standard binary string optimization
problems.
Learn More
Harik et al. extended the Compact Genetic Algorithm (called the Extended
Compact Genetic Algorithm) to generate populations of candidate solu-
tions and perform selection (much like the Univariate Marginal Probabilist
Algorithm), although it used Marginal Product Models [1, 4]. Sastry and
Goldberg performed further analysis into the Extended Compact Genetic
Algorithm applying the method to a complex optimization problem [5].
5.4.8 Bibliography
[1] G. R. Harik. Linkage learning via probabilistic modeling in the extended
compact genetic algorithm (ECGA). Technical Report 99010, Illinois
Genetic Algorithms Laboratory, Department of General Engineering,
University of Illinois, 1999.
[2] G. R. Harik, E. Cantú-Paz, D. E. Goldberg, and B. L. Miller. The
gambler’s ruin problem, genetic algorithms, and the sizing of populations.
In IEEE International Conference on Evolutionary Computation, pages
7–12, 1997.
[3] G. R. Harik, F. G. Lobo, and D. E. Goldberg. The compact genetic
algorithm. IEEE Transactions on Evolutionary Computation, 3(4):287–
297, 1999.
[4] G. R. Harik, F. G. Lobo, and K. Sastry. Scalable Optimization via Prob-
abilistic Modeling, chapter Linkage Learning via Probabilistic Modeling
in the Extended Compact Genetic Algorithm (ECGA), pages 39–61.
Springer, 2006.
[5] K. Sastry and D. E. Goldberg. On extended compact genetic algorithm.
In Late Breaking Paper in Genetic and Evolutionary Computation Con-
ference, pages 352–359, 2000.
216 Chapter 5. Probabilistic Algorithms
5.5 Bayesian Optimization Algorithm
Bayesian Optimization Algorithm, BOA.
5.5.1 Taxonomy
The Bayesian Optimization Algorithm belongs to the field of Estimation
of Distribution Algorithms, also referred to as Population Model-Building
Genetic Algorithms (PMBGA) an extension to the field of Evolutionary
Computation. More broadly, BOA belongs to the field of Computational
Intelligence. The Bayesian Optimization Algorithm is related to other
Estimation of Distribution Algorithms such as the Population Incremental
Learning Algorithm (Section 5.2), and the Univariate Marginal Distribution
Algorithm (Section 5.3). It is also the basis for extensions such as the
Hierarchal Bayesian Optimization Algorithm (hBOA) and the Incremental
Bayesian Optimization Algorithm (iBOA).
5.5.2 Inspiration
Bayesian Optimization Algorithm is a technique without an inspiration.
It is related to the Genetic