C LOSED -L OOP T URBULENCE C ONTROL :

P ROGRESS AND C HALLENGES

Steven L. Brunton1 and Bernd R. Noack2,3
1
Department of Mechanical Engineering & eScience Institute, University of Washington, Seattle, WA 98195, United States
2
Institut PPRIME, CNRS – Université de Poitiers – ENSMA, UPR 3346, Départment Fluides, Thermique, Combustion,
F-86036 Poiters CEDEX, France
3
Institut für Strömungsmechanik, Technische Universität Braunschweig, D-38108 Braunschweig, Germany

Abstract 4 Linear model-based control 12

4.1 Linearized input–output dynamics . . . . . . 13
Closed-loop turbulence control is a critical enabler of aero- 4.2 Model-based open-loop control . . . . . . . . 13
dynamic drag reduction, lift increase, mixing enhancement, 4.3 Dynamic closed-loop feedback control . . . . 14
and noise reduction. Current and future applications have 4.3.1 Controllability, observability, and
epic proportion: cars, trucks, trains, airplanes, wind turbines, Gramians . . . . . . . . . . . . . . . . . 14
medical devices, combustion, chemical reactors, just to name 4.3.2 Linear quadratic Gaussian . . . . . . . 15
a few. Methods to adaptively adjust open-loop parameters 4.4 Robust control . . . . . . . . . . . . . . . . . . 16
are continually improving towards shorter response times. 4.4.1 Sensitivity, complementary sensitivity,
However, control design for in-time response is challenged and robustness . . . . . . . . . . . . . . 16
by strong nonlinearity, by high-dimensionality and by time- 4.4.2 H∞ robust control design . . . . . . . 17
delays. Recent advances in the field of model identification 4.4.3 Fundamental limitations with implica-
and system reduction, coupled with advances in control the- tions for turbulence control . . . . . . . 18
ory (robust, adaptive, and nonlinear) are driving significant 4.4.4 Two degrees of freedom control . . . . 19
progress in adaptive and in-time closed-loop control of fluid 4.5 Balanced model reduction . . . . . . . . . . . 19
turbulence. In this review, we provide an overview of critical 4.5.1 Discrete-time systems and Gramians . 20
theoretical developments, highlighted by compelling exper- 4.5.2 Goal of model reduction . . . . . . . . 20
imental success stories. We also point to challenging open 4.5.3 Balanced proper orthogonal decompo-
problems and propose potentially disruptive technologies of sition . . . . . . . . . . . . . . . . . . . 20
machine learning and compressive sensing. 4.5.4 Eigensystem realization algorithm . . 21
4.5.5 Observer/Kalman filter identification . 22
4.6 Case study: Transition delay in a boundary
layer and stabilizing steady states . . . . . . . 22
4.6.1 Early work . . . . . . . . . . . . . . . . 22
Contents 4.6.2 Use of reduced-order models . . . . . 23
4.7 Case study: Wall turbulence control and skin-
1 Introduction 2
friction reduction . . . . . . . . . . . . . . . . 23
4.8 Case study: Cavity flow control . . . . . . . . 23
2 Turbulence control problem 3
4.9 Potential impact and challenges . . . . . . . . 24
2.1 The flow control plant and associated goals . 4
2.2 Linear dynamics . . . . . . . . . . . . . . . . . 5 5 Prototypes for linear and nonlinear dynamics 24
2.3 Turbulence control mechanisms . . . . . . . . 6 5.1 Galerkin model . . . . . . . . . . . . . . . . . 25
2.4 Actuators and sensors . . . . . . . . . . . . . . 7 5.2 Linear dynamics . . . . . . . . . . . . . . . . . 25
2.5 Achievable performance . . . . . . . . . . . . 7 5.2.1 Oscillator model . . . . . . . . . . . . . 25
5.2.2 Energy-based control design . . . . . . 26
3 Black-box, gray-box, and white-box models 8 5.3 Weakly nonlinear dynamics . . . . . . . . . . 26
3.1 State-space models . . . . . . . . . . . . . . . . 8 5.3.1 Mean-field model . . . . . . . . . . . . 27
3.2 Kinematics: employed state spaces . . . . . . 9 5.3.2 Nonlinear control design . . . . . . . . 27
3.2.1 Full-resolution description (white-box) 9 5.4 Moderately nonlinear dynamics . . . . . . . . 27
3.2.2 Modal representation (gray-box) . . . . 9 5.4.1 Generalized mean-field model . . . . . 28
3.2.3 Input–output (black-box) . . . . . . . . 11 5.4.2 Nonlinear control design . . . . . . . . 29
3.3 Dynamics: classification by system resolution 11 5.5 Strongly nonlinear dynamics . . . . . . . . . . 30
3.3.1 White-box models . . . . . . . . . . . . 11 5.5.1 Examples of strongly nonlinear dynamics 30
3.3.2 Gray-box models . . . . . . . . . . . . 11 5.5.2 Nonlinear control design . . . . . . . . 30
3.3.3 Black-box models . . . . . . . . . . . . 12 5.6 Enablers and challenges of nonlinear model-
3.3.4 Model-free approaches . . . . . . . . . 12 based control . . . . . . . . . . . . . . . . . . . 31

1
6 Model-free control
6.1 Open-loop forcing . . . . . . . . . . . . . . . .
6.2 Adaptive control . . . . . . . . . . . . . . . . .
6.2.1 Extremum-seeking control methodology
6.2.2 Examples of adaptive control in turbu-
lence . . . . . . . . . . . . . . . . . . .
6.3 In-time control . . . . . . . . . . . . . . . . . .
6.3.1 Control law parameterizations and
tuning methodologies . . . . . . . . . .
6.3.2 Case study: opposition control in wall
turbulence . . . . . . . . . . . . . . . .
6.4 Neural network based control . . . . . . . . .
6.5 Genetic algorithm based control . . . . . . . .
6.6 Genetic programming control . . . . . . . . .
7 Conclusions
7.1 Historical perspective . . . . . . . . . . . . .
7.2 Current practices . . . . . . . . . . . . . . . .
7.2.1 Linear control and transition . . . . .
7.2.2 Nonlinear control and separation . .
7.2.3 Model-free control and mixing . . . .
7.3 Industrial applications . . . . . . . . . . . .
8 Future developments beyond control theory
8.1 Bio-inspired sensing and actuation . . . . . .
8.1.1 Cheap hardware and local computations
8.1.2 Sensor and actuator placement . . . . .
8.2 Data-driven modeling and control . . . . . . .
8.2.1 Compressive sensing . . . . . . . . . .
8.2.2 Machine learning . . . . . . . . . . . .
8.2.3 Uncertainty quantification and
equation-free methods . . . . . . . . .
8.2.4 Design of experiments . . . . . . . . .
8.3 Advanced nonlinear models, controllers and
closures . . . . . . . . . . . . . . . . . . . . . .
8.3.1 Graph-theoretic flow control . . . . . .
8.3.2 Markov model-based control . . . . . .
1 Introduction
Taming turbulence for engineering goals is one of the old-
est and most fruitful academic and technological challenges.
One of the earliest examples are the feathers at the tail of an
arrow invented several thousand years ago. These feathers
stabilize the orientation of the arrow and make the trajectory
more predictable and increase its range. Meanwhile, mod-
ern turbulence control has applications of epic proportion.
Examples include drag reduction of road vehicles, airborne
transport, ships and submarines, drag reduction in pipes and
air-conditioning systems, lift increase of airfoils, efficiency in-
crease of harvesting wind and water energy, of heat transfer
and of chemical and combustion processes — just to name a
few examples.
Animal motion has inspired numerous technical ad-
vances in engineering flows [1]. The shape of sharks, dol-
phins and whales, for instance, yields a low drag per volume
[2]. It is not an accident that zeppelins and airplanes have
similar shapes. Dolphins are speculated to delay boundary
32 layer transition by a compliant skin. This form of transition
33 delay has been applied to submarines and is under active in-
33 vestigation. Under magnification, the skin of sharks exhibit
33 riblets [3]. Riblets have been found to reduce drag by up to
11 % in the laboratory [4]. In-flight tests of riblets on an Air-
35 bus passenger airplane have reduced fuel consumption by 2-
35 3 %. Some sharks decrease drag by ejecting lubricants during
high-speed chases of their pray. A similar skin-friction reduc-
35 tion is used in oil pipelines: one added polymer per 1 million
oil molecules reduces the drag by about 40 %. Eagles and
36 other birds have 5 feathers at the tip of their wings. These
36 feathers increase the lift by reducing the pressure short-cut
37 between the low pressure upper side and the higher pressure
38 lower side. Most modern passenger airplanes have winglets
for the same reason.
41 The environmental benefit of turbulence control can be
. 41 illustrated with an everyday example: automotive trans-
. 42 port. Today, the annual global CO2 emissions from cars ex-
. 42 ceed 22 billion tons and are expected to increase by 57% by
. 42 2030. A large portion of this emission is due to aerodynamic
. 43 drag [5, 6]. At a speed of 50 km/h the aerodynamic drag
. 44 accounts for 50% of the total resistance reaching 80% at 130
km/h. A drag reduction of around 25% is currently achiev-
44 able by active flow control [7]. Ad a speed of 120km/h, this
44 would reduce consumption by about 1.8 liter and would re-
45 duce CO2 by almost 2 kg per 100 km. In normal traffic, the
45 corresponding reductions are 0.15 liter in fuel and 0.73 kg in
46 CO2. For Europe, this drag reduction would mean a reduc-
46 tion of 23 million tonnes of CO2 emission in one year. To
46 mitigate pollution, the European government imposes strict
norms to car manufacturers. By 2020, the mean CO2 emis-
47 sion per vehicle must not exceed 95 gCO2/km. By 2025, the
48 limit is 75 gCO2/km.
Among the countless technologies that will benefit from
48 turbulence control, we highlight the potential benefits for en-
48 ergy and transportation. Increased lift and reduced drag due
49 to separation control and transition delay would result in
increased payloads and decreased runway requirements for
aircraft and improved efficiency in nearly all vehicles. Con-
sidering that transportation accounts for approximately 20%
of global energy consumption, a small improvement would
have a dramatic effect [8, 9, 10]. Active separation control
would also improve the safety of cargo trucks and trains in
strong cross-winds [11, 12, 13]. Hypersonic vehicles stand to
benefit from active control to prevent the undesirable ejection
of flames out of the combustion chamber, and subsequent
quenching. Finally, reducing the amount of turbulent fluctu-
ations on rotor blades would reduce vibration and improve
the life of rotor hubs on wind turbines and rotorcraft.
Strategies to control laminar and turbulent flow are clas-
sified in three categories [14]: aerodynamic shape optimiza-
tion, passive and active control. The first approach for in-
creasing flow performance is the optimization of the aerody-
namic shape. Potential flow theory, invented about 150 years
ago, provides a simple mathematical foundation. Meanwhile
adjoint-based shape optimization can be numerically per-
formed for the full Navier-Stokes equations. As a second
step, passive actuators may improve the performance. Such
a device represents a small change of the original configura-
2
tion. One example are the turbulators on wings of passen-
ger airplanes to delay separation. Such passive devices may
come with the penalty of parasitic drag. An alternative are
active control devices, like fluidic vortex generators, which
may be turned on and off but require energy for their opera-
tion. One advantage is a large dynamic bandwidth, e.g. the
excitation of particular frequencies. Active control may be
performed in a predetermined open-loop manner, e.g. peri-
odic blowing and suction, independent of the flow state. The
largest gains are, of course, realized in a closed-loop manner
when the actuation is informed by the sensors recording the
flow state.
Up to the 1990’s, manufacturers have often seen ac-
tive control as a remedy for a flawed aerodynamic design
[15]. Hence, industrial interest has been correspondingly
low. Meanwhile, aerodynamic design and passive devices
are considered as maturely developed and active control is
pursued to further increase the performance, particularly for
off-design conditions. Three trends foster closed-loop con-
trol. First, the power and reliability of actuators and sensors
have dramatically increased, while the price is decreasing.
Second, a sophisticated control logic can be performed un-
der real-world conditions with increasing computer power
and the advancement of mathematical theories. Thirdly, the
experimental demonstrations of the benefits of closed-loop
over open-loop forcing has become overwhelming [see, e.g.,
16, 17].
Most literature on closed-loop flow control falls in one
of three categories: stabilization of laminar flow, adaptive
control of turbulence and model-free tuning of control laws.
For the first category, there exists a mature theory for the
stabilization of laminar flows with in-time closed-loop con-
trol. Early experimental examples are described in [18, 19]
while most studies are based on Direct Navier-Stokes (DNS)
solutions [see, e.g., 20, 21]. ’In-time’ means that the actua-
tion responds on a time-scale much smaller than the natural
time-scale. Most corresponding publications are based on a
linearization of the evolution equation. The employed evolu-
tion equation may be a white-box model, e.g. DNS discretiza-
tion, resolving all features of the flows, a gray-box model,
e.g. POD models, just describing the coherent structures, or
a black-box model, e.g. transfer functions, representing only
the input-output behavior. The control logic based on white
box models are the most accurate. Gray-box and black-box
models are less accurate but allow online-capable control so-
lutions for experiments.
The second category is adaptive control of turbulence
usually based on a manipulation of periodic forcing. Most
experimental success stories belong to this class. ’Adaptive’
means that the change of the actuation parameter, like am-
plitude or frequency, is slow compared to the natural time-
scale. The third group consists of in-time control of turbulent
flows, for instance by tuning simple laws, e.g. opposition or
PID control [22]. The inherent nonlinearities of turbulence
pose a challenge for model-based in-time control.
Closed-loop control requires decisions on the hardware,
like the kind, number, location, and dynamic bandwidth of
actuators and sensors. Such decisions may be guided by
modern adjoint-based techniques for linearized equations,
3
i.e. for laminar flow. For turbulence, these decisions are
largely guided by engineering wisdom from the flow con-
trol processes and past experiments. The control laws may
be guessed based on the flow phenomenology. These aspects
will be touched in Sec. 2. The focus of this review is the con-
trol logic sketched in Fig. 1. This logic is based on model
complexity, e.g. model-free approaches (bottom right of this
figure), black-box (left), gray (middle) and white-box models
(right). Experimental flow control solutions require a delicate
compromise between simplicity and accuracy. Many plants
benefit from system reduction approaches. This spectrum of
models will be outlined in Sec. 3. The most complete model-
based control design techniques are available for linear mod-
els. These will be described in Sec. 4. The detailed review
of linear control theory may be surprising in a review about
control of nonlinear turbulence processes. We emphasize lin-
ear control for three reasons. First, it is a beautiful theory
showing clearly the effects of model accuracy and design pa-
rameters. Second, the prevention of turbulence, i.e. transition
control, can largely be done based on a linear theory. Third,
even some strongly nonlinear processes may be tamed with
linear control methods, if the nonlinearity is well understood.
Examples of such ’reducible’ nonlinear models are provided
in Sec. 5. Model-free approaches for ’non-reducible’ nonlin-
ear dynamics are reviewed in Sec. 6. In Sec. 7, we summarize
good practices of flow control and promising directions of
future research are discussed in Sec. 8.
This review focuses on modeling and control methods
for nonlinear dynamics associated with turbulent fluid flows.
Theoretical and computational aspects of optimal and robust
(linear) control are elaborated in [23]. A large spectrum of lin-
ear system approaches are presented in [24, 20, 21], and pe-
riodic excitation is reviewed in [25]. We illustrate the perfor-
mance of the methods by referring to select successful closed-
loop control studies. Preference is given to academic stud-
ies for simple geometries. There are additional books and
reviews in the related fields of flow control for wall flows
[26, 27, 28], for turbulent mixing [29, 30, 31], for turbulent jets
[32], for combustion [33], for cavity flows [34, 21], for bluff
bodies [35], and for actuators [36].
2 Turbulence control problem
Turbulence control comprises decisions on the flow control
plant, on the cost functional, on the actuation and sensing,
and, last but not least, on the control logic. The control
logic serves to minimize the cost functional under given con-
straints. For linear dynamics, there exist adjoint-based meth-
ods for the choice of the actuators [21] and the sensors [37].
For turbulent flows, the application of these methods is lim-
ited and the choice is generally based on experience and
engineering wisdom. In this section, we provide heuristics
for turbulence control decisions. Thereafter, a mathematical
foundation of the control logic is elaborated.
 Navier-Stokes equations
ray-box, and white-

Ul
w
strategy employed below, we as-
Figure 4: Model hierarchy for control design based on [74].
ctuate the flow with some input
re able to measure features of the following a classification of N. Wiener [74] and extending it
iables s 2 RNs . Once the inputs to include model-free approaches. These classifications are
are many choices for the model generally based on the system resolution of the model. Some Knowledge of the plant
ple, we may consider the full dis- control laws can be analytically derived from the Navier-
ations as a high-dimensional non- Input/output Flow data
Stokes equations. One example
Navier-Stokes
is energy-based control to
data
rential equations. Alternatively, stabilize equations
a flow (see Sec. 5.2). We refer to this control-design
outputs through either a statisti- as “ultra-white”. An expanded view of the various models Kinematics
empirical basis functions. Recent and modal decompositions is shown in Fig. 5. This section is
y reduction techniques and turbu- not meant to be exhaustive, but rather include methods that
Time-delay
citing implications for the future have either been applied Empirical Physical
with recent success or methods that Mathematical
d control. coordinates
have particular promise in the future.
affects nearly every downstream Modes
e many factors and tradeoffs that
eciding on a modeling strategy. 3.1 State-space models Dynamics
y of the model, execution time, Model Galerkin
In one of the most general frameworks, we will have a state
Black-box model based control design

ter regimes, spatial-temporal res- identification projection

a 2 R and a nonlinear function f advancing the state for-
Na
nces, and the up-front cost to ac-

Optimal adjoint based control

ward in time, along with a nonlinear output function m:
xample, direct numerical simula-
Plant model
at descriptive resolution, general- ȧ = f (a, b; µ), (2a)
nt computational capabilities are s = m(a, b; µ). (2b)
me execution for in-time control
models based on data from DNS In the following, we explore possible state-spaces for a Action mechanisms
al-time capable models, but these (Sec. 3.2) and dynamic models for f (Sec. 3.3). Here the dy-
ate and may only work for a small namics are assumed to be continuous,
Weakly and in general the flow
Moderately Strongly
rs. Fortunately, it may be possible stateLinear
a in the Navier-Stokes equation is twice continuously
nonlinear nonlinear nonlinear
on about the structure of the un- differentiable in space and once continuously differentiable
rms of linear combinations of full in time. The bifurcation parameters µ may change the qual-
models with addit ional terms and itative nature of the solutions, and the flow field may not Control design
nge. Black-box models based on Linear alwayscontrol
be continuously differentiable in µ. These parame-
Model-free
ally faster to generate and require ters include the Reynolds number and Mach number,
design controlamong
design
y lack the physical interpretation others.
nderlying modal representation. The nonlinear dynamics in Eq. (2) may be linearized at
odel hierarchy for control design, a steady fixed point as , where f (as , 0; µ) = 0, away from
LQR, LQG, ... Parameter tuning MLC Adaptive control
(optimal) (heuristic) (unsupervised) (quasi-steady)
8
Controllers

Figure 1: Turbulence control roadmap. For details, see text and the coming sections.

2.1 The flow control plant and associated
goals
In the sequel, flow is assumed to be within or around a steady
boundary with small unsteady actuators and sensors. Aca-
demic flow control configurations strive at geometric sim-
plicity for enhanced reproducibility and for ’clean’ under-
standable physical mechanisms. Examples include free shear
flows from a bluff body, a mixing layer or a jet and wall-
bounded flows in a channel or over a flat plate. Cavity noise,
suppression of aeroelastic oscillation and flame-holder com-
bustion serve as examples for multi-physics flows.
Configurations of industrial importance tend to be ge-
ometrically far more complex, like the flow around a car,
truck, train, airplane or wind-turbine. Internal flows in pipes,
diffusers, combustors, mixers, air-conditioning systems and
buildings are further examples. Each of these configurations
could profit from closed-loop control and the range of poten-
tial applications has epic proportion (see Fig. 2). Yet, many
flows with complex geometries can locally be approximated
by the above mentioned academic configurations. Thus, a
working flow control experience on simple test cases is ad-
vantageous in the design of efficient control for industrial
purposes.
The aerodynamic performance of transport vehicles and
wind-turbines is based on a force optimization, like drag re-
duction, lift increase or reduction of fluctuations to prevent
early material fatigue. Examples are transportation trucks
(PACCAR), airplane wings, wind turbine blades, helicopter
rotor hubs and reduction in structural loads. Combustors,
heat exchangers and chemical mixers profit from mixing en-
hancement. Noise reduction is a common request for greener
transport systems. Many of the above applications can be
idealized to aim at the stabilization of an unstable fixed point
or periodic orbit, which we may refer to as instability suppres-
sion. Some internal flows require an improved destabilization,
like the mixing enhancement in a combustor.
Most control objectives can be formalized in a rigorous
mathematical manner. Drag reduction, for instance, reduces
by definition the necessary propulsion or towing power. The

4
 Simple prototype flows Transport vehicles Energy systems Production etc.
(a) (f) (k) (o)

(p)
(b) (g)

(l)

(h)
(c) (q)

(m)

(i) (r)
(d)

(n)
(j) (s)
(e)

Figure 2: Applications of closed-loop turbulence control. (a) Homogeneous grid turbulence, with permission of T. Corke
and H. Nagib, (b) turbulent jet from Bradshaw, Ferris & Johnson 1964 [38], (c) Karman vortex street behind a mountain, photo
by Bob Cahalan, NASA GSFC, (d) coherent structures in a mixing layer from Brown & Roshko 1974 [39], (e) thunderstorm,
(f) automobile in a wind tunnel, photo by Robert G. Bulmahn, (g) high-speed train, (h) cargo ship, (i) passenger jet, (j) Blue
angles fighter jets, (k) automobile engine, (l) turbo jet engine, (m) aircraft engines, (n) wind turbines, (o) heat exchanger
flow (p) rotating mixer, (q) air conditioner, (r) chocolate mixing, (s) total artificial heart. Images (e) and (g)-(n) are from
http://pixabay.com. Images (c), (f) and (q)-(s) are from https://commons.wikimedia.org. Images (o) and (p) were made
using the COMSOL Multyphysicsr software and are provided courtesy of COMSOL.

optimal drag reduction may be defined by the maximum
energetic benefit, i.e. the saving in towing power subtract-
ing the investment in actuation power. Note that any well-
defined control problem requires a penalization of the actua-
tion power.
Many experimental studies demonstrate that the control
is effective for a well-defined operating condition in a noise-
free wind-tunnel environment. Engineering applications re-
quire additional effectiveness for the intended operating en-
velope, including various oncoming velocities, a range of an-
gles of attack, free-stream turbulence, just to provide few ex-
amples. Ideally, robustness of the control is included upfront
in an objective comprising a range of operating conditions,
like in linear H∞ control (see Sec. 4.4). At minimum, the level
of robustness needs to be assessed after a single point opti-
mization of actuation. An understanding of the turbulence
control mechanisms provides a first hint on the expectable
level of robustness.
2.2 Linear dynamics
Turbulence is known as the last unsolved problem of classi-
cal mechanics, largely because the effects of nonlinearity are
next to impossible to predict from first principles. Hence,
turbulence control can be seen as an even more Herculean

5
nonlinear problem as not only the unforced state needs to be
predicted but also the effect of a small actuation. Fortunately,
there exist a number of configurations for which a linear dy-
namics has been shown to be a good working assumption.
Examples include the following cases.
• Transition delay. The transition of a laminar into a turbu-
lent boundary layer is associated with a dramatic rise of
skin friction. Hence, engineering applications include
transition delay with closed-loop control. The laminar
state may still be stabilized based on a linearized model.
Evidently, stabilization of a laminar flow has benefits for
numerous other configurations.
• Drag reduction in wall turbulence. At high Reynolds num-
bers, active control at the wall does not have the author-
ity to stabilize the the laminar boundary layer. Yet, up to
11% drag reduction can be achieved with stationary ri-
blets which mitigate sweeps in the viscous sublayer [4].
Over 20% drag reduction can be obtained with linear ac-
tive control [40]. Arguably, linear control is applicable
because the sweep prevention in the viscous sublayer is
an effectively laminar process, like transition control.
• The in-time actuation response to large scale coherent struc-
tures may be described a linear model — extending the
examples of drag reduction in wall turbulence. Physi-
cally, such a locally linear model may be derived under
similar conditions as URANS simulations, i.e. if the ef-
fect of the unresolved stochastic velocity component on
the dynamically resolved coherent structures is roughly
represented by a temporally constant eddy viscosity. An
example is the mean-field model for oscillatory fluctua-
tions of turbulence (see Sec. 5.3).
• Adaptive control may be subject to a limited linear con-
trol. For instance, the change of cost function may
respond linearly to small changes of the amplitude
and frequency of periodic forcing. This is an implicit
working assumption of extremum seeking control (see
Sec. 6.2). Thus, tracking may be based on locally linear
dynamics.
• Another recently discovered example of linear dynam-
ics is the ensemble-averaged actuation response of a turbu-
lent shear flow [41]. The practical relevance of this ob-
servation still needs to be explored. Studies of forced
nonlinear chaotic systems indicate that the ensemble-
averaged effect of a Heaviside actuation may be de-
scribed by linear system while the amplitude depen-
dency is far from linear [42]. Moreover, the ensemble
averaged response may constitute a small portion of the
fluctuation energy and may not be very relevant for the
control goal.
2.3 Turbulence control mechanisms
In the following, common principles of turbulence control
are outlined. A control principle is understood as a phe-
nomenological rationale based on a simplified physical actu-
ation mechanism. A simple principle has a low-dimensional
plant mimicking relevant aspects of turbulence control, in-
cluding linear and nonlinear dynamics.
6
We start with stabilizing control. A very simple example
is opposition control. Let
da
=a+b
dt
be a plant with the unstable fixed point a = 0 and actua-
tion b. Evidently, the control law b = −2a will ’oppose’ the
natural evolution and stabilize the fixed point. A number of
flow control configurations mimic this behavior. Let us con-
sider Tollmien-Schlichting waves over a wall. The wall shall
have a membrane of which the vertical motion can be con-
trolled. A positive or negative wall-normal velocity fluctua-
tion of a Tollmien-Schlichting wave can be counteracted by a
membrane which moves in the opposite direction [18]. Simi-
larly, the transverse centerline motion of a channel flow may
be damped by blowing and sucking at opposite sides of the
channel wall in order to counteract this fluctuation. Thus, an
unstable 2D channel flow may be stabilized. Another exam-
ple of this category is skin friction reduction. Skin friction is
known to increase with sweeps and ejections, both associated
with wall-normal velocity fluctuation. A simple opposition
control scheme records this velocity fluctuation 10 plus units
away from the wall and opposes this motion by local suction
or blowing [43].
A slightly more complex principle is phasor control based
on an oscillatory process. A simple prototypic dynamical
system reads
da1 da2
= 0.1a1 − a2 , = 0.1a2 + a1 + b.
dt dt
Evidently, the fixed point a1 = a2 = 0 is unstable with re-
spect to an oscillatory instability with unit frequency and
growth-rate of 0.1. The control law b = −0.4a2 can be seen to
stabilize this fixed point by reducing the fluctuation energy
with targeted actuation at the correct phase. Phasor control
can also be consideredp as an opposition control with respect
to the amplitude r = a21 + a22 . The actuation reduces this
amplitude at phases where it is effective. The corresponding
energy-based control design will be elaborated in Sec. 5.2. A
large class of control laws for suppression of oscillation may
be written in the form
b = k r cos(φ − β), (1)
where φ is the flow phase, β is the phase lag and k the gain.
In the given example, φ is defined by the polar coordinates
a1 + ıa2 = r exp(ıφ), β = −90◦ , and k = 4. Phasor con-
trol (1) is an analogy for virtually any successful stabilization
of an oscillatory flow, regardless how the control law is de-
rived. The gains may be constant for assumed linear dynam-
ics or base-flow/energy dependent for weakly nonlinear dy-
namics. Examples are the stabilization of a laminar cylinder
wake with ZNMF actuators [44], the stabilization of a turbu-
lent wake in an experiment [45] and the suppression of cavity
noise with local forcing [34, 46].
A more complex case involves two or few nonlinearly
coupled ’clock-works’ based on the principle of construc-
tive or destructive frequency cross-talk. One example is high-
frequency forcing which mitigates a target instability [47, 48].
 Balanced model reduction 4.5.2 Sensitivity,
r Complementary Sensitivity,
and Robustness and Robustness
H robust vs. H 4.5
4.5.1 optimal HH robust
1 optimal control vs. H2 optimal
control: Linear control Gaus-
quadratic
4.5.1 H2 optimal control: Linear
1 2 2 quadratic Gaus- 4.5.3 H1 robust control 4.5.3 • S(⇣) H1- robust sensitivity control function
sian (LQG) sianwe (LQG) • S(⇣) - sensitivity function
Now
w that we have established conditions enabling that have established conditions enabling
We will often set We Ĉ • will=T (⇣) Koften andset
r- complementary D̂ Ĉ= =0, K r and D̂ = 0, where
where
sensitivity function
arbitrary pole placement Eq.of(2) thewith closed-loop T (⇣) - system, complementary
trarymay
We poleoften
placement modify We ofmay the (2)
Eq. closed-loop
oftenwith modifythe system,
addition of •the Kraddition of
is a linear-quadratic-regulator Kr issensitivity function
a linear-quadratic-regulator
(LQR) gain ma- (LQR) gain ma-
must we must tonow decidethem.onw where to place them. noise
whitenow
Balanced noise decide
modeldisturbanceonwhite
reduction
where wnoise place
d anddisturbance
measurement and
d noise 4.5.2measurement trix.
wr - We
•Sensitivity, may also
reference trix.
choose
Complementary
tracking • We wrÂ-may andalso
reference choose
B̂ according
Sensitivity, tracking Âto and theB̂ according to the
wn : w n : Kalman
and Robustness
mechanisms Kalman are far
filter, resulting in a combined estimation- from
filter, being
resulting understood. in a combined estimation-
Control mechanism Control method extended.
In the case For
of linear
broad-band dynamics, turbulence, sophisticated
no generic mathematical
simple
1 robust vs. H2 optimal
extended. For linear dynamics, sophisticated mathematical
Hcontrol
Control
Control mechanism
mechanism Control
Control method
method based controller based
known controller
as the known
linear-quadratic- as the linear-quadratic-
H2 optimal control: 4.5.1 Linear optimal
quadratic control:
Gaus- Linear quadratic Gaus- extended.
methods 4.5.3 For
have linear
H 1 robust
been cancontrol
dynamics,
developed sophisticated
for mathematical
d
Control
Control 2 mechanism
mechanism
d Control
Control 4.5.3
•method
method
S(⇣) H 1 robust
-control
sensitivity
extended.
recipescontrol
methodsfor
function the
Gaussian
For
have linear
control been law
(LQG).
dynamics,
developed Because for actuator
sophisticated
be offered. Yet, we
actuator
of the
and
and sensor
mathematical
present place-
sensoraprinciple
separation place-
sian (LQG) a = Aa sian
+ Bb b(LQG)
+ a w =
, Aa Proportional
+ Bb
(14a)
Proportional
Proportional + w
control Gaussian
, control (LQG).
(14a) methods
highly ment.
methods Because
have
For
promisinghave been
nonlinearof
been the
machine-learning separation
developed
dynamics,
developed for
for principle
actuator
heuristic
actuator
strategy inand
methods
and
Sec. sensor
sensor
6. for
Figureplace-
the
place-op-
at we have established conditions b enabling
d d ment. WeFor nonlinear
will often dynamics,
set heuristic andmethods for the op-
where
Opposition

dt Proportional control for linear systems, itĈ isheuristic

possible
= K[? to design
D̂for an
=the 0,optimal
Opposition

dt bb Proportional We for
will linear
control often ment.
systems,
setment.
Ĉ For
it
For
timization
= is nonlinear
possible
nonlinear
K of and
sensor dynamics,
to design
dynamics,
D̂placement
= 0, an
where
heuristic
exist rmethods
optimal methods
]. In thisfor the
survey, op-
op-
we
Opposition

y pole placement of the closed-loop system, PID control - complementary 3 summarizes

timization all
sensitivity
of
r discussed
sensor functioncontrol
placement principles
exist [? with
]. In associated
this survey, we
control

PID •
controlT (⇣) (14b)
control

WeCa may often + modify

s = CaPID
Eq. +control
(2)
Db
(14b) with
+ w the. addition of
Opposition

feedback isheuristics
a linear-quadratic-regulator
control gain and an (LQR)
optimal gain
we ma-
observer
Opposition

s = + Db the waddition
n. timization rof sensor placement exist [? ].]. In
In this survey, we
control

may often modify Eq. (2) with PID

PID of control
control K n
feedback
is a control
linear-quadratic-regulator
focus
methods. gain
timization Kon
The of
the
K and
sensor
control an
(LQR) optimal
placement
logic.
for gain
closed-loop Kobserver
existrma-[?
turbulence this survey,
presents
focus on the control logic.
control

r r
t now decide on where tonoiseplacedisturbance
them.
control

LQR, LQG control

e noise disturbancewhite wd and measurement
s
ss w
noise
LQR,
LQR,
LQR,
and
dLQR,
LQG
based •measurement
LQG
LQG
LQGseparately,
control
trix.
on waracontrol
control noise
We
-model
control may also
reference focus
focus
atracking
and tour de
they
choose on the
trix.
force
willÂcontrol
separately,
on the Webe may
control
trough
and both
B̂ logic.
and
the also
logic. they
control
stable
according choose
will tobe
strategies
and  and
both
optimal
the andstable
B̂ according
methods, and to the
weoptimal
Each of these noise Each
wn :inputs of these has noise
s a different inputs based co-has on a model
different co- Kalman filter, resulting in a combined estimation-
based
based
basedon on a model
onKalman
aamodel whenfilter,
model combined. will when
elaborate in combined.
later
resulting in a combined estimation- sections.
variance matrix: E(wvariance w T) = matrix:
V and E(w E(w d wndw
T ) T= V and E(w w T ) =
) = dbased controller The n resulting
n known based
controller, as controller
The resulting
the known known
controller,
more
linear-quadratic- asknown
generally the linear-quadratic-
more generally
H2 optimal control: Linear quadratic d d d Gaus- n
4.5.3 value. H1 robust control
V , where d E(·) is Vthen , where
expectation E(·) d is the
value. expectation
[ThisPID is Gaussian
control as a [This is
2.4
controller,
(LQG). Because asGaussian
Actuators a
optimallyH
of the (LQG).
and
controller,
balances
separation sensorsBecause
optimally
the
principle of
effect the
balances
of separationthe effectprinciple
of
ian (LQG) a = Aa + Bbbb + wd , a = Aa
n + Bb + wd , H 2 T(14a)
control

PID control 2
(14a)
control

not precise dt enough... notreally precise

bb need enough...
dtE(w (t)wreally
PIDPID
PID
(⌧ ) need
control
T control
)We
control
= E(w
will (t)w
often (⌧ )
set ) = for
Gaussian linear
and systems,
measurement it is
where possible
noise to
with design
process an optimal
distur-
control

for Gaussian
linear
d dsystems, Ĉ
measurement =
The itchoice
K
is possible noise D̂
to design =
with and 0, process
an optimal distur-
control

sss = Ca Phasor
d d
Phasor control
control (14b)
r
of the actuators sensors, their number, lo-
control

+ Db +iswabances. .
Phasor

yVdoften (t modify
⌧ ).] s =Eq.Ca V(2)d + (twith
Db ⌧ ).]
thewsaddition
+ ns . (14b)
of Phasor
Phasor
Phasor Krcontrol
control feedback
control nlinear-quadratic-regulator
control However, feedback
bances.
gainfrequency
K these
r andcontrollers
an control
However,
(LQR)optimal gaingainthese
are ma-K
known
observer and
rcontrollers an
to optimal
are known observer to
LQR, LQG control cation, range and amplitude level has a decisive
Linear-quadratic regulator LQR, (LQR):LQG control
Phasor

oiseLinear-quadratic
disturbance wd and regulatormeasurement (LQR): noise trix. We may also choose separately,
have and arbitrarily and
according they
poor to will be
robustness
the both stable
margins. and optimal
Instead,
Phasor

LQR,
LQR,LQR,based
LQG LQG
LQG separately,
on
control have
modelarbitrarily
aacontrol
control and effect they poor willÂ
on the be B̂
robustness
both stable
maximum margins.
and optimal
performance Instead,
of the control logic.
Phasor

EachPointer ofhas these noise based inputs based has on a modeldifferent co- inH
Multi-frequency Phasor

h of these noise inputs

Z 1 Tpointer
pointer
pointer a
diagram
diagram
diagram Z different co-
based
based onKalman
onon
a a
when
a
model model
model
H filter,
1 robust
combined. resulting Actuators
controllers when
a
may
1 robust
are combined.
combined
include
used controllers
estimation-
zero-net
when mass are
robustness used
flux when [50,
actuators
is robustness
51, is
diagram 1
ance matrix: E(wd w variance
pointermatrix:
Td ) = VdJ
diagram
Tand E(w E(w Tw dw T )T ) = Vd and
=Tbased E(wn wnT known
controller )52],= piezo-electric
important.
as The
the resulting
actuators
linear-quadratic- controller,
[53, 54], Festo-valves known more
(intermit- generally
J= aV Qa + = n
a Qa +(15a)
n d b Rb dt. important.
The resulting (15a) controller, known more generally
n , where b b Rb E(·)dt. 0 is the expectation value. [This is blowing),
Multi−frequency

whered E(·) tent as synthetic

aFigure controller, jets principle
[55,the 56,most
optimally 57, 58], plasmaschematic
balances actua-
the effectfor of
a =is Aa the expectation
Bbbb+b wd , value. [This is Gaussian as a H(LQG). Because ofthe the 4general
2separation shows of general
Energy−based c. H
Multi−frequency

0 + (14a) Energy−based 2Figure

c.controller, T4 shows optimally most balances theschematic
effect for
Multi−frequency
Multi−frequency

precise not precise

dtenough... really need E(ws1d1 (t)wd (⌧ ) ) Phasor enough...
s really
T need
Energy−based
Energy-based
Energy−based E(w control c.
(t)w
c. (⌧ ) tors
) = [59, 60,
Gaussian61, 62, 63, 64,
measurement 65], microelectromechanical
noise with processsys- distur-
= for linear d systems,
closedmeasurement looptems it is closed
possible
feedback to
control, loop
design feedbackan optimal control, encompassing H and
67,encompassing androughness el- 2
d
Gaussian
control noise [66, with 68,process distur- H2and
control

sscontrol Phasor Kcontrol (MEMS) 69, 70, 71, 72],

control

The
t ⌧optimal
).] s =control Ca The + VlawdDb optimal
(t is+bw n . 1s
⌧ ).] 11Kr a, where lawPhasor
(14b) is
Phasorb
K= =control r a, where Kgainr = K H bances. However, these controllers are aknown to
control

= Phasor feedback
control control and optimal
an optimal control observerstrategies. areThere innumber
control

rcontrol bances. optimal

However, controlthese strategies.
controllers Thereare are
known a number
to
control

1 B T X and X is the unique Low−frequency

Low−frequency H 1 c.
c. ements r on 1 the wall. Excellent overviews presented
R 1 B T X and Xregulator
Linear-quadratic isRtheLinear-quadratic
unique (LQR): solution to regulator
the alge- solution
(LQR): to the alge- ofhave arbitrarily
excellent books poor
expanding robustness on margins. Instead,
separately,
Low−frequency
Low−frequency
ss2 Low-frequency
High−frequency havecontrol and
c.
ofarbitrarily
excellent
c. c. they will
books
poor
[55, 36].be both
expanding
robustness
Similarly, stable on
sensors and
this
margins. may optimal
theory Instead,
measure 40].this pressure,
[39,velocity, theory [39, 40].
f these
braic noise
Riccati inputs
equation: braichas Riccati equation:
a different 2 co- High−frequency when combined. c. skin-friction, H robust
Here
and we controllers
temperaturediscuss in are
important
various used when
theoretical
frequency robustness
resolu-results re-is
Z 1 T Z s
1s High−frequency
High−frequency
s22 TT) High-frequency
2 H robust
1 control Here
c.
c. we
controllers discuss important
are 1 used when theoretical
robustness results is re-
e matrix: E(w d wTd ) = VdT andJ E(w n wa = T The
important. resulting
garding the tions.
controller,
various These important.
garding are described
known
types the
of more
optimal ingenerally
various text control:
books
typesofofexperimental
optimal fluid
control: H1
TJ = aexpectation
Qa +AbT X Rb = n Qa
(15a) + b Rb dt. (15a) H
+Tdt. 1 T 1
ere A E(·) Xis+the XA XBR 1 value.
B XA
X + [This
Q XBR
= 0.is BasXa +
(16a)
Open−loop H Q = 0. (16a)
control
controller, mechanics.
optimally robust Figurecontrol,
balances 4 shows
the and effect
H the LQG.
ofmost general schematic for
0 0 Open−loop 2 robust control,
Figure
control 4 showsand the H most2 LQG. general schematic for 2
cise enough... really need E(wd (t)wd (⌧ )T ) Open-loop =Open−loop
Open−loop control
control Up toclosed this day, loop the feedback
choice of the actuators
control, and sensors in H and
encompassing
Broadband

Gaussian
Adaptive control measurement noise with process distur-
dynamics

closed loop feedback control, encompassing and

Broadband
Broadband

Adaptive control control H2 experience

dynamics

2
The boptimal Kr control law is= where rexperiments •1isOften
based onconsidered
istimes engineering
turbulence is consideredand on the
are aa numberdistur-
Broadband

A isdual Riccati a,equation is b

Adaptive =
solved K forr a,the
control observerKtimes = turbulence
Broadband
dynamics

optimal controlequationlaw where Adaptive control Often a distur-

dynamics

⌧ ).] optimal control strategies. There

control

A dual Riccati =is solved

1 B T X and TX is theAdaptive
for the K observer
r Machine bances.
Machine H learning
learning
•
However,
optimal c.
c. control these controllers
H
strategies.
hypothetical actuation There are
mechanismknown
are a to
number
which shall be exploited.
unique control
solution1
to the alge- bance term inInstead,
aexpanding
slower dynamical system, such
ear-quadratic
B XKand
T regulator
Vngain
isTthe R
:unique (LQR):
Kf solution
= Y C Vto n : the alge- bance term in aofslower excellent dynamical
books system, 40].such onactuator
this theory
gain if the[39, 40].
f =X YC Machine
Machine have learning
of arbitrarily
learning
excellent c.c. bookspoor robustness margins.
For expanding
instance, the on sensors this theory
are placed [39,before the
c RiccatiZ equation: braic Riccati equation: Machine learning
H1 robust as
Herecontrollers the
we discuss rigid
actuation body
are as
equations
shallHere
used
important
the
when
counteract
rigid of
we robustness
theoreticaldiscussbody
an
upstream
equations
aircraft,
important
results space of
is re- theoretical
perturbations.
an aircraft,
The sen-
space
results re-
T1 T control shuttle, or rocket. In this case, turbulent fluc-
Y A + AY Y C TY T AT1 + AY
V CY + V Y = C 0.
T
V 1
CY +
important.
(17a) V = shuttle,
0. (17a) or
sors rocket.
are garding
placed In this
the
downstream case,
various ofturbulent
the types
actuator,offluc-
optimal
if the control:
actuation H1
J= a Qa + b 1RbTnTdt. d (15a) 1 T garding the various types of optimal control: H1
n d
T
A X + XA A
XBR B X + Q = 0. (16a) X + XA XBR B X +
Figure Q = 4 tuations
0.
shows (16a) may
mechanism
the most be
robust tuations
seen
exploits
general as
control,the may
inevitable
excited
and
schematic be H seen
and
structures.
LQG.
for as
oper- inevitable
For opposition and oper-
0 robust control, and H2 LQG. 2
TheFigure so-called 3: Kalman
Heuristics filter
of turbulence f is chosen
Kclosed control
loop ating
to mini-
feedback on control,
a time
control, sensors ating
scale and
encompassing on
that a
is time
actuators
Hshould
faster scale
than bethatat aissimilar
con- fasterloca- than con-
The so-called⇣ Kalman
Figure 3: filter
Figure ⌘ ⇣
Heuristics K 3: f is
Heuristics
of chosen
turbulence of to ⌘ mini-
turbulence
control. Here control are the 2 and
imalRiccati
controlequation
lawT is b AFigure
=dual
Figure K 3:
Riccati
3:
r a, where
Heuristics
Heuristics equation
T K ofr turbulence
of = is solved
turbulence control
•forOften
control
s
thetroller observer
times tion.
bandwidth. For phasor
turbulencetroller • Often
Insteadcontrol,
is consideredtimes
bandwidth.it is
of atrying important
turbulence toInstead
a distur- that
changeis the
consideredactuators
of trying toachange distur-
ual
mize E (a sensor
â) (amize is
signals solved
â) Eand (a
given b for
are â)
known
thethe (a observer
â)
covariance
actuation given H1known
signals. optimal control
covariance are strategies.
at a There
high-receptivity are point number
for the oscillatory instability,
XKfand XCisT V the unique gain solutionKf = Y C to theT Vn :alge- bance theterm nature in aofslower
the turbulence bance
the nature
dynamical term itself,in
ofsystem,
the atheslower
turbulence
controller
such dynamicalitself, the system,
controllersuch
=Y n: Vd andpresents Vn . of excellent books expanding
e.g. a point of on this
geometric theory [39,
separation 40]. where the sensors can
V d and
ccati Vn .
equation: turbulence a tour de force trough the control
as themay strate-
rigidbe designed
body equations to as
may the
obtain be
of rigid
designed
some
an bodyother
aircraft, equations
to obtain
objective
space someof an aircraft,
other space
objective
turbulence presents a tour de force troughHere the control westrate-discuss
strate- important
measure a clean theoretical
oscillation. results The number re- of rules could easily
[Note: Kalman turbulence
turbulence
gies Tand
[Note:
publishedpresents
1 AT +his
presents
methods, Kalman
aawetour
tour willde
famous
de published
force
force
elaborate trough
TKalman
trough
in
1 laterhis
the
the famous
control
control
sections. Kalman
strate- shuttle,
while or
robustly rocket. managing In thisthe case, turbulent
uncertain fluc-
turbu-
Y AT + AY gies gies
Lower
Y C and
frequency
V methods,
Y CY forcing
+ we
AY
V will
=may 0.Y elaborate
C
also V
(17a)
serve
n inCY later
garding
the +
same shuttle,
sections.
V d =
the
purposewhile
0.various or(17a)
robustly
rocket.
be types
extended. managing
In
of this
optimal
For case,
linear the
control:uncertain
turbulent
dynamics, H turbu-
fluc-
sophisticated mathemat-
filter in a journal giesof andfilter
and Fluid nT inEngineering.]
1 methods,
methods, a journal
we
wewill willof
d Fluid Engineering.]
elaborate
elaborate in
inlater
latersections.
sections. tuations
lent disturbance.may be
1
seen as inevitable and oper-
X + XA XBR [49, 45]. BSection X + 5.4 Q= 0. a (16a)
offers least-order robust dynamical tuations
control, lent disturbance.
system may ical be
H2methods seen have as inevitable
been developed and for oper-actuator and sensor
[Decent stability [Decentfor
margins stability
LQR, margins
but no guar- for LQR, but noand guar- LQG.
ating onfastera time scale that iselusive.faster For than con-
so-called Kalmanfor 2.3
2.3such The
filter
anteed
Actuators
so-called
frequency
Actuators
Kstability
f⇣LQGis chosen and
cross-talk.
Kalman
and
margins
sensors
Infilter
sensors
to mini- the
for ⌘ LQG
discussed
K f is
(famous
chosen ating
examples,
Doyle
on
to a
mini- placement,
pa-
time scale although
that
is
optimal
is
by far the
placement
than
more
remains
con-
popular control paradigm
anteed ⇣ stability
Riccati equation 2.3
the margins
2.3 is mize Actuators
Actuators
actuation ⌘
solved for
has a and
and(famous
destructive sensors
T sensors Doyle
frequency pa- • Often
cross-talk •
with H
times the is by far
nonlinear the
turbulenceInstead •
more H
dynamics, popular
troller
is considered
2 bandwidth.
heuristic controlmethods
a distur- paradigm
Instead
for the of trying
optimization to change
Eoffor the â)observer troller bandwidth. of trying to change
2
E (a T â)T target
eper)]. (aThe per)].
â)choice given (a known
the actuators (atheand
covariance giveninteraction
â) sensors, known
their covariance
number, lo- of sensor placementbecause exist of[21].its In simple
this mathematical
survey, we focus on formula-
the
= Y C Vn : The instability. However,
The choice of the actuators and sensors, their frequency bance thenumber,because
may
termlo- of
inlo-a slower its simple
dynamicalthe nature mathematical of
system,thethe formula-
turbulence itself, the controller
such
nd Vn .
The
also choice
choice
cation,
be
cation, andof
of the
d frequency
Vconstructive.
frequency
the
V actuators
actuators
n . range
rangeIn a and and
and
mixing
and
sensors,
sensors,
amplitude
amplitudelayer, the their
their
level
level
number,
number,
has
excitation
has aanature
decisive
oflo- of
decisive
the turbulence
control logic.
itself, controller
cation, frequency range and amplitude as themay rigidbe body equations
designed to obtain mayof somean beaircraft,
designed
other space to obtain some other objective
objective
Note: Kalman
cation,
effect
Kelvin-Helmholtz
effect
published
frequency
on
on[Note:the
thehis
range
maximum
Kalman
vortices
maximumfamous
and
leads to earlierlevel
amplitude
performance
published
performance
Kalman
level
of
histhe
vortex
of
has
the
has
famous aa decisive
control
pairing,
control
decisive logic.
Kalman
i.e.
logic. 11
T
A + AY Y effect
C T
effect
to on 1
onCY
Vnexcitation
Actuators
the the
themay maximum
maximum
+ Vinclude
d = the 0. performance
(17a) mass
performance
zero-net of the
offlux 11shuttle,
the control while
control
flux actuators,
or robustly managing the uncertain turbu- the uncertain turbu-
logic.
logic.rocket.
piezo- In this while
case, robustly
turbulent managing
fluc-
r in a journal of Actuators
Fluid
Actuators
filter in
may
Engineering.]
may
aofjournal
half
include
include
actuation
of
zero-net
Fluid
zero-net
mass
frequency.
Engineering.]
mass
flux
The
actuators,
tuations
actuators,
Kelvin-
piezo-
piezo-
may 2.5
be seen Achievable lent disturbance.
as inevitable performance
and oper-
Actuators
electric may include zero-net mass flux actuators, lent piezo-
— disturbance.
electric actuators,
Helmholtz instability
actuators,
[Decent
Festo-valves
— and
Festo-valves
stability thus the
margins
(intermittent
vortex
(intermittentfor pairing
LQR,
blowing),
blowing),
but may
no
syn-
syn-
guar-
Decent stability margins
electric
electric
thetic actuators,
actuators,
jets for
[17], LQR,
Festo-valves
Festo-valves
plasma but no
actuators, guar-
(intermittent
(intermittent
and roughness blowing),
atingelements
blowing), on syn-
a time
syn- on scale thatyet ischallenging
faster than con- in turbulence control
⇣called Kalmanbefilter mitigated
thetic Kjets
anteed
⌘wall.
by chosen
is
f [17], the
plasma
stability
excitation to mini-
actuators,
margins
of higher
forandLQG frequencies.
roughness(famous
With-
elementsDoyle on An important,
pa- •popular
Hof istrying
by fartothe
question
more popular control paradigm
ed stability thetic jetsfor[17], plasma actuators, and roughness • elements byon
T margins LQG (famous Doyle pa- iswith faris the
the more control paradigm
thetic
out the jets
a dynamic [17],
[13] plasma
offers
model, aanactuators,
excellent
careful and
summary.
observation roughness of Similarly,
troller
the elements
H
flow on
sensors
2 bandwidth. Instead
achievable 2performance. change
Evidently, any investment in
the wall.
given [13] offers
known an excellent summary. Similarly, sensors
].(a â) (a the theâ)
maywall.
wall.per)].
to[13][13] offers
offers anancovariance
excellent
excellent summary.
summary. Similarly,
Similarly, because sensors
sensors ofof its simple because
mathematical ofcontroller
its simple
formula- mathematical formula-
may measure
respect measure period velocity,
forcing
velocity, pressure,
may provide
pressure, skin-friction,
insight
skin-friction, into
the temperature
effective
nature
temperature in
the
in improving
turbulence a controlitself, strategy
the may be measured in terms of
Vn . may
may measure
measure
various
control velocity,
frequency
strategies. velocity,A pressure,
pressure,skin-friction,
resolutions.
sufficiently skin-friction,
strong periodic temperature
temperature
forcing atin
in the achievable additional performance. We constrain the dis-
various frequency resolutions.
various frequency resolutions.
may be designed to obtain some other objective
various
the right frequency resolutions.
te: Kalman published Upfrequency
Up to
tohisthis day,
thisfamous
day, may themitigate
the choice
Kalman
choice ofthe
of thetarget
the actuators
actuators
11
frequency.
while
and
and sensors The in
sensors
robustly in cussion 11 to aerodynamic
managing the uncertain turbu-
problems. The minimum skin fric-
loop Up
Up
may
experiments totobethis
this day,
closedday,
is the
on
basedtheachoice
choice
long
on of
ofthe
time
engineeringthe
scale actuators
actuatorstune and
toexperience the sensors
andactuation
sensors
and on in
in
the tion in a channel flow is conjectured to be associated with the
a journal of Fluid Engineering.]
experiments is based on engineering experience lentand and on the
disturbance.
experiments
experiments
amplitude
hypothetical isis based
to the based
minimal
actuation on engineering
onlevel.engineering
mechanism Alternatively,experience
experience
which in-time and on
on the
control the steady Poiseuille profile. Similarly, the minimum drag of a
cent stability margins hypothetical for LQR,actuation but no guar-
mechanism which shall shall be be exploited.
exploited.
hypothetical
hypothetical
may For destabilize
instance, actuation
actuation
the
the mechanism
mechanism
corresponding
sensors are which
which
oscillator
placed shall
shall
before which be
the exploited.
bemitigates
exploited.
actuator of cylinder wake may also be conjectured to be associated with
stability margins Forfor instance,
LQG the sensors
(famous Doyle arepa- placed before • the H 2theis actuator
by farof the of more popular control paradigm
For
For
the the instance,
instance,
target
actuator the
the
instability.shall sensors
sensors
counteract are
are placed
placed
upstream before
before the
perturbations. actuator
actuator of
The the steady solution. In fact, most drag reduction strategies
the actuator shall counteract upstream perturbations. because of The
itsaresimple mathematical formula-
the
the actuator
actuator
The
sensors areshall
present shall counteract
counteract
mechanisms
placed downstream areupstream
upstream
helpful perturbations.
perturbations.
of theconceptsactuator, which The
can
if actuation The formulated as minimization of the fluctuation energy.
sensors are placed downstream of the actuator, if actuation
sensors
explain
sensors are
mechanismare placed
many results
placed downstream
from the
downstream of
of the
literaturethe actuator,
and ifif actuation
can potentially
actuator, actuation Yet, control studies of the cylinder wake exhibit two other po-
mechanism exploits
exploits the
the excited
excited structures.
structures. For For opposition
opposition
mechanism
guide new
mechanism
control, exploits
experiments.
exploits the
the excited
Yet, the
excited structures.
binary For
categorization
structures. For opposition
in sta- tentially desirable flow states. One is the potential solution
opposition
control, sensors
sensors andand actuators
actuators should 11 be
should be atat aa similar
similar loca-
loca-
bilizing
control,
control,
tion. and destabilizing
sensors
sensors
For and
phasorandcontrol, control
actuators
actuators it is is an be
should
should over-simplification.
importantbe at aa similar
atthat similar
the loca- which is approximated in compliant wall actuation by [73].
loca-
actuators
Fortion.
many For phasor control, it is important that the actuators
tion.
tion.
are Forcontrol
For phasorgoals,
phasor likeititjet
control,
control, isisnoise reduction,
important
important thatthe
that the actuators Interestingly, Choi [35] reports a better performing optimal
theenabling
actuators
are at
at aa high-receptitivity
high-receptitivity point point for for the
the oscillatory
oscillatory instability,
instability,
are
are at a high-receptitivity
at a high-receptitivity
e.g. point for
point for the the oscillatory
oscillatory instability,
instability,
e.g. aa point
point ofof geometric
geometric separation
separation and and the
the sensors
sensors can can mea-
mea-
e.g.
e.g.aapoint
sure point of
ofgeometric
geometric separation
separation and the
andofthe sensors
sensors can
can mea-
mea-
sure a clean oscillation. The number of rules could easily7be
a clean oscillation. The number rules could easily be
sure
sureaaclean
cleanoscillation.
oscillation. The
Thenumber
numberof ofrules
rulescould
couldeasilyeasilybebe

5
55 5
control when the cost function for drag reduction contains

Model
the difference between the controlled flow and potential so-

free
Data
lution as opposed to the drag itself. Another solution is closer
to the Kirchhoff solution and is achieved by high-frequency
forcing [74]. In fact, the drag of high-frequency forcing may
even be lower than for the completely stabilized wake [75].

Black
box
Yet, both solutions require a significant actuation energy. In I/O
fact, the gain in towing power corrected by the actuation en-
ergy is less efficient than for complete stabilization. A fourth

Controllers
highly successful drag reduction strategy consist of an aero-

Gray
dynamic shaping of the dead-water region, so that the bluff

box
body and the wake are more streamlined [7].
ROM
For high-lift configurations, the potential solution can be
conjectured to yield the maximum achievable lift. Similarly,
the maximum achievable pressure recovery in diffusers can

White
box
also be estimated from the potential solution. Summarizing, CFD
the hypothesis on achievable performance has not only a the-
oretical value. The answer may also guide the control strat-
egy.
Finally, we mention mixing enhancement and noise re-

white
Ultra
duction problems in which the cost function explicitely de- Navier-Stokes equations
pends on the history of the fluid motion. For such La-
grangian control tasks, an intuition about achievable maxi-
mum mixing or the minimum noise emission is still in its Figure 4: Model hierarchy for control design based on [76].
infancy.

derlying modes, often in terms of linear combinations of full

3 Black-box, gray-box, and white-
box models
Regardless of the modeling strategy employed below, we as-
sume that we are able to actuate the flow with some input
variables b ∈ RNb and we are able to measure features of the
flow with some output variables s ∈ RNs . Once the inputs
and outputs are set, there are many choices for the model
that links them. For example, we may consider the full dis-
cretized Navier-Stokes equations as a high-dimensional non-
linear set of ordinary differential equations. Alternatively,
we may relate inputs to outputs through either a statisti-
cal description or a set of empirical basis functions. Recent
advances in dimensionality reduction techniques and turbu-
lence closures also have exciting implications for the future
of turbulence modeling and control.
The choice of model affects nearly every downstream
control decision. There are many factors and tradeoffs that
must be balanced when deciding on a modeling strategy.
These include the accuracy of the model, execution time,
generality in other parameter regimes, spatial-temporal res-
olution relative to disturbances, and the up-front cost to ac-
quire such a model. For example, direct numerical simula-
tion (DNS) is unparalleled at descriptive resolution, general-
ity, and accuracy, but current computational capabilities are
decades away from real-time execution for in-time control
strategies. Reduced-order models based on data from DNS
or experiments provide real-time capable models, but these
models are expensive to create and may only work for a small
range of training parameters. Fortunately, it may be possible
to leverage physical intuition about the structure of the un-
flow fields, to modify the models with additional terms and
extend their predictive range. Black-box models based on
input–output data are typically faster to generate and require
less measured data, but they lack the physical interpretation
that goes with having an underlying modal representation.
Figure 4 outlines a model hierarchy for control design,
following a classification of N. Wiener [76] and extending
it to include model-free and ultra-white approaches. These
classifications are generally based on the system resolution
of the model. Some control laws can be analytically derived
from the Navier-Stokes equations. One example is energy-
based control to stabilize a flow (see Sec. 5.2). We refer to
this control-design as “ultra-white”. An expanded view of
the various models and modal decompositions is shown in
Fig. 5. This section is not meant to be exhaustive, but rather
it includes methods that have either been applied with re-
cent success or methods that have particular promise in the
future.
3.1 State-space models
In one of the most general frameworks, we will have a state
a ∈ RNa and a nonlinear function f advancing the state for-
ward in time, along with a nonlinear output function m:
d
a = f (a, b; µ), (2a)
dt
s = m(a, b; µ). (2b)
In the following, we explore possible state-spaces for a
(Sec. 3.2) and dynamic models for f (Sec. 3.3). Here the dy-
namics are assumed to be continuous, and in general the flow
state a in the Navier-Stokes equation is twice continuously

8
differentiable in space and once continuously differentiable
in time. The bifurcation parameters µ may change the qual-
itative nature of the solutions, and the flow field may not
always be continuously differentiable in µ. These parame-
ters include the Reynolds number and Mach number, among
others.
The nonlinear dynamics in Eq. (2) may be linearized at
a steady fixed point as , where f (as , 0; µ) = 0, away from
critical values of the bifurcation parameters. The linearized
model may also be converted into a frequency domain repre-
sentation, as explored in Sec. 4.
3.2 Kinematics: employed state spaces
There are numerous choices for the underlying state space in
Eq. (2), some of which are shown in the ’Kinematics’ column
of Fig. 5. This choice depends strongly on the availability
of measurements and the desired model resolution; more-
over, it should be considered whether or not the method is
data-driven or if it requires knowledge of the governing equa-
tions. The following represents a non-exhaustive set of pos-
sible state-spaces, defining a in Eq. (2). Note that many of
these state-spaces may be used in model-free approaches.
3.2.1 Full-resolution description (white-box)
A full description of a fluid flow may include a high-
resolution spatial or spectral discretization of the velocity
field
a = Du(x, t).
Here, D is a discretization operator, resulting in a high-
dimensional state vector representation of a continuous field.
Such descriptions are the basis of white-box models, which
describe every relevant feature of the flow. These represen-
tations are typically very high dimensional, sometimes ex-
ceeding the capacity of computer memory. For example, a
high Reynolds number three-dimensional unsteady flow will
exhibit important spatial structures that span many orders
of magnitude in scale. The Reynolds number can be esti-
mated from the ratio between the largest-scale structures to
the smallest structures in the flow. Thus, for a generic ge-
ometry, the state dimension will scale with Re9/4 , along with
the memory cost [77, 78, 79]. The computational cost will
scale with Re3 because of the addition of multiple temporal
scales, which generally scale with Re3/4 . For a channel flow,
the scaling may even be worse with Reynolds number, as Re3
in space and Re4 in space and time [80, 81]. If a spatial dis-
cretization is required with 1000 elements in each direction,
then a three-dimensional simulation will contain 109 states
for every flow variable (velocity, pressure, etc.).
The highest-order fully-resolved simulation to date is
a wall-bounded turbulent channel flow with Reτ = 5200
(Reynolds number based on the friction velocity), containing
2.4×1011 states [81]. This simulation is about 3.5 times larger
than the previous record holder [82], and it uses slightly over
3/4 of a million processors in parallel. Even with Moore’s
law, it will take nearly 40 years for this type of computa-
tion to become a lightweight ’laptop’ computation [83], and
9
decades longer before being useful for in-time control, since
the parallel code takes 7 real seconds per simulated time-
step, as benchmarked in [81]. However impressive and use-
ful for design and optimization, it is unclear that this level of
resolution is even necessary for many control applications.
3.2.2 Modal representation (gray-box)
Instead of resolving every detail of the flow field at all scales,
it is often possible to represent most of the relevant flow fea-
tures in terms of a much lower dimensional state. This state
represents the amplitudes of modes, or coherent structures
that are likely to be found in the flow of interest. Galerkin
models based on modal expansions constitute one class of
gray-box models, which resolve the coherent structures of
the white-box models while accounting for small scale fluc-
tuations with sub-scale closures.
The proper orthogonal decomposition (POD) is one of
the earliest and most successful modal representations used
in fluids [84, 85], resulting in dominant spatially coher-
ent structures. POD benefits from a physical interpretation
where modes are ordered hierarchically in terms of the en-
ergy content that they capture in the flow. There are numer-
ous methods to compute POD, and the snapshot POD [86]
is efficient when a limited number of well-resolved full-
state measurements are available from simulations or exper-
iments. Snapshot POD is based on the singular value de-
composition (SVD) [87, 88, 89, 90], which is both numerically
stable and efficient. POD is known under other names: Prin-
cipal components analysis (PCA) [91], the Hotelling transfor-
mation [92], Karhunen–Loève decomposition [93], and em-
pirical orthogonal func tions [94]. POD has been widely used
for flow control, as in the case of using proportional feed-
back control to reduce turbulent fluctuations around a turret
for aero-optic applications [95]. Extensions of POD specif-
ically designed for closed-loop feedback control, known as
balanced proper orthogonal decomposition (BPOD) [96, 97],
will be discussed in Sec. 4.5.3.
A recent technique, known as dynamic mode decompo-
sition (DMD) combines features of the POD and the discrete
Fourier transform (DFT). The resulting spatial-temporal co-
herent structures oscillate in time at fixed frequencies, pos-
sibly with growth or decay [98, 99, 100, 101]. Like POD,
DMD is a snapshot based method, making it appealing for
simulations and experiments alike. DMD requires time-
resolved snapshots of the same quality as needed for a
Fourier transform, although recent methods have investi-
gated sub-Nyquist sampled DMD [102]. Finally, DMD has
a strong connection to the Koopman operator, which is an in-
finite dimensional linear operator describing the evolution of
an observable function of a nonlinear dynamical system on a
manifold [103, 104, 105, 100, 106, 107].
It is also possible to construct a modal representation
of growing and decaying features in the flow based on sta-
bility modes of the linearized Navier-Stokes equation and
linearized adjoint equations [108, 109, 110, 111]. The least
damped part of the spectrum determines the coherent struc-
tures and their transient dynamics. Although completeness
of the stability modes of the linearized Navier-Stokes equa-
 3. Dusek’s jet simulation at Re=100,following a loop
classification of N.cretization
Wiener [18] and extending
model-based
(LQR):
near-quadratic control
regulator
following (LQR):
a classification of N. closed
Wiener
where each of feedback
[18] andloop
closed
the (A, control,
extending
feedback
B, C, D) encompassing
it depend
control, on 2 and
encompassing
resolvethe lin-
H
all flow Hit2 and
physics and nonlin-

encompass

tuations may be seen as inevitable

a
this results in a dynamical system for the contr

The resulting controller, known mor

timal control techniques (LQR, Kalman estimation,

as the rigid body equations of an air

to design
Kr and Kf may be designed separately refe

optim

itself, th

This means that any state a may be estimated

ana
an ac

syst

state a
feedbb

byt

a suitable time-history of the measurements s.

ar
Note that the input to the controller is the s

linea

is co
any

general sc

excellent books expanding on this the

con
Gaussian measurement noise with pro
outputs state

respectively, and the closed-loop system will

the various types of optimal c

2ivity, Complementary Sensitivity,

ating on a time scale that is faste

shuttle, or rocket. In this case, tur

Instead of tryin
b. s
Sensitivity, Complementary Sensitivity,

is consider
BKr andtracki
the state â estimates the full-state of the syste
Why this would be a terrible id

bances. However, these controllers ar

and H2 LQG. implies th

Here we discuss important theoretic
s
robustness margi
are used when r
r + Kf DKr ) â +

o
to include model-free approaches. These classifications • are

stable
In the most general framework, we will have a state

to

balances
Machine learning s

bance term in a slower dynamical

to include model-free approaches. Hetc.)optimal
These control
H classificationsstrategies.
d There
are strategies. are a number

signal
earization to point optimal
a and control
earities.
bifurcation ProblemsThere
parameter for are acases)...
number
.real-time implementa-

d or sensor

closed-loop
s

the
4. transient control of separation for
LQG, fully turbu-
the robust controllers

There
[32, 33, 34,

A
3.2 Kinematics:

have arbitrarily poorObservability,

obustness H1 (5a)

for
and Robustness 3.2 Kinematics: employed
full state state spa
â = Aâ + Bb + K (s ŝ),

will be denoted

the
may
bTand adt.
nonlinear function advancing the

measure
2 R+ based
N
generally based on themodel.
system resolution of the model. An

dynamical
f control
aaturbulence Figure 2:the
General framework for feedback control.

C = B AB A2 B · · · ANa 1 B
Model free
T(15a) a
nRbclosed-loop
J = generally control
on are
system(15a)of Figure
resolution
excellent 2:
of General
the
books framework
expanding An
tions are
on for feedback
thetheory
this computational
[39, control.
40].load[39,and

The system
dt. Qa Rb This of excellent
approximation books
will bedt expanding
approximately on this theory
valid 40].

input
35]. These controllers penalize the worse-case sce-

principle
lent0 boundary layerin[38].

the

if
an
actuationfor

poles of Aof reference

dynamics
state forward time, along with
expanded a nonlinear view output
of the various models and modal decompo-

to an arbitrary
both
The input to the controller The are the
input system
to the measure-
controller are the system measure- (5b)

on
estimation. Extremum

the turbulence
expanded view of the various models and modal decompo-

observable
rS( systems.
nsitivity For
function example,
) - sensitivity function w one may sta- narionear error Here
fixed and we discuss
Here
provide
points and we
robustimportant
away discuss ŝ important
performance.
from = C
theoretical
critical Inâaddi-
+ Dbresults
theoretical
values of re-results re-
There areunderlying
numerous cho

for linear systems, it is possible

strategies.
Input There are numerous choices for the sta

and
are
J 4.3 sig- Dynamic feedback controllers

framework
function: w J

D̂s.
â = Ââ + B̂s,
ments and the controller
sitions is shownoutputs in an
Fig. actuation
5. These sig-
methods are not meant to be

controlleroptimally
ments and the controller outputs an actuation

most
closed loop feedback control,
where (14a) s, seeking control

relies
ptimal control sitionslaw is
is shown (14a) in Fig.
where 5. These methods ares, not meant to be
Controller
ble steady-state
K a, Kdand delay ther transi-
= tion,r garding
significant the effortvarious has gonetypes intoofthe optimal
design control:
and

inputs

control
b + w , garding the various types of optimal control:
Feedback

Aa + Bb + w , b = K a, K the = bifurcation parameter where H

has full rank Na . Controllability

times turbulence
r d r Output det df
Eq. /d
(2), =
some 0. of which are H Eq.
shown (2), insome
the of which
“Kinematic a

be
4.1 Linearized input–output dynamics Turbulent Turbulent

may
2. Grey-box models, e.g. reduced-order models

controller
controller
mplementary sensitivity nal b. Theto
function exogenous inputs b. may refer
[(typicallyto a refer- for this problem...)] which leads of the form in E

.
solution -in
complementary
to the alge- sensitivity function
theexhaustive, nal butw The
rather exogenous
include inputs
methods
Dalong = LQG.0 may
that refer
have to a refer-
either been

state r,ofdisturbances
T ( as )and isboundary
the unique solution alge- wwith

r
TX X exhaustive, but rather include methods that have either been

thethe
nce,
B the layer or chan- modeling robust
It is of sensor
control,
robust andand actuators,
control, LQG.
and their Given linearOpposition
dynamics

+
possible to represent
H the state-space model

K
H ofenable
Fig.strongly
5.a This on choicethe ad

separation

will
Data
d of Fig. 5. noiseThis choice depends

observer
bCa +w . + wn(14b) (14b)state b; ),System System (1a) 2 2

CANa 1
+nequation:
Db .
withence
ȧ =r, fdisturbances
(a, d or
ence sensor
state tonoise
disturbances orforThe coherent
or sensor structure dynamics, The may

troller bandwidth.
Riccati applied with recent success
r, n. methods that have particular
n.

Ĉâ
state

Eq. (2).
r
applied recent success or
placement methods in thethat flow.have particular ten want a control
feedback controller based

controllers

matrix:
s case, thewe laminar solution becomes

gain
Often, are interested in linearizing Eq. (1) about

is
K
Feedback

2
â = (A Kf C BK

in Eq. (2) as a transfer function in the frequency

poles of
of measurements and t

exogenous

CA
P (⇣) of measurements and the desired model resolut

inaccuracy

O = ofCA
is the

the
output s cost = function z may Often
measure times any
Often turbulence
cost
times zreal-time
associ- islinear
turbulence control
considered isany inacost
experiments
considereddistur- and state esti-
a distur-

C
• in output •cost function may measure associ-

function
(1b)

they

control
promise the future.

state

..
.
promise in the future. g(a, b).There are many instances when control

2 shows
3bust control feedback: For example, if t

rank:
ost-critical robust Reynolds control numbers, but it

of
b =over, Krit a.should

domain.
H2 controller,
has a
H
inputs steady
a fixed
different
has a point
co-
different a s or
co- periodic orbit,domain leading (in to
terms of a Laplace variable over, ⇣): itsystem,
should be considered iswhether be
or not the consi
2: General
(16a) B ated d mation. However, +acceptable accuracy only
T 1 T
X X++QXA = 0. XBR X +with 0. (16a) of reference
Q =inaccuracy bance termtracking,
in aterm expense
âslower dynamical such

the
atedhave with inaccuracy
bance of
in areference
slower tracking,
dynamical expense
ssystem, such

of
be steered
control

combined.
(6a)

Controllability,
T ) =Figure 5: strategies may significant and direct impact
the âfor then the poles of the closed
the
= (A atK f C) fcontrollable,
Klevels + (B K f D) b.

choice
ed aby
Tand feedback
linear system
and control.
T ) of equations: Schematic illustrating popular choices the various of kinematic and dynamic descr
spacearbitrarily byif
data-driven or it requir

and

system in
V E(w w = data-driven or if it requires knowledge of the gover

general
w ) = V E(w w achieved for one of few dominant frequencies

the nature
control,
of control, etc.

full
dt

time-history
following
b of control, etc. s

etc.
nwill setfor r and 0,D̂where b as the rigid body equations s of an aircraft,of anspace
s, the output

n
d d d often
Ĉthe=setd KHeren
Ĉ we K
= nhave
D̂r =n and
assumed =continuous wheredynamics,
nonlinear
0, turbulent as
and the
flows, rigid
even body
away equations
from laminar aircraft,

full
may be placed choice of K

b
The

to place
ved observer 1
s, and
(3a)
to

alhas encompassed
Riccati equation aissignificant
is solved forthemod-
theturbulent
observer system , and choices for designing the controller

bility ofHtheoptimal
eistion value. [This
expectation value. [This isgain P (⇣) = C (⇣I A) B + D.
tions. The K.
following represents tions. aThe following rep
non-exhaustive

by the
3.1 State-space models and few known nonlinear mechanisms.

dt
3.1 State-space models
cost

d
in general the Navier-Stokes equationFeedback is twice con-
stability

robust
r-quadratic-regulator
a linear-quadratic-regulator (LQR) (LQR) ma- gain ma- shuttle, Feedbackor rocket.
shuttle, or In this
rocket. case,
In turbulent
this case, fluc-
turbulent fluc-

important.
separately,
Kr â.

fixed-point solutions:

the
Gramians
The abilityFigure

• Often
control,
b

The
The full-state estimate â will converge to an esti-
input

frequency
ated with
toT describe: )T(t)w dthe(⌧relevant

is
general
eed Sec. 3) T = low-

with
feedback
K really
f E(w=Y C
need
d (t)w
VE(w (⌧ sible state-spaces, defining sible
da instate-spaces,
Eq. (2). Note defin
tha
dn d ) = )and
=)linear (2a)
(2).

tinuously differentiable
for in space
+systems, isand
ittoController
possibleonce continu-
to design an optimal

garding
also
We may choose also and
choose B̂aÂ
d according Aa to Bb,
accordingthe the tuations
for Controller
linear may
systems,
tuations be seen
it
may as
is possible
be inevitable
seen to asdesign and
inevitablean oper-
optimal and oper-

matrix
B̂ mate of the
output true state if the eigenvalues of

Similarly,
most
a = Aa BKr a
signal Figure

robust
3. Black-box models,a e.g. transfer functions, A have
ments

hanisms. Performance dt issues such as

system may
when
In 1.one In of the
mean-field most general
models, frameworks,
exciting one we
oscillatory will have a state these
structures that span many b
state-spaces may
b.

In one
ously of the most
differentiable
feedback general
in time.
control frameworks,
4.2
Notice
gain Model-based
that there
and we an is will
a set
optimal have
open a state
loop
observer these
control state-spaces may be used in model-free appr
poles given
(17a) feedback control dt
ongain r and an optimal observer
ence

r,YAresulting
man T++ filter,
VdAY inY aC
= 0.resulting combined
T
V in 1
estimation-
a=combined Vord the estimation-
vorticity (17a) K
field rating on atingaK time scale Kthat
a stable.
time is
scale faster
that hibit than
is fastercon-than
important spatialcon-

a
Eq.

the
The

s CY + =
+ 0.

1
arbitrarily
(2b) are similar dynamic bandwidth as grey-box mod-
nal

Ca Db. f Cstate

by:
sturbance
(LQR):
egulator rejection,
(LQR): R Nan and
and noise
a atten-
nonlinear function Rmode
Na with
and
advancinga linear
nonlinear control
the may
function for-kill other
advancing oscilla- the state for-
the

= number
(A BK ) a.

as
of

lable ifof
of parameters that
separately, may and change
they2 the
will qualitative
be both stablena- and optimal

H
a 2 a Internal f state
separately, and and they will be bothf magnitude
stableof and optimalin scale. The Reynolds is, rrough
b =

suitable
ollerd controllerknown as known the linear-quadratic-
as the linear-quadratic- troller bandwidth.
troller Instead
bandwidth.
The of trying
Instead to change
trying to change

The
What it would look like els.
to designA black-box
the ideal Controller
model
sys- may be derived for structures to the
in

ability
is chosen
balanced to ture
with
ward mini-
robustness
in
of time, istochosen
along model
with a ward
nonlinear •
atory
in modes
D!(x,
time,output along,
[36, Dr 37],
with
function a nonlinear output function

the
o-called Kalman filter call to mini- =propagator t) ⇥ u(x, t).
Kfwhen
solutions; combined.
we these bifurcation when parameters.
combined. given g: ing, the ratiog: of the largest-scale
mine

8
QG).ssianBecause (LQG). of the separation
Because of the separation principle principle the nature theofnature
the turbulence
ofgrey-box
theetc. itself,
turbulence
3.2.1 the controller
itself,
and the
Full-resolution controller 3.2.1 Full-resolution
models in However,
vice flow.description
theversa. full-state
Oneforad- (white-bo
measurements
tem

tem, invert 6plant dynamics,

4.4
d time-delays. structures Thus, a generic geometry
dt.For â) this reason, The there
resulting controller, known more generally
dt

en known covariance Figure 2: General framework The for feedback

resulting control.
controller, a known more generally
inputs has a different co-

(15a)

R B X and X is the unique solution to the alge-

A X + XA XBR B T X + Q = 0. (16a)

A dual Riccati equation is solved for the observer

(17a)

mini-
covariance

famousofKalman

stability margins for LQR, but no guar-

for LQG (famous Doyle pa-

model Sensitivity,

Ĉ = Kr and D̂ = 0, where
gain ma-

based controller known as the linear-quadratic-

E(wd wd ) = Vd and E(wn wnT ) =
⇣ ⇤ ⌘kvalue. [This is
(11b)d (⌧ )T ) =

noise to the
Kalman filter, resulting in a combined estimation-

control (LQG). Because of the separation principle

T
+ bTâ)Rb (15a) Figure 2: General framework for feedback control.
d

given known covariance

in
E Rb
Qa dt.
(a (a (15a) Here, D is a2.discretization ensemble averages operator, of resulting
turbulent inflow
vantage high-
responses
of black-box ther
models infeasible
are easy or prohibitively
experi- expensiv
ent asThe aH controller, optimally balances the effect (2a) ofb; µ), dimension will scale with Re 3
, along with the mem
d Vpush to move away from ȧop- (2a)
r

H 22Kinematics,
input Theto=theinput fcontroller
(a, to b;theaslinear,
µ), aare H2 the
controller controller,
ȧ
Dynamics,
system
are =the optimally
fsystem
(a,
measure- A balances
Control
full
measure- the
description effect
design, b of=of K(s)
a fluid
A full flow may
description4 includ of
K

n. dimensional state may vector

Why be representation
this as
would in of
Colonius’s
be a continuous
a field.
experimental
terrible
mental idea (in
identification, jet many estimate
particularly may
for be
weakly-obtained from meas
3.2 Employed state spaces 8 • The computational â cost will scale with Re because

them. function
echniques
his famous (LQR,Kalman Kalman Gaussian
estimation,
Such measurement
descriptions are noise8
Gaussian
the withofmeasurement
basis process
white-box distur-
models, noise whichwithsig- process distur-
where

 and B̂ according
ments and the controller
to
Note: Kalman
aw isrb = Kr a, where
K a, where published
K = his famous s,
Kr = However, theses mentsKalman
= g(a,
s, and
noise b). thestudy
cases)... outputs
controller
, san actuation
outputs
= (2b)
an
g(a, sig-
nonlinear
b). resolution
actuation dynamics. dition spatial
of This or
multipleis spectral
(2b)
of temporal discretization
resolution
course, only
scales if spatial
the
[21, system
22]. of
or
If eith
sp
of
a spe
(11a)

(12a)

(12b)

mea-
adjoint

(13a)
d (13b)

discrete
since

BPOD,

Now that we have established conditions enabling

arbitrary pole placement of the closed-loop system,

controlquadratic Gaus-

(LQR) of
Navier-Stokes
e C T V ⇤ 1 CY + V = 0.

r bances. controllers are known to

eA tE(wd (t)w

he
neering.] robust controllers [32, describe
33, 34, every relevantbances. feature ofHowever,
the flow. These
these representa-
controllers are(a known to
box

chosen
n a journal H of 1.
Fluid Velocity
Engineering.] field
nal (a The
= nal exogenous
u(x, The
t)), inputs
exogenous may
inputs refer mayto a refer-
refer locity
to a field
refer- cretization = is required or the
locity
with vorticity
is observable. An observer dynamical sy
u(x, t)) field (a field
elements in (a
each =

hisdegrees
hesolutionunique to the alge-
solution
1 to thehave b.
alge-arbitrarily b. w w 1000 = u(x, t)

Eq. (2) with the addition

or LQR,stability
trollers but no margins
penalize guar- the worse-case tions are typically
sce- poor
3. robustness
veryhave
Dusek’s high jet known equationat
margins.
dimensional,
arbitrarily
simulation Instead,
sometimes
poor Re=100,robustness exceed-margins. Instead,

filter in a journal of Fluid Engineering.]

Decent Here the for
dynamics LQR,
ence statebut
are enceno guar-
assumed
disturbances
Here
state of the to be continuous,
d orwhen
dynamics
disturbances sensor are noise
assumed
or 4.
and
sensor Model-free
in The
toabe
n. isnoise r⇥u(x, approaches,
then
n.continuous,
The a Such e.g. based
three-dimensional
descriptions
and in as
constructed on qualita-
simulation
are the will
basis
follows:t)). Such descr1 contain
of whit
a,

r, t)). r⇥u(x,
These mayT be approximated by1 full-state

wrong,

d and measurement
: r,are drobustness

sensitivity
Complementary
White
regulator (LQR):

ing
robust the capacity
controllers computer
used memory. For example, high
At + b Rb dt.

H 4.3 Dynamic robust feedback

controllers controllers
are used when robustness is
.

dGprovide
(famousrobust Doyle pa-

BT,
and

H
r

stability margins 2. modal

performance.
for
general the important. LQG coefficients
flow In
(famous addi-
state of
Doyle a Galerkin
pa-
in4.costthe expansion,
Navier-Stokes equation tiveisforsteady
els,ex- state
which for every
maps, flow
are variable
generally (velocity,
restricted pressure, etc.).
turbu- describe everyisd relevant feature of the fl
K

output cost afunction mayflow measure of any a cost associ-

k

general the state in the Navier-Stokes equation els, which describe ever

H2 optimal control
output function z may measure any cost associ-
vs.

z
1 E(·) is the expectation

Reynolds number three-dimensional

transient control unsteady
separation flow will
fully

linear-quadratic-regulator
is

Discretization important.
A t

matrix, to existing one or few-parametric open-loop

â) given

Tt effort
Bhas gone into
= 0.the (16a) design and
really need

+ Q(16a)

4.5.2 Sensitivity, state-space

1 T

reduction:
BRX +Q =X0. ated with inaccuracy
ated with of
inaccuracy reference oflayer tracking,
reference expense
tracking, expense â = Aâ + Bb + K (s
(2)

Given lent lineargeneral

boundary dynamics of the
[38]. form
mostingeneral Eq. (2),schematic we of- for dt f
continuous

Figure =2s,shows the most Figure 2 schematic for

f

on where to place
identification
=

shows the
T

published
3. sensor signals

• S(⇣) - sensitivity function

are slightly

atheir control.
e

nsor and actuators, along ofwith

Parameter
projection
control, etc.
offeedback
control, etc.
Eq.

Wo ⇡ Wo = OOn .

ten control,
want a feedback controller based on full-state
Galerkin

closed loop encompassing and

e BB e Tt dt

closed loop feedback2control, H ŝ = C â + Db

8 encompassing H2 and
e 0 C Ce dt

8
⇡ Wc = CC ⇤ ,

filter
C

ved
entary isfor the observer
Sensitivity, 8

complementary
on solved for the observer
⌘

Linear
is

he flow.
Sensitivity, Complementary Sensitivity,

often set control

⇤

reduction

robust control
J =A⇤ t ⇤aT Qa

symmetric

optimal control
4.1feedback:strategies. There are a number
For example, if There
the system is

wchoose
eA t BB

H Linearized input–output dynamics

in

H b optimal =ideally Kr a. control strategies. are a number

Each of theseAtnoise⇤ A

4. sensor signals + time delay coordinates,

law
C

ERA] anteed stability margins

and Robustness [(typically for this problem...)]

and Robustness
many instances when linear controlbookscontrollable, D = 0(grey-
Stability modes

of excellent expanding on this theory 3.4

[39, Approaches
40]. to system reduction
system
braic Riccati equation:

then the poles of the closed-loop system

Kalman

of excellent books expanding on this theory [39, 40].

POD

for linear systems,

for) linear it is
systems, possible
C is it to
is auto- design an
possible to design an optimaloptimal
k

model

with ⌧ given by C(⌧ = 0, where the

enough...

Y
⌘

to
Linear-quadratic

4.5 H1 robust vs. Linear

k

(a â)T (a

Often, we are interested in results

linearizing Eq. (1) about
gain Kf = Y C Vne:
control

have- significant and direct impact fordiscuss box

re- models)

Heuristic
Here we important theoretical
Kalman
for
box

optimal
A t
Z 1

and H1 control:
+nsensitivity
= 0.+ Vfunction may beK placed Here we discuss
arbitrarily important
anbyobserver
choice of theoretical
Kr . results re-

control
S(
CYT)V correlation feedback
function, control
feedback gain
control and
r gain an roptimal
s and optimal observer
Wc BPOD

(17a)

We willpredictive
V1dCY (17a)
E Gramians
DMD

We may also
= 0. garding
Gramian:

a steady fixed point or periodic orbit, leading to

⇤

ulent flows, even away from laminar d the various typesgarding of optimal a control: types
variance matrix:

H d
AY

the bevarious of optimal

surfacescontrol:

Open-loop based control

trix.disturbance
notation

(on theH
model
separately, and
separately, they and will be both
they ofwill stable both and optimal
stable
1. Response and optimal âlevel = of (Ablack-box
Gray

direct
e

sitivity function
balanced

r is amodify
Kf C) â + Kf s + (B
SVD and

( ) - complementary sensitivity function a linear system equations:

so-called

robust control, and LQG.

1 ⇣

utions: H robust d
control, and LQG. dt

decide
is chosen to 5.
mini- sensor signals + derivatives (a
Z 1

filter is chosen to mini- 2 H

The optimal
X

control
Vd (t ⌧k=0

K when combined. when combined. = (s, ṡ)). a = Aa models BKr aare model-free approaches).
+
X

k=0

(4a)
).]

2
[Mention[Decent

f
0

not precise

f
Vd dand Vn .

⌘
Vn , where

-
0

2 optimal
sian (LQG)
[Mention[Note:

ddt times

PIDGaussian
(⇣)
• Often times turbulence
T

is considered a distur- The full-state estimate â will converg

⇣

en known covariance The resulting The controller,

resulting known
controller,
Often moreknown generally
turbulence more generally
is considered(2a) (4b)distur-
a
the

given known covariance w J

YA

w J
controllability...]

•
Balanced

eld
aH+ models, ,exciting (14a) one oscillatory
and observability

â)
T

a) robust
+ wdcontrol a == Aa + Bb,
emp.

Model
Bb
Wc =

on

(A BK ) a.
Wo =

bance term in a slower dynamical system, such

often
mate of the true state a if the eigenva
T
may kill Turbulent as a oscilla-
H2 controller, Turbulent
optimally dtbalances
bance term4 in thea slower
effectthe r
of
dynamical
effect of system, such
per)].

we must now
as a H2 controller, optimally balances
time A...mize

linear control other

of

4.5.1 H4.5.3
]
1

Nonlinear
The

and . where as the • rigid body equations of an aircraft, space

noise
[Decide

b) D̂
Db = 0,n Ĉ (14b) System System s = Ca + Db. (2b) K C are stable.
A

will a +often +setw and GaussianGaussian where

measurement as the
noise full-state
with rigid body
process equations
distur- of an aircraft, space

We mayK
measurement noise with process distur-
surements

sublished
[36,(LQR) 37], his = K D̂ = 0, f
ator his famous gain Kalman
famous
ma-
r
Kalman shuttle, or rocket. However,
In this case, turbulent measurements
fluc- of a are ei-
use

a linear-quadratic-regulator (LQR)bances. gain ma-However, shuttle, orare rocket. In

arethis case, to turbulent fluc-
system:

neering.]
uid
o-
nputs
d Engineering.]
has
according achoose
different
to the and co-bances. tuations
However,
may ther
be seen
these
infeasible controllers
these
as tuations
inevitable
controllers
or prohibitively known
andbeoper-
toknown
expensive, and so6and an oper-

white
box

B̂may
Geometric

We also according to the may seen as inevitable

4.4.1

averages of turbulent flow responses

B̂
they

T ) = V and E(w w T ) =have arbitrarily have poor

arbitrarily robustness
poor margins.
robustness
linear Instead,
margins. Instead,
(10a) control

wor
margins
ombined
an
= LQR,
filter, but
for no
LQR,
estimation- b guar-
but no guar- ating on a
( b estimate
time
, s ) scale â may
that is be obtained
faster thans from
con- measurements s. con-
as in resulting in experimental
a combined estimation- atingwhen on a robustness
time scale is that is faster than
d d n n
ear, Colonius’s robust jet controllers
robust arecourse,
controllers used are used when robustness is
Black

is (famous
xpectation
G
ns
he for LQG value.
Doyle
(famous
linear-quadratic- pa-[This
Doyle is Hpa-1troller H bandwidth.
1 This is Instead
of of
Volterratrying
only to
series
if thechangesystem of equations (2)
Feedback Feedback
y=controller known asT the linear-quadratic- troller bandwidth. Instead of trying to change

7
ally
he , separation
need principle
E(wBecause
d (t)wd (⌧of ) )the =important.
the nature important.
of the turbulence itself,SVM theofcontroller
sian (LQG). Controller
separation principle is observable. Controller the An observer
nature the dynamical
turbulence
nonlinear itself,may
system be
the controller
signal s, the output is the actuation signal b, and
the state â estimates the full-state of the linear sys-
tem in Eq. (2). The separation principle states that
Kr and Kf may be designed separately to deter-
mine the stability of the state dynamics for a and â,
respectively, and the closed-loop system will have
poles given by the poles of A BKr and A Kf C.
The most general observer dynamical system is

(8a)
(8b)

and
The general controller will be denoted by K(⇣)

The ability to place poles of the closed-loop system

arbitrarily with choice of Kr relies on the controlla-
bility of the system in Eq. (2). The system is control-

(9a)

has full rank Na . Controllability implies that the

system may be steered to an arbitrary state a with

Similarly, the full state is observable if the observ-

This means that any state a may be estimated with

Figure 2 Figure shows the
2
constructed showsmost as general
the most
follows: schematic
general schematic for for
tulator
simulation at Re=100,
(LQR): 8 closed loop closedfeedback loop feedback control, encompassing control, encompassing H2 and H2 and
a suitable time-history of the measurements s.

mentary
Complementary Sensitivity, Sensitivity, 8 Machine-learning
H1 optimal H1 controloptimalstrategies. control d There are a number
ontrol of separation for fully turbu- â strategies.
= Aâ + There are(s a number
Observability,

ŝ),control(5a)
suitable time-history of the control input b.

ss Bb + K
⇤

f
Figure 2: General framework for feedback control.
C = B AB A2 B · · · ANa 1 B
free

a
darya)+ blayer T
Rb dt. [38]. (15a)of excellent Figure 2:
books
of excellentGeneral expanding
booksframework on this
dt expanding for theory
feedback
on this [39, control.
40]. [39, 40].
theory
function The input to the controller The
Here areinput
wethediscuss
system
Here to the measure-
controller
weimportant
discuss are
ŝ important
= the
Câsystem
theoretical +theoretical
Dbresultsmeasure- re-results re- (5b)
Extremum-
Input-
Model

b = Ĉâ + D̂s.
â = Ââ + B̂s,

=is b =ments Kr a, and theKcontroller

s, where r garding the
= outputs
ments s,
garding an
and
various
Output
actuation
the controller
the typesvarious sig-of typesoutputs
optimal an
of control: actuation
optimal H sig-
control:
1
seeking
H1 control
ed
sitivityinput–output
nal functionThe dynamics
exogenous inputs may refer
[(typicallyto a refer- for this problem...)] which leads
O = 6 CA 7 .

ntary
e- sensitivity
b. function
unique solution to the alge-robust control, nal wb. The exogenous
and H2 LQG. inputs
D = 0
w may refer to a refer- Opposition
3
6 CA 7
7
2 7
7
5

robust Data control, and H LQG.

n. The d2 or sensor noise n. The
CANa 1

ence state r, disturbancesence d orstate sensor r,tonoise

disturbances control
lable if the following matrix:

nterested in linearizing Eq. (1) about

output cost function z may • Often measure times any
Often turbulence
cost
times associ- is measure
turbulence considered isany acost
considered distur- a distur-
C

output •cost function z may associ-

..
.
ability matrix is full rank:

trol1 Ba
in the frequency domain.

point T or periodic orbit, leading to

s
R a) ated
X +with 0. (16a) of reference
Q =inaccuracy bance
ated with term indaterm
tracking,
inaccuracy
bance expense
âslower =in of(A dynamical
areference
slower system,
tracking,
dynamical such
+ Koffexpensessystem, such D) (6a)
4.4 Controllability,

of equations: Figure 5: Schematic illustrating popular choices at the K f C) â

various levels + (B and
kinematic Kfdynamic b. description of the turbulent
2

6
6
6
4

and = K of
D̂ = control,
and0,D̂where etc.
= 0, whereas of the
control,rigid asetc.body
the dt rigidequations
body equations of an aircraft, of anspace aircraft, space
eris solved
r
forsystem P , and choices for designing the controller K.
the observer
dt
d

ator
d (LQR) gain
atic-regulator ma-gain ma-shuttle, or
(LQR) rocket.
shuttle, orIn thisestimate
rocket. case,
In thisturbulent
âcase, fluc-
will turbulent fluc-
Gramians

The full-state converge to an esti-

d
oseB̂a for
=
andlinear
Aa
accordingB̂ +systems,
toBb,
the
according it
tois possible
the (2a)
tuationsto design
may
for lineartuations an
be
systems, optimal
seen
it
may as
be inevitable
is possibleseen to as
design and an oper-
inevitable optimal
and oper-
dt 1 tion has onlygain
beenK shown
mate of the true state a if the eigenvalues of A
a)
ombined
ngVnin afeedback
combined control
estimation- estimation-
(17a) andforan
feedback
rating geometrically
on optimal
control
a time
ating on simple
observer
gain
scale
a configura-
that
time
K andis
scale an
faster
thatof non-normality
optimalthan
is observer
con-
faster than of the evolution operator [115, 116]. One
con-
⇥

CY +
s = CationsV d = 0.
+ Db. (2b) assumedKf Cand are stable.
r
given by:

[112,they
113],will
it is generally has also been nu- feature of non-normality can be large transient growth due
own asseparately,
he linear-quadratic- and
the linear-quadratic-
merically corroborated.
be bothbandwidth.
troller
For
stable
separately, and
troller
Stokes
and optimal
they will
Instead
bandwidth.
eigenmodes,
be both
arisingof
stableof
trying
Instead
from
and
to
to the
optimal
change
trying to change
destructive interference of nearly parallel eigenvectors,
ni-
lter K is
when chosen
combined.to mini- when combined.
he ⌘ separation
ause f
principle
of the separation principle
a linearization around thevanishing
naturethe of thecompleteness
nature
flow, turbulence itself,
of the turbulence
and or- the
even controller
itself,
withthe controller
very similar eigenvalues. This phenomena may be
ce
) given knownThe resulting
covariancecontroller, known 6 more generally
The resulting controller, known more generally
thogonality can be shown for a large class of boundary con- especially pronounced in shear flows, such as channel flows,
as a H2ditions
controller,
[114]. optimally
as a balances the effect
H2 controller, of
optimally balances
andthe effect offor flow control.
is important
an Gaussian In 8
measurement
many fluid 8
noise
systems,with
Gaussian process
measurement
large transientdistur-noise
energy with
growth process
The distur-
modal decompositions discussed above are approx-
lished his famous Kalman
bances. may
However, these bances.
be experienced, controllers
even arestable
in linearly
However, known to controllers
systems,
these because imations
are basedtoon data snapshots of the full system, most
known
d Engineering.]
r- have arbitrarily poor robustness margins.poor
have arbitrarily Instead,
robustness margins. Instead,
rgins for LQR, but no guar-
a- H 1 robust controllers areH used when
robust robustness
controllers is used when robustness is
are
for LQG (famous Doyle pa- 1
important. important. 10
Figure 2 shows the most Figuregeneral2 schematic for
shows the most general schematic for
closed loop feedback control, encompassing
closed loop feedback and
H2control, encompassing H2 and
y,
mplementary Sensitivity,
H1 optimal control strategies. There are
H optimal a number
control strategies. There are a number
likely from a white-box model or experiments. We then
have many of the same problems as in the white-box models:
since the data may be exceedingly large, even simple tasks
such as computing the inner product of velocity fields are
cumbersome. Fortunately, the expense to compute a modal
decomposition is a one-time upfront cost during the train-
ing phase, and resulting reduced-order models are generally
much faster in the execution phase. There is a new software
package in Python, called modred from ’model reduction’,
that efficiently computes various modal decompositions and
reduced-order models for systems with large data [117].
The oldest class of gray-box models are based on vor-
tex representations starting with Helmholtz’s vortex laws in
1869. Von Kármán’s kinematic model [118] of the vortex
street is one famous example. Some of these vortex models
have been used for control applications. The Föppl (1906)
vortex model of the cylinder wake has, for instance, been
used to design a controller stabilizing the wake [119]. Suh’s
vortex model [120] of a recirculation zone has been used for
flatness-based control targeting mixing enhancement [121].
Most vortex models, however, are of hybrid nature, i.e. vor-
tices are continually produced, merged or removed. This
implies that the dimension of the state space as well as the
meaning of the coordinates continually changes. Such hy-
brid models are a challenge for almost all control methods.
We shall not pause to elaborate these Lagrangian gray box
models as there exist excellent textbooks on the topic [see,
e.g., 122, 123, 124] and the control applications are sparse.
3.2.3 Input–output (black-box)
Models that are built purely on input–output data are re-
ferred to as black-box models because they are opaque with
respect to the underlying structure of the fluid. However,
what black-box models lack in flow resolution, they make
up for in rapid identification and low-overhead implementa-
tion. These models may be based on a state space comprised
of the sensor and actuator signals
h iT
a = sT , bT ,
a time-history of sensor signals using time-delay coordinates
h iT
a = s(t)T , s(t − τ )T , s(t − 2τ )T , · · · , b(t)T , · · · ,
or these signals and their derivatives
 T
d T d T
a = sT , b T , s , b , ··· .
dt dt
3.3 Dynamics: classification by system res-
olution
Some of the modal representations above are fundamentally
linked to a dynamic model. For instance, DMD results in
spatial-temporal coherent structures along with a low-order
model for how these modes oscillate and/or grow or decay.
Likewise, BPOD results in a balanced linear model. For other
representations, it is necessary to build a model on top of
11
the data separately. The various methods to determine f in
Eq. (2) are illustrated in Fig. 5 under the ’Dynamics’ column.
3.3.1 White-box models
For white-box models, the discretized Navier-Stokes equa-
tions are used to resolve all flow physics and nonlineari-
ties. When appropriate, e.g. near a fixed point or periodic
orbit, the linearized Navier-Stokes equations may be used.
As discussed earlier, there are significant hurdles to real-
time implementation of these models based on the compu-
tational load. In addition, full-state estimation may be re-
quired when using a white-box model to control an exper-
iment where measurements are limited. Estimation may be
statistical, such as linear stochastic estimation (LSE) [125, 126,
127, 128, 129, 130, 131, 132], or dynamic, as in the Kalman fil-
ter [133, 134, 135, 136]. However, these models are able to
accurately capture important dynamic events, such as bifur-
cations, which define the phenomenological landscape.
There are a variety of methods in computational fluid
dynamics (CFD), including the finite element method, spec-
tral elements, finite volumes, finite differences, immersed
boundaries, point-vortex methods, and many more. As the
Reynolds number increases, DNS becomes prohibitively ex-
pensive, and it is necessary to use turbulence models for
smaller scale fluctuations. Finding turbulence closures to
these models is an active field of research, and common
methods include Reynolds averaged Navier-Stokes (RANS)
and large eddy simulations (LES). These may be thought of
as off-white box models, since they still resolve many orders
of magnitude in scale. However, these models must be used
with caution and experience.
3.3.2 Gray-box models
Reduced-order models may be obtained in terms of the dy-
namic interaction of coherent structures, such as those ob-
tained through POD, BPOD, DMD, or global stability analy-
sis. The resulting models enable real-time control in exper-
iments since the computational burden is limited and state
estimation is often feasible based on limited measurements.
However, acceptable accuracy is typically only achieved
near the training parameters, for one or a few dominant
frequencies, and for a few known nonlinear mechanisms.
More robust mathematical reduced-order models may be ob-
tained for simple geometries from Hilbert space considera-
tions without advance data, but at the price of a significantly
increased dimension [137, 138].
Galerkin projection of the Navier-Stokes equations onto
a modal basis is a common technique to obtain non-
linear reduced-order models. The ’traditional’ Galerkin
method [139] requests the same orthogonal modes for the
expansion and as test functions for the projection. POD-
Galerkin models are the most prominent corresponding ex-
ample in fluid mechanics [140]. Although BPOD has an as-
sociated linear model, it is possible to use BPOD modes for a
nonlinear Galerkin projection. Gray-box models will be dis-
cussed in more detail in Secs. 4 and 5.
3.3.3 Black-box models
Transfer functions and state-space models identified purely
from input–output data (i.e. measurements of the sensor and
actuation history) are black-box models. They have similar
dynamic bandwidth as gray-box models, and in fact may
have identical input–output characteristics. However, results
and predictions lack the clear interpretation available when
model states represent modal amplitudes, as is the case with
gray-box models. A black-box model may be derived from
a gray-box model, and the opposite may also be true if ad-
ditional snapshot information is available. For example, it
may be possible to relate the internal model states with linear
combinations of the flow snapshots, resulting in a gray-box
model. A major advantage of black-box models is easy ex-
perimental identification, particularly for linear and weakly-
nonlinear dynamics.
There are many techniques available to obtain black-
box models, and a full exploration is outside the scope of
this review. However, the eigensystem realization algorithm
(ERA) [141, 142] is a promising algorithm that produces bal-
anced models from input–output data. Other methods are
discussed in [143, 144].
The ERA, discussed more in Sec. 4.5.4, results in a linear
state-space model based on impulse response data; i.e. sen-
sor measurements in response to an impulsive delta function
input in the actuation. The impulse response of a multiple-
input, multiple-output (MIMO) system will be given by h(t),
a function of time with Ns rows and Nb columns. It is possi-
ble to predict the sensor measurements s(t) in response to an
arbitrary input signal b(t) for linear systems by convolution
of the actuation signal with the impulse response:
Z t
s(t) = s(0) + h(t − τ )b(τ )dτ.
0
The notion of convolution may be extended for generic non-
linear systems, such as in Eq. (2), by a Volterra series [145,
146, 147]. To simplify notation, we consider a single-input,
single-output (SISO) system:
N Z
X t Z t
s(t) = s(0) + ··· hk (t − τ1 , · · · , t − τk )
k=1 0 0
×b(τ1 ) · · · b(τk )dτ1 · · · dτk .
The functions hk are called Volterra kernels, and there are
existence and uniqueness theorems for a large class of non-
linear systems [148]. There are also uses of Volterra series in
geometric control theory [149]. Notice that the first integral
in Eq. (4) for k = 1 is the linear impulse response from Eq. (3).
As a simple example to demonstrate Volterra series, con-
sider a SISO system with linear dynamics and a quadratic
output nonlinearity1 :
d must be balanced with robustness to model uncertainty and
a = Aa + Bb
dt
s = (Ca)2 .
1 This example was adapted from notes by Nicholas R. Gamroth.
The output s(t) is the square of the convolution in Eq. (3):
Z t 2
s(t) = s(0) + h1 (t − τ )b(τ )dτ ,
0
Z tZ t
= s(0) + h1 (t − τ1 )h1 (t − τ2 )b(τ1 )b(τ2 )dτ1 dτ2 ,
0 0
Z Z
t t
= s(0) + h2 (t − τ1 , t − τ2 )b(τ1 )b(τ2 )dτ1 dτ2 .
0 0
Here, h2 (t − τ1 , t − τ2 ) = h1 (t − τ1 )h1 (t − τ2 ), so that static
output nonlinearities have simple higher order kernels.
Volterra series have been used with back-stepping and
boundary control of PDEs for fluid flow control [150]. They
have also been used for general fluid modeling [151], to cap-
ture aerodynamic and aeroelastic phenomena [152, 153, 154],
and to model and control plasma turbulence [155, 156, 157].
Support vector machines (SVMs) [158, 159, 160] are
a new class of supervised nonlinear models that have a
tremendous amount of potential for the control of complex
dynamical systems. SVMs are supervised models that take
input data into a high-dimensional nonlinear feature space,
which may then be mapped back down to inputs. They are
related to the Volterra series above. Knowledge of the kernel
is a central part of the algorithm, and in practice, a suite of
kernels may be tested to optimize model performance. SVMs
have not penetrated the turbulence control literature signifi-
cantly, but we expect this method to become more prominent
in fluids in the future.
3.3.4 Model-free approaches
Model-free approaches do not rely on any underlying model
description relating inputs to outputs. Instead, these ap-
(3) proaches are often based on qualitative steady state maps
and are generally restricted to existing one or few-parameter
open-loop control. Steady-state maps typically assume
working periodic control and the maps relate input param-
eters to outcomes in some objective function. However, re-
cent methods allow the identification of controllers using ma-
chine learning and adaptive control.
(4)
4 Linear model-based control
Many results in closed-loop turbulence control are specific
to linear systems. For example, one may stabilize an un-
stable steady-state and delay the transition to turbulence in
the boundary layer or channel flow. In this case, the steady
(laminar) solution, which becomes unstable for post-critical
Reynolds numbers, may be stabilized by feedback control.
Such work has encompassed a significant modeling ef-
fort (see Sec. 3) to describe the relevant low-order flow mech-
anisms to be suppressed or utilized. Performance issues such
as bandwidth, disturbance rejection, and noise attenuation
time-delays in sensing, actuation, or computation. For this
reason, there has been a recent push to move away from
H2 optimal control techniques (LQR, Kalman estimation,
LQG, etc.) to the robust H∞ controllers [161, 162, 163, 164].
12
These controllers guarantee robust performance by penaliz- For linear systems, the output frequency will be the same
ing worst-case performance in the design process. Progress as the input frequency ω, the magnitude is given by |P (ζ)|,
has also been made in the design and modeling of sensors and the phase is given by ∠P (ζ). For example, consider a
and actuators [36], along with their placement in the flow. single-input, single-output (SISO) linear system, where an
There are many instances when linear control strategies input sinusoid sin(ωt) will excite an output measurement
may have significant and direct impact for nonlinear turbu- A sin(ωt + φ), where A and φ are the magnitude and phase
lent flows, even away from steady fixed-point solutions. For angle of the transfer function evaluated at ζ = iω: A = |P (ζ)|
example, in mean-field models, exciting one oscillatory mode and φ = ∠(P (ζ)). In the context of controls, the system P is
with linear control may suppress other, more energetic, os- known as the plant. The roots of the denominator of P are re-
cillatory modes [48, 47]. In addition, ensemble averages of ferred to as poles and the roots of the numerator are referred
turbulent flow responses may be linear as in [165, 41]. Other to as zeros. The plant is unstable if any poles have a positive
examples include the transient control of separation for the real part; if P is based on Eq. (6), then the poles of P corre-
fully turbulent boundary layer [41]. spond to eigenvalues of A.
The main goal of this section is to develop an overview Both the state-space and frequency domain representa-
of the linear control framework and provide context for the tions are useful. It is often beneficial to design specifications
related literature in flow control. Ideas such as control topol- and assess controller performance in the frequency domain,
ogy (feedback vs. feedforward, open-loop vs. closed-loop), although it is often simpler to achieve these goals by manip-
controllability and observability, state estimation, pole place- ulating the state-space system [166]. Generally, an effective
ment, and robustness (sensitivity, bandwidth, stability), are open-loop plant will have high gain at low frequency for dis-
introduced with an emphasis on physical interpretation and turbance rejection and reference tracking, low gain at high
engineering implications. Since[(typically
turbulentDflows for this
= 0 often have problem...)]
frequencywhich
for noiseleads Similarly,
attenuation, and a goodthe fullmargin
phase state isatobservable if the
to
very fast time-scales, many of the topics above are consid- crossover for stability. ability matrix is full rank:
ered in light of computational complexity4 Linear and the model-based
latency control Open-loop
of the control decision, leading dto a discussion on reduced- 2 control 3
â Many
= (Aresults Kf C) â + K f4.2
in closed-loop s + Model-based
(B K
turbulence
f D) (6a)
b.
controlopen-loop
are control 6 C 7
order models. dt
specific to linear systems. Fora example, 6 CA 7
Controller
This section is not meant to be an exhaustive overview With model of one may sta-
the form in Eq. (5) or Eq. (6), 6 it may
CAbe 2 7
of either linear control theoryThe
or itsfull-state
application
bilize an estimate
in flow â
unstable con-will possible
converge
steady-state and todelay
an an esti-
the O=6
transi- control law to 7.
to design open-loop achieve
6 some .. 7
trol. For more details on control
matetheory, there
oftion
the are
true two
state recom-
if the
to turbulence, as indesired
a eigenvalues
the boundary of
layerwithout
specificationA or chan- 4
the use of measurement-based 5 .
mended texts, [166] (practicalK engineering),
f C are
and [167] (math- feedback or feedforward control. For instance, ifCA
nelstable.
Na 1
flow. In this case, the laminar solution becomes perfect
ematical theory). There have also Combined
been a number withof excellent
full-state feedback
tracking of thenumbers,
b = reference
K â, inputit wr is desired in Fig. 6, un-
recent reviews involving various aspects unstable for post-critical
of the subject of lin- derReynolds
r but
this results
may in
bea stabilized
dynamicalby system
feedback certain
for the circumstances
controller: it may
control. Thisbe possible
means to design
Figure
that any 3:aOpen-loop
con-
state a may be control
estima
ear closed-loop flow control [168, 34, 20, 169, 170, 171]. troller by inverting the plant dynamics P : K(ζ) = P −1 (ζ).
Such work has encompassed a significant mod- a suitable time-history of the measurement
In this case, the transfer function from reference wr to output
eling effort (see Sec. 3) stoisdescribe the relevant Although the definitions of controllab
4.1 Linearized input–output d dynamics given by P P −1 = 1, solow-
that the output
whereperfectly
each ofmatchesthe (A, B, C, D) de
â =order (A fluid Kfmechanisms.
C BKr +the KPerformance
DKr ) â +
freference. K(7a)
issues
However,f s, such observability
perfectas control
4.4.1 is abovepossible
Balanced
never aremodelbinary, there are d
reduction
dt
Often, we are interested in linearizingbandwidth,
Eq. (2) aboutdisturbance
a steady real-world earization point ain
s and bifurcatio
rejection,systems,and noise controllability
atten-
and this strategyThis should and be used with willbased
observability, on
b = K â.
fixed point a , corresponding to a desirable flow state. This4.4.1
s r (7b) approximation be appr
uation must be balanced Balanced
with since
caution, robustnessmodel reduction
to model
it generally ficult
relies on it is to
4.5anear
number
H1 control
robust or estimate
of significant 4.5.2
vs. H optimal Th
a Sensitiv
state a. co
leads to a linear system of equations: fixed points and2 away from
uncertainty and assumptions
time-delays. For on
this the plant
reason, of controllability and observability
.
there
P First, effective control based andof aRobgiv
Note that the input to the 4.5 controller is
1 robust
Hmove the sensor
vs.requires
H2Hoptimal the
control
Now bifurcation
that haveparameter where de
d
a = signalAa + Bb, hasthebeen a recent push
(5a)
ontoplant inversion
awayb,from extremely
is quantified
2 op-
by we
precise knowledge
eigenvalues established
ofand eigenvecto condi
dt s, output is the actuation signal and
P and well-characterized, predictable It is possible
disturbances;
arbitrary to represent
there is •ofS(⇣)
pole placement the
senss
the -closed
timal control techniques (LQR, Kalman estimation, controllability Gramian
the state estimates the Now
full-state that
of we
the have
linear established
sys- conditions enabling
s = Ca + Db, â
LQG, etc.) toseparation
(5b) little room for model errors or uncertainties,
the robust controllers
H1 measurements [32, 33, weinmust
34,closed-loop
Eq. (2) asas
now a transfer
there
decideare noon function
where toPplac (⇣)
tem in Eq. (2). The arbitrary
principle
sensor pole placement
states that to of the
determine if performance
domain system,
Z 1
(in is
terms as ex-
of a T (⇣) - varia
•Laplace com
where each of the matrices (A, B, C,35]. D) depend on the lin- penalize the worse-case sce-
Kf These controllers ⇤
earization point as and bifurcation Kr and parameter mayµ.be The statewepected
designed must
separatelyand no
now tocorrective
decide deter- feedback
on where to mechanisms
place
Wc them. = to modify eAt BB the⇤ eA t dt
nario
minebetween
the errorof
stability andtheprovide robust performance.
actuation
state dynamics strategy
for a and â,In addi- 4.5.1 H2 optimal
to compensate. 0 • w r - referen
1
vector a refers to the difference the current flow P1 (⇣) = control:
C (⇣I Linear ⇣ ⇤ qu
A) B ⌘+
and the fixed point a . The respectively,
s tion, significant
system is linearlyand stable effort
the closed-loop
if all has gone
For into
open-loop
system will have the design
control and
using plant inversion, X the plant k
must also along
be stable. Ittheir
is impossible sian
to (LQG)
⇡ fundamentallye A t k
BB e ⇤ A t
of the eigenvalues of A are poles modeling
in the given
left-half of of
the sensor
complex 4.5.1
and
by the poles of A BKr and A Kf C.
P actuators,
H 2 optimal control:
with Linear quadratic Gaus- 4.5.3 H 1 robu
4.4.1 having
plane, Balanced model
negative reduction
real parts. The
The placement
lineargeneral
most in the
approximation flow. change
observer 4.5.2 sianthe
dynamical
dynamics of a linear system
(LQG)
Sensitivity,
system is ComplementaryWe4.2
through
mayModel-based
k=0open-loop open loop co
Sensitivity,
often modify Eq. (2) with th
in Eq. (5) will be approximately validby:
nearThere
fixed points control, and thus an unstable plant cannot be stabilized with-
with- instances We will often
given are many when
and linear control white noise disturbance
Robustness wdlikeandto meas
4.5purely robust eigenvalues
vs. H2 optimal control We out
may feedback.
often Attempting
modify Eq. and
(2)to observability
stabilize
with the • What
an
addition it would
Gramian:
unstable of plant look des
out H1imaginary andstrategies
away frommay critical
have val-significant and direct impact for K r is a linear-q
by inverting the dynamics will w n:
typically tem,have disastrous
invert plant
ues of the bifurcation parameter (i.e. nonlinear
when det df /dµ
d turbulent
6
= white
flows,noise
0). • S(⇣) - sensitivity
even disturbance
away from laminar function
wd and measurementZ noise trix. We may et
dynamics, al
Now that ifwe
Fortunately, haveisestablished
control effective, theconditions
flow state shouldenabling
â = stay
Ââ
w + : B̂s, (8a)
1
A t ⇤Kalman
⇤
filter, r
fixed-point dt solutions: n Wo• Why = function d e would
this C Cebe dt At
arbitrary
close pole placement
to as , where of the closed-loop
the linear approximation is valid. system,
Similar • T (⇣) - complementary sensitivity 0 a = based Aa + aBb terribl
+ wd
controlle
linearization maydecide
be applied near a periodic (8b) cases)...1dt⇣
Inorbit of
we must now on where to 1.
place them.b the = flow. Ĉ â + D̂s. ⌘k
mean-field models,• exciting one oscillatory
dController X
seA⇤= t Gaussian (LQGk
It is possible to represent the state-space model in Eq. (5) wr - reference
a = Aa + Bb + wd , Plant
tracking ⇡ (14a) Ca C ⇤+CDb eA+ twn
as a transfer function P (ζ) relating
mode with linear control may dt kill other oscilla- for linear system
Thethe frequency
general of sinu-
controller will be denoted by s K(⇣)
• P k=0
(14b)
tory modes [36, 37], = Ca + Db + wn .
4.5.3 H1 robust control Each of these noise feedback contro
soidal input forcing to
4.5.1 H2 optimal control: the magnitude
Linear
in the and phase
quadratic
frequency of
domain.the
Gaus- out- inputs has a
put response; here ζ = iω ∈ C is a Laplace variable: separately, and
sian (LQG) 2. ensemble averages of turbulent flow responses variance
These maymatrix:be approximated
E(wd wd ) by T = V d and
full-st
EachWeof will these often noiseset inputs has a different
rVand 0,co- where
P (ζ) = C (ζI − A)−1 B + D.may be linear,(6) as in Colonius’s experimental
Figure
Ĉ =surements
K
T ) =jetVcontrol
6: Open-loop of
n , where
D̂the=E(·)
topology. direct thewhen
issystem combined
in Eq.
expectation (2) and
v
We may often modify Eq. 4.4(2) Controllability,
with the addition ofObservability, variance
K ismatrix:
a E(w and w
linear-quadratic-regulator
d d system: d and4.3 E(w Dynamic
(LQR) n w T) =
gain feedback
ma- The contro
resultin
noise study , r not precise enough...
n really need E(w
white noise disturbance wd and measurement noise Vn , trix.
Gramians where WeE(·) mayisalso the choose
expectation  and value. [This isto theas a H2 control
B̂ according
Vd Given ⌧ linear
).] T dynamics
13 precise enough... really need E(w (t of the form
wn : 3. Dusek’s jet simulation not Kalman filter, resulting in a combined
at Re=100, d (t)w (⌧ ) )=
d estimation- eGaussian meas
The ability to place poles ofVthe(tclosed-loop system Linear-quadratic
ten want ⇡ Wregulator
Wca feedback CC ⇤ , (LQR):
c =controller ba
d based ⌧ ).]controller known as the linear-quadratic-ebances.⇤ Howe
arbitrarily with choice of K relies
4. transient controlr ofLinear-quadratic on the
separation(LQG). controlla-
for fully turbu-(LQR): feedback: W b
o Z =
⇡ WK a.
= For
OO example
.
d Gaussian regulator
Because of thecontrollable,
separation then principle
r
1 ohave arbitrarily
a = Aa + Bb
bility of +
thewsystem
d , boundary
lent (14a)
in Eq. (2). The[38].
layer system is control- the
T poles of T the clo
 consequences. For instance, consider the following unsta- yield:
ble plant with a pole at ζ = 5 and a zero at ζ = −10:
P (ζ) = (s + 10)/(s − 5). Inverting the plant would result in d
a = Aa − BKr a, (7a)
a controller K = (s − 5)/(s + 10); however, if there is even dt
the slightest uncertainty in the model, so that the true pole is = (A − BKr ) a. (7b)
at 5 − , then the open-loop system will be:
If the system is controllable, as defined in the next section, the
s−5 closed-loop poles, given by eigenvalues of A − BKr , may be
Ptrue (ζ)K(ζ) = .
s−5+ placed arbitrarily by choice of Kr . Pole placement has been
This system is still unstable, despite the attempted pole can- used to control combustion instabilities [173].
celation. Moreover, the unstable mode is now nearly unob- However, full-state measurements of a may be either in-
servable, a concept that will be discussed later. feasible or prohibitively expensive, and so an estimate â may
In addition to stability, the plant P must not have any be obtained from measurements s. This is of course, only if
time delays or zeros in the right-half plane, and it must have the system of equations (5) is observable. An observer dynamical
the same number of poles as zeros. If P has any zeros in the system may be constructed as follows:
right-half plane, then the inverted controller K will be un-
stable, since it will have right-half plane poles. These plants d
â = Aâ + Bb + Kf (s − ŝ), (8a)
are called non-minimum phase, and there have been general- dt
= 0 for this problem...)] which
izations to plant leadsthat provide
inversion Similarly,
bounded theinverses
full state
to is observable if the
ŝ observ-
= Câ + Db. (8b)
ability matrix is full rank:
these systems [172]. Similarly, time-delays are not invertible,
near model-based and if P has control
more poles than zeros, then the resulting
Open-loop con- Typically D = 0 for this problem, meaning that there is no
control
2 3 feedthrough from actuators to sensors, although exten-
direct
A troller will not be realizable
(6a) and may have extremely large ac-
C
sultsKinf C) â + Kf s +turbulence
closed-loop (B Kf D) b.
control are 6 CA 7 exist when D 6= 0; see [166]. Keeping D in the calcula-
tuation signals b. There are also generalizations that provide sions
o linear systems. For example,
regularized one may where
model inversion, sta- optimization schemesController
6 are tions, Plant Eqs. (8a) and (8b):
7 and combining
2 6 CA 7
eunstable
estimatesteady-state
â will
applied converge
and penalty
with to anthe
delay esti-
termstransi-
added to keep the resulting O = 6 ac- 7. (10a)
6 .. 7 d
rue state if the
tuation eigenvalues
signal
urbulence, as in the boundary layer or chan-
a b of
bounded. A These regularized open-loop 4 con- . 5 â = (A − Kf C) â + Kf s + (B − Kf D) b. (9)
trollers are often significantly more effective, with improved dt
ble.In this case, the laminar solution becomes CANa 1
dfor with full-staterobustness.
feedback The estimate â will converge to the full state a if the eigen-
post-critical Reynoldsbnumbers, = Kr â,but it
a dynamical system Combined,
for these restrictions on the plantFigure
the controller: P imply that valuescontrol
3: state
Open-loop A −topology.
of estimated Kf C with are stable. Again, it is possible to place
tabilized by feedback control. This means that any a may be
model-based open-loop control should only be used when the poles of A − Kf C arbitrarily if the system is observable.
work has encompassed the plant isa well-behaved,
significant mod- a suitable
accurately time-historybyofathe measurements s.
characterized Combined with full-state feedback b = −Kr â, this re-
ort (see Sec. 3) model, to describe whenthe relevant low-
disturbances Although
are characterized, and thewhendefinitions of controllability and
where each of thethe (A, B, sultsC,inD) depend on
a dynamical systemthe forlin-the controller:
id mechanisms.
Kf C BKadditional r Performance
+ Kf DK r ) âissues
feedback +K (7a)
control
f s, asobservability
suchhardware 4.4.1 above model
is unnecessarily
Balanced areex-binary, there are degrees of
reduction
earization point as and bifurcation parameter . 4.5.2 Sensitivity, Complementa
dth, disturbance pensive.
rejection, Otherwise,
and noise (7b) atten-controllability
performance goals must and observability,
be modest. d based on how dif- and Robustness
K r â. 4.4.1 Balanced model reduction This approximation will âbe
4.5.2 =approximately
Sensitivity,
(A − Kf Complementary
C − BK valid r ) â + Kf s, (10a)
r + Kf DKSensitivity,
must be balanced with robustness to model 4.5 H1 robust vs. H2 dt
Open-loop model inversion is often
ficult used
it is to in manufactur-
control or estimate optimal
a state a.control
The degree
ing and robotics, where systems near fixed points and away b andfrom critical
=ofRobustness values of
nty and totime-delays. ofare constrained
controllability and
andwell-observability a−K given
r â. state • S(⇣) - sensitivity (10b)function
e input the4.5 H1For
controller
characterized
this
robust
is inthe
reason,
vs.
sensor
a standard H2 there
optimal
operating Nowcontrol
the bifurcation
that
environment. we have parameter
established where conditions
det df /d enabling
= 0.
a recentispush
output to move away
the actuation signalfrom b, and H2 op-is quantified by eigenvalues and •ofS(⇣)
eigenvectors of the
- sensitivity function
arbitrary
controllabilityIt is pole
possible
Gramian to represent
placement The thethe
input tostate-space
the controller
closed-loop model
is the sensor•signal
system, T (⇣)s,- the
complementary
output is sensitivi
ntrol techniques Now
timates the full-state (LQR,
that of Kalman
we
the have
linear estimation,
established
sys- conditions enabling
4.3 Dynamic closed-loop in must
Eq. (2)now
feedback
we ascontrol
adecide
transferonfunction the actuation
where P (⇣)
to place in the
signal b,
them. frequency
and the state â estimates the full-state
c.) to the robustarbitrary 1 controllers
Hprinciple states[32,
pole placement 33,
of34,the closed-loop system, • the
of T (⇣) - complementary
linear system sensitivity • function
wr - reference
principletracking
). The separation that domain (in Z 1 of a Laplace
terms variable ⇣):in Eq. (5). The separation in
se controllers penalize
we must
Feedback
may be designed separately to deter- nowthe
control worse-case
decide
addresseson sce-
where
many toofplace
the them.
aforementioned
Wc = Atis-
e BB• ew dt control
⇤ A t theory states that the closed-loop system, combining
⇤
(11a)
or and sues withperformance.
open-loop control. Namely, closed-loop feedback, -1reference tracking
of provide dynamics for a andInâ,addi- 4.5.1 H2 P
the state robust stateLinear
restimation and full-state feedback, will have combined
bility optimal 0 control: quadratic Gaus- (3a)
⌘k poles of A−BKr 4.5.3 H robust control
(⇣) = C (⇣I A) B + D.
nificant effort as illustrated
has gone into intheFig.design
7, uses and sensor measurements X to
1 correct poles given ⇣ ⇤by the and A−K1f C. Thus, Kr
and the closed-loop for model
system will have sian (LQG) A t k ⇤ A t
(11b) independently to satisfy different
gy of 4.5.1 H2 uncertainties,
optimal control:rejectLinear
disturbances,
quadratic and ⇡stabilize
Gaus-eun- 4.5.3 and
BBK Hf1emay robustbe designed
control
thesensor
poles ofand A actuators,
stable BK system
along
r anddynamics.
A K with their
f C.Depending on the plantk=0 dynam- stability criteria for
We will often set Ĉ = Kr and D
the state-feedback and estimation, and
ntgeneral
in the flow. 4.5.2
observer
sian
dynamical
(LQG)
Sensitivity, Complementary
systemrobust We4.2 may Model-based
Sensitivity,
often
is performance may be achieved We modify open
Eq. (2)loopwith control
the addition of Kr is a linear-quadratic-regulator (
ics and system latency, theywill
may often
then be setcombined
Ĉ = Kwithout r and interference.
D̂ = 0, where
e are many instances and Robustness
when linear control white noise disturbance and measurement noise
design the ideal sys- trix. We may also choose  and B̂ a
through w
We may feedback.
often modify Eq. (2)and with • What
the addition
observability itGramian:
would
of look Kr like
is a to
d
linear-quadratic-regulator (LQR) gain ma-
s may have significant For theandlineardirectdynamics
impact for in Eq.w(5), n : full-state measure- Kalman filter, resulting in a combi
white
• S(⇣) noise disturbance
- sensitivity wd and measurement
function tem, invert noise planttrix. dynamics,
We may
4.3.1 etc.also chooseobservability,
Controllability, Â and B̂ according to the
and Gramians
r turbulent
enablingd flows,
ments even
s = away
a and from
a laminar
negative feedback controller b Z = 1−K r a based controller known as the li
w
â = Âân + B̂s,: (8a) ⇤
d eA t C ⇤The Kalman filter, resulting in a combined estimation-
int solutions:
system, - complementary sensitivity W =
function
• Why this wouldAa Ce At
dt
ability
be+a Bb terribleto place (12a)
poles of the Gaussian
closed-loop (LQG).
system in Eq.Because
(7) of the sep
dt • T (⇣) o
0 a = based + wd ,idea
controller known(in (14a)
many
as the linear-quadratic-
m. b = Ĉ â + D̂s. (8b) cases)...1 ⇣ dt arbitrarily with choice of K r relies on forthe controllability
linear systems, of the
it is possible to de
d one oscillatory ⌘k
mean-field models, • wrexciting
- reference
+ = Aa + Bb + wd ,Plant⇡ X
tracking
aController (14a)s eA =⇤ Gaussian
tCasystem
+ ⇤Db in(LQG).
C C systems,e+Eq.
A wt(5).
n k Because
. The system
. it (12b)
ofisthe
(14b) separationif and
controllable principle
only if the
feedback control gain Kr and an op
de with linear control may dt kill other oscilla- forfollowing
linear matrix has isfull
possible
row rank to N design
a: an optimal
ral controller will be -denoteds by=K(⇣) Ca + Db + w • . P (14b)
k=0 separately, and they will be both sta
modes
c Gaus- [36, 37], n feedback
Each of these noise inputs has a differentr2 co- control gain K and an optimal observer

ncy domain. 4.5.3 H1 robust control C =and B ABwill ATbeBboth when
· · · stable
A and combined.
Na −1
.
B optimal (11)
emble averages of turbulent flow responses variance
These may matrix: E(wdseparately,
be approximated andthey
wdT ) =byVdfull-state E(wmea- n wn ) = The resulting controller, known
Each of these noise inputs has a different co- when
We will often set Ĉ = Kr surements
be linear, as variance
in Colonius’s experimental T jet
andV D̂ =of0,
n , where the where
Tdirect
E(·) is the
system incombined.
expectation
Eq. (2) value.
Controllability and [This
adjoint
implies that the is system as amayH be steered to optimally
controller, an balanc
dition
ollability, of matrix: E(wand
r is a linear-quadratic-regulator
KObservability, d wd ) = Vd and 4.3
(LQR) E(w n wn )ma-
Dynamic
gain =feedback The controllers
resulting controller, known more2ofgenerally
e study , not
system: precise enough... really need
arbitrary state
E(w a
d with
(t)w d (⌧ ) T) =
suitable time-history the control
Gaussian measurement noise within-
nt noise Vn , where
trix. We may alsoischoose
E(·) the expectation
 and B̂ value. [This
according is asputaH controller, theoptimally
full state isbalances
observablethe effect
stateof
ans Figure 7: Closed-loop VGiven
controld (t ⌧ ).] to
topology.
linear
the
dynamics of
b.2 Similarly,
the form in Eq. (2), we of-
if any a may
bances. However, these controller
not
Kalman precise
ek’s jet simulation at Re=100,
enough...
filter, resulting really
in aneed E(w
combined d (t)w (⌧ )
estimation-
d
T ) = Gaussian measurement noise with process distur-
o place poles based of system as the ten Linear-quadratic
want W a feedback regulator
c ⇡ Wccontroller
e ⇤ (LQR):
= CC , based on(13a) full-state have arbitrarily poor robustness ma
Vdthe (t closed-loop
⌧ ).] bances.
controller known linear-quadratic-
feedback: 14eFor example, ⇤However, if the
these controllers are known to
system is H1 robust controllers are used whe
ith choice of K
sient controlGaussian relies on
Linear-quadratic
ofr separation the controlla-
forBecause
fully regulator
turbu- (LQR): b
W = ⇡
oZ 1 r oK Wa. = OO . (13b)
(14a) (LQG). of the separation principle have arbitrarily poor robustness margins. Instead,
ystem
boundary in Eq. (2). [38].
layer The system is control- controllable, then
Joptimal
= the T
a Qa poles of
+robustTthe closed-loop
b Rbcontrollers
dt. system
(15a) important.
for linear systems, Z it is possible to design an H 1 are used when robustness is
llowing
(14b) matrix: 1 may beon placed 0arbitrarily by choicevs. of discrete
Kr . Figure 4 shows the most genera
feedback control gain T Kr and T an[Decide
optimal notation
observer forimportant.
continuous
J= a Qa + b time Rb dt.A... emp. Gramians (15a) are slightly wrong, since closed loop feedback control, encom
be estimated from a suitable time-history of the measure- The total energy expended during this minimum-energy ac-
ments s. This is true when the observability matrix has full tuation to steer the system to a in time tf is give by:
column rank Na : Z tf
  kb(τ )k2 dτ = aT Wc (tf )−1 a. (17)
C
0
 CA 
 2 
  Thus, if the controllability Gramian is nearly singular, then
O =  CA  . (12)
 ..  tremendous actuation energy is required to control the sys-
 . 
tem. Conversely, if the eigenvalues of the Gramian are large,
CANa −1 then the system is highly controllable.
Although the definitions of controllability and observ- It is therefore possible to determine a hierarchy of con-
ability above are binary, there are degrees of controllability trollable states, in order of controllability, by taking the eigen-
and observability, based on how difficult it is to control or value decomposition of Wc , which is positive semi-definite.
estimate a state a. This is a more physically intuitive notion, The eigenvectors corresponding to the largest eigenvalues
that some states are easier to control than others. The degree are the most controllable states. This eigen-decomposition
of controllability and observability of a given state in time tf is closely related to the singular value decomposition (SVD)
is quantified by eigenvalues and eigenvectors of the control- of the controllability matrix from Eq. (11), but defined from a
lability Gramian: discrete-time system approximating Eq. (5). This connection
Z tf to the SVD will be important in model reduction, and will
Wc (tf ) =
∗
eAτ BB ∗ eA τ dτ (13a) be discussed in Sec. 4.5.1. A similar hierarchy of observable
0 states may be determined.
tf /∆t  k  ∗ k
X
≈ eA∆t BB ∗ eA ∆t ∆t, (13b) 4.3.2 Linear quadratic Gaussian
k=0

and observability Gramian: Now that we have established conditions enabling arbitrary
Z tf pole placement of the closed-loop system, we must decide
∗ on where to place them. It is mathematically possible to
Wo (tf ) = eA τ C ∗ CeAτ dτ (14a)
0 make closed-loop eigenvalues arbitrarily stable (i.e. arbitrar-
tf /∆t  k  k ily far in the left-half complex plane) if the system is control-
X
≈ eA
∗
∆t
C ∗ C eA∆t ∆t. (14b) lable. However, this may require expensive control expendi-
k=0 ture with unrealistic actuation magnitudes. Moreover, very
stable eigenvalues may over-react to noise and disturbances,
Here, the matrices A∗ , B ∗ , and C ∗ denote the adjoints of
causing the closed-loop system to jitter, much as a new driver
A, B, and C. Often the Gramians in Eqs. (13) and (14)
over-reacting to vibrations in the steering wheel.
are evaluated at infinite time, and if the time dependence
In a linear-quadratic regulator (LQR) controller, the con-
is not explicitly mentioned, we assume tf = ∞, so that
troller Kr is chosen to place the closed-loop poles to mini-
Wc , Wc (∞) and Wo , Wo (∞). In this case, if the matrix
mize a quadratic cost function J. This cost function balances
A is stable, so that all eigenvalues have negative real part,
the desire to regulate the system state to a = 0 with the added
then the Gramians are unique solutions to a Lyapunov equa-
objective of small control expenditure:
tion:
Z ∞ 
AWc + Wc A∗ + BB ∗ = 0, (15a) J= aT Qa + bT Rb dt. (18)
A∗ Wo + Wo A + C ∗ C = 0. (15b) 0

Q is a symmetric positive semi-definite matrix that is chosen

Solving these equations may be quite expensive. Empirical
to penalize deviations of the state a from the set-point a = 0.
snapshot based methods, discussed in Sec. 4.5.1, are faster.
Similarly, R is a symmetric positive definite matrix that is
The matrices Wc and Wo are both symmetric and posi-
chosen to penalize control expenditure. Often, Q and R are
tive semi-definite. The system is controllable (resp. observ-
chosen to be diagonal matrices, and the magnitude of the di-
able) if and only if the controllability (resp. observability)
agonal elements may be adjusted to tune the control perfor-
Gramian is invertible. The controllability of a state a is mea-
mance by adjusting relative penalty ratios. For example, to
sured by a∗ Wc a, which is larger for more controllable states.
increase the aggressiveness of control, the diagonal entries of
Physically, if a∗ Wc a is large, then it is possible to steer the
R may be decreased with respect to those of Q.
system state far in the a direction with a unit control input.
The optimal control law that minimizes J in Eq. (18) is
Similarly, the observability of a state is measured by a∗ Wo a.
given by b = −Kr a, with Kr = R−1 B T X. X is the unique
This connection between the Gramians and energy expen-
solution to the algebraic Riccati equation:
diture to move the system toward a state a at time tf can
be made precise with the notion of minimum energy control. AT X + XA − XBR−1 B T X + Q = 0. (19)
The minimum-energy control input b(t) to steer the system
to a(tf ) from a(0) = 0 is given by: In LQR, a balance is struck between the stability of the
 T closed-loop system and the aggressiveness of control. Tak-
b(t) = B T eA(tf −t) Wc (tf )−1 a(tf ). (16) ing control expenditure into account is important so that the

15
 the poles of
es when A BK
linear
4.5.1 Hr2 and
control A K
optimal C. These
f 35].
control: signal
Linear thewe
s,quadratic
controllers must
output penalize
Gaus- now
is thedecide
actuation
the on where
worse-case signal tob,place
sce- and them. 1
⇣ ⇤ ⌘k
tem,thisinvert
results plant
inâa dynamics,
dynamical k=0
etc.
system sian 4.5.3
for(LQG)
the 1 robust control
Hcontroller: This means that any
X1
• state
w r - reference
Amay t k betrackingestimated
eneral observer
nt and direct impact dynamical
for(LQG)
sian
system is the state estimates
nario error and provide robust performance. In addi- the full-state of the linear sys- ⇡ We
a
e 1will often set
BB ⇤
eA Ĉt with = K (11b)and D̂
Why
tion, tem
thisand in
would
significant Eq. (2).
be
observability
effort The
a separation
terrible
has Gramian:
Wegone may idea
into We principle
(in
often
thewillmany
modify
designoften 4.4.1
states set
Eq.
and that
(2)
Ĉ Balanced
a
with
= suitable
K the
P
and
(⇣)
model
addition
D̂
=
time-historyC
=
(⇣I
reduction
of0, of A)
where the B +
measurements D. s. (3a) 4.5.2 r Sensiti
ven away from laminar • r k=0 Kr is a linear-quadratic-regulator (L
K and 4.5.1
f may
optimal control:
H2designed Linear quadratic Gaus- the4.5.3
Although definitions 1 robust control
of controllability and and Ro
We may often modify Eq.
d (2)r with
cases)...
modeling K
of sensorthe andbe
addition whiteof
actuators, noise K separately
is awith
r disturbance
along towdeter-
linear-quadratic-regulator
their
4.5 d andH measurement
robust vs. (LQR)
H noise gainHtrix.
optimal ma- controlWe may also choose  and B̂ ac
d mine the stability Z
of sian
the (LQG)
state dynamics for (7a)and 1 and
observability observability
above
2 Gramian:
are binary, there are degrees of
white noise disturbance wd and measurement
â
placement
(8a)
= (A
in the flow.K f C noise
BK
wn :A
1 r⇤ + K DK
trix.
f We ) â +
r may also K af s, chooseâ,4.2 ÂModel-based
and B̂ according open toKalman
Weobservability,
will loop control
the filter, resulting in• aS(⇣)combin
= -0,sen
â = Ââ
exciting
dt
+ B̂s,
onewoscillatory
dt respectively, Wo and = the closed-loope t C ⇤ CeAtsystem dt will (12a)controllability and
have Z 1 often set basedĈ = on K howr and dif- D̂ w
n: There
b = areKmany â.We instances
may often when Kalman
modify linearEq. filter,
control
(2) Now
resulting
(7b)
with that
the we
in a
addition have
combined
of established
K estimation-
is aconditions
based enabling
controller
linear-quadratic-regulator
⇤ known
Atthe ideal sys- (LQR) as the gainlin
4.3 (8b) Dynamic feedbackr 0
controllers f C.ficult • What it would look like Ato t design
⇤
degree•(12a)
polesmay given by significant
the poles
1 ⇣ of and Aarbitrary asditplacement
is to W control of or estimatee Ca state dt The
r
ol mayb = kill Ĉ
other
â + oscilla-
D̂s. A⌘based BK d rimpact
controller Kknown pole the o =
linear-quadratic-the closed-loop
Gaussian Ce(LQG). a.
system, ofTthe(⇣) -sepa
co
strategies have
The mostwhite general
X noise observer
and
disturbance
A⇤ t
direct
k
⇤ a dA
dynamical w =and for
k measurement
Aa
tsystem + Bb is of+ w
tem, ,noise
invert plant
controllability (14a)
trix.
and We
0 may also
dynamics,
observability etc.choose
of a ÂBecause
given and B̂ according
state
d ⇡ e Gaussian
C dt C e (LQG). we (12b)
must now
sensorBecause of the separation
. decide on where to
principle place
for linear them. systems, it is possible to des
= nonlinear
aGiven Note
linear
Aa + that turbulent
dynamics
Bb
given the
+by: wdinput
, ofwnflows,
: to form
the the even
(14a) in away
controller
Eq. (2),from
is
wethe laminar
of- is+quantified by(14b)Kalman
eigenvalues
X ⇣filter,
1 ⌘k
⇤ resulting in a tof combined
k the• wr -estima refer
l controller will be denoted dt by K(⇣)
ten want fixed-point
signal s, the
a feedback solutions:
output
controller
k=0
is thebased actuation on
s systems,
LQG
for full-state
linear
signal
= Ca +it Db
b, and is possible
wn . to design ⇡an optimal eAandt eigenvectors
feedback C ⇤
control C eAgain . r (12b)
K and an op
urbulent
y domain. flow responses • Why
controllability this based
would
Gramian be controller
a terrible known
idea (inas the
many linear-quad
s = CaState
feedback:
+ Dbestimator
the bstate
+ wn .
= âKestimates dthe(14b)
r a. For example, full-state if
dLQRthefeedback
ofB̂s,
the linear
system control
is sys-gain KPlant and an optimal Gaussian
k=0 separately, and they will be both stab
observer (LQG). Because Gaus-of the4.5.3
onius’s experimental jet r cases)... separation pri
1. (Kalman
In mean-field
These maymodels,
be â Each =exciting
approximated ofa +
Ââ one
these
= by oscillatory
Aa noise
full-state
+ Bb 4.5.1
inputs
+w (8a)
mea- ,Hhas2 optimal (14a)
a different control: co-Linear when quadratic combined. H1 rob
tem in
controllable, thenEq.the filter)
(2).polesThe of separation
dt the closed-loop separately,
dtprinciple system and
states that they dwill be both stable
These may and
Z
for optimal
linear
be systems,
approximated it is possible
by full-state to design
mea- an op
Each of these may noisebeK inputsmode surements
has with a of the
linear
different control variance
direct co- system
may matrix:
killinother
Eq.E(w (2) and
oscilla- T ) =sian
d wdadjoint Vd and (LQG) E(wn wn ) = At T 1
The
⇤ A resulting controller, known m
swhen combined. ⇤
ability, Observability, andplaced and arbitrarily
Kf may be bydesigned
choice
b = of Ĉseparately
â +.= D̂s.Ca to
+ Db deter- +w (8b)
n. • P (14b)
surements
W = offeedback the e tgain (11a)
is direct
K e BB system
control dtin Eq. Kr(2) andand anWeadjointwill often
optimal obs
t Re=100, variance matrix: E(wd w Tr tory
) = system:
modes
V d and E(w
[36, 37], TV)n=
n wnstate
, wherer E(·) is the expectation value.c [This as a H2 controller, optimally balance
ns mine
d the stability of the dynamics The for resulting
a and We â, controller,
may often known
modify
system: more
Eq. 0 generally
(2)
Tseparately, with the
and addition
they will of
be both K stable is aandlineaop
d
The not precise enough... really need
a general controller isr a will be denoted by K(⇣)
Vn , where E(·) is the respectively,expectation value. E(w (t)w (⌧ ) ) 1 = Gaussian ⇣measurement ⌘k r
noise with p
the[This X effect
d d
2. ensemble and
= Each
Aa
averages of
closed-loop
ofBK these
turbulent as
noise a H⇤inputs
system
flow (4a)
controller,
will have
2responses has aoptimally
white different
noise balances
disturbance co- ⇡ w the
when and t of
measurement
combined.
A k ⇤ A⇤noiset trix. We may
aration for fully turbu- in the E(w
dt frequency domain. V (t ⌧ e ).] (13a) noiseT with process d e bances. BB e (11b)
place poles ofnot theprecise enough... really need W ⇡ = CC , T measurement ⇡ Wce =However, these controllers
T = W
closed-loop system poles given by d (t)w (⌧ )c of)dmatrix: cGaussian Wcdistur- CC ⇤ , (13a)
may be linear, =the aspoles
variance r and dn :and (LQR):
din Colonius’s
A experimental fjet
) a.BK E(w
r Linear-quadratic dw A
d) =
⇤(4b)
Kw V C.
regulator E(wn wn ) = k=0 The resulting controller, known Kalman morefiltergen
h]. choice of K
(A BK e 4.3 Dynamic feedback havecontrollers
arbitrarily poor robustness mar
Vrd relies
(t ⌧on ).] the controlla- noise The most studygeneral W
, Vn , whereobserver
o ⇡ E(·)W =
obances.
dynamical OO .
is the expectationHowever,
system (13b)
these controllers
is value. [This is are known ⇡ to
+ e
OO ⇤
(13b)
as W
a Ho W
controller,
robust
o = optimally.
controllers based
balances
are used contro
the eff
when
tem in Eq. (2). The Linear-quadratic
system is control- regulator
However, 4.4
given by: Controllability,
(LQR):
full-state measurements
not precise enough...
Observability,
of a
have reallyare
Z ei-
arbitrarily need poorand
E(wTdGiven robustness T
(⌧dlinear margins.
2
Instead,
H 1
1 (t)w
and ) )=
dobservability dynamics Gramian:
Gaussian of + the form
measurement
important. in Eq. (2),
(14a)noise wewithof-
Gaussian process (LQ d
utput
owing dynamics
matrix: 3. Dusek’s Gramians
jet simulation at Re=100, T a = Aa
on(15a) + Bb w ,
Z ther infeasible or[Decide
prohibitively
Von notationexpensive, for H and
continuous
J1 so an
=robust vs.Qadiscrete
acontrollers + b Rb aredt. [Decide
used awhen notation
robustness foriscontinuous onvs.full-state
discrete time
d
1 d (t ⌧ ).] ten want
dt feedback bances. controllerHowever,
Figure based 4these
shows controllers
the for
most linear
are syst
know
general
estimate â may time beA... obtained
emp.
d place from measurements
Gramians are slightly
important.
0
s.wrong, since A...s emp. Gramians
Z+ are
wnslightly wrong,
(14b) since they
â Linear-quadratic regulator (LQR): b== Ca r a.Db
inearizing Eq. (1) about J =⇤ aT Qa ThebT Rbability to poles
(15a) ofB̂s,
the closed-loop system feedback: 1 For+example, . poor iffeedback
the system ismargins.
+ dt.control Khave arbitrarily robustness feedback cont
Ins
is 4. transient of separation for fully(2) turbu- (8a)
= Ââ + ⇤ tclosed loop control, encomp
AB A2 B · · · ANa 1 B 0This (9a) of course,they only
use A if ]the system of equations Figure 4 shows the most use W A
general ] = schematic e A
C ⇤
for Ce At
dt (12a)
eriodic orbit, leading to arbitrarily
lent An boundary
dt
with
d choice
layer The
[38]. of optimal
K relies control
on the law is
controlla-
b = controllable,
K a, where
o dthen K the = poles
robust of the closed-loop
controllers
optimal control are system
used separately,
when
strategies. robustn an
degrees of There
is observable.
r
Z 1 may r Hr H1
observer dynamical system isbethe [Mention 1
0SVD and symmetric matrix,
Figure 8: Linear-quadratic [Mention
bilityGaussian bSVD
controller. and symmetric
1The Kalman matrix,
filter degrees of
is a dynamical system that
of the system = in Eq. (2). closed
Theand loop
Tsystem feedback
Kis (8b)
T Each
unique
control- ofcontrol,
these
solutionbe encompassing
noise
to thearbitrarily1 takes
inputs
alge- ⇣ Hhas sensor
and achoice measurements
different r .co-expanding
TD̂s.
RĈâ B+ X X f may placed important.
X by
2 of ⌘excellent ofbooksK when combin on this
Thesoptimal
Na . Controllability impliescontrol
that law
constructed
and the actuation signalthe is b as follows:
controllability...]
= K
b toifestimatea, where K
the full-state =J = a
The Qa +
LQR gain b Rb dt.
Kr isstrategies.
a matrix:
matrix that (15a)
controllability...]
multiplies the
k
tfull-state
nto
optimal control There are Aproduce k
d )a= number
⇤
lable the
r following braic
rmatrix: â.
Riccati0H1 equation: variance E(w ⇡d w
T
eAV
Figure d and 4 CE(w
shows
⇤
Cwe w
ethe
T t =
n )most (12b) The
. important
general result
schemat
steered to an R arbitrary state is with4.1 Linearized
â[Mention input–output
balanced model dynamics
reduction: BT, [Mention balanced model reduction: BT, BPOD,theore
Here discuss
X and the bunique solution to the alge- Vn BPOD,
1 T
an
B actuation Xsignal
a =dThe−K generalthat⇥ iscontroller
optimal with
will respect
ofdenoted
beŝ), to the
excellent byquadratic
books
⇤K(⇣) cost E(·)
, expanding
where function on isthis indtheory
the Eq. (18).
expectation [39, 40]. value. [This is encompassing
aistory
+ Bb,of thebraic (2a) â
r
= Aâ + Bb + K (s (5a) a closed
k=0
= loop
Aa garding feedback
BK the
a various
control, as
types
(4a) a ofH contr
optima H
1(9a) ERA] dt
2 N 1
control inputequation:
Riccati b. ERA] C = B AB A B T · · · A
The optimal
f
Acontrol law
a
iswe
B T
where
r 2
dt we
in the
Often, frequency
are interested domain. in linearizing X + XA Here
Eq. (1) bXBR=discuss
about notKprecise
rB important
a, X+ enough...
QK= theoretical
r =0. really (16a)
H results
need
optimal re- control
E(w
robust d (t)w d (⌧
control, )Tand
) = H2 There
strategies. Gaussian
LQG. are amea
nu
a + Db. (2b) ŝ = C â + RDb (5b) These may be
1
(4b)
of approximated
a steady
1 T and Xgarding is the leading
unique
the various Vsolution
d (t types ⌧ to
).] theof alge-
optimal = (AH1BKrby
control: ) a.full-state mea-
controller doesn’t overreact 1 Tfixed
to
point
high-frequency
asBorX periodic
noise and
orbit,
dis- general
to
observer 11 dynamical system, excellent
given books
by: expanding onbances. this theory How [3
AT X + XA 10 (16a) Often times turbulence
have is consid
XBR B X
a linear system of braic+ Q = 0.
equations: A dual
Riccati Riccati
robust equation
equation: control, and H2 surements
isLinear-quadratic
solved LQG. for the of the direct
observer
regulator system
(LQR): • in Eq. important
(2) and adjoint
turbances,9 doesn’t exceed maximum actuation amplitudes, Here we discuss theoretical arbitrari
resu
4.4 Controllability, gain Observability,
Kf = Y C T Vn : and However,
system: full-state measurements bance term of in a aare ei- dynamica
slower
and Riccati
isn’t prohibitively Zd considered garding the various types ofso H1 robust
optimal control co
A dual equation isexpensive. Gramians
solved for dthe observerA T
X + XA •XBROften 1times
B T turbulence
X + Q ther
= 0. is
infeasible â
(16a)
Ââa+distur-
1 =or prohibitively B̂s, asexpensive,
the rigid (22a)
and
body an
equations of an
a = Aa + Bb, (2a) dt T robust T control, and H LQG. important.
gain Kf In = practice,
Y C T Vn :to make Eq. (5) more dt realistic, it must Y AT + AY
be aug- banceYterm CT V inn a1 CYslower J =dynamical
estimate
+ Vd â = maya Qa
0. be (17a)
+
system, b Rb
obtained
c ⇡
such dt.
efrom shuttle, 2 (15a)
measurements
⇤ or rocket.(13a)
(22b)
In
s. this case,
The ability to place 0 b = WĈâ + D̂s.W c = CC , Figure 4 s
mented with the addition of white s poles
noise =disturbance
Ca of + theDb. closed-loop
wd and as the rigid system
(2b)body This equations of an aircraft,
is of course, only if space
Oftenthe timestuations
system of
turbulence may be
equations is seen
(2) as inevita
considered
measurement noise arbitrarily
w : with A
choice dual of Riccati
K equation
relies on the is solved
controlla- for the observer W • ⇡ W e
=
o −ating OO ⇤
. closed
(13b) loopa fe d
Y AT + AY Y Cbility T n 1
Vn of CY + V = 0. (17a)
The
r
so-called shuttle,
Kalman Fororthe rocket.
filter
The LQG
optimal
K In chosen
is this
controller,
control case, we
to
law turbulent
set
mini-
o
is Â
b = fluc-
bance
= AK term
a, K where
finCaon − a time
BK
slower
K = + scale that
rdynamical is fa
system
the system d gain in K Eq.f (2).= YThe T
C⇣ Vsystem n: isK control- ⌘ f r r
1 optimal
HInstead c
1 ,B f ,X r , and D̂ = troller
and solution 0, recovering bandwidth.
alge-the of of try
mize E (a tuations â)T (a f may be seen
DK 9B̂ TX = and Kas inevitable
Ĉ is=the −K unique oper- to the
d lable if the following matrix: R
â) rgiven known
[Decide covariance
on notation as thefor rigid
continuous body equations
vs. discrete of excellent
an aircraft, bo
ating form
(20a) on1 a
oftime
braic Eq. (10) scalewith thatKisr as the LQR
faster than gain
con-matrix the nature and of Kthe asturbulence itself
+ VRiccati equation:
The so-called Kalman a = K Aa + Bb + Bw wmini-
f is chosen to d ,T
⇣ dt filter ⌘ YA Vd + and AYVn . Y C T Vnthe CYKalman =time
dfilter 0. gain
A... (17a) emp.
matrix. Gramians shuttle, areor rocket.wrong,
slightly
may
In fthis case,Here
be designedsince
turbulent
to obtain some we d
T ⇥ troller⇤ bandwidth. Instead of trying to change
tuations may be seen as inevitable andv
mize E (a â) (a s â)= given Ca known
+ Db
C = B AB nA B · · · A + covariance
w . 2 [Note: N Kalman
(20b)
a 1 B published
(9a) resulting
The his
they famous
use
controller,
A ]Kalmanknown more generally
while as
robustly an H garding
managing the
the un
filter in a journal
the nature
of Fluid
ofEngineering.]
the
A Tturbulence itself, the 1controller
X + XA
d
XBR B T
atingX + on Q a= time
0. (16a)
scale
2
that is faster than
Vd and Vn . The so-called Kalman filter K
controller,
f is chosen
optimally to
[Mentionmini-
balances SVD the and
effectsymmetric
of Gaussian
lent matrix, degrees
measure-
disturbance. robust
of contro
⇣ may⌘be margins designedfor to obtain some other
guar-objective
trollerAlthoughbandwidth. Instead
The matrix
[Note: Kalman has full his
Bw determines
published rank
the famous
spatial . Controllability
Namize distribution
E (a â)Tof(a
Kalman [Decentimplies stability
how that
â) ment
while given
robustly the
noise
known with
managing
LQR, process
covariance
but
controllability...]
the
nodisturbances.
uncertain turbu- LQR con- of trying to ch
anteed stability margins A fordualLQG Riccati
(famous equation
Doyle is
pa- solved
the naturefor
• H the
of is observer
the
by turbulence
far the more itself, Often
•popular
the cont tim
con
disturbances enter system
the state. may
filter in a journal of Fluid Engineering.] V and V .Each be of steered
these to
noise an arbitrary
inputs has state
a trollers
a with may have [Mention
decent balanced
stability model
margins, reduction:
there
2 is no BT,
guar- BPOD,
per)]. lent disturbance. gain TV : because of its simple bance
mathema ter
for LQGmay be designed to obtain some other obje
d n K = Y C
differentstability
[Decent co-variance marginsmatrix:
suitable fortime-history
LQR,d (t)w
E(w but dno (τ T
of)guar-
the )= control
Vd δ(tinput − τ ) b. antee on stability f ERA]margins n controllers, as famously
[Note: Kalman published his famous Kalman while robustly managing as the rigt
the uncertain
anteedandstability
E(wn (t)w margins
T
n (τ ) )for = LQG − τ ), where
Vn δ(t(famous Doyle E(·) pa- is the expec- • H is demonstrated
by far the more in [161].
popular This means
control that
paradigm even small uncertain-
filter in a journal of Fluid 2 Engineering.]T T 1
lent disturbance. (17a) shuttle, o
per)].tation value and δ is the Dirac delta function. The additionbecause of ties,of such its as
Y A unmodeled
simple + AY dynamics,
Y C V 11
unexpected
CY + V = disturbances,
0.
nomathematical
10 n formula- d
disturbances and sensor noise is shown in [Decent
Fig. 8,stability
and alsomargins in or time-delaysfor LQR, butmay guar-
destabilize the closed-loop system. tuations
Fig. 9 with wr = 0; wr is the reference anteed
input.stability margins for LQG The (famous
so-called Doyle pa- filter• K
Kalman H2 is is by far thetomore
chosen mini- popular control atingpara on
⇣ ⌘ because f
In linear-quadratic estimation (LQE), a dual problem toper)]. 11 of its simple mathematical trollerform ba
LQR is solved, resulting in an optimal full-state estimator,
4.4 mize Robust E (a control T
â) (a â) given known covariance
the natur
as in Eq. (9), that balances the relative importance of mea- The notion Vd and Vn . may be d
of robustness and 11 performance are central in feed-
surement noise and process noise. The process noise may be back control. [Note: TheKalman
limitations published
of LQG his control famous haveKalman motivated while rob
an additive stochastic term, or structural uncertainty in the significant filter inadvances a journalinofthe Fluid Engineering.]
development of controllers with lent distu
model. A dual Riccati equation is solved for Y in the ob- robust performance. [Decent stability Robustnessmarginstypically for LQR,refers but no to the guar- abil-
server gain Kf = Y C T Vn : ity toanteed maintain stabilitycontrol margins for LQG despite
performance (famousmodel Doyle uncer- pa- • H2 is by f
tainty, per)].
unmodeled nonlinear dynamics, and unforeseen dis- because
T
Y A + AY − Y C Vn CY + Vd = 0. T −1
(21) turbances, time-delays, etc., which are all important for tur-
bulence control. A complete discussion of robust control 11 is
beyond the scope of this review; instead, our goal is to build
The so-called Kalman filter Kf may be used  in the observer  an intuition and provide a glimpse of the powerful robust
in Eq. (9), and it is chosen to minimize E (a − â)T (a − â) control machinery for flow systems [174]. For a more com-
given known covariance Vd and Vn . plete overview with excellent attention to engineering con-
The optimal state-feedback (LQR) and optimal state- siderations and practical control design, see [166].
estimation (LQE) may be designed independently and then
combined. The separation principle guarantees that when 4.4.1 Sensitivity, complementary sensitivity, and ro-
combined, the state-feedback and state-estimation will re-
bustness
main stable and optimal. The resulting controller, combining
estimation-based full-state feedback, is known as a linear- As discussed in the previous section, real systems will always
quadratic-Gaussian (LQG) controller, shown in Fig. 8. include disturbances wd and sensor noise wn , as illustrated
The combined LQG controller may be written as a more in Fig. 9. This diagram also includes a commanded reference

16
 he most general observer dynamical system is given 4.4.1 H2 optimal control: Linear quadratic Gaussian
(LQG)
d
â = Ââ + B̂s, (8a) [Decide on notation for continuous vs. discrete time A...
dt emp. Gramians are slightly wrong, since they use Ad ]
b = Ĉâ + D̂s. (8b) We may often modify Eq. (2) with the addition of white
noise disturbance wd and measurement noise wn :
he general controller input w will be denoted
r , that the controller by K(⇣) should in track.
the When designing a For performance and robustness, we want the maximum
ency domain. robust controller, it is important to understand how these ex-d apeak = ofAa S, + MBb S = wd ,∞ , to be as (13a)
+ kSk small as possible. From
ogenous inputs affect the outputs s, the error signal e, and thedt Eq. (25), it is clear that in the absence of noise, feedback con-
actuation signal, b. Inand general, systems will be particularly strol=improves Ca + Db + wn . (i.e. reduces
performance (13b)error) for all frequen-
Controllability, Observability, Gramians
sensitive to disturbances and noise at certain Each frequencies.
of these noise inputs cies where has a|S| < 1; thus
different control ismatrix:
co-variance effective when T ≈ 1. As
bility to place poles ofWe themay closed-loop
express the system outputarbitrarily
s in termsE(w of transfer func- explained T in [166] (pg. 37), all real systems will have a range
d wd ) = Vd and E(wn wn ) = Vn , where E(·) is the ex-
T
hoice of Kr relies tions onon thethe inputs wr , w
controllability ofdthe
, and system
wn : in of frequencies where |S| > 1, in which case performance is
pectation value. [This is not precise enough... really need
). The system is controllable if the following matrix: T degraded. Minimizing the peak MS mitigates the amount of
⇥ s = Pd w d + P ⇤ K(wr − s − wE(w n ),
d (t)wd (⌧ ) ) = Vd (t ⌧ ).]
degradation experienced with feedback at these frequencies,
C = B AB A =⇒ 2
B · · (I · +AP K)sB= Pd wd +(9a)
Na 1 Linear-quadratic regulator (LQR):
P Kwr − P Kwn . improving performance. In addition, the minimum distance
Z 1
of the loop transfer function L to the point −1 in the complex
ll rank Na . Controllability
Therefore, we implies
have:that the system may J = plane aT Qa + bT by RbM dt.−1 . The larger (14a)
is given this distance, the greater
ered to an arbitrary state a with suitable time-history 0 S
−1 −1 the stability margin of the closed-loop system, improving ro-
control input b. s = (I + P K) P K wr + (I + P K) Pd wd control law is b =
| {z } | {z The1}optimal bustness. These areK r a,two
the where
majorK r = to minimize M .
reasons S
imilarly, the full state is observable if the observability B T X and X is the unique solution to the algebraic
T S R The controller bandwidth ωB isRic- the frequency below
x is full rank: cati equation [Expensive!!]:
2 −3 (I + P K)−1 P K wn . (23) which feedback control is effective. This is a subjective defi-
| {z }
C
T A X + XAnition.
T
XBR Often, 1 ω
BTB Xis+theQ frequency
= 0. where
(15a) |S(jω)| first crosses
6 CA 7 -3 dB from below. We would ideally like the controller band-
6 2 7
is the sensitivity and T is the complementary sensitivity. If width
(10a) A dual Riccati equation is solved
to be as forlarge
the observer
as possible gainwithout
Kf = amplifying sensor
S6 7
O = 6 CA 7 .
we6let L. = P7 K be the loop transfer function,Ythen C T we have:
V : noise, which is typically high frequency. However, there are
D = 0 for this problem...)] 4 .. 5
which leads
n
Similarly, the full state is observable if the observ-
−1
fundamental bandwidth limitations that are imposed for sys-
CANa 1 S = (I ability
+ L)matrix , is full rank: Y AT (24a) + AY tems YC that
T
Vhave
n CY
1
time+V d = 0.or right half
delays (16a)plane zeros [166].
near model-based control −1
T = (I + L) Open-loop
L. Thecontrol (24b)
⇣ so-called
means that any state a may be estimated with a suitable 2 3Kalman filter Kf is chosen to minimize
(A K C) â + K s + (B K D) (6a)
b. C ⌘
results in
history of closed-loop
f
the measurements
f turbulence f control are T
7 â)4.4.2 given ∞ robust
Hknown controlVdesign
covariance
s.
Since we are interested in minimizingEthe (aerror
6 â) e
CA (with-
(a d and Vn .
clthough
to linear the systems. For example, one may sta-and observ- Controller
6 7 Plant
definitions of controllability
out noise), the following expression is moreOuseful: 6
[Note: CA 2
Kalman 7 published his famous Kalman filter inarea known to have ar-
tate estimate â will converge to an theesti- = 6 7 . (10a)
As discussed above, LQG controllers
ynabove
unstable aresteady-state
binary, there andare delay degrees transi-
of controllability journal 6
of ..
Fluid
7
Engineering.]
he true state a in
turbulence, if the
the boundary
eigenvalues wof
4 . (25) bitrarily poor robustness margins. This is a serious problem
5
bservability,asbased =layer or
Aschan-
on howe difficult r − it=isSw to rcontrol
− SPdor wd + T w n .
WeCA will Naoften setinĈturbulence
= Kr andcontrol, D̂ = 0where in Eq. the
(8), flow
whereis wrought with un-
w.table.
In this case, the laminar solution becomes
1
ate a state a. The degree of controllability and observ- K is a linear-quadratic-regulator (LQR) gain matrix. We
ned
le for with full-state
post-critical The feedback
disturbance
Reynolds b = plant
numbers, Kr â, Pd and
but it the system plant r P are often certainty and time-delays. H∞ robust controllers are used
ys of a given state system
in a dynamical
is quantified by eigenvalues and eigen- may also choose  and B̂ according to the Kalman filter,
for the controller:
closely related. For example, This meansFigure
disturbances thatandany3: Open-loop
control a maycontrol
state inputs bewhen topology.
estimated robustnesswith is important. There are many connections
sstabilized by feedback
of the controllability control.
Gramian resulting
h work has encompassed may both abesignificant amplifiedmod- by a natural
a suitable convective
time-history ofinthe
instability a combined
in between
measurements estimation-based
H2 and H∞ control,
s. controller andknown
we refer the reader to the
Z 1 as the linear-quadratic-Gaussian (LQG). Because of the sepa-
the
ffort (see Sec. 3) to describe flow. the relevant
⇤ low- Although the definitions of excellent
controllability reference
and books expanding on this theory [166, 167].
W = At
eK BB ⇤ A
e) âwe t
dtK where
(11a) each of the
ration (A, B, for
principle C, D) depend
linear systems, on the it lin-
is possible to design an
Kcf C BK0rPerformance
uid mechanisms.
(A +Typically,
DK issues
+ will(7a)
such
s, choose
as the controller
observability above K so
are that
binary, the there are Figure
degrees 10 shows
of the most general schematic for closed-
f r f
earization point as and bifurcation parameter .
idth, disturbance open-loop
rejection, transfer
and noise function
atten- controllability
L = P K hasoptimal
anddesirable feedback
observability, prop- control
loop
based on gain
feedback
how K dif- and an
control, optimal
encompassingobserver H 2 and H∞ optimal
Kr â. X ⇣ A t ⌘k
1 ⇣ (7b) ⌘k This approximation will be approximately valid
r
separately, and at they will bestrategies.
both stable In and optimal when
⇤
must be balanced ⇡ erties withe inrobustness
the BB frequency⇤
toeAmodeldomain.
t ficultForit(11b)
isexample,
to control or estimate
small gain a statecontrol
a. The degree the generalized theory of modern con-
near fixed points and away from critical values of
ainty and time-delays. high frequencies
to thek=0 For this there of controllability
will attenuate
reason, sensor noise. and
combined. observability
Similarly, high trol, of a given
the goal stateis to minimize the transfer function from ex-
the input controller is the sensor the bifurcation parameter where det df /d = 0.
en a recent push gain
to at low
move awayfrequencies
from H will
op- is quantified
provide good by eigenvalues
The
reference resulting
tracking and eigenvectors
ogenous
controller, of
inputs
known the w (reference,
more generally disturbances,
as a H2 noise, etc.) to the
he output is
bservability the actuation signal b, and
Gramian: 2 It is possible to represent the state-space model
ontrol techniques performance.
(LQR, Kalman These
estimation, controllability
are intimately related Gramian optimally cost
to the sensitiv-
controller, balancesfunction J (accuracy,
the effect of Gaussian actuation
measure- cost, time-domain per-
estimates the full-state
Zity of the linear sys- in Eq. (2) as a transfer function P (⇣) in the frequency
tc.) to the robust 1H S and complementary
controllers [32, 33, 34,sensitivity T . In particular,
ment noise from
with formance,
process etc.).
disturbances. Both 2 andLQR
Although
H H control
∞ con- design result in
(2). W The separation
=
1
e principle
⇤
A t ⇤
C Ce states
At
dt that domain
(12a) Z 1 of a Laplace variable ⇣):
(in terms
Eq. (25), should be small at low frequencies, and should controllers that minimize different norms on this fundamen-
hesemay controllers penalize the worse-caseto deter-sce- Wc =trollersemay have
⇤ A decent stability margins, there is no guar-
o S T
At ⇤
K f be designed 0 separately BB e t dt (11a)
rror and provide be small
robust at large
performance. frequencies;
In addi- note that S antee
+ T on= stability
I, from tal
margins input–output
for LQG transfer
controllers, function.
as famously In fact, the symbol H2
tability of the state X1 dynamics
⇣ ⇤ ⌘kfor a and ⇣ â, ⌘k 0
P (⇣)1= C (⇣I A) ⇣B + D. 1
(3a)
gnificant effort
ly, and the⇡closed-loop Eq.
has (24).
goneA into t
e systemCwill the ⇤ design
C have e A and t
. (12b) demonstrated
X k in [32]. refers
⇤
⌘tok a Hardy space with bounded two-norm, consisting
ngbyofthe sensor ⇡ eA t BB ⇤ of eAstablet
and (11b)
strictly proper transfer functions (meaning gain
n polesand Aactuators,
ofk=0 BKr and alongA with Kf C.their
ent general
ost in the flow.observer dynamical system is 4.2 Model-based k=0wd open loop control
rolls off at high frequency). The symbol H refers to a Hardy
Mention SVD and symmetric matrix, degrees of con- 4.4.2 [Sensitivity, Complementary Sensitivity, and ∞
ere are many instances when linear control space with bounded infinity-norm, consisting of stable and
bility...] • What it would
and observability Gramian: Robustness]
look like to design proper the idealfunctions
transfer sys- (gain does not grow infinite at high
ies may have significant and direct impact for
igenvalues and eigenvectors of W are left singular tem, invert plant dynamics,frequencies). etc. The infinity norm is defined as:
ear turbulent
d flows, even away from laminar c Z• 1 Pd ⇤
s of C
oint d â = Ââ + B̂s,
solutions: (8a) eA t Cbe ⇤
CeaAtterrible (12a)
dt •W o =this would
Why •0 n d
dt idea (in many
kP k∞ , max σ1 (P (iω)) . (26)
b = Ĉ â + D̂s. (8b) cases)... 1 ⇣ ⌘ ω
mean-field models, +exciting one oscillatory b
Controller Plant X• S(⇣) ⇤- s sensitivity function
robust vs. H2 optimal control
k
H with w r e ⇡ +
e A t ⇤
C C e A t k
. (12b)
ode 1 linear control may kill other oscilla- Here, σsensitivity
1 denotes the maximum singular value. Since the
neral controller will be denoted by K(⇣) • P • +T (⇣) - complementary function
y modes
that [36, 37],
wedomain.
have -
established conditions enabling arbitrary
k=0
k · k∞ norm is the maximum value of the transfer function
uency • wr - reference tracking
placement of the closed-loop system, we must now These de-may be approximated byatfull-state any frequency, mea- it is often called a worst-case scenario norm;
semble averages of turbulent flow responses • Time delays
n where to place them. surements of the direct system + in Eq.therefore,
(2) and minimizing
adjoint the infinity norm provides robustness
ay be linear, as in Colonius’s experimental jet
trollability, Observability, and system:4.3 Dynamic feedback + controllers
to worst-case exogenous inputs.
ise study ,
mians If we let Pw→J denote the transfer function from w to
Given linear10 dynamics of the form in Eq. (2), we of-
usek’s jet simulation at Re=100, wen= CC ⇤ , J , then the(13a) goal of H control is to construct a controller
y to place poles of the closed-loop system ten want aW feedback
c ⇡ Wcontroller c based on full-state ∞
to minimize the infinity norm: min kPw→J k∞ . This is typi-
with choice
nsient control of of relies on
Figure
Krseparation 9: thefor controlla-
Feedback fully turbu-control with feedback:
disturbancesbW =o and Kr a.
⇡ W For
o = example,
noise.
e
OO . ⇤ if the (13b)system is
cally difficult, and no analytic closed-form solution exists for
e system
nt boundary in Eq.
layer(2).[38].
The system is control- controllable, then the poles of the closed-loop system
following matrix: may be placed
[Decide on notationarbitrarily by choice vs.
for continuous of Kdiscrete
r.

inearized input–output time A... emp. Gramians are slightly

17 wrong, since
Na dynamics
⇤ d
2
B AB A B · · · A 1 B (9a) they use Ad ] a = Aa BKr a (4a)
we are interested in linearizing Eq. (1) about [Mention SVDdt and symmetric matrix, degrees of
y fixed
ank Na . point as or periodic
Controllability orbit,that
implies leading
the to controllability...] = (A BKr ) a. (4b)
ay be steered
r system to an arbitrary state a with
of equations: [Mention balanced model reduction: BT, BPOD,
 Pw→J etc. Typically, a loop shape should have large gain at low
frequency to guarantee accurate reference tracking and slow
w J disturbance rejection, low gain at high frequencies to atten-
Generalized uate sensor noise, and a cross-over frequency that ensures
plant desirable bandwidth. The loop transfer function is then ro-
bustified so that there are improved gain and phase margins.
H2 control has been an extremely popular control
paradigm because of its simple mathematical formulation
b s and its tunability by user input. The advantages of H∞ con-
trol are increasingly realized in flow control, eminent exam-
Controller ples being the collaborative research centers (Sfb 557 & Sfb
1029) lead by Rudibert King [16, 17, 175]. Additionally, there
are numerous consumer software solutions that make imple-
mentation relatively straightforward. In Matlab, mixed sen-
sitivity is accomplished using the >>mixsyn command in
Figure 10: General framework for feedback control. The in- the robust control toolbox. Similarly, loop-shaping is accom-
put to the controller are the system measurements s, and the plished using the >>loopsyn command in the robust control
controller outputs an actuation signal b. The exogenous in- toolbox.
puts w may refer to a reference wr , disturbances wd or sensor
noise wn . The cost function J may measure cost associated
with inaccuracy of reference tracking, expense of control, etc. 4.4.3 Fundamental limitations with implications for
turbulence control
As discussed above, we want to minimize the peaks of S and
the optimal controller in general. However, there are rela-
T . Some peakedness is inevitable, and there are certain sys-
tively efficient iterative methods to find a controller such that
tem characteristics that significantly limit performance and
kPw→J k∞ < γ, as described in [164]. There are numerous
robustness. Most notably, time-delays and right-half plane
conditions and caveats that describe when this method can
zeros of the open-loop system will limit the effective control
be used. In addition, there are computationally efficient al-
bandwidth and will increase the attainable lower-bound for
gorithms implemented in both Matlab and Python, and these
peaks of S and T . This contributes to both degrading perfor-
methods require relatively low overhead from the user.
mance and decreasing robustness.
Selecting the cost function J to meet design specifica-
Similarly, a plant will suffer from robust performance
tions is a critically important part of robust control design.
limitations if the number of poles exceeds the number of ze-
Considerations such as disturbance rejection, noise attenu-
ros by more than 2. These fundamental limitations are quan-
ation, controller bandwidth, and actuation cost may be ac-
tified in the waterbed integrals, which are so named because
counted for by a weighted sum of the transfer functions
if you push a waterbed down in one location, it must rise in
S, T , and KS. In the mixed sensitivity control problem, vari-
a another. Thus, there are limits to how much one can push
ous weighting transfer function are used to balance the rela-
down peaks in S without causing other peaks to pop up.
tive importance of these considerations at various frequency
Time delays are somewhat easier to understand, since a
ranges. For instance, we may weight S by a low-pass filter
time delay τ will introduce an additional phase lag of τ ω at
and KS by a high-pass filter, so that disturbance rejection
the frequency ω, limiting how fast the controller can respond
at low frequency is promoted and control response at high-
effectively (i.e. bandwidth). Thus, the bandwidth for a con-
frequency is discouraged. A general cost function may con-
troller with acceptable phase margins is typically ωB < 1/τ .
sist of three weighting filters Fk multiplying S, T , and KS:
Following the discussion in [166], these fundamental
  limitations may be understood in relation to the limitations
F1 S
of open-loop control based on model inversion from Sec. 4.2.
 F2 T  .
If we consider high-gain feedback b = K(wr − s) for a sys-
F3 KS
∞
tem as in Fig. 9 and Eq. (25), but without disturbances or
Another possible robust control design is called H∞ noise, we have
loop-shaping. This procedure may be more straightforward
than mixed sensitivity synthesis for many problems. The b = Ke = KSwr . (27)
method consists of two major steps. First, a desired open-
We may write this in terms of the complementary sen-
loop transfer function is specified based on performance
sitivity T , by noting that since T = I − S, we have
goals and classical control design. Input and output com-
T = L(I + L)−1 = P KS:
pensators are constructed to transform the open-loop sys-
tem to the desired loop shape. Second, the shaped loop is b = P −1 T wr . (28)
made robust with respect to a large class of model uncer-
tainty. Indeed, the procedure of H∞ loop shaping allows Thus, at frequencies where T is nearly the identity I and
the user to design an ideal controller to meet performance control is effective, the actuation is effectively inverting the
specifications, such as rise-time, band-width, settling-time, plant P . Even with sensor-based feedback, perfect control is

18
unattainable. For example, if the plant P has right-half plane
zeros, then the actuation signal will become unbounded if
the gain K is too aggressive. Similarly, limitations arise with
time-delays and when the number of poles of P exceed the
number of zeros, as in the case of open-loop model-based in-
version.
As a final illustration of the limitation of right-half
plane zeros, we consider the case of proportional control
b = Ks in a single-input, single output system with plant
P (ζ) = N (ζ)/D(ζ). Here, roots of the numerator N (ζ) are
zeros of the plant and roots of the denominator D(ζ) are
poles. The closed-loop transfer function from reference wr
to sensors s is given by:
s(ζ) PK N K/D NK
= = = . (29)
wr (ζ) 1 + PK 1 + N K/D D + NK
For small control gain K, the term N K in the denominator
is small, and the poles of the closed-loop system are near the
poles of P , given by roots of D. As K is increased, the N K
term in the denominator begins to dominate, and closed-loop
poles are attracted to the roots of N , which are the open-loop
zeros of P . Thus, if there are right-half plane zeros of the
open-loop plant P , then high-gain proportional control will
drive the system unstable. These effects are often observed
in the root locus plot from classical control theory. In this
way, we see that right-half plane zeros will directly impose
limitations on the gain margin of the controller.
The limitations imposed by a time-delay have significant
implications for turbulence control. First, time-delays will be
inevitable for many sensor/actuator configurations in con-
vective flows [176], limiting the frequency of disturbances
that may be effectively suppressed with feedback. Second, as
turbulence time-scales may become extremely fast, the time it
takes to compute a control action will introduce a latency that
is just as deleterious as a time-delay in the plant. Time-delays
and flow time-scales should be primary considerations when
designing feedback controllers for turbulent flows.
4.4.4 Two degrees of freedom control
With the addition of reference and disturbance feed-forward
control, it is possible to improve the control performance at
frequencies where feedback control is ineffective due to large
sensitivity. This more sophisticated two-degrees-of-freedom
control is illustrated in Fig. 11.
Again, we may compute the transfer function from in-
puts to output:
s =(I + P K)−1
[P (K + Kref )wr + (Pd − P Kd )wd − P Kwn ] .
Then the error becomes:
e = s − wr = −SSref wr + SSd Pd wd − T wn , (30)
where the additional feedforward sensitivity functions are
Sref = I − P Kref , (31a)
Sd = I− P Kd Pd−1 . (31b)
19
wd
+ +
Kref Kd Pd
wr + e + Plant +
K + P
- +
s
+
+
wn
Figure 11: Two degrees-of-freedom control with reference
tracking and disturbance rejection.
Two-degree of freedom control may be intuitively un-
derstood as the combination of a fast feedforward controller
to get close to a desired reference value, followed by a slow
feedback controller to mitigate model uncertainty and re-
ject unexpected disturbances. There are explicit bounds on
model uncertainty that determine when combined inverse-
based feedforward and feedback will outperform feedback
alone [177].
4.5 Balanced model reduction
The high-dimensionality and short time-scales associated
with turbulent flows makes it infeasible to implement the
model-based control strategies above in real-time. More-
over, solving for H2 and H∞ optimal controllers may be
computationally intractable, as they involve either a high-
dimensional Riccati equation solve, or an expensive iterative
optimization. Instead, reduced-order models provide effi-
cient, low-dimensional representations of the most relevant
flow mechanisms. These models result in efficient controllers
that may be applied in real-time for many systems. An alter-
native is to develop controllers based on the full-dimensional
model and then apply model reduction techniques directly to
the full controller [178].
Model reduction is essentially data reduction that re-
spects the fact that the data is generated by a dynamic pro-
cess. If the dynamic process is a linear time-invariant (LTI)
input–output system, then there is a wealth of machinery
available for model reduction, and performance bounds may
be quantified. Many of these methods are based on the singu-
lar value decomposition (SVD) [88, 179, 87], and the minimal
realization theory of Ho and Kalman [180, 181]. The general
idea is to determine a hierarchical modal decomposition of
the flow state that may be truncated at some model order,
only keeping the most important coherent structures.
The proper orthogonal decomposition (POD) [182, 85]
orders modes based on energy content, and it has been
widely used for a range of fluid dynamic reduced-order
models, many for control. POD is guaranteed to provide
an optimal low-rank basis to capture the maximal energy
or variance in a data set. In some cases, a large number of
POD modes may be required to represent non-normal energy
growth, as in wall-bounded shear flows, such as in [183].
Instead of ordering modes based on energy, it is possible
to determine a hierarchy of modes that are most controllable
and observable, therefore capturing the most input–output
information. Such balanced models give equal weighting to
the controllability and observability of a state, providing a
coordinate transformation that makes the controllability and
observability Gramians equal and diagonal. It is observed
that for many high-dimensional systems, the control input
may only excite a few controllable modes, with the remain-
ing modes being stable. These models have been extremely
successful in the context of flow control, especially for sys-
tems with non-normal growth. However, computing a bal-
anced model using traditional methods is extremely expen-
sive computationally. In this section, we describe the balanc-
ing procedure, as well as modern methods for efficient com-
putation of balanced models. A computationally efficient
suite of algorithms for model reduction and system identi-
fication may be found in [117].
4.5.1 Discrete-time systems and Gramians
Until now, we have considered continuous-time dynamical
systems, as in Eq. (5). However, the discussion of model re-
duction is somewhat simplified using a discrete-time model:
ak+1 = Ad ak + Bd bk ,
sk = Cd ak + Dd bk .
The index k may be thought of as the k-th sample of a
continuous-time system in Eq. (5), sampled every ∆t by
an analogue-to-digital converter. Alternatively, the discrete-
time system may be a numerical time-stepper. The discrete-
time system may be related to the continuous-time system
R ∆t
by the following: Ad = exp(A∆t), Bd = 0 exp(Aτ )Bdτ ,
Cd = C, and Dd = D. Then ak = a(k∆t), and similar for
bk and sk . The assumption that b is constant during each ∆t
interval is called a zero-order-hold in control theory.
The Gramians may be approximated by full-state mea-
surements of the direct system in Eq. (32) and adjoint system:
direct: ak+1 = Ad ak + Bd bk ,
adjoint: ak+1 = A∗d ak + Cd∗ sk .
It is then possible to determine empirical Gramians without
solving the Lyapunov equations in Eq. (15).
Wc ≈ Wce = C d C ∗d ,
Wo ≈ Woe = O d O ∗d .
C d and O d are snapshot matrices constructed from impulse
response simulations of the discrete direct and adjoint sys-
tems in Eq. (32). The method of empirical Gramians is quite
efficient, and was used in [181, 184, 185]. Note that there are
Ns adjoint impulse response experiments required. This be-
comes intractable when there are a large number of outputs
(e.g. full state measurements).
4.5.2 Goal of model reduction
The goal of model reduction is to obtain a related system
(Ar , Br , Cr , Dr ),
ãk+1 = Ar ãk + Br bk , (35a)
sk = Cr ãk + Dr bk , (35b)
in terms of a state ãk ∈ R with reduced state dimension,
Nr
Nr  Na . Note that bk and sk are the same in Eq. (32) and
Eq. (35). A balanced reduced-order model should map in-
puts to outputs as faithfully as possible for a given order.
It is therefore important to introduce an operator norm to
quantify how similarly Eq. (32) and Eq. (35) act on a given
set of inputs. Typically, we consider the infinity norm of the
transfer functions P (ζ) and Pr (ζ) obtained from the full sys-
tem (32) and reduced system (35), respectively:
kP k∞ , max σ1 (P (iω)) . (36)
ω
To summarize, we seek a reduced-order model (35) of low
order, Nr  Na , so the operator norm kP − Pr k∞ is small.
4.5.3 Balanced proper orthogonal decomposition
In balanced truncation (BT) [181], a coordinate transforma-
tion is sought that makes the observability and controllability
Gramians equal and diagonal. The balancing transformation
is given by the matrix B in the eigendecomposition:
(32a)
(32b) Wc Wo B = BΣ2 (37)
where Σ is a diagonal matrix containing Hankel singular val-
ues (HSVs). It is then possible to obtain a reduced-order basis
by choosing the first Nr columns of B corresponding to the
Nr largest HSVs. It has been demonstrated that modes with
small energy content may be important for control of a given
input–output system [97, 183]. Therefore, instead of truncat-
ing based on energy content, we consider truncating based
on Hankel singular values.
In practice, it may be extremely expensive to compute
the Gramians Wc and Wo by solving a high-dimensional
Lyapunov equation, and the subsequent eigendecomposition
is also expensive. Instead of solving for Gramians directly, it
is possible to construct empirical Gramians using snapshots
(33a) of direct and adjoint simulations, as discussed in Eq. (34).
(33b) Strong connections have been established between POD and
balanced truncation [181, 184, 185, 96], and in [96], POD is
used to obtain low-rank approximations to the Gramians.
However, the early methods combining POD and BT do not
(34a) scale well with the number of output measurements. In fact,
the number of adjoint simulations required is equal to the
(34b)
number of output measurements, Ns , which may be quite
large [186].
In [97], Rowley introduced the method of balanced
proper orthogonal decomposition (BPOD) to address the
20
aforementioned issues associated with snapshot based bal- One of the key benefits of balanced truncation is that
anced truncation. There are two major advances introduced there is an upper bound on the error of a given order trunca-
in this method that make it practical to very large dynamical tion:
systems: n
X
1. Method of snapshots – Instead of computing the balanc- Upper bound: kP − Pr k∞ < 2 σj .
ing transformation by solving the eigendecomposition j=Nr +1
in Eq. (37), it is possible to construct the transformation
by computing the singular value decomposition of A BPOD model may not exactly satisfy this upper bound
due to errors in the calculation of empirical Gramians. In ad-
Od C d , (38) dition, there is a lower bound that is universal to all models
with Nr states:
reminiscent of the method of snapshots from [187, 188,
86]. Lower bound: kP − Pr k∞ > σNr +1 .
2. Output projection – To avoid computing Ns adjoint
simulations, it is possible instead to solve an output- 4.5.4 Eigensystem realization algorithm
projected adjoint equation:
Balanced POD requires expensive full-state measurements
ak+1 = A∗d ak + Cd∗ Ur s (39) and adjoint simulations, which are not feasible for experi-
ments. The eigensystem realization algorithm (ERA) makes
where Ur is a rank-Nr proper orthogonal decomposi-
it possible to obtain equivalent balanced models directly
tion truncation.
from input–output measurements. ERA is based on the mini-
First, define the generalized Hankel matrix as the prod- mal realization theory of [180], and it was developed to iden-
uct of the direct (C d ) and adjoint (O ∗d ) snapshot matrices from tify structural models for various spacecraft [141]. It has
Eq. (11) and (12), for the discrete-time system: been shown that ERA models are equivalent to BPOD mod-
H = Od C d (40) els [189]. ERA is a system identification method based on
 mc −1  impulse response measurements and does not require prior
Cd Bd Cd Ad Bd ··· Cd Ad Bd knowledge of the high-dimensional model in Eq. (32) or ac-
 Cd Ad Bd
 Cd A2d Bd ··· Cd Amd Bd
c  cess to adjoint simulations.

= .. .. .. .. . Similar to BPOD, we take the SVD of the Hankel matrix
 . . . . 
mo −1 mo mc +mo −2 H. However, instead of collecting snapshot matrices C d and
Cd Ad Bd Cd Ad Bd · · · Cd Ad Bd
O ∗d to obtain modes Φ and Ψ, we proceed directly to the
2
Next, we factor H using the SVD: reduced-order model, without modes. Introducing another
   ∗ Hankel matrix H 0 advanced one time-step in the future
∗   Σ 1 0 V1 ∗
H = U ΣV = U1 U2 = U1 Σ1 V1 .
0 0 V2∗ H 0 = O d Ad C d , (44)
(41)
a model is constructed as follows:
For a given desired model order Nr  Na , only the first Nr
columns of U and V are kept, along with the first Nr × Nr Ar = Σ−1/2 r Ur∗ H 0 Vr Σ−1/2
r , (45a)
block of Σ. This yields a bi-orthogonal set of modes given Br = first p columns of Σr V , 1/2 ∗
(45b)
by:
Cr = first q columns of U Σ1/2 r . (45c)
direct modes: Φr = C d Vr Σr−1/2 , (42a)
It is simple to verify that (43a) and (45a) are equal.
adjoint modes: Ψr = O ∗d Ur Σr−1/2 . (42b) The expressions (43b) and (45b) for Br are equivalent, since
−1/2 ∗ 1/2
These modes are bi-orthogonal, Ψ∗r Φr = INr ×Nr , and Σr Ur H = Σr Vr∗ . Similarly, the expressions for Cr are
Rowley [97] showed that the direct and adjoint modes, equivalent.
Φ and Ψ, establish the change of coordinates that bal- H and H 0 are constructed from impulse response sim-
ance the empirical Gramians: B = Φ , C d V Σ−1/2 and ulations/experiments, without the need for storing direct or
B−1 = Ψ∗ , Σ−1/2 U ∗ O d . Moreover, Φr and Ψr are the first adjoint snapshots. However, if direct snapshots are available,
Nr -columns of the balancing transformation. for example, by collecting velocity fields in simulations or
Now, these modes allow us to project our original sys- PIV, then it is possible to construct direct modes.
tem onto a (balanced) reduced-order model or order Nr : ERA and BPOD balance the empirical Gramians, so un-
less we collect a very large amount of data, the true Grami-
Ar = Ψ∗r Ad Φr , (43a) ans are only approximately balanced. However, instead of
Br = Ψ∗r Bd , (43b) collecting long tails of data, it is possible to collect data until
the Hankel matrix is full rank, balance the full-rank identi-
Cr = Cd Φr . (43c)
fied model, and then truncate. This is more efficient than
2 The powers mc and mo in Eq. (40) signify that data must be collecting snapshots until all of the transients have decayed3 .
collected until the matrices C d and O ∗d become full rank, after which
the controllable/observable subspaces have been sampled. 3 This idea is developed in [190], inspired by [191].

21
4.5.5 Observer/Kalman filter identification
Observer/Kalman filter identification (OKID) was devel-
oped to compliment the ERA for lightly damped experimen-
tal systems with noise [192, 193, 194, 142]. This method
addresses the general problem of identifying an approxi-
mate impulse response (i.e. Markov parameters) from arbi-
trary, noisy input–output data. Typically, one would ob-
tain reduced-order models according to the following gen-
eral procedure:
1. Collect output response to a pseudo-random input
2. This information is then passed through the OKID algo-
rithm to obtain the linear impulse response,
3. The impulse response is passed through the ERA to ob-
tain a reduced-order state-space system.
The OKID method identifies an approximation to the
impulse response of a system from noisy input–output data
using an asymptotically stable Kalman filter. Although this
method is quite powerful, it has not been broadly adopted
because of the algorithmic complexity as well as nonintuitive
theoretical elements. However, a recent method extending
dynamic mode decomposition (DMD) to systems with con-
trol inputs provides similar models as ERA/OKID, but with
a more intuitive and computationally simple algorithm [195].
This follows recent results that DMD and ERA may be seen
as equivalent, up to a coordinate transformation, for certain
types of input–output data [101].
4.6 Case study: Transition delay in a
boundary layer and stabilizing steady
states
For many canonical flows, such as pipe flow or Couette
flow, the underlying laminar steady solution is stable for all
Reynolds numbers [196, 197, 198], and turbulent fluctuations
arise as transient events due to perturbations and wall rough-
ness. However, many fully turbulent flows are character-
ized by saturated nonlinear dynamics, and the existence of
a desirable underlying unstable fixed point or periodic or-
bit. For example, in the laminar-to-turbulent transition of a
flat plate boundary layer or channel flow, the unstable fixed
point corresponds to steady, laminar flow with significantly
decreased drag. The existence of these remnant unstable
steady-states is consistent with the Hopf-bifurcation route to
turbulence [199]. In addition, linearization may also work in
the viscous sublayer in wall turbulence.
Linear control is ideal for such systems, because success-
ful regulators keep the system in a state that is near the fixed
point, where the linear assumption remains valid. Therefore,
linear control has largely been applied to transition delay and
stabilizing unstable steady-states. It is also possible to desta-
bilize a system to enhance mixing by reversing the sign of
a stabilizing controller [200]. It is worth noting that there
are numerous examples where linear model-based control
has been successfully applied to flows that are in a nonlinear
regime. Often times turbulence is considered a disturbance
term in a slower dynamical system, such as the rigid body
22
equations of an aircraft, space shuttle, or rocket. In this case,
turbulent fluctuations may be seen as inevitable and operat-
ing on a time scale that is faster than controller bandwidth.
Instead of trying to change the nature of the turbulence itself,
the controller may be designed to obtain some other objec-
tive while robustly managing the uncertain turbulent distur-
bance. Here, we consider the problem of actively controlling
the turbulent dynamical system.
The field of linear model-based flow control has expe-
rienced exceptional growth, largely due to the close align-
ment of challenges in engineering fluid dynamics with the
sophisticated machinery in model reduction and control the-
ory. There are a number of excellent reviews of linear flow
control [20], for transition delay in spatially developing wall-
bounded flows [170, 201, 171], and for the control of cavity
oscillations [34]. A good overview of drag reduction by con-
trol of turbulence boundary layers is also found in [10], with
a quantification of potential fuel savings in transportation.
The discussion here is not meant to be complete, but rather
is designed to clarify the choices and historical progress of
linear model-based flow control, especially in turbulence.
4.6.1 Early work
Early progress in model-based flow control is closely tied to
advances in computational resources, enabling the simula-
tion of large-scale dynamical systems, such as in fluid dy-
namics. Currently, the majority of model-based flow control
employing H2 and H∞ techniques are applied to numerical
simulations. In [202], gradient-based optimal control tech-
niques were introduced to control the flow in a channel using
boundary condition control. Adjoint methods have also been
used extensively for aerodynamic shape optimization based
on CFD [203, 204, 205, 206].
A suboptimal control approximation was introduced by
Choi et al in 1993 [207] using a very short time horizon that
allowed convective terms to be neglected, resulting in a con-
troller based on a Stokes flow model. This method was then
applied to a turbulent channel flow by Bewley and Moin
in 1994 [208], resulting in a 17% drag reduction. Subop-
timal control with more physically realizable measurement
was analyzed for a turbulent channel flow in [209]. Without
the suboptimal approximation, a model-predictive control
(MPC) framework may be used for receding-horizon optimal
control, although this is computationally intensive. MPC has
been used in [210] and [211], and is discussed in more detail
in a recent review [171].
In Bewley and Liu [212], both H2 and H∞ controllers
were developed and applied to single wavenumber pairs in
a plane channel flow with three dimensional perturbations.
In [213], robust control was applied to the extended prob-
lem of 2D multi-wavenumber control with localized actua-
tion and sensing in plane Poiseuille flow. Högberg et al [214]
extended the analysis of [212] to include multiple wavenum-
bers and state estimation using a Kalman filter based on wall
measurements. Their three-dimensional DNS used bound-
ary condition forcing, and it was shown that state-estimation
was not as effective as full-state feedback.
Högberg and Henningson [215] demonstrated the effec-
tiveness of linear H2 optimal control in spatially developing
boundary layers, even with moderately strong nonlinearities
present. In a follow-up work [216], they extended this analy-
sis to include full-state estimation based on a number of wall
measurements, similar to [214].
4.6.2 Use of reduced-order models
As discussed earlier, the development of controllers for high-
dimensional linear systems may be extremely computation-
ally intensive since they depend on the solution to Riccati
equations, involving O(Na3 ) operations, or expensive itera-
tions. Further, once a high-dimensional controller has been
designed, it is expensive to compute the resulting control ac-
tion, introducing unwanted latency in the control loop. This
latency significantly limits control bandwidth and may pre-
vent real-time control of high-dimensional systems with fast
time-scales. These issues are especially pronounced in flow
control.
In the timely work of Åkervik et al [218], reduced-order
models were developed by using a few non-normal global
eigenmodes of the linearized Navier-Stokes equations as a
basis for Galerkin projection. Based on the reduced-order
model, an LQG controller is able to attenuate global oscilla-
tions in a separated boundary layer. The use of non-normal
global modes has been important to capture the transient
energy growth associated with wall-bounded shear flows.
Yet, these approaches have limited applicability to unstable
advection-dominated flows. Transient growth implies a sen-
sitivity issue: a small difference in the initial condition or a
small perturbation in the system can give rise to significantly
different solutions. Hence, unavoidable truncation errors
of reduced-order models based on global modes may cause
similar differences in the solutions. The sensitivity problem
is less pronounced at lower Reynolds numbers or for weakly
unstable flows.
After the seminal work of Rowley [97], introducing
a computationally efficient method to compute balanced
reduced-order models, the use of balanced models has be-
come a mainstay in model-based linear flow control. The re-
sulting models are able to capture transient energy growth
with significantly fewer modes than POD, as demonstrated
in [183] on the transitional channel flow. BPOD was also
shown to outperform standard POD/Galerkin methods for
the separation control of an airfoil at low Reynolds num-
ber [219], using an immersed-boundary method [220, 221].
The paper of Bagheri, Brandt, and Henningson [222]
provides one of the most complete and clearly presented nar-
ratives on closed-loop control based on balanced models. In
this work, they develop models using snapshot-based bal-
anced truncation. Snapshot based methods are necessary
when the state dimension Na is large, since the A matrix
is size Na2 , which may not be representable in system mem-
ory. The matrix-free methods advocated in [222] use the CFD
time-stepper to find the next flow state, rather than explic-
itly computing a large A matrix. These methods may be
used to solve large eigenvalue problems, for instance using
Arnoldi iteration, thus bypassing the creation and factoriza-
tion of a large matrix. In this paper, H2 control is developed
23
to suppress two-dimensional boundary layer perturbations.
[223] presents a three-dimensional generalization of [222] us-
ing LQG to delay transition in a boundary layer.
As described in Sec. 4.5.4, the eigensystem realization al-
gorithm (ERA) produces balanced models that are equivalent
to BPOD, but without the need for adjoint simulations [189].
ERA has been rapidly adopted in flow control because of the
fact that it is based purely on input–output data, making it a
system identification method, rather than a model reduction
method. ERA was used to study and control cavity flow os-
cillations and combustion oscillations [224, 225]. In Semeraro
et al [217], ERA models were used to develop LQG control to
suppress three-dimensional Tollmien-Schlichting (TS) waves
in a transitional boundary layer using a volume force actua-
tion. This paper is well-written and thought provoking, and
an illustration of the closed-loop control topology is shown in
Fig. 12. Interestingly, one of the earliest experimental demon-
strations of active feedback suppression of turbulence was
applied to cancel TS waves using a downstream heating el-
ement and a phase shifted measurement feedback, resulting
in an increase in the transition Reynolds number [18].
4.7 Case study: Wall turbulence control
and skin-friction reduction
There is a tremendous industrial motivation to reduce tur-
bulent skin-friction drag. This problem has thus received
significant attention, and corresponding progress has been
made [40, 10]. Reduced-order models for control based on
LQG and the robustifying loop-transfer recovery were used
to reduce skin-friction drag in a a channel flow [226, 227], re-
sulting in eventual relaminarization of the flow. Reduced-
order models were also used with blowing/suction at the
wall to reduce skin friction in simulations [228], and reduced-
order Kalman filters were developed based on balanced trun-
cation models [133].
In addition to the mechanisms and techniques dis-
cussed, there have been many efforts to reduce turbulent
skin friction drag [229, 230, 231, 232, 233] using various tech-
niques such as traveling-wave-like actuation including blow-
ing/suction and wall deformation. Increasing heat trans-
fer while decreasing drag is also an important area of re-
search [234]. In addition, system identification and control
were used for the linear feed-forward control of turbulent
boundary layer fluctuations by exploiting coherent struc-
tures and predicting their behavior downstream [235]. The
same reduction in turbulent boundary layer fluctuations was
explored using model-based control in an experiment using
an array of synthetic jet actuators [236].
4.8 Case study: Cavity flow control
The work surrounding the control of open cavity flows rep-
resents one of the most complete and compelling success
stories in active flow control [237]. Cavity flow control has
many applications, such as the suppression of oscillations in
weapons bays in high-speed flight [238], which is an inher-
ently high Reynolds number phenomena. A more complete
 Transition delay in a boundary layer flow using active control 293

Domain to minimize
disturbance energy
C1

Sensors Actuators z(t)

C2
B2

Disturbance
B1
Controller

Figure 12: Schematic of the closed-loop controller for transition delay of a flat plate boundary layer, reproduced from [217].
F IGURE 2. (Colour online) Sketch of the control configuration. The disturbance (B1 ) consists
Here, ψ correspond to sensors s and ϕ correspond to actuators b.
of an optimal initial condition. A low-order LQG controller is designed; based on the output
extracted by a spanwise row of sensors (C 2 ), the controller computes the control signals
feeding of
discussion a row of actuators
the cavity flow control(B 2 ). The
problem objective
is discussed in ofborhood
the controller is transition
of 20%, and to minimize the disturbance
delay cited in the literature,
energy in the region
reviews [239, 34, 240]. marked with the grey colour. The inset figure shows
it may be somewhat surprising that activethatclosed-loop
all of the con-
sensors are wired to all of the actuators (so-called centralized control, see Appendix). trains, or
Suppression of cavity oscillations has benefited from a trol is not being utilized on mainstream aircraft,
combination of new physical insights combined with simu- automobiles. There are a number of reasons why the studies
lations [241], advances in advanced Galerkin modeling [242, above are largely numerical. First, many model-reduction
243], and the practical implementation of control in experi- techniques require extensive and invasive information about
ments [244, 245, 246, 247]. Exciting new methods, such as the plant, although ERA is less invasive. Second, the time-
The based
control optimal, localized
on neural networks [248,initial
46] andcondition triggers
the use of scales a with
associated wavepacket of and
real experimental TSindustrial
waves tur-
(Monokrousos
ERA et al.
[225] have also been 2010). The temporal behaviour
investigated. bulence are described
extremely fast,by so the signal w
that reducing time 2 R is
(t)delays
is In
thesome
herently
regimes, oscillations are self-sustained and in- a significant challenge. Time delays may originate from la-
disturbance
nonlinear. signal
To identify excitation,
models assumedtency
suitable for control, asinvolved
a unit-variance white
in the computation of a noise process.
control decision, and
et al [244]Bstabilized
n⇥m
The matrix
Rowley 2 2 R the represents theNyquist
oscillations using spatial they
distribution
may also ariseoffrom
theconvective
actuators,time located
scales fromin a
actua-
plots to tune the phase of controllers based on experimen- tors to sensors.
spanwise array of m elements and modelled as volume forcing localized close to the Finally, the development of advanced sensor
tally obtained frequency response data. A model was then and actuator hardware will be a major enabling factor in the
walls. for
obtained The thecontrol
closed-loopsignal
systemis represented
using by (t)
system identifica- 2 Rmimplementation
practical . of these methods.
tion,The p
output
and model signals
for the in (2.2b) are extracted by the array of p localized
(t) 2wasRderived.
unstable system
sensors C 2 2 Rp⇥n , which are placed a short distance upstream
5 Prototypes of the and
for linear actuators.
non-
The Potential
4.9 noise contamination of the signal is modelled by the unit-variance white noise
impact and challenges
p linear dynamics
g (t)
All , whose
2 Rmodels
of the level
discussed is and
above, set therefore
by thetheconstant
cor- ⌘. A large value of ⌘ introduces high
levels ofcontrollers,
responding noise corruption
depend on theon the measurements
placement of actua- Despite the whiletools
powerful
(t), a for
small
linearvalue indicates
model reduction and
tors and sensors, as these directly effect B and C in Eq. (5). control, the assumption of linearity is often overly restrictive
high fidelity of the information extracted by the sensors C 2 . Turbulent fluctuations are inherently
Actuator and sensor placement was recently investigated for real-world fluids.
The output
for boundary signal delay
layer transition 2 RkImportantly,
z (t)[249]. in (2.2c) this isnonlinear,
used to andassess
often the the performance
goal is not to stabilize anofunstable
the
controller. Output projection is performed (Rowley 2005): the signal is obtained byat-
study provides a clear comparison of feedforward (sen- fixed point but rather to change the nature of a turbulent
sor upstream of actuator) versus feedback (sensor down- tractor. Moreover, it may be the case that the control input is
projecting
stream the control.
of actuator) velocity field on itaissequence
In particular, shown that ofeitherk =a bifurcation
10 proper orthogonal
parameter itself, or decomposition
closely related to one,
(POD) modes, capturing ⇡93 % of the perturbation energy. The modes
disturbance-feedforward control is effective sometimes, but such as the control surfaces on are generated
an aircraft.
is sensitive to additional disturbances and uncertainty. Feed- The degree of nonlinearity is most easily characterized
by a dataset of snapshots collected from the impulse response to the initial
back, on the other hand, effectively rejects disturbances and in a Galerkin modeling framework (Sec. 5.1). The subsequent
condition
B1 and are
compensates represented
for unmodeled by However,
dynamics. the rows sen-C 1sections
if theof
k⇥n
2 R (Sec. . By adopting
5.2–5.5) this approach,
provide prototypic the
examples of linear,
k
signal z (t) 2 R represents an approximation of the energy content of the system;
sor is too far downstream, the time-delay between actuation weakly, moderately and strongly nonlinear dynamics as evi-
and sensing dramatically limits robust performance, which denced in many fluid flows and corresponding control strate-
isbased onwith
consistent thetheoutput z (t),
discussion a fictitious output isgies.
above. defined
Sec. 5.6as
concludes with enablers and show-stoppers of
Given the performance of drag reduction, in the neigh- nonlinear model-based control design.
" # " #
C 1 0
z (t)0 = u(t)240 + 1/2 (t), (2.3)
0 R

where the diagonal entries of the matrix R 2 Rm⇥m correspond to the control penalty
5.1 Galerkin model Here, the quasi-constant cB i measures the distance to the
fixed point a ≡ 0, the quasi-linear term with the coefficients
In this section, different degrees of system non-linearity are incorporates slow base-flow changes, and hBi represents a
B
lij
introduced. For simplicity, we consider an incompressible quadratic nonlinearity in the fluctuating modes. hB
i has slow
velocity field u(x, t) in a finite steady domain x ∈ Ω. Let and fast components. If the volume force is in feedback with
a = [a1 , a2 , . . . , aNa ]T represent an Na -dimensional approxi- the fluctuations, cB
i + hi ≈ 0 in a short-term average. In this
B
mation of the the flow state. The coordinates ai , i = 1, . . . , Na f
Na
may, for instance, represent the coefficients in a Galerkin ex- P
case, fi = B
lij aj + gi b, i = 1, . . . , Naf may be a suitable
pansion j=1

XNa linear representation of the fast dynamics.

u(x, t) = us (x) + ai (t) ui (x), (46) The slow modes are slaved to an approximate manifold
i=1 in state space defined by dai /dt = fi (a) ≈ 0, i = Naf +
where us represents the steady Navier-Stokes solution and 1, . . . , Na and are driven by the Reynolds stress contributions
ui , i = 1, . . . , Na , are suitable expansion modes. of the fast modes, or, equivalently, by the slow component of
Let b = [b1 , b2 , . . . , bNb ]T characterize hBi .
P b the actuation. One In the following sections, four prototypic examples are
example involves Nb volume forces N i bi (t) gi (x) with the
individual fields gi , i = 1, . . . , Nb . To simplify the discussion, discussed: A) an oscillation around the fixed point; B) a self-
only a single component is assumed, Nb = 1, and b = b1 excited amplitude-limited oscillation; C) frequency cross-talk
denotes its amplitude. with two different frequencies over the base-flow deforma-
The sensor signals s = [s1 , s2 , . . . , sNs ]T may represent tion; and D) the remaining irreducible cases.
velocity components. In this case, they are affinely related to
expansion coefficients via (46). The sensing plays no role in 5.2 Linear dynamics
this section. Focus is placed on finding a proper linearized
system whenever possible. First, a small oscillatory fluctuation around a steady solution
For the sake of simplicity, we assume the structure of is considered. Examples include the flow over a backward-
a Galerkin system with a single volume force as actuation. facing step [250] at subcritical Reynolds number with noise,
Then, the dynamics in Eq. (2) take the form transition delay of a boundary layer [251], or stabilization of
a cylinder wake [19, 252]. In Sec. 5.2.1, a control-oriented os-
Na
X Na
X cillator model is presented as a least-order description. Sec.
fi (a, b) = lij aj + qijk aj ak + gi b. (47) 5.2.2 exemplifies the powerful method of energy-based con-
j=1 j,k=1 trol design for this model.
The constant term of the dynamics vanishes since the basic
mode us is assumed to be the steady Navier-Stokes solution. 5.2.1 Oscillator model
Near the fixed point a ≈ 0, the quadratic term may be The considered flows can be described by
neglected yielding the linear dynamics (5) of Sec. 4. Physi-
cally meaningful linearizations may also be effected far away u(x, t) = us (x) + uu (x, t), (48a)
from the origin. The critical element is the notion of ’fast’
uu (x, t) = a1 (t) u1 (x) + a2 (t) u2 (x). (48b)
and ’slow’ modes. The fast modes describe the evolution of
coherent structures and may best be considered as a fluctu- Here, ui , i = 1, 2 correspond to the cosine and sine contri-
ation. Let us assume i = 1, . . . , Naf represent the indices of bution of the first harmonic or the real and imaginary part
the fast modes. The slow modes have significantly lower fre- of the unstable complex eigenmode. Higher harmonics are
quencies and may best be attributed to a base-flow variation. neglected. By construction, the stable or unstable fixed point
Let the remaining indices i = Naf + 1, . . . , Na represent such is as = 0.
slow modes. By a trivial operation, the dynamics have the The linearized version of the dynamics (47) reads
form
f
Na
X d
B B a = A0 a + B b, (49)
fi = ci + lij aj + hBi + gi b, dt
j=1
where
where  
σ u −ω u
Na Na A0 = ,
X X ωu σu
cB
i = lij aj + qijk aj ak ,  
f f 0
j=Na +1 j,k=Na +1 B = .
g
Na
X
B
lij = (qijk + qikj ) ak , Without loss of generality, the modes can be rotated so that
k=Na +1
f
the gain in the first component vanishes. A similar equation
Na f holds for the measurement equation. As discussed in Sec. 4.1,
X
hB = qijk aj ak . the matrices A0 and B depend on the fixed point as and the
i
j,k=1 bifurcation parameters µ.

25
 5.4 Moderately nonlinear dynamics
✓
Ansatz: B = p0
tg
u(x, t) = us (x) + uu (x, t) + ua (x, t) + u (x, t),
(25a)
The linear dynamics (49) may also be an acceptable ap- References [58] & [59].
proximation for turbulent flows with dominant oscillatory
uu (x, t) = a1 (t) u1 (x) + a2 (t) u2 (x), (25b)
behavior. In the triple decomposition, the velocity u a is
field
(x, t) = a3 (t) u3 (x) + a4 (t) u4 (x),
partitioned into the mean flow u0 , the periodic oscillation uu
Unstable 5 (25c)
oscillatory
and an uncorrelated stochastic fluctuation ur . The u periodic
(x, t) = a5 (t) u5 (x) + a6 (t) u6 (x). 5.4 Moderately nonlin
trajectory (25d)
fluctuation may be described by two modes (48b), again. (49)
is obtained by substituting the velocity field u = u + u + u
u r

in the Navier-Stokes equation, projecting onto the modes ui ,

Evolution equation: Ansatz:
and filtering out anything but the dominant frequency. In
this case the growth-rate σ u = 0 has to vanish and the am-
plitude selection mechanism cannot be resolved by the two- 0 1 0 t) 1 = us (x) + uu (x
dimensional Galerkin model. In particular, the fixed point 0 a1 u(x,a1
of the Galerkin model represents the mean flow which is not u
d B C
B a2 C = A(a5 , a6u
B(x, C
B a2t) C = a1 (t) u1 (x) +
the steady solution of the Navier-Stokes equation. Yet, the ) + Bb (26a)
model may be good enough to predict the right actuation for dt @ a3 A u@a a3 A
(x, t) = a3 (t) u3 (x) +
increasing or decreasing the fluctuation near the limit cycle.
a4 a
u (x,4t) = a5 (t) u5 (x) +
5.2.2 Energy-based control design a5 = cu a21 + a22 (26b)
Control design of the linear system (49) can be performed
with the methods of Sec. 4. Here, we illustrate the idea of a6 = ca a23 + a24Evolution equation: (26c)
energy-based control which is particularly suited for nonlin-
ear dynamics. Moreover, energy-based control has a kine- 0 1
matic relation to phasor control and an energetic relation to Stabilizing
a1 12
opposition control. actuation
The growth-rate σ u is assumed positive and small d B C
B a2 C = A(a
enough so that the time-scale of amplitude growth is small Figure 13: Phase portrait of oscillatory linear dt @dynamics
a3 A
compared to the time-scale of oscillation. In this case, the (49). The dashed black trajectory corresponds to the unac-
state can be approximated by a1 = r cos θ, a2 = r sin θ, tuated dynamics while the solid red trajectory corresponds
a4
where r and ω = dθ/dt are slowly varying functions of time. to actuated dynamics with (51). The chosen parameters are
The amplitude evolution is given by σ u = 0.1, σ c = −0.1, ω u = 1, g = 1 implying k = 0.4. a5 = c u
dr2 da1 da2 a6 = c a a
= 2a1 + 2a2 = 2σ u r2 + 2g a2 b.
dt dt dt When a2 = 0, no change of the energy is achievable. Sec-
The control goal is an exponential decay of the amplitude ondly, the construction of the control law is based on design-
with σ c < 0, i.e. ing a dissipative term ga2 b in the power balance. Hence, the
dr2 ! actuation mechanism may be considered an energetic oppo-
= 2σ c r2 .
dt sition control. Thirdly, the actuation command b scales with
Eliminating the time-derivative in both equation yields fluctuation amplitude r. Its phase lags 270◦ with respect to
the first coordinate a1 . Hence, on a kinematical level, the ac-
dr2 c 2 u 2 tuation describes a phasor control. Figure 13 describes the
= σ r = σ r + g a2 b.
dt corresponding unactuated and actuated solution of (49).
The described energy-based control design is very sim-
The control command b increases (decreases) the energy r /2 ple and immediately reveals the physical mechanism. It is
2

if it has the same (different) sign as a2 . To prevent wast- easy to generalize for nonlinear systems, particularly if the
ing actuation energy with the wrong phase, the linear ansatz fluctuation is composed of clean frequency components. Re-
b = −k a2 is made. The gain k > 0 is determined by sub- lated approaches are called Lyapunov control design and
stituting a2 = r cos θ in the energy equation and averaging harmonic balance.
over one period. The resulting control law reads

σc − σu
b=2 a2 . (51)
g 5.3
The gain increases with the difference between the natural
and design growth rate σ c − σ u and decreases with the forc-
ing constant g in the linear dynamics. The factor 2 arises from
the fact that the actuation is only effective in the [0, 1]T direc-
tion.
It may be emphasized that the achieved decay of the
fluctuation energy is an average value over one period.
Weakly nonlinear dynamics
As a refinement to the linearization, mean-field theory is re-
capitulated [253, 254] providing an important nonlinear am-
plitude selection mechanism. The onset of vortex shedding
behind a cylinder is one well investigated example fitting
this description [255]. The following two sections outline the
dynamical model and corresponding control design, respec-
tively.

26
5.3.1 Mean-field model harmonic, via the quadratic Navier-Stokes term, generates
not only a mean-flow deformation but also a second har-
Qualitatively, mean-field theory describes the feedback
monic which changes, in turn, the mean flow. The nonlin-
mechanism between the fluctuations and the base flow. The
ear interaction of the first and second harmonics produces a
fluctuation gives rise to a Reynolds stress which changes the
third harmonic, etc. Dus̆ek et al. [256] derive a corresponding
base flow. The base-flow deformation generally reduces the
harmonic expansion from the Navier-Stokes equation in the
production of fluctuation energy with increasing fluctuation
neighborhood of a supercritical Hopf bifurcation. Let  > 0
level until an equilibrium is reached. The resulting evolution
be the small amplitude of the first harmonic, then n-th har-
equations are also referred to as weakly nonlinear dynamics,
monics are shown to scale in geometric progression i.e. with
as they describe a mild form of nonlinearity.
n . Hence, higher harmonics may be neglected near the onset
The fluctuation has the same representation as in
of fluctuations. Even for periodic flow far beyond the onset,
Sec. 5.2. However, the base flow is allowed to vary by an-
the second harmonic is observed to be one order of magni-
other mode u3 , called the 0-th, base-deformation or shift
tude smaller than the corresponding harmonic component.
mode. This mode can be derived from the (linearized)
A similar observation holds for turbulent flow with domi-
Reynolds equation and it is assumed to be slaved to the fluc-
nant periodicity.
tuation level. The resulting velocity field ansatz reads
Intriguingly, even turbulent flows with dominant peri-
u(x, t) = us (x) + uu (x, t) + u∆ (x, t), (52a) odic coherent structures may be described by (53). One ex-
u ample is the wake behind a finite cylinder [257]. In this case,
u (x, t) = a1 (t) u1 (x) + a2 (t) u2 (x), (52b) the rationale is the velocity decomposition with an added un-
u∆ (x, t) = a3 (t) u3 (x). (52c) correlated stochastic fluctuation ur : u = us +a3 u3 +uu +ur .
In this case, (53) remains valid if ur scales with the oscillatory
The evolution equation is given by fluctuation level and the base flow deformation is slaved to
    this level as well.
d a1 a1
= A(a3 ) +Bb (53a)
dt a2 a2

5.3.2 Nonlinear control design
a3 = αu a21 + a22 , (53b)
The mean-field model and variants thereof have been suc-
where cessfully used for the model-based stabilization of the cylin-
der wake at Re = 100 [44, 258, 259]. In these studies, an
A(a3 ) = A0 + a3 A3 ,
 u  energy based control design as discussed in Sec. 5.2 has been
σ −ω u used to prescribe a fixed decay rate of the model. Figure 14
A0 = u u ,
ω σ illustrates an unactuated and actuated transient.
 
−β u −γ u It may be noted that (53) can be considered a linear pa-
A3 = ,
γu −β u rameter varying (LPV) model with the shift-mode amplitude
  a3 , or, equivalently, with the fluctuation level r2 as the pa-
0
B = . rameter. Thus, the linear control design of Sec. 4 is immedi-
g
ately applicable for the linear system corresponding to the ac-
Without loss of generality, αu > 0. Otherwise, the sign of the tual value of this parameter. King et al. [260] elaborate other
mode u3 must be changed. A nonlinear amplitude saturation nonlinear control design techniques for the mean-field model
requires the Landau constant to be positive β u > 0. employing input-output-linearization, Lyapunov-based syn-
For a3 ≡ 0, (53) is equivalent to (49). However, (53) has thesis and backstepping. The mean-field model has also been
p
a globally stable limit cycle with radius r∞ = σ u /αu β u in the basis for optimal control of the Navier-Stokes equation
the plane a3 = a∞ ∞
3 = σ /β , and with center [0, 0, a3 ].
u u [261], using the simulation to improve the model and the
In the framework of weakly nonlinear stability the- model as a surrogate plant for control design.
ory, the growth-rate is considered a linear function of the
Reynolds number σu = κ(Re−Rec ), where Rec corresponds 5.4 Moderately nonlinear dynamics
to its critical value. The other parameters are considered to
be constant. This yields the famous Landau equation for Some oscillatory flows may be tamed by direct mitigation
the amplitude dr/dt = σ u r − βr3 , β = αu β u and a corre- with models and methods described in the previous section.
sponding equation for the frequency. In particular, the Lan- Not all plants, particularly turbulent flows, have the actu-
dau equation
√ explains the famous square-root amplitude law ation authority for such a stabilization. However, periodic
r ∝ Re − Rec for supercritical Reynolds numbers assum- forcing at high frequency has mitigated periodic oscillations
ing a soft bifurcation. We refer to the literature for a discus- in a number of experiments. Examples are the wake stabiliza-
sion of the hard subcritical bifurcation with quintic nonlin- tion with a oscillatory cylinder rotation [74], suppression of
earity [254]. Kelvin-Helmholtz vortices in transitional shear layers [262],
Mean-field theory explains the stabilizing feedback reduction of the separation zone in a high-lift configuration
mechanism between the harmonic oscillatory structure and [48], and elongation of the dead-water region behind a back-
the mean-field deformation. A refined weakly nonlinear ex- ward facing step [47]. Also a periodic frequency at 60–70%
pansion also takes higher harmonics into account. The first of the dominant shedding frequency may substantially delay

27
 quency cross talk is key to the control goal. A prominent ex-
Ansatz: ample
processes
ample is high-frequency isto
high-frequency
(clock-works).
forcing forcing
There
mitigate vortex existtoan
shedding. m
were were tion for event-based control.
u(x, t)us (x) + uu (x, t) + ua (x, t) + u (x, t),
= (25a) uu u• Turbulent jet-noise
u uucontrol
u 2 [62].
2
u A(a ) = A + a A A1 1 = u c a21 +Aa1221 = ua ca a23c+ aa241 + a2 (28)
A(a3 ) u = (x,
At)0 +a
3
=3 A3a1 (t) u1 (x) + ✓ 0 3 3
a2 (t) u2 (x), ◆ (25b)
a ✓ ◆ Reference: [60], [36, 61].
u u
u (x, t) u = ! a3u(t)Au0 3 (x)
=
! evolution
+ a4 (t)u u4 (x), Reference: [60], 5.6
(25c)
equation: [36, 61]. Implications for control
A0 = u u ! u
!
u (x, t) = a5 (t) u5 (x) + ✓ a6 (t) u6 (x). ◆  a(25d)   
✓ ◆ a1
5.5 Strongly nonlinear dy
u u 1

A3 =
u u
A3 = u 5.5u Strongly

d  a2 


 = nonlinear
A(a5 , a6 ) 


a2 dynamics

 + B b, (55a)
u u dt a3 a3 
Evolution equation: ✓ ◆ A simple a4 version of strongly nonline
✓ ◆ 0 A simple version a4 of strongly nonlinear


dynamics is described
0 by a5 = α by u
a1 +(??),
2
a2 ,but with
2
1 (??), but with
B = (55b)
B =
g0 a1 1 g 0
a1 a
a6 = α a3 + a4 , 2 2

(55c)
X N
B
d B a2 C C B a C X N
= A(a5 , a6 ) B C + Bb
2
@ a3 where (26a) A = A0 + ai Ai . A = A0 + (29) a
References [58] & [59]. References
dt @ a3 A[58] & [59]. Limit A
i=1
cycle A(a5 , a6 ) = A0 + a5 A5 + a6 A6i=1
a4 a4  u
,

u 2 2 Now, the quadratic σ −ω Now,
nonlinear
u
0 the
gives 0quadratic nonlinear gives r
rise to an energy cascade,
a5 = c a1 + a2 (26b) ωu u
0 
Inertial 5.4 Moderately nonlinear dynamics  σi.e. non-trivial
0  frequency cross talk.
.4 Moderately manifoldnonlinear dynamicsa6 = ca a23 + a24
i.e. non-trivial frequency
A 0 =
(26c)
 0 0 crossσ a talk.
This
−ω a 
,
This classification 0 is
0 basedω u
σon a spectrum ofbased
classification
u is on
periodic
nsatz: Ansatz: 
processes (clock-works). −β uu −γ
processes
There
uu
exist
0
(clock-works).
an0 analogueThere
 exis
classifica-
 γ uu tion uufor event-based  control.
tion for event-based
A5 =   120
control.
−β 0 0 ,
s u a 0 −β au −γ au 
s u(x,
u t) = a u (x) + u (x, t) + u (x, t) + u (x, t),
u(x, t) = u (x) + u (x, t) + u (x, t) + u (x, t), (25a) (25a)
• Turbulent jet-noise 0 •
control
0 Turbulent
γ[62]. −β
au jet-noise
au control [62].
u  
uu (x, t) = a (t) u (x) u + (x,at)(t) u
= (x),
a1 (t) u1 (x) +(25b)
a2 (t) u2 (x), (25b)
−β ua −γ ua 0 0
1 1 2 2
 γ ua ua 
Implications5.6 Implications for contr
−β 0 0 
a
u (x, t) = a3 (t) u3 (x) ua +
(x,at)(t) u= (x), a4 (t) u5.6
a3 (t) u3 (x) +(25c) 4 (x), A6 = (25c)
for
0 control
−β
0 −γ , aa aa
4 4
Figure u +
14: uPhase (x,at)(t)ofu
=weakly
a5 (t) u5 (x) +(25d)
a6 (t) u6 (x). (25d)0 0 γ aa −β aa
u (x, t) = a5 (t) 5 (x) portrait
6 6 (x). nonlinear dynamics.  
The dashed black trajectory corresponds to the unactuated 0
dynamics while the solid red trajectory corresponds to actu-  0 
B =  
Evolution equation:  0 .
Evolutionated
equation:
dynamics. The chosen parameters of (53) are σ u = 0.1,
ω u = 1, αu = 1, β u = 1, γ u = 0 and the forced decay g
0 rate σ1= −0.1. The globally
c
00stable11 limit cycle lies on the0 1
parabolic inertial manifold, a
shown 1 in gray. Ita1should be noted that a vanishing actuation implies
a1 a1
4 = a6 = 0 and yields the mean-field model of (53)
d B C d BBB a2a C C B aa C
3 = a
2CC = B modulo
a 2 C
B 2 C = A(a5 , a6@ B A(a , a )
(26a) @ the
5 6 A index (26a) In the sequel, we assume that
+ Bbnumbering.
dt @ the A
a3vortex dt ) @a3a3AA + Bb a3fixed point is unstable (σ u > 0) and the limit cycle is
formation in wall-bounded shear-layers [49] and
a4a stable
a4 (β uu > 0). Similarly, a vanishing natural fluctuation
a4
D-shaped cylinders [45]. The following
4 two sections outline
a = a2 = a5 yields another mean-field model, again mod-
a corresponding modeling 2 1
2 and 2 control strategy. u 2
a5 = c a1 + a2u a 5 = c a + a
(26b) ulo index numbering.
1 2 (26b)
In the sequel, we assume that the ac-
a 2 2 tuated structures vanish after the end of actuation, implying
a 2 2 a 6 = c a +
(26c) a aa
and aa (26c)
in the model.
5.4.1 a6Generalized
= c amean-field
3 + a4 model 3 4 σ < 0 β > 0
An interesting aspect of (55) is the frequency cross-talk.
Here, a generalized mean-field model for such frequency- The effective growth-rate for the natural oscillation reads
cross talk phenomena is reviewed from [48]. Let uu denote 12

the natural self-amplified oscillation represented by two os-12


A11 = σ u − β uu αu a21 + a22 − β ua αa a23 + a24 . (57)
cillatory modes u1 , u2 . Analogously, the actuated oscillatory
fluctuation ua is described by two modes u3 , u4 . The base- The forcing stabilizes the natural instability if and only if
flow deformation due to the unstable natural frequency ω u β
ua
> 0. Complete stabilization implies A11 ≤ 0. From
and stable actuation frequency ω is described by the shift- (57), such complete stabilization is achieved with a threshold
a

modes u5 and u6 respectively. The resulting velocity decom- fluctuation level at the forcing frequency
position reads
σu
a23 + a24 ≥ .
αa β ua
u(x, t) = us (x) + u∆ (x, t)
+uu (x, t) + ua (x, t), (54a) Thus, increasing the forcing at higher or lower frequency can
u
u (x, t) = a1 (t) u1 (x) + a2 (t) u2 (x), (54b) decrease the natural frequency.
The generalized mean-field model has been fitted to nu-
ua (x, t) = a3 (t) u3 (x) + a4 (t) u4 (x), (54c)
merical URANS simulation data of a high-lift configuration
∆
u (x, t) = a5 (t) u5 (x) + a6 (t) u6 (x). (54d) [48] with high-frequency forcing. This model also accurately
describes the experimental turbulent wake data with a stabi-
Generalized mean-field arguments yield the following lizing low-frequency forcing [263] as shown in Fig. 15.

28
 [58] & [59]. dt @ a3 A
ReferencesAnsatz:
Evolution equation: Evolution equation: quency c
u(x, t) = us (x) + uau4(x, t) + ua (x,
ample
t) + is
u
were
were were
uu (x, t) 0 = a1 (t)1 u1 (x) + a2 (t) u2 (x), u 0
0 1 0 a5 = c a
31
a1 5.4 Moderately
A(a )(x, =
3Inertial
a
5.4 Moderately nonlinear
u
= A0 +✓
A(a3 ) manifold non
A
dynamics
t) = 0 +a
a a(t)
1
a3 Au3A(a
3 3 Au 3 (x)
a1 3 ) u =◆ A + a04 +
(t)au 3A4 (x),
3
A1 15
◆ a2
a
✓t)d uB a C
◆ !5 (x) +✓aa66 (t) = u c B
d B C uA0 = BB !
(x, = (t)
C0C
u
B A3
u u (x).
2 a ! Refe
B a2 C = A0 A(a u a2A
5
u 6
= , a uAnsatz: B ! u =
C = A(au 1 5 ,ua6 )
(26a)◆@ b
!6@
) equation: AA +◆Bb
@a3amanifold
!
dt Ansatz:
u
@ a3 A 5 dt ✓
Evolution
✓Inertial u3 ◆ u ✓ u u
A = 0
u
a
1u4
u
A = 0 5.5
u1 S
a4 A3 = 3
ua
1 au4 3 u u
a1
d B ✓◆a2 C ◆ ✓ u◆Bs 2 A C simple
u(x, s✓ 0 uu(x, a t) = = a0 u (x)
c a a2+C +a 2u
a5 t) =a2
=B c B
u u
= 1 a(x)
2
= +0B+a 2u
dt g@ a23g A
C(x,
u
B=5t) = +
A(au5 , a(x,) Bt)
g @ a3 A b
6 1 +
(26b)
by u +2B
(??), (
uu (x, t) = = caa1a(t) 2 u1a2 (x)
4
u + (x,aa6 2t)(t)
=u= c(x), aa1 (t) 2
a(26c)
a4+ ua1 2(xN
a + a 2 3 4
References
6 [58] & [59].
3References
4 a[58] & [59].
ua (x, = t)c = u
a + aa (t) u (xi
2 2
(a) Natural wake with vortex shedding
a
uReferences
(x, t) [58]
=& [59].
3
Figure 12: Phase portrait
23
a (t) u (x)5 + a (t) u1 (x),
of 3weakly nonlinear
4 4
2 dynam-
3

ua =
(x, t) c a
a
= + aa24 (t) uThese p
5.4(24).
(x,Moderately nonlinear dynamics 5 (x
6 3
ics
u t) =Figure 12:
a5 (t) Phase
u 5 (x)portrait
+a 6 (t)ofuweakly
5 Now,
6 (x). nonlineatemthe
(L
5.4 Moderately ics 5.4 Moderately
nonlinear
(24). dynamics nonlinear dynami 12 t
i.e. quenc
non-t
Ansatz: This
ample
Ansatz:
Ansatz:
wereEvolution
u(x, t) = equation:
Evolution equation:
us (x) + uu (x, t) + ua (x, t) + u (x, t),
processes
(25a)
tion for e
were
u(x,
u
u (x,
t) t)= = s
u (x) a1+ (t)uu(x,
u(x,
u
1 (x) t) +
t) += a2 (t)
u a uu
(x,
s
(x)
t)2+ +
(x),
u0
u
u(x, (x, (25a)
t), 1
a
t) + u(25b)(x,
• t)Turb5
+A
A(a 3) = A 0 + a3 A 3
=0= 1 0 31
u
ua (x,
uu (x, t) t) a1 (t)au 3 (t)
uu(x,3✓ at)
(x) A(a = ) au1u (t)
= uA1 (x)
◆4 (x), 0 + a+1a a32A
a(25b)(t) u2(25c)
(x),
1 (x) + u+ u
2 (t) a432 (t)
(x),
a ! ✓ ◆ R
a A10 u3= a
u (x, at) = ua3u (t) uB 3 (x) u a14C
+ (25c) (t)! u45.6
u (x), Im
(b) Actuated partially stabilized wake u u (x, (x,
t) t)= = a3 (t) a (x)
(t) u + (x) 4 (t)
u
+ A0 = ◆ BBa
u
a (x),
4 (t) d (x). (25d)
d B a2 C(x)
u (x, t) =Ba5 (t) uC
5
u (x,
5 !
✓ at)(t)u=
6 6
a5u(t) u5 (x) B ! +a
u 2 CC
a26 (t)C
(25d) 6=
uu (x).
A
Figure 15: Flow visualization of the experimental wakeEvolution @ A
equation:
5 += 6 A(a
u 6 (x).5 ,dta 6 @
)
✓ @ a 3 A A + ◆ Bb
5.5
hind a D-shaped body without (a) and with symmetric low-
be-
dt aA33 = u u au 3 u
0 ◆A = 1 au4
frequency actuation (b), reproduced with permission Evolution
from a14 1 Evolution
equation:
a
✓
0
3 0
equation:
a 1 a4 u A sim
Mark Pastoor. The D-shaped body is indicated in gray, the ✓ ◆ by (??
red squares mark the location of the pressure sensors and the 0d B B1 a2BC =
Ca = =A(a g 00u 1 B
B 21 a2 C 2
C 0 a 0
5 (26a)= 1c
blue arrows indicated the employed ZNMF actuators. @
dt a1 a3 A 5 c
5 ,aBa16 )a@ =
a1 1 a3 2g+ a A + Bb a 1

d B C d B B B a2a C Ca B a2 C
a 2C2 B C+
References
B a
[58] &=a[59]. a = 2A(a (26a)6 @ a=
, a
a ) Ac
2 C 4 B C 4 5 6
@ A A(a= a6@
5 ,dt )c@ a AA+
a + Bba
Table 1: Parameters of the generalized mean-field model il-dt a 3 References
6 [58] 3
&
a 3 [59].
3 4 3

lustrated in Fig. 16. a4 a5 = cu a21a+ 4a a2

4
2
(26b)
a4
Now,
a = u 2a
c a 22
+ a524 = cu a(26b)
a
2 2
1 + a2 (26c)
Unstable oscillator 5.4 Moderately
Stable oscillator 5.4
a 5 =6
nonlinear
Moderately
c a 1 + a 2 dynamics
3
nonlinear a 2 dynamic
2
i.e. no
σ u = 0.1 ω u = 1 σ a = −0.1 ω a = 10 a6 = c a3 + a4 a 2 2 a 6 = c a +
(26c)
3 a 4 T
β uu = 1 γ uu = 0 β au = 0 γ auAnsatz:
=0 Ansatz: proce
β ua = 1 γ ua = 0 β aa = 0 γ aa = 0 12f
tion
u a
α =1 α =1 g=1
u(x, t) = us (x) + uu (x, t) + u12
a
(x, t) + u
u(x, t) = us (x) + uu (x, t) + ua (x, t) + u (x, t),
(25a) • T
u
u
u (x, t) = a1u(t)(x, t) +=a2 (t)
u1 (x) a1 u
(t) u1 (x) + a2 (t)(25b)
2 (x),
u2 (x),
a
5.4.2 Nonlinear control design
a
u (x, t) = a3u(t)(x, t) +=a4 (t)
u3 (x) a3 u
(t)
4 (x),
u4 (x),5.6
u3 (x) + a4 (t)(25c)
The model above may guide in-time control [264] and (x, t) = au
u adap- 5 (t)(x, t) +=a6 (t)
u5 (x) a5 u
(t) u5 (x) + a6 (t)(25d)
6 (x). u6 (x).
tive control design providing the minimum effective actua- Figure 16: Phase portrait of moderately nonlinear dynamics
tion energy [265]. Evolution
(55). The dashed equation:
black trajectory corresponds to the unac-
Figures 16 and 17 show an unactuated and stabilizing Evolutiontuatedequation:
dynamics while the solid red trajectory corresponds
transient with the parameters of Tab. 1. For simplicity, all to actuated dynamics with 0 (51). 1 The chosen parameters are a
0 1
nonlinear terms of the forced oscillator are assumed to van- enumerated a 1 1
0 1 in Tab. 1. B 0 1
ish, since the linear term is already stabilizing. For the same a1 d B a2 C C a1 B a2 C
B C
reason, all nonlinear frequency terms are set to zero as the @ a3 AB a2 C = A(a 5 , a 6 ) @ a3 A +B
d B a C
B It should
2 C be dt B Cnonlinear
frequency cross-talk is not affected by small frequency vari-
@ a3 Anonlinear= noted
A(athat linear,
5, a 6 ) @ weakly + Bb and (26a)
ations. In principle, (55) can be generalized for an arbitrarydt moderately a4 show
systems a3 A actuation re- a4
different
number of frequencies. sponsea4which may be tested in experiments.
a4 For the linear
u 2
a52 = cu a21 + a22
a5 = c a1 + a2 (26b)
29
a 2
a62 = ca a23 + a24
a6 = c a3 + a4 (26c)
 ] & [59].
3 a3dt
4 au 3 u frequency cross talk.
i.e. non-trivial
a4 A3 = a
This
a4 classification
u4 u is based on aa4spectrum of periodic
✓ ◆ (clock-works).
processes u There exist A = A0 +
an analogue classifica-
ely nonlinear dynamics u 2
a5 = c B a1=12
+tion20 a 5 = c
a2g for event-based control.a 2
12
+
(26b)
1 a 2
2 (26b)
a 2 2
a a 2
= c a +a a
2• Turbulent
6 = c
jet-noise a +
control a
[62].
(26c) 4
3 (26c) g
Now, the quadratic nonlinear
References
6 [58] & [59].
3 4 i.e. non-trivial frequency cross
us (x) + uu (x, t) +
1.6
ua (x, t) + u (x, t),
(25a)
This classification is base
(25b) 5.6 Implications for control
1.4
1 (t) u1 (x) + a2 (t) u2 (x), processesStrongly
(clock-works). There
5.4 Moderately nonlinear dynamics
control.12
1.2

3 (t) u3 (x) + a4 (t)1 u4 (x), (25c) 12 tion for event-based

Weakly
Moderately nonlinear
nonlinear
Ansatz:
5 (t) u5 (x) + a6 (t)
0.8 u6 (x).
Control
Control
(25d) Linear nonlinear • Turbulent jet-noise contro
on
off A
ation:
0.6
u(x, t) = us (x) + uu (x, t) + ua (x, t) + u (x, t),
B(25a)
u
(25b) C5.6 Implications for c
0.4
1 0u (x,1t) = a1 (t) u1 (x) + a2 (t) u2 (x),
0.2 a 1
C ua (x, t) = a3 (t) u3 (x) + a4 (t) u4 (x),
0 B a2 C
(25c) D
C A(a5 , a6 ) 0B 0.2 C0.4
= t)+ Bb = a5(26a)
0.6 0.8 1 1.2 1.4
A @u a(x,
3 A (t) u5 (x) + a6 (t) u6 (x). (25d)
a4 Figure 18: Venn diagram for the classification of nonlineari-
Figure 17: Phase Evolution
portrait ofequation:
the shift-mode amplitudes a5 ties. Prototypic examples are for (A) the subcritical flow over
a5 = cu and
a21 a+ a22 the slow dynamics
6 , i.e.
0
in (55).
1(26b)
Same transient solu- backward-facing step with noise excitation [250], for (B) the
0 1
tions as in Fig. 16. supercritical onset of vortex shedding [255], for (C) the sup-
a 2 2 a1 a1
a6 = c a3 + a4 B
d B a2 C C (26c) B pression
C of Kelvin-Helmoltz vortices by high-frequency forc-
a2[47]Cand for (D) the decay of 2D turbulence [266].
system, a global superposition
@ principle
A = theA(a
for actuation )B
5 , a6 re-
@
ing + Bb (26a)
dt a32 a3 A
sponse holds: Let b (t) and b (t) be two open-loop actua-
1

tion commands and a1 (t) and a4 a2 (t) be the a4

12corresponding turbulence by optimal initial conditions is a beautiful con-
solutions. Let the linear combination b = λb1u+ µb 2 with2
2
figuration illustrating the complexity associated with strong
a =
real coefficients λ and µ be a new actuation
5 c a
command. 1 + a2
Then nonlinearity [266].
(26b)
a = λa + µa is a corresponding new solution.a The2weakly2
1 2
Mathematically, strong nonlinearity may be cast in a
a6 superposition
nonlinear system has a similar local = c aprinciple
3 + a4 (26c)
form similar to the linear dynamics in (5), but with
for an infinitesimal perturbation of the stable unforced limit
cycle. In contrast, the frequency cross-talk mechanism of the Na
X
moderately non-linear system is not resolved by a local lin- A = A0 + ai Ai . (58)
earization around the stable unforced limit cycle. The lin- 12
i=1

earization of (57) removes the frequency cross-talk.
5.5 Strongly nonlinear dynamics
Sections 5.5.1 and 5.5.2 outline examples of strongly nonlin-
ear dynamics not fitting in the previous categories and exam-
ples of corresponding control design, respectively.
5.5.1 Examples of strongly nonlinear dynamics
In the case of moderately nonlinear dynamics, different fre-
quencies interact over the slowly varying base flow. The
corresponding solutions are well described by the local lin-
earization around the short-term averaged flow. In other
words, the solution lives on a manifold in state space and
evolves according to a locally linear dynamical system. This
may be a useful approximation even for turbulent flows with
one or few dominant frequencies. The turbulence cascade is,
however, dominated by triadic interactions involving non-
vanishing frequencies. The corresponding energy flow from
large scales (low frequencies) to small scales (high frequen-
cies) is not well characterized by moderately nonlinear dy-
namics. No linearization is able to describe such a cascade.
A similar behavior applies to the inverse cascade from dom-
inant to larger scales, e.g. via vortex pairing. We shall call
such dynamics ’strongly nonlinear’ implying that they are
irreducible to even locally linear systems. The decay of 2D
The variation of the matrix A can neither be ignored nor con-
sidered slowly varying — in contrast to linear, weakly non-
linear or moderately nonlinear behavior.
We arrive at the following classification of system dy-
namics (see Fig. 18). The dynamics are either linearizable
near the fixed point, or they are locally linearizable with one
or few frequencies, or they belong to the reducible cases. The
boundaries between two neighboring cases are, of course, a
bit blurred depending on the considered error tolerance of
the model.
5.5.2 Nonlinear control design
The above classification is based on a spectrum of periodic
processes (clock-works). There exists an analog classification
for event-based control. Model-based control strategies for
strongly nonlinear dynamics are scarce. One programmatic
general approach is suboptimal control in which the optimal
control actuation is chosen just for the next time step [43]. In
[267] a corresponding Lyapunov-based control strategy has
been found effective to suppress jet noise events. For some
well understood configurations, the physical mechanism can
be exploited. For instance, skin-friction of wall turbulence is
known to increase with sweeps and ejections. Hence, a wall-
normal blowing or suction counteracting the wall-normal ve-
locity at 10 plus units has been found to be a very effective
closed-loop opposition control [43].

30

Steady fixed point
PSD
! !
Limit cycle dynamics
PSD
! !
Multi-frequency dynamics
PSD
! !
Broadband frequency dynamics
Power spectral
density (PSD)
Frequency, [ ! ] ! modes at the actuated frequency ω a under periodic forc-
Figure 19: Input/output characteristics of different dynam-
ics. Left: actuation command; right: sensor signal with-
out forcing (black), and sensor signal under periodic forcing
(red). From top to bottom: a stable fixed point with periodic
excitation (linear dynamics); a stable limit cycle with locking
periodic forcing (weakly nonlinear dynamics); a stable limit
cycle with high-frequency forcing (moderately nonlinear dy-
namics); and broad-bad turbulence under periodic forcing
(strongly nonlinear dynamics).
Figure 19 displays examples of the single input (one ac-
tuator) and single output (one sensor) system for the four
kinds of system dynamics. The first row shows the excitation
of a frequency for a stable linear system. The second exam-
ple is the energization of a stable limit cycle. The third row
shows the suppression of a low frequency by high-frequency
forcing. And the last example indicates how a single excita-
tion frequency can change the whole frequency spectrum.
There is a strong increase of complexity as one moves
from taming few frequency peaks to broadband dynamics.
For the latter, even developing reduced-order models suit-
able for control design constitutes an unattainable goal with
modern techniques.
31
5.6 Enablers and challenges of nonlinear
model-based control
In this section, enablers and challenges of nonlinear model-
based control are discussed. The beauty of the linear (49),
weakly nonlinear (53) and moderately nonlinear models (55)
is that each mode and each dynamical system coefficient has
a clear physical meaning. Moreover, the nonlinear systems
have an inherent tunable robustness. Evidently, more sophis-
ticated models with more frequencies can be constructed in
a similar spirit. Alternatively, a POD model with, say, 100 di-
mensions may be more accurate but the modes and the sys-
tem coefficients have no physical meaning. Moreover, each
mode and each coefficient acts as a noise amplifier for esti-
mation and control design tasks. Thus, more accurate mod-
els for a given operating condition tend to be less effective
for control design [44].
In the sequel, a validated recipe for control-oriented
least-order modeling is provided giving preference to the
most simple methods. Focus is placed on the generalized
mean-field model which incorporates the other models as
special cases. In addition, an experimental plant is assumed,
which gives access to less data and thus makes model iden-
tification more difficult.
• Modes — The domain of the modes should be large
enough to provide instantaneous phase information
and small enough to ignore pure convection effects. The
size of the recirculation region or one or two wave-
lengths are good indicators. The first two modes u1,2
may be a pair of Fourier cosine and sine modes at the
natural frequency ω u of the unactuated flow. Similarly,
the second pair u3,4 may be Fourier cosine and sine
ing. The dominant POD or DMD modes of unactu-
ated and forced flow are good candidates [48]. Filter-
ing techniques may be equally suited [268]. The shift
modes u5,6 are more difficult as the unstable steady so-
lution us is generally not accessible from experiments.
However, base-flow changes from modulations and un-
forced transients may provide u5 , for instance via a
POD of low-pass filtered flows [257]. Now, us may be
inferred from the fixed point of a calibrated Galerkin
model with the averaged flow u0 and the expansion
modes u1,2,5 . The second shift mode u6 points from
us to the average actuated flow hua i. An orthonormal-
ization completes the construction of the basis.
• Coefficients of the dynamical system — The 4D-Var method
is a powerful technique that is generally applicable for
parameter identification [269]. One significant advan-
tage is that no time-derivative information from flow
snapshots is needed. One actuation off-on-off transient
(from unforced to forced to unforced state) can provide
information about unforced and periodically forced so-
lutions as well as growth-rates and frequencies. Gener-
ally, the parameter identification problem tends to be ill-
posed, i.e. significantly different dynamical system co-
efficients can yield similar Galerkin solutions. Hence,
a critical enabler is a regularization, i.e. a weak pe-
nalization of the difference from a reference dynamical
 system. This reference system may be obtained for a
Navier-Stokes-based Galerkin projection (ignoring the
third flow dimension for 2D PIV data) or from a simple
propagator calibration technique.
• Sensor-based estimation — Good sensor placement can
easily be inferred from the actuation mechanism. A
destabilizing sensor-based control, for instance, re-
quires knowledge of the phase of the forced structures
and the amplitude of the natural shedding for gain
scheduling. Developing more general techniques for
sensor optimization is a recent area of active research.
No generally applicable robust optimization strategy
has emerged yet (see Sec. 8).
• Control design — A control law for the model can eas-
ily be designed following conventional wisdom of con-
trol theory. However, any control design should respect
the narrow limitations of the model. For instance, the
generalized mean-field model can be stabilized in a tiny
fraction of the shedding period. The shed vortices in the
experiment, however, need a minimum time to leave
the observation domain. In particular, the model is only
applicable for slow transients with the design frequency
content. For most shear flows, the resolvable frequen-
cies are hard-coded by the wavelength of the vorticity
in the expansion modes and the convection of the mean
flow.
These recipes for model-based control design follow increas-
ingly powerful data-driven methods. The danger of over-
fitting needs to be mitigated by a cross-validation with data
not used for the parameter identification. The modeling
may also be based on first principles, i.e. a Navier-Stokes-
based Galerkin projection, subgrid turbulence closures, actu-
ation modes, deformable expansion modes and other auxil-
iary methods. This path provides additional physical insight
at the price of larger effort and the necessity of a more exten-
sive experience [270].
Reduced and least-order Galerkin models can provide a
crisp analytical description of the actuation mechanism and
thus guide the design of control laws. As such, they can
serve as light houses for terra incognita. However, the con-
struction of robust control-oriented models is more often an
art rather than a fool-proof methodology. By construction,
Galerkin models are elliptic, i.e. a local change of the flow
is immediately communicated via the expansion modes in
the whole domain. Moreover, Galerkin models can build
up unbounded fluctuation energy. Shear flows, however,
are hyperbolic, i.e. dominated by convection. Excited struc-
tures convect out the observation domain. Thus, the fluc-
tuation level is naturally stabilized. Galerkin models lack
this convective stabilization mechanism. This can be con-
sidered the root-cause of the narrow dynamic bandwidth of
Galerkin models. Another related challenge is the change
of flow structures under natural or actuated transients. In
simple cases, such as structures with dominant frequencies,
these changes may be tracked in deformable base-flow de-
pendent modes [271, 272]. In broadband turbulence, as in
turbulent jets, there is no rationale for such cure. Physically,
an evolving vortex configuration convects downstream and
32
it constitutes a significant challenge to robustly embed a rel-
evant ensemble of such vortex dynamics in a modal frame-
work.
Some challenges of Galerkin models can be avoided by
choosing a suitable sensor-based state space (see, e.g., Sec.
3). The structure of the linear, weakly nonlinear and mod-
erately nonlinear models has been derived purely from the
linear-quadratic structure of the Navier-Stokes equation and
from frequency filtering arguments. Hence, similar models
may be constructed from the sensor history. These sensor-
based models keep all relevant information which are acces-
sible in the control experiment and bypass Galerkin mod-
eling problems with the observation domain, the unknown
steady solution, base-flow dependent expansion modes, etc.
ERA/OKID is a powerful realization of this path for linear
dynamics (see Sec. 4.5.4).
6 Model-free control
Often, developing a detailed dynamical system model for a
given set of high-dimensional nonlinear phenomena may not
be the best use of time and resources. After an attractor has
been identified and dynamics painstakingly determined, ap-
plying control strategies will usually shift the attractor signif-
icantly, rendering models inaccurate. The obvious exception
is linear stabilization of a fixed point, whereby effective con-
trol makes the model more accurate. Alternatively, one may
apply adaptive or model-free approaches to control a com-
plex high-dimensional system.
There are a wide range of model-free control options,
and we explore a number of methods that have been widely
applied in turbulence control. First, open-loop forcing is per-
haps the most pervasive model-free control strategy, based
on its simplicity. Next, adaptive control may be used as a
slow parameter tuning feedback wrapped around a working
open-loop control to modify controller behavior in response
to changing environmental conditions. Extremum-seeking
control is among the most widely used adaptive controllers
for flow control. Third, in-time control may be achieved by
first specifying a given parameterized control structure, such
as PID control, and then employing tuning methodologies
to improve performance. Finally, machine-learning control
constitutes a growing collection of data-driven techniques for
structure identification and parameter identification of con-
trollers.
Machine learning is a rapidly developing field of com-
puter science whereby a complex system may be learned
from observational data, rather than first-principles mod-
eling [273, 274, 275, 276]. There is a tremendous poten-
tial to incorporate data-driven modeling techniques, espe-
cially for the control of high-dimensional complex systems,
such as turbulence. Machine learning control, the use of
machine-learning techniques to determine effective output–
input maps (i.e. the controller), is a relatively new innova-
tion [277]. The specific machine learning methods discussed
here include adaptive neural networks, genetic algorithms,
and genetic programming control.
Figure 20 provides an organization of these model-free
 Physics-based control
Structure assumed
Structure of
control law
Open-loop In-time
forcing control
+ programming
identification
Parameter
Adaptive Gradient
control search
Online
Find local extrema
Figure 20: Overview of model-free control methods discussed in Sec. 6. Model-free control involves the choice of control
law structure as well as the optimization of controller parameters.
control methods. In each method, the structure of the con-
trol law must be determined and the parameters identified
to optimize the control law. Many of the methods (open-
loop forcing, in-time control, and neural networks) specify
the structure of the control law, while other methods (adap-
tive control, gradient search, and genetic algorithms) iden-
tify optimal parameters once the structure has been identi-
fied. Thus, once a given structure has been assumed, there
is a choice of parameter identification algorithm. Among the
methods discussed here, genetic programming control is the
only method where both the structure and the parameters
of the control law are identified. Other important considera-
tions include whether or not the method adaptation or learn-
ing occurs predominantly online or offline, and whether the
parameter optimization finds local extrema or has the possi-
bility of exploring the global parameter space.
It is important to note that model-free control method-
ologies may be applied to numerical or experimental systems
with little modification. All of these model-free methods
have some sort of macroscopic objective function, typically
based on sensor measurements (past and present). The ob-
jective may be drag reduction, mixing enhancement, or noise
reduction, among others.
6.1 Open-loop forcing
Periodic forcing is widely used in turbulence control, largely
because its ease of implementation and the lack of the need
for a model of the flow [25]. Periodic forcing may be used to
modify the dominant frequency of a flow, resulting in a lock-
on with the forcing frequency. Alternatively, the forcing may
modify the broadband frequency content, exploiting nonlin-
ear frequency cross-talk.
In either case, open-loop forcing does not take advan-
tage of sensor measurements, including reference or distur-
33
Machine learning control
Structure identified
Neural
networks
Genetic
control
Genetic
algorithms
Predominant offline use
Possible global exploration
bance measurements for feedforward control, and down-
stream sensors for feedback control. This limits the ability of
open-loop forcing to reject disturbances, adapt to slow varia-
tions in flow parameters, or compensate for un-modeled dy-
namics. Most importantly, open-loop strategies are unable to
stabilize unstable flows, regardless of the forcing strategy.
6.2 Adaptive control
6.2.1 Extremum-seeking control methodology
There are numerous adaptive control techniques, although
extremum-seeking control (ESC) [278, 279] has gained the
most traction in fluid dynamics. ESC is attractive for com-
plex systems because it does not rely on an underlying model
and it has guaranteed convergence and stability under cer-
tain well-defined conditions [278, 279]. Extremum-seeking
control may be applied to track local maxima of an objec-
tive function, such as mixing enhancement or drag reduc-
tion, despite disturbances and varying system parameters.
Although adaptive control may generally be implemented
in-time, it is overwhelmingly used in flow control as a slower
feedback tuning the parameters of a working open-loop con-
troller. There are a number of reasons for this, including
sensor and actuator bandwidth, the fact that turbulent fluc-
tuations may be viewed as a fast disturbance, and the fact
that many objective functions, such as mixing, require inte-
gration over a slow time-scale. However, this slow feedback
has many benefits, such as maintaining performance despite
slow changes to environmental conditions.
ESC is an advanced method of perturb-and-observe,
whereby a sinusoidal input perturbation is used to estimate
the gradient of an objective function J to be maximized (or
minimized). The objective function is typically based on sen-
sor measurements, s, although it ultimately depends on the
 J
b Plant s Objective J
dynamics function

b̂ R ξ JHP High
+ k pass
+ filter

b̂

Controller
M sin(ωt)
Figure 21: Schematic illustrating the components of an
extremum-seeking controller. A sinusoidal perturbation is
added to the best guess of the input b, passing through the
plant, and resulting in a sinusoidal output perturbation. The
high-pass filter removes the DC gain and results in a zero-
mean output perturbation, which is then multiplied (demod-
ulated) by the same input perturbation. This demodulated
signal is finally integrated into the best guess b̂ for the opti-
mizing input b.
b̂ < b∗ b∗ b̂ > b∗
Figure 22: Schematic illustrating extremum-seeking control
on for a static objective function J(b). The output perturba-
tion (orange) is in phase when the input is left of the peak
value (i.e. b < b∗ ) and out of phase when the input is to the
right of the peak (i.e. b > b∗ ). Thus, integrating the product
of input and output sinusoids moves b̂ towards b∗ .
left of the optimal value b∗ and which is mostly negative if b
is to the right of the optimal value b∗ :
ξ = M sin(ωt)JHP . (61)

Finally, the demodulated signal ξ is integrated into b̂, the best

choice of input signal b, via the plant dynamics relating b to
s. Often, for extremum-seeking control, the variable b may
be a parameter that describes the actuation signal, such as
the frequency of periodic forcing.
A schematic of the extremum seeking control architec-
ture is shown in Fig. 21 for a single scalar input b, although
the methods readily generalize to vector-valued inputs b. A
schematic objective function J(b), for static plant dynamics,
is shown in Fig. 22.
In extremum-seeking control, a sinusoidal perturbation
is added to b̂, the best approximation of the input that maxi-
mizes the objective function:
b = b̂ + M sin(ωt). (59)
estimate of the optimizing value:
d
b̂ = k ξ, (62)
dt
so that the system estimate b̂ is steered towards the optimal
input b∗ . Here, k is the integral gain, which determines how
aggressively the actuation responds to a non-optimal input.
The demodulated signal ξ measures the gradient of the
objective function, so that the algorithm converges more
rapidly when the gradient is larger. To see this, first assume
constant plant dynamics, so that J(s) is simply a function of
the input J(b) = J(b̂+M sin(ωt)). Expanding J(b) in the per-
turbation amplitude M , which is assumed to be small, yields:
J(b) = J(b̂ + M sin(ωt))

This input perturbation passes through the system dynamics, ∂J 
resulting in an output signal J that varies sinusoidally about = J(b̂) + · M sin(ωt) + O(M 2 ).
∂b b=b̂
some mean value. To remove the mean, the output is high-
pass filtered, resulting in JHP . A simple high-pass filter is The leading-order term in the high-pass filtered signal is
represented in the frequency domain as JHP ≈ ∂J/∂b|b=b̂ · M sin(ωt). Averaging ξ over one period
yields:
ζ Z 2π/ω
HP(ζ) = , (60) ω
ζ + ωHP ξavg = M sin(ωt)JHP dt
2π 0
where ζ is the Laplace variable, and ωHP is the filter fre- Z 2π/2 
quency, chosen to be slow compared with the perturbation ω ∂J 
= M 2 sin2 (ωt) dt
frequency ω. It is possible to multiply the high-pass filtered 2π 0 ∂b b=b̂

output signal by the input sinusoid, resulting in a demodu- M 2 ∂J 
lated signal ξ, which is mostly positive if the input b is to the = .
2 ∂b  b=b̂

34
 Fig. 9. Experimental results from an ESC with combined EKF. Topmost frame:
pressure measured in the combustion chamber; second frame: map output and
controller index – (1) for on and (0) for off; third frame: loudspeaker gain; bottom:
delay time.

Thus, for the case of trivial plant dynamics, the average sig- 160
nal ξavg is proportional to the gradient of the objective func-
tion J with respect to the input b.
control off
In general, J may be time-varying and the plant relating
b to s may have nonlinear dynamics that operate on a faster 140
timescale than the perturbation ω, complicating the simplis-
tic averaging analysis above. This general case of extremum-
seeking control applied to a nonlinear dynamical system was

p in dB
analyzed by Krstić and Wang in [278], where they devel- 120
oped powerful stability guarantees based on a separation of
timescales and a singular perturbation analysis. It is also pos-
sible to modify the basic algorithm outlined above by adding
a phase φ to the sinusoidal input perturbation in Eq. (61). 100
In [278], there was an additional low pass filter ωLP /(ζ + ωLP )
placed before the integrator to extract the DC component of control on
the demodulated signal ξ. Finally, there is an extension to
extremum-seeking called slope-seeking, where instead of a 80
zero slope, a specific slope is sought [279]. Slope-seeking is 0 100 200 300 400 500
preferred when there is not an extremum, as in the case when f in Hz
control inputs saturate. Often extremum-seeking is used for
frequency selection and slope-seeking is used for amplitude Figureof23:
Fig. 10. Spectra Acoustic
acoustic pressure
pressure reduction
for the in combustor
uncontrolled and theexper-
controlled case.
selection. iment with modified ESC algorithm. The main peak is re-
duced by about a factor of 60 when control is applied. Re-
converges towards
produced froma[287].
second minimum at Kls = − 1.2 and ! = 8 ms.
6.2.2 Examples of adaptive control in turbulence
Not surprisingly, the difference between the two optimal time
Extremum-seeking has been widely applied in turbulencedelays corresponds to half the period of the fundamental frequency
separation and increase the pressure ratio in a high-pressure
control, largely because of its ease of use and equation-
of the instability (1/2 · 94 Hz = 5.3 ms). At t = 170 s, the ESC also starts
free implementation. ESC was used in [280, 281] to reduce axial fan using an injected pulsed air stream. The use of a
the drag over a bluff-body in an experiment at moderate! 0 =
from 6 ms, filter
Kalman and at t = 225the
improved s, it starts once
controller speed again fromof! 0 = 5 ms,
by a factor
Reynolds number (Re = 20, 000). The objective function 10 over traditional ESC. The implication
thus, converging towards the first minimum. is that the controller
weighted drag reduction against energy expended by theFig. may compensate for disturbances and changes in environ-
ac- 10 shows the spectrum of the acoustic pressure for the
of the mentaland
tuation, a rotating cylinder on the upper trailing edgeuncontrolled conditions that are 10 times faster than before.
the controlled
backward facing step, to obtain efficient drag reduction. The external perturbationcase.
used Inin the
ESC uncontrolled
may be infeasi- case dis-
tinct peaks at 94 Hz and higher harmonics can be seen. The peak
In [282], ESC was used for separation control on a high- ble for some applications, such as optimizing aircraft control
amplitude
lift configuration. The experiment consisted of spanwise surfaces
at 94forHzefficient
is 157 flight.
dB. In In [289],
the atmospheric
controlled turbulent
case, this amplitude
pressure sensors and pulsed jets on the leading edgeisof the fluctuations
reduced by 35 dB.were Dueused as the
to the perturbation
complete for optimization
stabilization of the system
single-slotted flap for actuators. This work demonstrated en- of aircraft control based on ESC. The strategy of using natu-
the higherral harmonics also disappear.
system perturbations for ESC is promising. ESC has also
hanced lift over open-loop forcing, both at large angles of
attack where the flow was separated, and at moderate an- been used for parameter tuning in PID controllers [290] as
gles of attack where flow remained attached. They 6. the well as toand
use Conclusion tuneoutlook
PI controllers to stabilize a model of nonlin-
slope-seeking extension of ESC. This work also developed ear acoustic oscillation in a combustion chamber [291].
both SISO and MIMO controllers. The convergence speed of ESC can be improved by a more accu-
ESC has also been used to control thermoacoustic modes 6.3 In-time control
rate estimation of the local gradients. In this contribution no plant
across a range of frequencies in an industrial scale, 4 MW gas
turbine combustor [283, 284]. It has also been utilized dynamics were
for 6.3.1 considered
Control – the convergence
law parameterizations wastuning
and sped up by a
separation control in a planar diffusor that is fully turbulent methodologies
and stalled [285], and to control jet noise [286].
Often, the structure of a control law may be decided by an
The ESC relies on a separation of timescales where the
expert in the loop with engineering intuition based on pre-
system dynamics are the fastest, the periodic perturbation is
vious experience. The control structure is often chosen for a
moderate, and the high-pass filter cut-off is slow. Changes
combination of flexibility and simplicity, as in the ubiquitous
to the plant dynamics, either varying parameters or distur-
proportional-integral-derivative (PID) control:
bances, are assumed to the the slowest timescale. It is pos-
sible to improve the performance of ESC in some instances Z t
d
by adding additional filters and phase delays. For exam- b(t) = k P s(t) + k I s(τ )dτ + kD s(t).
t0 dt
ple, extended Kalman filters were used as the filters in [287]
to control thermoacoustic instabilities in a combustor ex- The PID control above is parameterized by three constants, kP ,
periment. The dramatic performance improvement of the kI , and kD , the proportional, integral, and derivative gains.
modified ESC algorithm from [287] is reproduced in Fig. 23. When the controller is parameterized, as is PID control, it is
Kalman filters were also used in [288] to reduce the flow possible to tune the controller to improve performance and

35
meet specifications (rise-time, bandwidth, overshoot, etc.) by
identifying locally or globally optimal parameters. Without
an underlying system model, there is tremendous freedom
in choosing these parameters. This parameter tuning may be
based on intuition, trial-and-error, or on a physical mecha-
nism, as in the case of opposition control in the next section.
6.3.2 Case study: opposition control in wall turbu-
lence
In 1994, Choi, Moin, and Kim introduced a method of ac-
tive feedback control to reduce the drag in a fully developed
turbulent boundary layer flow [43]. In DNS, a 20-30% reduc-
tion in drag was achieved by imposing a surface boundary
condition (blowing/suction) to oppose the effect of vortices
in the near wall region. This opposition control was an early
model-free control approach based on intuition about flow
physics and drag mechanisms, rather than a mathematical
model. [27, 208, 40, 292, 10] provide reviews of opposition
control. One of the earliest experimental demonstrations of
active feedback suppression was Liepmann and Nosenchuck
in 1982 [18], where they canceled T-S waves using a down-
stream heating element and a phase shifted measurement
feedback. This may be seen as a predecessor of the popu-
lar opposition control, and the experiments were quite suc-
cessful, resulting in a significant increase in the transitional
Reynolds number for a flat plate experiment.
There are many extensions to opposition control. The
best performance of skin-friction reduction [43] was achieved
when sensors were located at y + = 10, which is not nearly
as practical as measurements at the wall. This led to the
development of a neural network architecture to optimize
the mapping from surface measurements to opposition con-
trol [22], as shown in Fig. 24. A drag reduction of about 20%
was achieved in a low Reynolds number turbulent channel
flow. Some studies suggest that opposition control does not
scale favorably with increasing Reynolds number [293], al-
though there are studies suggesting significant potential drag
reduction, even at high Reynolds numbers, assuming perfect
damping of near-wall fluctuations [294]. The method of op-
position control is also strongly dependent on the amplitude
and phase of the actuation response [295].
Opposition control has also been generalized to utilize
piston actuation [296], and has been used to reduce drag in a
DNS of turbulent channel flow using wall deformation [297].
Drag reduction has also been achieved in DNS of pipe flow
using opposition control [298, 299], and using sub-optimal
control [300]. Opposition control has also been investigated
in the context of stochastically forced non-normal dynamical
systems [301].
Recently opposition control has been explained in the
context of resolvent analysis, whereby the Fourier trans-
formed Navier-Stokes equations are viewed as an input–
output system [302]. In this approach, the convective terms
are viewed as the input and the turbulent flow field is the
output, and a singular value decomposition (SVD) of the
resolvent operator indicates the forcing that results in the
largest gain in the response.
36
y
z
y
z
Figure 24: Illustration of the benefits of opposition control
(bottom) in contrast to unforced system (top). Contours of
streamwise vorticity are plotted in a cross-flow plane. Neg-
ative contours are indicated with dashed lines. Reproduced
with permission from Lee, Kim, Babcock and Goodman [22].
Copyright 1997, AIP Publishing LLC.
6.4 Neural network based control
Many model-based open-loop controllers in fluid dynamics
are based on the inversion of a model, as discussed in Sec. 4.2.
When an input-output function is not well-approximated in a
simple linear or quadratic framework, it is often necessary to
employ more sophisticated methods, such as artificial neural
networks (ANNs). ANNs are a construct in machine learning
that attempts to mimic the computational flexibility observed
in the brains of animals. In particular, a number of individual
computational components, or neurons, may be connected in
a graph structure with inputs and outputs. By exposing this
network to example stimulus, it is possible to train the net-
work to perform complex tasks, though either supervised or
unsupervised reinforcement learning. There is a tendency
to use a gradient search to determine network weights, al-
though there are many variations in the literature [303, 304].
In turbulence, there are examples where neural net-
works have been used for both modeling and control [305,
306, 248]. As mentioned earlier, neural networks have also
been used to optimize opposition control [22]. Interest-
ingly, the POD, also known as principal components analysis
(PCA), may be trained in a neural network [307]. The neural
network framing of PCA also allows for powerful nonlinear
generalizations [308, 309]. Recent work has demonstrated
the application of network-theoretic tools more generally to
fluid modeling [310], resulting in a graph theoretic model of
the vortex dynamics in a flow.
 In general, neural networks are adaptable and may ap-
proximate any input–output function to arbitrary precision
with enough layers and enough training. However, they are
often susceptible to local minima and may result in overfit
models when trained on too much data. In recent years, sup-
port vector machines (SVMs) [158, 159, 160] have begun to re-
place neural networks for a number of reasons. First, SVMs
result in global solutions that have simple geometric inter-
pretation. SVMs also scale favorably for systems with very
large input spaces. However, multi-layer neural networks
have seen a recent resurgence in activity with the associated
field of deep learning [311, 312, 313]. The fact that these algo-
rithms have been developed to scale to extremely large data
sets (i.e., by Google, etc.), is promising for the mining of high-
Reynolds number turbulence data.
6.5 Genetic algorithm based control
An important class of machine learning algorithms are based
on evolutionary algorithms that mimic the process of opti-
mization by natural selection, whereby a population of in-
dividuals compete in a given task and rules exist to propa-
gate successful strategies to future generations. Evolution-
ary algorithms are typically employed to find near globally
optimal solutions when there are multiple extrema and gra-
dient searches won’t work. They may also provide an alter-
native to the extremely expensive Monte Carlo search algo-
rithm, which does not scale well with high-dimensional pa-
rameter spaces. In this section, evolutionary algorithms are
employed for parameter identification of controllers in the
genetic algorithm (GA) [314, 315, 316]. In the next section,
evolutionary algorithms are employed for both parameter
and structure identification of controllers in genetic program-
ming (GP) [317, 318]. The implementation of evolutionary
algorithms for engineering control is relatively recent [277].
In both genetic algorithms and genetic programming,
an initial generation of candidate parameters or controllers,
called individuals, is randomly populated and the perfor-
mance of each individual is quantified by some cost func-
tion for that particular simulation or experiment. The cost
function balances various design goals and constraints, and
it should be minimized by an effective individual. In the
case of genetic algorithms, the individuals correspond to pa-
rameter values to be identified in a parameterized model, as
shown in Fig. 25. In genetic programming, the individual
corresponds to both the structure of the control law, and the
specific parameters, as shown in Fig. 27.
After an initial generation is populated with individu-
als, the performance of each individual is assessed based on
their performance on the relevant cost function. Individuals
resulting in a lower cost function have a higher probability
of being selected for the next generation. Successful individ-
uals advance to the next generation according to a handful of
rules, or genetic operations:
• Elitism (optional) – a given number of top-performing
individuals are copied directly to the next generation.
Elitism guarantees that the best individuals of the next
generation will not perform worse in a noise-free envi-
ronment.
37
000
001
010
011
100
101
110
111
p2,max 111
110
100110 101
100
011
parameter 1 parameter 2 010
001
p2,min 000
p1,max
p1,min
Figure 25: Illustration of a possible binary representation of
parameters used in genetic algorithms. This example has two
parameters, each represented with a 3-bit binary number.
• Replication – a statistically selected individual is copied
directly to the next generation. Replication, also called
(asexual) reproduction in GP, has a positive memory
function.
• Cross-over – two statistically selected individuals ex-
change randomly selected values or structures and then
advance to the next generation. Cross-over has an ex-
ploitation purpose and tends to breed better individu-
als.
• Mutation – some portion of a statistically selected indi-
vidual is modified with new values or structures. Muta-
tion has an explorative function as it can discover better
minima of the cost function.
The top-performing individuals from each generation are ad-
vanced to the next generation using these four genetic oper-
ations, and a handful of new random individuals are added
for variety. This is illustrated conceptually for the genetic al-
gorithm in Fig. 26. These generations are evolved until the al-
gorithm converges or performance is within a desired range.
There are no guarantees that the evolutionary algo-
rithms will converge, although they have been successful in
a wide range of applications and may converge to a nearly
globally optimal solution. There are a number of choices that
can improve the performance and convergence time of these
algorithms. For example, the number of individuals in a gen-
eration, the number of generations, the rate of each genetic
operation and the schedule for advancing top-performers all
determine the quality of solution and speed of convergence.
Genetic algorithms generally involve a large-scale pa-
rameter identification in a possibly high-dimensional space.
Thus, these methods are typically applied to tune control
laws with pre-determined structure. Early efforts in using
machine learning for flow control involved the application of
genetic algorithms for parameter optimization in open-loop
control [305]. Applications included jet mixing [319], opti-
mization of noisy combustion processes [320], wake control
and drag reduction [321, 322], and drag reduction of linked
bodies [323]. These early genetic algorithms optimized cost
functions by specifying input forcing parameters without
taking into account sensor feedback. However, the method
was also applied to tune the parameters of H∞ controllers in
a combustion experiment [324].
 Daniel
•Dickinson • K. [456] Breuer
[333] and [450, • 451, APS
Spohnberg 452,
DFD 453,2008,
,[330],
454] DMD[443]free turbulence
[331]
networks), are especially In particular, difficult the number to control. ofdistribution.
driver nodes (or leaders) may be
DFD ••2008,
• Videler et alDMD[443] (science and[335, nature 327?bio-flight)
?••] K. [326] [455] random graph with more random homogeneous To leaders) graph
remove with
degreethe more
DC be component
homogeneous of this degree distributio
signal, the out
• APS
Other Other Other
Willcox References
Videler [445, References
References
et446,al 447,
(science •448, Willcox
and449] Breuer
nature [445, [450,
446,
bio-flight)
451,
447, In448, 452,449]
particular,
[455]
453, 454] Extremum-seeking mathematics
the number of driver quite nodes large (or for these 4 may
systems, as compared with a regular or
Extremum-seeking mathematics
Then, this signal passes through the system, 4 4 and the out-
•Wang Moss • bat [338,
[328] 337] • compressed DMD (Mathelin) [444] Degree
heterogeneous heterogeneous
heterogeneous to whichdegree network
degree distributions
degree Degree is
distributions
sent controllable
distributions through to (such which a (such[481]
as (such
high-pass sparse,
network as assparse,isscale-
sparse,
filter controllable
of scale-
scale-
the form [481]
h(⇣) = ⇣/(⇣
• Daniel [456] • and Spohnberg Network et theory quite
references: large for these systems, as compared with a regular or
Other References
• compressed •• DMD K. Breuer (Mathelin)
• Daniel[333]
Dickinson [450, 451,
[456][335,
[444]
and
• 452, • 453,
K.
Spohnberg
Willcox
327? ?
Videler
• Breuer
454]
][445,
, [326] [450,
446,
al451,
447,
(science
448,452,
random449]
and 454]
heterogeneous
453, nature
graph
putfree bio-flight)
J(s)
with
turbulence
free freealso
degree
more turbulence [455]
Extremum-seeking mathematics
has
turbulence
homogeneous
a
networks),
distributions random
bsinusoidal
= b̂networks),
networks), +M 4are graph
degree(such
where sin(!t)
perturbation
especially
are with
are as ! especially
distribution. f is more
especially
sparse, the difficult at homogeneous
frequency
scale-
cutoff difficult to
difficult control.
frequency. to(55)
!.to degree distribution.
control.
control.
• Moss • [330],
•bat APS • •APS
[338, [331]
DFD APS
337] 2008,
DFD DFD DMD[443]
2008, 2008, DMD[443]
DMD[443] •• Daniel Sam and Aditya [457] free To
turbulence Inremove particular,In
Innetworks), thethe
particular,
particular, DCnumber component
the the number Degree
number of driver ofto of this
driver which
nodes
driver signal, nodes network
(or nodes the
leaders)
(or(or is leaders)
output controllable
leaders) mayisbe may may be [481]
be
• Willcox
Other
Other [445,
References
References
•• Videler 446, 447, et al 448, (science449] and nature
Videler [456]
bio-flight)
et al (science and [455] Spohnberg
and Degree
nature to which
bio-flight) Then, network
[455]bthis b̂signal isare especially
controllable
passes 44 through difficult
[481] Finally, the system, to
the control.
high-pass and thefilteredout- signal is multiplied
•
• K.•BreuerAPS
Dickinson DFD •Network
[450,
Wang
2008,
• Moss
compressed
• •compressed
[333]
451,
[328]
DMD[443]
theory
compressed
[335,
452,
bat
DMD
453,
[338,
327?
• 337]
references:
454]
K.
?(Mathelin)
DMD DMD
•Breuer
] , [326] (Mathelin)
(Mathelin)•
• Moss
[450,
Eurika
[444]bat
451,
[444] and
[444]
452,
Bernd
[338,Spohnberg
453,
337] free
heterogeneous
heterogeneous
454]
[379]In particular, sent
quite
put J(s)
degree
the
degree
throughlarge
quite Extremum-seeking
quite
number a
for
large
=
also hasare
mathematics mathematics
Extremum-seeking
distributions
distributions
high-pass
large these+for
of
M
for
driverthese
a sinusoidal
sin(!t)
systems,
these b filter = systems,
nodes
(such
b̂
(such
systems,of
original
+ as M the (or
perturbation
as
compared as
sin(!t) form
as
sparse,
leaders)
sparse,
as compared
perturbation compared
h(⇣) with scale-
may
scale-
= (55)
a
with
⇣/(⇣
at frequency !. be regular
with +
sin(!t a ! af ),
+ or
regular
regular )
(55) or
with or a possible add
•• Daniel • Dickinson [456] and[333] Spohnberg • Daniel
[335, 327? ?[456]
] , [326] and free turbulence
turbulence where
random networks),
!networks),
random is the
graph
random cutoff
graphwith
graph are especially
more
with especially
frequency.
with more homogeneous
more difficult
difficult
homogeneous homogeneous to todegree control.
control. degree distribution.
degree distribution.
distribution.
[330], [331] Videler et al (science and nature quite large
bio-flight) for these
[455] systems, as compared phase with tosignal, a demodulate
regular or the signal. The result is a sign
•
Extremum-seeking mathematics b = b̂ + M sin(!t)
f
•••APScompressed
APS DFD
DFD 2008,
2008, •DMD DMD[443]
DMD[443] (Mathelin) [444] Network 449] control refs[458, Then, To this
remove signal the passes DC through
component the system,
of this and the the out- output is
• •Videler Wang Sam
Willcox
Other
Other
Other
•[328]
et ••al (science and Aditya
[445,
Willcox
•Willcox
batand References
References
References 446,
[445,
nature [457]
[445, 447,
446,
337] bio-flight)
446, 448,
447, ••449]
447, 448, 448,
[455]
Dickinson 449] [333]
337] [335,
InIn327?particular,
particular, ?459, 460,
] , [326] the
the 461,
number
number 462? of,of463] driver
driver nodes
nodes (or
(or leaders)
leaders) maymay be
be[481]
compressed
compressed
Moss
Network
DMD
• Wang
DMD
[338, [328]
theory
(Mathelin)
(Mathelin) •449]
references:
[444]
[444]
• Moss
Daniel [456] batand [338, Spohnberg Extremum-seeking
quiterandom
quiteput large J(s)graph
large also
sent
for
Finally,
for469]
original
Degree
heterogeneous
heterogeneous
heterogeneous
with
has
through
these
these
Then,
Degree theto which
Degree
more
asystems,
sinusoidal
systems,
perturbation
high-pass
this
a high-pass
to mathematics
to
signal
homogeneous
as
which
degree
degree
degree
as
network
which
perturbation
compared
compared
filtered
passes
filter
network
distributionsnetwork
isof
distributions
distributions is
either
⇤ ) with
degree signal
controllable
through
the
with
with
Then,
isat
form
is
a
is
controllable
mostly distribution.
aathisfrequency
regular
multiplied
44controllable
4 the
(such
(such
regular
possible
(such
h(⇣)
signal
[481]
system,
positive
=as as
oror !.
⇣/(⇣
additional
passes
by
[481]
and
sparse,when
sparse, +(55)
thethescale-
!f scale-
through
out-
),b is left of the optimum
!. thewhen system, (5th
••• Willcox
• [330],
• Daniel [456] • •
[331][445,
Eurika
K.
and • Breuer
•446,
K. and
K.
Spohnberg Breuer447,Bernd
[450,
Breuer 448,451,
[450, [379]
[450, 452,
451, 451, 453,
452,• 452,
• More
454]
453,
Wang 453, 454]refs
[328] 454] [464, 465, 466,
To
467,
Degree
remove
468,
where
put
free
free
tofree
the which
DC turbulence
turbulence
J(s)
turbulence
is
also
network
component
the cutoff
b has= sin(!t
networks),
networks), b̂
networks),
is a +
controllable
frequency.of
M
sinusoidal
this
+ sin(!t)
b are
are and
are
signal,
perturbation
especially
especially a
especially
[481] signalthe
b
output
=
that b̂
difficult
difficult +
difficult is at M
is
frequency
mostly sin(!t)
to
to negative
control.
control. b isandto th
• Cost Dickinson•APSAPS[330], DFDDFD
[333]
Probability
[331] [335, 327?
2008,
2008, Moss
DMD[443]
DMD[443]
? ] , [326] [333] [335, 327?
Dickinson
• Generationbat [338, 337]
random
random ? ] , [326] graph
graph
phase with
Towith!
tof more
remove morehomogeneous
demodulate homogeneous
the DC thecomponent signal. putof
degree
degree
theGeneration
The optimum of
distribution.
distribution.
result
also this has is(or
signal,
point.
Operation
aa signal
sinusoidal
the
This that
output
demodulatedperturbation
is at
signal freque
is the
• Willcox
Willcox
• K. Breuer [445,
• [445,
•• NetworkSam
• • •
Videler
[450, APS
446,
446,and451, 447, DFD
447,Aditya
etcontrol
al
452,448,
448, 2008,
(science453,
•
[457]
449]
449]
refs[458, DMD[443]
and 459,
454] nature Mesbahi
460, 461,
bio-flight) work
462? ,[470]
463][455]
[455] sent through In
InIn particular,
particular,
particular,
a high-pass the
the
the
filter number
numbernumber of of of J(s)
driver
driver
driver nodes
nodes
nodes (or
(or leaders)
leaders) may
may be
be
• • •Videler Videler et et al al (science
(science •and
•and naturenature bio-flight)
bio-flight) [455] Finally,
isExtremum-seeking
the high-pass isof the filtered form
mathematics signal is⇣/(⇣ b̂multiplied by the the⇤ system, and(55)
bExtremum-seeking
Extremum-seeking mathematics mathematics
• Moss bat
Network [338, 337]
theory references: [330], [331] Degree
Degree toThen,
to
either which
which mostly thisnetwork
network signal
positive passes
is controllable
controllable
when through Then, b is
h(⇣)
left [481]
this the
[481]
kinto of + b= system,
signal =
the 1possible +
optimum
+
passes
M and!sin(!t)
f ),
(individuals) thepoint
=through
out- theou ou
• Wang
function Eurika
[328]
Network
compressed
••••compressed
compressedand
of selection
Bernd theory DMD
DMDDMD
•454]
[379]
Wang
•(Mathelin)
references:
Dickinson
(Mathelin)
(Mathelin)
k [328]
[333] [444]
[444]
[444] [335, 327? in ? ]general
,where
[326] !put Elitism
is
original
sent
quite
quite
quite
the
through
large
large
large
=perturbation
cutoff
also b̂ +has M for
for for
frequency.
athese
sin(!t) high-pass
these
these
a issinusoidal sin(!t systems,
systems,
systems, To
+ grated filter remove
) as
perturbation
as as
with
of the
compared
compared
compared aour
form DC
at best
(55) with
h(⇣) component
with
frequency estimate aa⇣/(⇣
additional regular
regular b̂+of! offor
bor),this
, drivingsignal, thethe input
•• K.
K. Breuer
Breuer
Videler • ••et [450,
[450,
More
Daniel
al 451,
451,
refs
Daniel
(science [456]
Daniel 452,
452,
[464, andand
[456] 453,
453,
465,
[456] 454]
466,
Spohnberg
and
nature and 467,
Spohnberg
Spohnberg •
bio-flight) Network
468, 469][455] science f
⇤
and J(s) a signal that mostly put J(s)
negative also whenhas a sinusoidal
is to the rightperturbation
!. ath(⇣)
frequency !
• Dickinson
•• Sam and
Videler
Videler et
[333]
• [330],
et
• al
Aditya
Network
al• • •
(science
•
[335,
(scienceSam
Willcox
•Willcox
Willcox
327?
[457]
[331]and Aditya [457]
control
and
and
?
[445,
[445,
[445,
] ,
nature
nature•
[326]
446,
446,
446, Wang
refs[458,
•447,
447,
447, [330],
1448,
bio-flight)
bio-flight)
Network
0 0[331]
[328]
448,
448,
459, 1449]
449]
449]
460, 1 461,
[455]
[455]
theory references:
0 462? , 463]Then,Finally,
b
To
of
this
phase
the
random
random
signal
Extremum-seeking mathematics
where
random
the high-pass
remove optimum
tothe
Finally,
passes
! graph
graph
graph
demodulate
f is
DCpoint.
the
throughthe
with
with
with
filtered
cutoff
component This
high-pass
the more
more
To
the
more
Then,
frequency.
signal.
signal remove
demodulated
sent
towards
where
system,
homogeneous
homogeneous
homogeneous
this
ofis
filtered
through
1this signal
The multiplied
0and the
the
0is result
signal,
1DC
signal thesignal
the
a
optimal
passes
10
high-pass
b degree
degree
degree
byathrough
isthe
component
cutoff
out- is is
value.
signal
the
multiplied
output
then
filter
distribution.
distribution.
E(1)
frequency.inte-
the
that
of
of
isthis
by
the
system,
the
form
signal, andthe
=
theoutput
out-
⇣/(⇣
– Small-world Extremum-seeking
1mathematics
Degree to which network is controllable
••
• Wang•• Daniel [328]
Eurika ••[456]Mesbahi
andMoss and
•Bernd bat
•Moss Mosswork [338,
Spohnbergbat
[379] bat [470]
337]
[338, [338, 337] 337]
0•[331] Sam and 0 Aditya 0[457] Extremum-seeking
1473],
networks
0J(s)original
1Universality
1 0also is
sent [471],
either
perturbation
through
Degree
Degree
mostlyScale-free
a sin(!t
to
0positive
high-pass
which
to which put
1 perturbation
b0 + mathematics
net- filter
b̂1bJ(s) )+
sent whennetwork
0=
network
with b̂throughalso
of b̂ba+ the The
is⇤possible is
has left
!
form
controllable
is controllable
f theory
aawith of sinusoidal
high-pass the
h(⇣) additional isoptimum [481]
=relatively
⇣/(⇣
[481]perturbation
filter +of
R(2) !straightforward
point the), form
(55) (55) ath(⇣)
(55) frequencyto analyze
⇣/(⇣ !.
=multiplied + !f
•Network theory references: ⇤hasoriginal ourperturbation = M
bsin(!t = sin(!t) Finally,
M sin(!t) the ahigh-pass
possible filtered
aadditional signal is
[472,put grated ainto sinusoidal best estimate bat frequency
Eurika and Bernd Network
[379] 1453, 0, [326]
1454] theory
1 references: case +of M +0of ,sin(!t)1drivingno )0 1component
dynamics 1 the
0 !.input and signal f
quadratic cost function.
••Daniel
• •[330],Daniel [331] [456]
•[456]
• More
•••and
Network •
•K.
and
Dickinson K.
K.
• Breuer
Breuer
Breuer
refs
Spohnberg
Spohnberg [464,
science
Dickinson
Dickinson [333] [450,
[450,
[450,
465,
in
[335,
[333]
451,
•451, [330],
451,
466,
general
[333] 327?
[335,
452,
452,
452,
467,
[335, ?
327?
453,
]453,
468,
327? ?
454]
454]
]? ,works
469]] ,
[326] [326] phase bto
where and
Replication of
demodulate
! a networks
signal
is the thethat
cutoff [474]
To
signal. is mostly
remove
frequency. where The negative the
result is DC the is when a cutoff signal b is to
that
frequency. the
of right
this signal, the output isTo e
• Moss
Network bat [338,
•• Sam control and
337] refs[458,
Aditya 459,
[457] 460, • 461,Eurika 462? and , 463] Bernd To [379] remove the
towards DC phase component
f
the optimal to demodulate of
value. this the
signal, original
this !
signal.f theperturbation
to systems The
output result with is is a
sin(!t
nonlinear signal + that
) with
dynamics, a possible
Wang add
and
•••Videler
• Network
Videler
Videler et
et control
al
et[328] (science
alal[470]
(science
(science • Sam
refs[458,and
and
and and
nature
0nature 459,
1nature Aditya
460, bio-flight)
1bio-flight)
bio-flight) [457]
461, 462? [455]
[455]
[455], is463]either of
mostly the Then,optimum
Finally, positive
Then,
b this
the
=Then, b̂ signal
+ point.
when
high-pass
this this
M sent
signal passes
sin(!t) This
signal through
is filteredleft
passes demodulated
through
passes of a the
signal
through high-pass
through optimum
the is system, signal
multiplied
the the filter
system, is
(55)
point
and
system, ofthen the
C(2,6)
the
byand inte-
form
out-
the
and the the h(⇣)
out- out- = ⇣/(⇣ + ! ),
•• Moss
Mossbat
•• Dickinson
Network More •refs
bat •[338,
theory
Mesbahi
[338,
Wang•
–
[333]
Eurika
337]
•337]
Wang [328]
references:
[464, and
Wang
Small-world
[335,
work
465, Bernd [328]
327?
466, 467, ? ]
[379]
Network
networks
, [326]
468, 469]
1
• Network
theory 0–1Network
[471], references:
Scale-free
control
review
sent
refs[458, ⇤
b1 1
paper [475]
through
net- and 459,
a
aoriginal
signal
460,J(s)
grated
put
Extremum-seeking
Extremum-seeking
Extremum-seeking
is
high-pass
The
461,
either
theory
that
into
⇤ put
put 462?
also
perturbation
bbJ(s)=
mostly
filter
isb̂b̂+
our
=J(s)
is
mostly
has
+ , best
also 463]
M
of
relatively
Masin(!t)
also has
positive
the
where
estimate
b
negative
sinusoidal
sin(!t) has
sin(!t
form
aoriginal a!
straightforward
sinusoidal
Finally,
phase
f b̂isof
sinusoidal
+
when
h(⇣)
leveraged when the
0 =
b ,cutoff
)perturbation
with ⇤
1 b mathematics
mathematics
the is
driving
0 left
to
⇣/(⇣ 1demodulate
singular
ispossible
b aperturbation
perturbation to
0
high-pass
+ toof0
frequency.
!
the the
f the
analyze
),
(55)
(55)
atsin(!tright
input
frequency
filtered
optimum
the
perturbation
additional
atisat
in the
signal.
signal
frequency
frequency
signal
point
!. a!.possible
The
theory is multiplied
!.
result
and is
a a
fby
sign
separat th
•• Dickinson Network
• •
[333] • • More
Daniel
Daniel
Daniel
works
[335, refs
[456]
[456]
[456]
science
[472,
327? [464,
?and
and
and
in
473],
] , 465,
Spohnberg
Spohnberg
Spohnberg
•
general 466,
Eurika
Universality
[326] 467, and
– 468,
All
of Bernd 469]
scale-free
networks [379]
0 1
where 0
networks
[474] ! 0 is the
case
are cutoff
of
sparse
b no and dynamics a
frequency.
[476] 0
signal 1 0
and that 1 0 a is 0quadratic mostly is time-scales either perturbation negativecost mostly function.
argument. when positive To
b +to
extend
C(2,6)when ) with
the bright
is left of the addition
optimum
Dickinson •
• [330],[333] • [457] [335,
[331]
•[330],[330], 327?
[331] [331] ? •] ,Sam [326]and 1 1 Aditya 0 1 0 0[457] of the Then,
f
optimum this
towards
To
phase remove
To signal
To point.
tothe
remove the passes
optimal
demodulate
remove This DC the through
the demodulated
value.
component
DC the DC Finally, signal.
component the
component system,
the
of 0Thesignal 1high-pass
this 1
of result 1and
of is1this
signal,
this then 0theissignal, a out-
filtered
inte-
the signal
signal, output
the signal
thethat
output isis multiplied
output is is is a by the
• Sam •• Wangand Aditya
Mesbahi [328] work
Network [470] control refs[458, • More
459, 460, controlrefs
461,[464, 462? 465,
, 463] 466, Finally, 467, 468,
the
this 469]
of
high-pass
to the
systems optimum filtered
with point.
signal
nonlinear phase Thisisb⇤system, multiplied
dynamics,
and to
demodulated ademodulate signal by Wang the
that the
signal is!.signal.
and is
mostlyKrstic then The inte-
negative result when signal
b is to th th
•• Wang
Wang[328]
•
[328] •••–•Moss Mesbahi
Moss
Moss
Network bat
bat
bat[338, work
[338,
[338,
review [470]
337]
337]
337] • There
paper Network [475] refs[458, put 459, Then,
Then,
J(s)460, is this
this
sent461,
also
either signal
signal
The 462?
has
through theory
mostly
sent passes
,passes
athrough463]
sinusoidal
througha is
high-pass
positive through
through
original
relatively a perturbation
high-passwhen filter thethe perturbation
straightforward system,
of is filterthe left at form of andand
frequency
the the
the to
optimum
form out-
out-
analyze with
in
point the a possible additional
Network – Small-world
NetworkNetwork theory theory •
references:
theory Eurika
networks 1
references:
references: and
1 0 0Bernd
have
[471],
1 0 been [379] many
Scale-free original grated
powerful net- into
perturbationadvances
leveraged our sent best in
singular estimate
network a high-pass
perturbation b̂ con-
with of is b ⇤
either,
b̂b̂ab̂+ driving
possible
b filter theorymostly
1 1 of the 1 the
additional 1
and input
positive0form
sin(!t
h(⇣) 0 apoint. signal
=
h(⇣)+
separationwhen ⇣/(⇣
h(⇣) )
M(4)
= +
=⇣/(⇣
b !⇣/(⇣
is
off ),
+
left +!
(55)
(55) ! ), ),
c1 s1 s2 signal is poin
of the optimum
• Eurika
•• [330],
Networkand • Bernd
[331] Morescience [379]
refs in[464,general 465, 466, 467, • Mesbahi
468, 469] work 0
[470] 1 put
put
1 To 1 0
J(s)
J(s) 0
remove also
also
⇤ the grated
has
has DC a asin(!t into0+
sinusoidal
sinusoidal
component our
1 1 )best bb1b=
perturbation
1 perturbation
of
= estimate
= 0 this
+of +M Mthe
M
signal,
sin(!t)
sin(!t)
b̂sin(!t)optimum
at offrequency
at
⇤
frequency
bthe , driving output !.!. b=
the is Thisf1 (s
input demodulated
)+
2 signal
f f
the
Network
works science
[472, in trol
general theory surrounding multi-agent
towards case
where
the and of a
where
systems
optimal no signal
where dynamics
is in the that
the
value. is cutoff
is
the
past is
thephase
and cutofffrequency.
mostly
twocutoff a quadratic to negative
frequency. frequency.demodulate cost when function. the issignal.
to To
the The
extend
right result is a signal that
•473], •Universality
More ?refs [464, ofrefs[458,
networks
465, 466, [474]
467, 468, 469]
• Dickinson
•••–Dickinson
Dickinson [333]
[333]
[333] [335,
[335,
[335, 327?
327?
327? ]],,,[326]
??]sparse [326]
[326] b ! ! ! ⇤
and a is signal that b
is mostly negative
•• [330],
[330],[331] [331] All scale-free networks
Network 1are control [476] phase
459, 460, to461, demodulate
time-scales
462? , 463] the
argument. signal. The bthis result ainto signal 0that b⇤when b is to
thethe righ
f f f
• Sam and
•Sam Sam Aditya
and and [457]
Aditya
Aditya 460,[457] [457]
0•462? 1 1, 463]
0 ToTo remove
remove thetheatowards
Crossover DCDC component
component the optimal of ofhigh-pass value.
this grated signal,
signal, the the our output
output best estimate
is isand
bC(1,4) b̂ofofthe ,the
driving input
• Network Network – control
Small-world
• Mesbahi
theory
• refs[458,
work
references: 459,
networks[470] 461, decades.
[471], Network In0particular,
Scale-free science net- innetworks
is
sent
general
either
through
The
mostly
ofthis
are the
theoryoften to high-pass
Finally,
positive
systems
optimum Finally,
characterized
isThen,
Then, relatively
Then, when
the
Finally, this
this
this filter
with
high-pass
point. the the
signal
bsignal
signal
is
of
ishigh-pass either
nonlinear
This
by
straightforwardleft
the
of apasses
passespasses
of
filtered
the formmostly
demodulated
the optimumdynamics,
filtered
through
through
1
h(⇣)
through
optimum
signal
filtered 0 to0=
positive 1
signal
analyze is
⇣/(⇣
the
the
point.
0
signal
signal
the Wang
point
when
multiplied+
system,
system, isin
system, is
!is
This then
),
fthe isand
multiplied
multiplied and left
Krstic
by
inte-
demodulated thethe
the by out-by
out- the optimum
signal is pointthen int
• • • –Wang
Wang
Wang Network
– [328]
[328]
[328]
Small-world review More • paperMesbahi
networks [475] work [471], [470] sent
Scale-free sent through
through net- aa high-pass
high-pass The theory
filter
filter is
ofof relatively
the
the form
form towards straightforward
h(⇣)
h(⇣) = the
= ⇣/(⇣⇣/(⇣ optimal + + ! ! to
), ), analyze
value. in the
There have been many •powerful
0refs [464, 465,
advances 1 in466, network 467, 468,
con-469] ⇤
Network
Network •theory
theory references:
references: large collection of individuals where leveraged
isdynamics
foriginal
of(represented
grated the cutoff
original
original
into singular
perturbation
byand
our frequency.
perturbation
perturbation
nodes),
best bperturbation
estimate aand
athat asin(!t signal ofinto theory
⇤with that
s,perturbation
driving a)is
with and 1mostly
possible
with the aaextend aseparationnegative
additional
possible
possible
ff b
input = s1(56) 2of
when
(cadditional
additional + f!. 2 (s b 2is))to the right
Eurika and Bernd [379] case !no anegative
quadratic cost bc)bfunction. To b̂signal
• More refs works
[464, •
465, •Eurika
[472, Eurika
466, 473], and
467, and Bernd Bernd
Universality
468, 469] [379] [379] of
0 0networks
00 [474] put
put
put J(s)
J(s)
J(s) also
also
also has
has
has b =sin(!t sinusoidal
afsinusoidal sinusoidal
(s 2b̂ )+ sin(!t 11+ 1 perturbation
+
perturbation
s0 )2 the 1 0 at at frequency
frequency !.
a+quadratic C(1,4) ⇤
• Samtrol and •theoryNetwork
Aditya [457] works
science in[472,general 473], Universality Small-world
– sparse ofin b⇤ where
networks and
where
networks a[474]
signal
!!ff is is thethat
the
[471], case
cutoffis
cutoff ofmostlyno
frequency.dynamics
frequency.
Scale-free net- grated
1 and when The is 1 our to theory best cost right estimate
is function.
relatively of , driving
Tostraightforward
bextend the input
to analyzesign
• –[330],
•• [330], All
surrounding
[330], scale-free
[331]
[331]
[331] networks
multi-agent
•
Mesbahi Network
each are systems
execute
work science
[470] [476]
in
their the 1
own past
general1 0 settwo
1
this 0 of 0
Finally,
to localtime-scales
phase
towards
systems the
To
ToTophase phase
protocols theto
high-pass
with
remove
remove
remove argument.
demodulate
optimal to1
in
nonlinear to
the
the 1
demodulate
the 1
responseDC
DC of
demodulate
filtered value.
DC 1 the0
the dynamics,
component
component 0
component optimum
signal signal.the the is
signal. signal. The
multiplied
Wang ofpoint.
ofof result
this
this The
this The
and This result
signal,
signal,
signal, is
by demodulated
resulta
Krstic the signal
is
the
theis
a a that
signal
output signal
output signal
that
is
isthat is then inte-
• Sam
Samand
• •Mesbahi and
– •Aditya
Aditya
Network
work Network•
[470] • [457]
[457]
Network
Network
review control control
paper •
refs[458,
control [475]refs[458, 459,
refs[458, 460,
459, 459, 461,
460, 460, 462?
461, 461, , of
462?463] the
462? , optimum
,
463] 463]
Finally,
Finally, the
thepoint.
this
high-pass to
high-pass This systems demodulated
filtered
filtered with towards
signal
signal nonlinear case signal isis the
multiplied
multiplied
of no is optimal
dynamics, then
dynamics inte-
byby value. Wang
the
the and a and (57)
quadratic Krstic cost function. To e
decades. InSmall-world
particular,
–[379] Network networksreview to1are 1 0often
paper
external 00 works
[475] 0characterized
stimulus. [472, 473], This by Universality
original a
analogy isMutation
either isofis
holds
perturbationThe networks
mostly
eithereither
theoryquite mostly positive
ismostly
well [474]
relatively +bfor ⇤ positive
positive when
awith straightforward when abour when is 0of
possible 1
left 0isbof 1is left 1
the left
additionalto1ofoptimum ofthe
analyze the C(2,3)
b̂baoptimum ⇤ pointpoint
optimum +!!fffpoint
leveraged singular
sent
sent through
through perturbation b̂aaanet- grated
high-pass
high-pass theory into filter
filter and of best athe
the estimate
separationform of fin the
,(sdriving the input signal
• Eurika andhave Bernd
–Network networks [471], Scale-free net- sent through sin(!t high-pass filter of bthe form form
•• Eurika
Eurika There
large and
and Bernd
MoreBernd
collection Network
Network
refs
More
been
[379]
More[379] [464,
of
refs
many
theory
theory
theory
refs 465, •
[464,
individuals
[464,
powerful
references:
references:
references:
Network
466,
465, 465, 467, 466,
advances
– 468,
(represented
466, Small-world
science
467, 467, 469]
468, 468,in by
in
469]
network
general
469]nodes), grated
networksoriginal
original con-
that into[471], our ⇤ best
perturbation
perturbation leveraged
⇤ estimate
Scale-free sin(!t
sin(!t singular of
+ + b b = )= ), s)with
perturbation
driving
with f (c (s The
this a +a ) + f the
possible
possible theory
to c (s sinput
systems )) theory
s is
additional
additional relatively
signal and
with
h(⇣)
h(⇣)
h(⇣) = =
= b⇣/(⇣
nonlinear
⇣/(⇣
straightforward
separation
1(58)(56)
+
2) +
),),
cof1 f2 (sto
dynamics, 2 )analyze
Wang in th
and
• and baand
⇤
signal
and af is a
signal that
signal isthat mostly
that is is
mostly negative
mostly negative negative awhen bwhen isthat tob the b is toright
to the right
1 0when is the right
–trol Alltheory scale-free
• • networks are sparse
number [476]of large ingraph dynamical
• Network science in general b systems,
case of b no !!fincluding
dynamics and ani- acase quadratic
Network review phase
time-scales
paper to
[475] demodulate
argument.
where
where the
is the
the signal. towards
cutoff
cutoff The frequency.
frequency. the result optimal 0cost is function.
value.
signal aTo extend
1 2 1 1 2
works – All [472, 473],
scale-free Universality
networks –0systems
are of networks
sparse [476] [474] where ! is the cutoff 1
frequency.
2
of no
2
1dynamics
2
and quadratic cost function.
surrounding multi-agent 1461, 0 0works 0 1 [472, the
473], past
towards
Universality
phase two theto optimalof time-scales
networks value. [474] argument. is1 0 C(2,3) and aTo exten
f
Network
• Network each•control control
execute
••••Sam Sam
Sam refs[458,
and
their
and andAditya Aditya
own
Aditya 459, set 460,
[457]
[457]
[457] of local 462? protocols ,463]
463] in 477], responsephase to ofdemodulate
demodulate
the ...optimum
of the thetheoptimum signal.
signal.
point. This The
point. Thedemodulated result
result
leveraged This is
demodulated aasingular signal
signal signal that
thatperturbation
is then
signal inte-
(57) theory separat
•• Network control
Mesbahi • refs[458,
refs[458,
Mesbahi
Mesbahi work 459,
459,
[470]
work work 460,
460,
[470] –
[470] mals
461,
461,
Small-world flocking
462?
462? ,,463] [458,
networks is multi-robotic
either
[471], this
mostly toof
Scale-free the
cooperative
systems
Finally,
Finally,
Finally,
positive optimum net- with thecontrol
the
the
when point.
nonlinear
high-pass
high-pass
high-passb is The left This
theory of dynamics,
filtered
filtered
filtered the demodulated isoptimum relatively
signal
signal
signal Wang isis signal
and
straightforward
multiplied
isnonlinear
multiplied
point is(59)
Krstic is
then then
by inte-
by the
the inte-
to analyze in the
There have
– Small-worlddecades.been – In
many particular,
Network
networks powerful review networks paper
advances
[471], are
[475]
Scale-free –
in oftenAll
network characterized
scale-free
net- con- is The
networks
is by
either
either a
theory are
mostly
mostly is
sparserelatively
positive
positive [476] straightforward
when
when b b is is this
left left ofto
of
time-scales thesystems
to
the analyzeoptimum
optimum with
argument. in the
point
point dynamics, Wang and Krst
Morerefsto external
refs[464,There
•[464, stimulus.
haveand
465, been
466, 467, This
many 468, analogy
powerful
0
systems
469] 1– 1 holds
Network
0 advances
1
[469], 1 quite
review
sensor well paper
in networks
networkfor a
[475] con-
[467, grated
leveraged 463],
gratedinto biological
grated our
singular b
into =
into best
f
our regula-estimate
perturbation
(s
1 our caseb2best ) +
best of c
sbestimate no s
estimate s
b̂dynamicsof theory 1 ⇤ 1 ,of 0
driving of 1
b21and ⇤
)sand 0
⇤
b,extend s,the
driving 0 aathe
driving (56)
input
separation
quadratic bthe the signal
sinput input
costof ssignal
signal
function. and aTo extend o
⇤
and asignal
signal original
original
original
that is perturbation
perturbation
perturbation
mostly negative when f21b2)b̂fsingular )2with with
with
cis to aa2aperturbation
possible
possible
possible
right =
M(2) (c
additional2 +
additional 1 s2 )
•••More Eurika and Bernd [379] works [472, 473], by Universality of networks [474] ))b̂b(s (58)
•• EurikaEurika and Bernd
Bernd [379]
[379] sin(!t
sin(!t +c+ 1(60) (56)
1sin(!t +
bof == 1= (c 1f 2
More works large
refs collection
[464,
Network
[472, 465,
465,
473], 466,
466, of 467,
science
Universality individuals
467, in 468,
468,
general 469]
469]
of (represented
networks [474] nodes),
case ⇤⇤
andthat
and no aadynamics
signal that
that and is
is a
mostly quadratic
mostly bnegative
negative fcostleveraged (s )when
2function. when + 1+ (s (s2+ is is ))
To to to 1the the right
right theory separation
trol theory number surrounding
• •
trol–oftheory • Network
Network
large
All multi-agent science
science
graph networks
surrounding
scale-free dynamicalin systems
in
generalgeneral
multi-agent in
systems, the past
systems includingtwo b b
in powerful
the ani-
past two
1 2 b b
each •execute their ownare set tory
There
–aofNetwork –are
networks
local have
All sparse been
scale-free
protocols
[476]
[478,many 479],
networks
in and
,of
aresponse
this to the the
are
systems
internet
towards
time-scales
advances
optimum
sparse phase
phase
phase
with
[462,
the
towards
towardspoint.
[476] inargument.
nonlinear to 460],
optimal
to
to
network the
This the
demodulate
demodulate todemodulated
optimal
demodulate thisname
optimal
dynamics,
value.
con- tovalue.
time-scales the
the the value.
systems signal.
signal.
signal.
Wang signal with
argument. and The
The The
isisthen
nonlinear
then
Krsticresult
result
result inte-(57)bis=adynamics,
is afsignal
signal
(s(59)
1(61)
that
that
(57)cWang and Krstic
–Mesbahi work [470]
decades.
•••Mesbahi
Mesbahi malsIn workparticular,
••review
flocking
work
decades. Network
Network
Network
[470]
[470] networks
[458,
In control
control
control
477],
particular, refs[458,
often
refs[458,
refs[458,
multi-robotic
networks 459,
characterized
459,459, are 460,
460,
review
460,
cooperative
often 461,
461,
461, by462?
462?
paper462?
characterized ofof the
,,[475]
463]
control the
463]
463] optimum
optimum
by a systems point.
point. This
This demodulated
demodulated signal
signal is then inte-
inte- 2) + 1 s1 s2
Network – Small-world
– paper
– Small-world
Small-world [475]networks trol few.
theory
networks
networks Similarly,
[471], surrounding
[471], in
Scale-free
[471], a fluid
multi-agent
Scale-free
Scale-free net- we may
net- net- is
isisview
The either
either
either theory
The packets
in The
mostly theory
mostly
mostly
the is
theory
past of
relatively
positiveis
positive
leveraged
positive vor-
two is
relatively ⇤ straightforward
relatively when
when when singular straightforward straightforward
is
is is left
left perturbation
left of
of of to the
the
the analyze to
optimum to
optimum theory in
analyze
analyze the in
and
point
point inthe the
a separation of
large Figure to external
There
collection 26: have Genetic stimulus.
been operations
many This analogy
powerful are used
advances holdsto advance
inquitenetwork wellgenerations
leveraged for ainto
grated
con- intoour
singular of our individuals
best
perturbation b = s1inb̂(c
estimate b̂bbb̂genetic
theory of + bsbf1bff ,= (s driving
and algorithms.
s)a)) as2cthe bb b
)f1the
separation input Operations signal
of (58) b = fare 2elitism
•••of individuals (represented by nodes), that
systems [469], grated
grated into our ⇤best
best estimate
estimate of
bof
⇤ ⇤
,,1(s 2driving
driving sthe input
input signal
signal (s
1(62)(60)
(56) ) +(58) f3 (c3 + s2 )
insensor networks There [467,
have 463],
been ofbiological
many powerful regula- advances in network con- = (s + sf(s 2 )function.
2
= 2
Network
•••Network
Network
– All
(E), trol
large
science
science
science
scale-free
number
replication
theory
More
More
More
of
collection
works
inin
networks
large
(R),
surrounding
refs
refsgeneral
refs
works
general
general[472,
works[464,
[464,
[464,
graph
crossover
of[472,
are
individuals
465,
465,
473], 465,
[472,
sparse
dynamical
multi-agent
– 466,
466,
ticity
466,All
decades.
Universality
473],
(C), 473], [476]467,
467,
and
467, as(represented
scale-free 468,
468,
nodes
In468,
Universality
Universalitysystems,
mutation
systems
469]
469]
469]
particular, inof
networks
networks
in
aof
including
(M).
the
by
graph nodes),
networks
networks
networkspast
arethat
[474]
For
time-scalestowardsani-
two
sparse
each[474] that
move
are
[474] case[476]
often
individual
argument.
the bcollectively
b⇤of
case
⇤
boptimal no
case
and
and
and ofdynamics
aaaof
characterizedof
no
value.
no
signal
signal
signal according
dynamics
generation
dynamics
that
that and
that =
time-scales
by is is a1ab(c
is and
k
quadratic
mostly
mostly
mostly
2 and
22
+
)+ 1
1,
(c
squadratic
+argument.
a1cnegativequadratic
negative
1
after negative
21 + cost
2 2 2
the
(s 2 cost ))when
when cost
when
elitism = function.
b function. To
is
fbb1step,isto
(s 2to
extend
)+ To
the
the
a cTo extend
rightextend
sright
genetic s2 (5
each execute tory networks their execute own [478, set479], of local and protocols
the internet in[462, response totowards
towards the
the optimal
optimal value.
value. (59)
1 1
mals
decades.
each
•flocking Mesbahi
••is–Mesbahi
Mesbahi
In particular,
Network [458,
their
work
workwork 477],
review
trol
ownlarge
[470]
[470]
[470]
networks
theory
to set
multi-robotic
paper
global
are
ofsurrounding
collection
[475]
local
often
rules of460],
protocols
(i.e.,
cooperative
characterized
multi-agent
governing
individuals inname
control
by
response
a
systems
physicalthis
(represented
The theory
of
of to
ofthis in
thethis
the
the
is
the
equations)
systems
to to
optimum
optimum
byoptimum
relatively
past
systems systems
nodes), with two
based point.
point.
point.nonlinear
with
straightforwardthat on This
with nonlinear
ThisThis nonlinear dynamics,
demodulated
demodulated
demodulated
to
dynamics,
analyze
dynamics, Wang signal
signal
signal
in
Wang
the
band
Wang
=isis Krstic
s1then and(61)
(63)
(57)
then
(c and
2 + (59)
Krstic
inte- fKrstic
inte- 2 (s2 ))
(5
to
There have–been operation
external
– – stimulus.
Small-world
Small-world
aSmall-world
few.many to Similarly, chosen
external –
powerful –
This randomly
Network
Network
networks
networks
networks There
analogy
in aadvances
stimulus. fluidreviewreview
have according
holds
[471],
[471],
[471],we
This paper
been
inmay paper
quite [475]
many
Scale-free
Scale-free
Scale-free
analogy
network view to [475]
well a pre-determined
powerful
for
net-
packets
holds
con- net-
net- quite a ofsetadvances
vor-
well The
The probability
theoryin
theory network
isis relatively
relatively distribution.
con- straightforward
straightforward The individuals to to analyze
analyze participating
in in the
the in this (56)
systems
large collection [469], sensor
of
decades.
networks
individuals
local
each [467,
In
(represented
rules
execute particular,
463], (diffusion,
their
biological
by
networks
own
nodes),
etc.)regula-that of for
are
as often
well
local a as
leveraged characterized
their
grated
grated leveraged
leveraged
protocols
grated b =into external
singular
fb1in
into
into =
(s our 2f
our by
singular
our singular
response (sbbest
best
)quadratic
1 + ab2=
inputs
bperturbation
best c= )1=s+ fsestimatesperturbation
estimate
1estimate
1b sc(s 21
(c perturbation
(c
= f 2)
2f+ (s + (s 2f fs)b̂b̂2theory
b̂1(s )ofof
sof
(c + 23)b bb+
)) ctheory
⇤⇤ ⇤ b and
stheory
,,,fdriving
driving
driving= (s
)extend fa)1and
(56) (sseparation
and(60)
the
2 the ainput
c)a1+inputcc1separation
separation
s(62)
2(64)
(58) of
1 scsignal
2signal
3 (60) c4ofof
number
trol theory operation ofworks
works
works
ticity
surrounding largeas are
••[472,
number•[472, graph
[472,
Network
–Network
Network
nodes All selected
473], 473],
473],
– in
–Alldynamical trol
science
science from
Universality
science
Universality
aUniversality
scale-free
multi-agent
of large
All graph
scale-free theory
in
in
large
scale-free in
that
networks
systems
graph systems,
generation
general
general
general
ofofsurrounding
of
move
dynamical networks
networks
in
collectionnetworks
networks
networks
summed are the including
collectively
k
sparse
past
are
across of
arewith multi-agent
[474]
[474]
[474]
systems,two[476]
individuals
sparsesparse
the ani-
probability
accordingcase
including
[476]
entire [476] case
case ofofno
systems
of
(represented
network nonodynamics
related
ani- dynamics
in the
dynamics
(i.e.,
time-scales to
by
convectionpast and
their and
nodes), and two
argument. a a aquadratic
fitnessquadratic
due that to (e.g.,
in-
1 1cost
cost 2cost inversely
2 function.
1function.
function. 3 2 1 To To 2To
proportional 2extend
extend
b = s (c to
+ the
f (s cost
)) (5
tory networks [478, 479],set and to the external
internet stimulus.
[462, 460], This
to name analogy holds towards
towards
towards time-scales
time-scales quite thethe
thewell argument.
optimal
optimal
optimal argument.for value. avalue.
value. (61)
1 2 2 2
(57)
malsfunction).
decades. In
each
flocking
to global
particular,
execute
mals[458, rules 477],
networks
flocking
their
(i.e., own
decades.
multi-robotic
governing
are
[458, often477],
each
ofIn local
particular,
cooperative
physical
characterized
duced multi-robotic
execute
protocols
velocity equations)
their networks
by control
own
from a
in based
cooperative set
the
response
this
are
ofthis
this to
often
on to tosystems
control
local
Biot–Savart systems
characterized
systems
protocolslaw). with with
with in nonlinear
nonlinear
by a
nonlinear
response dynamics,Wang
dynamics,
dynamics, Wang Wangand and and(57) Krstic Krstic
Krstic b = f1(65) (s(63)
(59) 2) + (61)c1 f2 (s2 )
(5
––– Network
to Network
Network
a external
There few. Therehave
There review
review
review
–
Similarly,–been
–have
stimulus.
have paper
Small-world
Small-world paper
paper
Small-world
many
been inlarge
been [475]
amany
This [475]
[475]
fluid
powerful
many number
networks
networks
networks
we
analogy powerful
powerful may
advances of
holds large
view [471],
[471],
[471], in graph
packets
quite
advances
advances Scale-free
Scale-free
Scale-free
networkwell
in in dynamical
of for
network
network vor-
con- net-
a net- net-
con- con-systems, The
TheThe theory
including
theory
theory is isis relatively
ani-
relatively
relatively straightforward
straightforward
straightforward to
to analyze
analyze inin the
the (58)
largesystems
collection [469],
local ofrules sensor
individuals
systems networks
(diffusion,
[469], etc.)
(represented
sensor tocollection
[467, as 463],
networks
externalwell byIn of
biological
as
nodes),
[467,
these individuals
their
stimulus. 463],
large regula-
external
that biological
This
multi-agent leveraged
leveraged
(represented
leveraged
inputs
analogy regula- singular
holds
systems, singular
bysingularnodes),
quite
it b perturbation
is =perturbation
often sbthat
perturbation
well 1= s1+
(cpossible
for (ccbaf b=
12 2+ = theory
(s theoryto
cfand theory
s2f1b1)) 1s
b1(s (s
= = b2caa)2)32a
2 s= and
and and
)fquadratic
+ 1+
(c cc124f1c2(s
f(s fa)a(c
+ s22a+ separation
s)separation
(s 3s separation
+c2s+ bcs)s1=s)(58)
)cost ss12(cof (62)
of
2s of+
1 f2 s(s 2 ))
(56)
(64)
2(66)
(60) s3 (62) s4(56)
(56)
ticity
trol
number theoryas ofnodes works
works
surrounding
large works
in grapha [472,
[472,
[472,
graph 473],
473], mals
473],
that
multi-agent
dynamical flocking
Universality
Universality
Universality
move collectively
systems
systems, [458,
ofof
of
in 477],
networks
networks
networks
the according
past multi-robotic
[474]
[474]
two [474] case
case
case of
ofof
cooperative no
nono dynamics
dynamics
dynamics2
control and and 1quadratic
quadratic 31 1 1 2 1 2cost
cost 21 2function.
function.
function. ToTo extend
extend (6
eachtory networks
execute All
–––summedtheir trol
Allscale-free
All trol
scale-free
scale-free
[478,
tory own theory
across theory
479],
networksset networks
networks
networks
the and
of surrounding
surrounding
entiretheare
each
local
[478, are
are sparse
internet
network sparse
execute
protocols
479],
number sparse
and multi-agent
multi-agent
of
[476]
[476]
their
[462,
(i.e.,
the [476]
inlarge own
460],
convection
response
internet graphtoincluding
systems
systems setname
[462, of
due
dynamical
in
in460],the
local
to theani-
time-scales
time-scalespast past
toprotocols
time-scales
in- name two
systems,
two
argument.
argument. in response
argument. b = f
(63)
1 2(s ) + c f
1 2 2 (s )
(59)
6.6 to
decades.
mals Genetic
globalflocking
decades.
decades. rules
In programming
(i.e.,
particular,
[458,
In In governing
477],
particular,
particular, networks
manipulate
systems
multi-robotic physical
networks
networks are control
[469],
often
are
the
equations)sensorlarge-scale
characterized
cooperative
are often often networks
based
control
characterized
characterized
behavior
by on a [467,
by by a thisincluding
with
this
this
463],
a toleader
to
tosystems
biological systems
systems nodes
Control ani-
regula- with
with
with thatlaw nonlinear
nonlinear
nonlinear dynamics, dynamics,
dynamics, (59)Wang Wang b = sand 1(67)
and (c (57)
(65)
(61) + (63)
2Krstic
Krstic 1 s(57)
s(57) 2)
a few.
to There
external Similarly,
duced
stimulus. avelocity
few. in
This Network
–––Similarly,
apowerful
fluid
Network
Network
fromanalogy toweexternal
the review
may
review
review
Biot–Savart
in holdsa toryfluidview
enact stimulus.
paper
paper
paper
quite we law).
ain [475]
packets
[475]
[475]
well
may
larger forThisof
view a vor-
supervisory analogy
packets controlholds
of vor- quite
protocol well forcontrol aa system- (6
There
There have
have
have been
been
local
large
systems been many
many
rules many
collection
large [469],
large powerful
powerful
(diffusion,
sensor
collection
collection ofmulti-agentof
mals
advances
advances
etc.)
individuals
networks
of advances as
individuals
individuals
flocking
well inin
networks
(represented
[467, network
network
as
463],
[458,
network
their
(represented
(represented [478,477],
con-
con-
con-
external
by
biological 479],
nodes),
by
multi-robotic
by and
inputs
regula- that
nodes),
nodes), the internet
that that
cooperative
leveraged
leveraged
leveraged bto=
[462, create
bbb= f460],
b1=
= fsingular
singular
(sfsingular
f(s
12(s )to
12 (s
+2s)bname
)1c2b+c)=
+)1=f+ cperturbation
perturbation
scperturbation
1s cb(ssfb1(c
s13= s(c
(c
= b1s)2sf c32+=3s 1+1+
(s ss(c s11f2cs4s2)(c 22(s )+ )2theorytheory
theory
+f))32(c b(s2=
f (s)) + and
and
f2and
(60) (s
s(56)
))
1 )2(64)
(56)
2(56)
afa) 1separation
+separation
cf12(68)
f(62) (s
2(58)
(66) f32 )(64) of
fof4(58)
(58) (60)
number ticity as
of large nodes In graph in
these
ticity as a graph
large
–dynamical
–– nodes number
that move
in asystems, graph of large
collectively
systems,
wide including
that minima graph
move it is oftendynamical
according
ani-
collectively
that ispossible systems,
to
according including ani-
= 1f (s 1 + 2 c
2
1 1
2 s12 2 3 4 2 32
Functions (6
systems [469], sensor networks afavorable [467, [458,
may470,
463], 480].
biological Theregula- fact that
1 2 2
trol
trol
troltheory
theory
theory surrounding
surrounding
surrounding All
AllAll
multi-agent
multi-agentscale-free
scale-free
scale-free
multi-agent networks
networks
anetworks
systems
systems
systemsfew. ininof are
are
are
the
the
Similarly,
inlocal the sparse
sparse
sparse
past
past
past two
two
intwo[476]
[476]
[476] fluid we time-scales
time-scales
time-scales
view packets argument.
argument.
argument. of vor- b = s1 (c2 + s1 s2 )
mals toflocking
decades.global
decades. InIn
summed
each
tory
rules
manipulate
[458, to
particular,
particular,
execute
networks
each each
(i.e.,
477],
global
across
governing
the their
[478,
execute
execute
networks
networks
the
large-scale
multi-robotic
rules
entire
own
479],
their their
mals
(i.e.,
and
own
physical
are
are tory
network
setown
flocking
behavior
often
of
theset
cooperative
governing
often birds
networks
local
set of
equations)(i.e.,
internet
[458,
and
characterized
characterizedwith
physical
convection
protocols
local [462,
477],
control
fish
[478, based
leader 460],
protocols
protocols
equations)
often
479],
by
by
in
multi-robotic
on
nodes
act
due
response
to
andasthe
aboth
a
name
in to
in
that
based
local
in-
response
response
internetcooperative
flowson [462, with460], control
large-scale to according
name coher-
(61) (57)
(57) (65)
(57) b = f1(69) (s (59)
(67)
(63) 2) + (65)f3(59)
(59) (c3 + s2 ) (61)
decades. Genetic a duced
In
to toprogramming
particular,
external
few. There
There tovelocity
Similarly,
external
external
have
have from
networks
stimulus. been
been in a(GP)
the
stimulus.
stimulus. many
many are
This
fluid [318,
ticity
Biot–Savart
often
analogy
we
This This
powerful
powerful
317]
may as
analogy nodes
law).
characterized
analogy achieves
holds
view
advances
advances in
holdsquite a
packets
holds byin
in grapha
well
quite of
quite
network
struc-
that
for
vor-
well well a move for
forbiological
con- a acollectively (62)
(6
systemslocalcollection
large
large rules
[469], enact
collection sensor There
(diffusion,
aof larger
local
of
have
networks
rules
individuals
individuals etc.)
supervisorybeen as
[467,
(diffusion, many
systems wellacontrol
463],
(represented
(represented few. powerful
as[469],
etc.)ence, their sensor
protocol
biologicalas and
Similarly,
byby well advances
external
that
nodes),
nodes), tonetworks
regula-
as in inputs
create
their
leaders
athat
that
in
fluidanetwork
network
can[467,
system-
external we 463],
strongly may con-
con-
inputs influence
view b=
packets regula-
band
bb= s= c11(c c(c
manipulate
sof
s1s12based +
vor-
(c
2 22 fbs+ sb+ c=13s= ff2f Actuation
)2bf(s
bf cb=
bsc(s (s==
4 (s
= b2))
f))f)2f3sc= f)1+ (s
(s
1(s+ f(s f212)s4c3)2(s)341(c+)+ + f2+ )ccc(s 1+
+ 1c1s s12sb1s1cf)1s2s1= s2)2f(s s(s
2(62)
1) (c
(58)
(58)
2) 2(66)++ s1⇥s(70) 2 )(60)(56)
(68)
(64) (60)
(66)
(56) (60)
large
summed
ture
collection and
number
ticity
across trol In
asparameter
number
trol
trol these
of
number of
nodes
theory
theory
the theory entire
large
individuals
large of inoflargeidentification
surrounding
surrounding multi-agent
graph
a graph
large
surrounding
network tory (represented
graph dynamical
that
graph
networks
to global
multi-agent
(i.e., move of
systems,
dynamical
dynamical
multi-agent
multi-agent
convection [478,input–output
by rules it
nodes),
systems,
collectively is
systems
systems
479],systems
due
(i.e.,
often
systems,
systems, and that
including
in
to in- in
in
the maps
governing
possible
according
the
the including
including
the ani-
past
past
internet past us-
to physical
two
two ani-
two
[462,ani- equations)
460],isto name
= 1 (c2 +on
1 1 2 2
1 12
2 (s2 ))
2 2 11 3
2 2 32 4 2 22
b = (58)
f (s ) + f (c + s ) (6
tory networks wide [478, minima
summed479], and that
acrossthe isof favorable
internet
the entire [462,
the [458,
network 460], 470,
large-scale to 480].
(i.e., name The
convection
acoherent fact motion that
due to[458,in- 477], 1 2 3 3 2
(63)
each
each
each
execute
execute
ing
execute mals
to their
their
manipulate
evolutionary
global flocking
their
mals
decades.
decades. mals
decades.
own
own
rulesown the
flocking
flocking
InIn
set
set
[458,
In(i.e.,set of
large-scale
algorithms.
particular,
particular,
local
local
477],
governing
of
[458,
particular, [458,
ticity
local local as
protocols
protocols
behavior
In
multi-robotic
477], 477],
networks
networks
networks physical
protocols
nodes
rules
genetic
multi-robotic
multi-robotic are
are
in
with
are in
in response
inequations)
response
(diffusion,
cooperative
often
often
often
graph
leader
programming
response cooperative
characterized
characterized
that
etc.)
nodes
based
cooperative
characterized control
move
as
that
con-
on well collectively
control
control
byby
by aaa as their promising according
external inputs when (63) (59)
(59) (67)
(59) c1 c2(71) (61)
(69)
(65) c3 (57) (67)
(57)c4(61)
(61)
a few. duced velocity
Similarly,birds and
in
ducedfrom afish the
fluid often
velocity Biot–Savart
we a
act may
from few.
as local
view
the
tocontrol Similarly,
law).
global flows
packets
Biot–Savart
considering rules with in of
(i.e., a
law). fluid
large-scale
vor-
network-based
governing we may
coher- view
fluid flow
physical packets control.
equations) of vor- (6
to
to external
external stimulus.
stimulus.
enact
systems a larger [469], This
Thissupervisory
sensor analogy
analogy networks holds
holds
summed quite
quite
protocol well
across well to for
theforentire
create aabiological
aanodes), network
system- (i.e., convection sbasedff11due
on
)c)to in- sb2122)) = 2(64)
)f1(60) (s (68) ) + f3(72) (c 3 +(58)
(62)
(70) s(62)
(68)2 )(62) (64)
trol local
(GPC), rules
a systems GP (diffusion,
is used
[469], etc.)
to
sensor as
iteratively networksit[467,
well as 463],
learn atheir
[467, biological
and external
463], refine regula-
inputs
apossible
non- regula- (66)
to asexternal systems
stimulus.
large
large
large collection
collection
collection [469],
This sensor
analogy
ofof
of networks
individuals
individuals
individuals holds quite [467,
(represented
(represented
(represented well463], for biological
by
byby nodes),
nodes), regula-
that
that
that b= bb = f= 11(s 2(ss(s
)22+ 2+ f1b+fs+ = (c
c⇥ cb1bs1b2fs3fsb= f2== +
4(c2= b(s
(s css2s31f s21s221+
= (c
)(c
)(c s(c cs21224sf (c
+ +s34+ + 2f f)f+ s2s2(s (s
4(s s)))
)) (60) 2 (58)
ticity
number
number In
nodes these
of
ofence, in
large
large and graph
graph
graph Inmulti-agent
that theseleaders
that
dynamical
dynamical ticity
move
large cansystems, as
strongly
local
systems, nodes
collectively
multi-agent
systems, rules is
There in
often
influence
(diffusion,
including
including graph
according
systems, possible
have and
it ani-
isthat
been
etc.)
ani- to
manipulate
often moverecent
as well collectively to
advances
as their according
in
external understanding
b = inputs
f (s )
13 3
+ c
1 12
f (s
3
21
)
12 2 1
c (60) c⇥ c c (6
wide
tory
summed toryminima
networks
tory across networks
networks that
[478,
the is
entire
[478,favorable
479],
[478, and
network
479], duced
479], the
and [458,
and velocity
internet
(i.e.,
the 470,
the convection480].
[462,
internet
internet from 460],the
The due
[462, Biot–Savart
fact
to toname
460], that
in-
toto name law). 1 2 1 2 2 4 The degree distribution 2 of3 a network is the distribution(65)
on[462, 460], name 1 4
number
to mals linear
manipulate
global
mals rules
flocking
flocking
of
the mapping
large
each
each
each
the
large-scale
(i.e.,[458,
[458,
graph
execute
execute
execute
large-scale
governing
manipulate 477],
477],
from dynamical
coherent the the
their
their
their
behavior
physical
multi-robotic
multi-robotic
sensors
own
own
own
tolarge-scale
global
motion
summed
systems,
set
set
set
rules
with
when[458,
equations) toof
of
cooperative
of the
leader
behavior
across
cooperative
local
local
(i.e.,
477],
such actuators
including
local basedthe
protocols
aprotocols
protocols
governing
nodes
is promising
withnetwork
entire
control
control
ani-
that
leader to in
networkachieve
in
in
physical
iswhen response
response
nodesresponse
controllable equations)
that
(i.e., convection and basedwith due onhow to in- many (65) (61)
(61) (69)snode s2is(67) (63)
(71) s3 (59) (69)
(59)s4(63)
(63)
birds
a few. and fish
Similarly, often act
inthe a Theas
fluid local we In
flowsmay these with
view large multi-agent
large-scale
packets coher- systems, it is often possible to many other nodes each (61) 1 connected to; this is often vis
(6
local mals
enact
rules
systems
systems flocking
some duced
[469],
toto
to
control
considering
a[469],
larger
(diffusion, sensor
velocity
aenact
sensor
afew. few.
external
external
[458,external
supervisory etc.)477],
objective.
anetworks
larger
networks
from
Similarly,
Similarly,
network-basedstimulus.
stimulus.
as multi-robotic
stimulus.
control
welllocal
supervisory
[467,
[467,
Biot–Savart
in
as in
duced
a This
This
rules
their
463],
afluid
This
463],
fluid
resulting
fluid
protocol leader
we
analogy
analogy
cooperative
analogy
(diffusion,
flow to
external
control
velocity
biological
biological
law).
we
or
may may
control
control.
createprotocol
“driver”
from
holds
holds
holds
inputs view
etc.) view
control
aregula-
system-
the
regula- to quite
quite
law
nodes
packets
quite
as create isof
packets
well
Biot–Savart
well
well
well
in
vor-
deter-
aas offor
for
system-
the
ofvor-
for
their
systemaavor-
a
law).external [480]. c1 binputs
bA= c=f2key
1ss11(c cf(c observa-
3222s + + b1cf4s= 3s11 sbsb⇥ fbsf22bf= =
1)
= (s
)= bfsfas f= f f)1(s(sa+
1(s f(s 2)f )42(s
s1histogram.
)3+ )+ + (c
2+ )cc3c1f+ 1f1f +f3fc22f(c 1(ss(s
3(s (c )(66)
2+
c))3)2(62)
Recent +s2(70)
(62) cs)/32results
) c4 (68) (64)
(72)
indicate (64)
(70)
(60)
(60) (64)
that all scale-free (66) ne
ence,
ticity In and
as
ticity
number
number
number these
ticity that
nodesas large
ofas
of leaders
nodes
oflarge in
nodes
large
large a graph
multi-agent
in can
ina
graph
graph
graph a
graph manipulate
strongly
that
graph
dynamical
dynamical
dynamicalmove
systems,
that thatinfluence
move movethe
collectively
it is
systems,
systems,
systems, large-scale
and
often manipulate
collectively
collectively according
possible
including
including
including behavior
to
according
according ani-
ani-
ani- with leader nodes that 12 4 2 131 2 22 4 2 23 22
s1 (62) (6
summedsystems
torywide
tory minima
mined
across
networks
networks [469], There
theby sensor
that
wide
[478,
[478, have
is
sequential
entire minima
479],
479], networks
favorable
network
and
and been summed
that
the
the [467,
recent
[458,
mathematical
(i.e.,
is favorable
internet
internet 463],
across
470,
convection
tion
In advances
[462,
these
[462, biological
480].
in the
operations
[458,
this due
460],
large
460], entire
Thelinein to
470,
toto regula-
fact network
understanding
in-
of
name on
480].that
research
multi-agent
name combina-
The (i.e.,
isfact
systems,that convection
that
large,it4on is sparse
often duepossible bto=in-s1 (cto
networks 2 + with s1 s2 )are inherently sparse,
(67) with s2 heterogeneous
s3 s4 degree distribution
(67)
theglobal
to
manipulate large-scale
to toglobal rules
global the coherent
(i.e.,
large-scale
rules rules governing
(i.e.,
(i.e., motion enact
behavior
governing
governing [458,
physicala larger 477],
with is
equations)
physical
physical leaderpromising
supervisory nodes
equations)
equations) based when
control
that
based on
based protocol
on The degree
to create distribution a system- of a network is the distribution (63)
(63) (71)f1 off2how (65)
(69) f3 (61) (71)
(61) f4(65)
(65)
mals
mals
mals flocking
flocking
flocking [458,
[458,
[458, 477],
477],
477], multi-robotic
multi-robotic
multi-robotic cooperative
cooperative
cooperative control
control
control
aatory
duced birds
few.
few. networks
and
velocity
tions when
Similarly,fish
Similarly,from
of
considering
[478,
such
often
birds
sensors the
inin 479],
aaBiot–Savart
andaact network
and
fluid as
fluid and
fish
network-based
local
we
we the
duced
often
constants, is
mayinternet
flows
law).
may controllable
act velocity
manipulate as with
which
view
view
fluid
wide
local[462, from
packets
flowpackets
minima the460],
and
large-scale
flows
may the
control.
to
with
with
beof
of
large-scale
that
name
coher- how
Biot–Savart
large-scale
represented
vor-
vor-
is
many
behavior
favorable
law). coher-
in[458, with
many leader
other
470, 480]. nodesnodes The each that
fact node that is connected to; this is often (63) visualized (68)
(6
local
enact
leader arules
local
systems
systems
systems
orlarger
local “driver”(diffusion,
rules supervisory
rules
[469],
[469],
a[469], (diffusion,
(diffusion,
sensor
sensor
sensor
nodes etc.)
in
Incontrol
networks
networks asetc.)
etc.)
networks
the
itthese
well
system protocol
as
large as as[480].
well
[467,
[467,
[467, their
well to
as
463],
463],
463],
multi-agent A external
create
astheirtheir
biological
biological asystems,
external
biological
key observa- inputs
system-
external regula-
regula- itinputs
inputs
regula- is s1 bb = s=
+ 2ff11⇥
to(s s(s 322) c1+
f)resultss+14 ff fb3bb3c+ 2(cc= =
(c= 13c 3s fs1+s⇥ c1+1c ssc22f
(c
(c(c 2)c +)4+ c+
43s css1311sscs2242s)c))14 (68) s2(64) (64) (72)
s3 s4(70) (66)(62) (66)
(72)
(62) (66)
aence,
Infew.
theseaand Similarly,
that
large leaders
multi-agent
ence, in and canfluid
that stronglywe
systems,
leaders may isview
influence
can often
strongly packets
and
possible manipulate
influence of to vor- and manipulate asoften possible 2 13 2
ticity
ticity as
as recursive
nodes
nodes in
in a atree
graph
graph structure
that
that move
move where
enact each
collectively
collectively
a larger branch according
according
supervisory is a signal
control and protocol a histogram.
to create a Recent
system- indicate 3 2 that all scale-free networks (7
summed
wide There
minima
summed
tory
tory
tory summed across
networks
networks
networks have that the
across
across been
is entire
[478,
[478,
[478,favorable
the the recent
entire
479],
479],
479], birds
network
entire and
andand advances
[458,
network and
the
the
the(i.e.,
network 470, fish
convection
internet(i.e.,
internet
internet in
480].often
(i.e., understanding
The
convection
[462,
[462,
[462, act
convection dueas
factwith
460],
460],
460], local
tothat
due
to
toto in-
due flows
name
name toare
name to
in-in- with large-scale
4 The degree distribution b = f (s coher- ) + of 343
f (c + s ) f
(69) 1 (64) f 2 f 3 f 4
(67)(63) (67) (67) (69)
to the
manipulate
toticity
global
global tion
large-scale
the
asrulesnodes
rules in thethis
coherent
large-scale
(i.e.,in
(i.e., line
agoverning
graph
large-scale
governing of research
motion
behavior that manipulate
coherent [458,
move
physical
physical is
with
wide that
477],
leader large,
the
collectively
motion
equations)
equations)
minima is [458,sparse
large-scale
promising
nodesthat according
477],
based
based is networks
that behavior
when
isononpromising
favorable with
[458, when leader
470, 480].nodes
inherently The that
sparse,fact
1 2
with heterogeneous
that a network 3 2 is the distribution
degree distribution. (65)
(65) of (71) how (63)
the merging
when
duced
birds a aa and
duced
few. such
velocity
duced
few.
few. points
fish avelocity
velocity
Similarly,
Similarly, from
often
Similarly, denote
network actthe
from as
from
inin
inisaaathe
amathematical
controllable
ence,
Biot–Savart
localthe
fluid
fluid
fluid flows and
Biot–Savart
Biot–Savartwewe
we law).
may
maymayand
that
with operation,
law). with
leaders
large-scale
law).
view
view
view howcanas
packets
packets
packets illus-
many
strongly
coher- of
of of vor-
vor-
vor- influence and manipulate + ⇥
(7
enact considering
to
local
localaglobal
larger
rules
rules rules network-based
supervisory
(diffusion, considering
(diffusion, (i.e., control
governing
etc.)
etc.) enact
fluid
protocol
network-based
as
as physical
well
well aas
flowas larger control.
to
their
their createsupervisory
fluid
equations)external
external aoften
flow system- control.
based
inputs control
inputs onlocal protocol many
to create other af system-nodes each node is connected to; this
fs13334+ is
(70) often
fss2s(66) (65) f4(72)visualized
(68)(64) (68) (68) (70)
trated leader
ence, in In
andFig.or
these 27.
“driver”
that In large Thelarge
leaders
these sensors
nodes
multi-agent
can
large birds
in and
stronglythe
multi-agent and system fish
constants
large-scale
systems, influence [480].
systems, is areact
Ais
coherent
it collectively
often
and it as
the
key “leaves”,
observa-
motion
possible
manipulate
is often flows to[458,
possible with a large-scale
477],
to cc11f2 cRecent
is 1promising 22f3cc+
ccoher- 3sf
when 14c⇥ cb4b4bss= =
2= 1sf f1f1s1s1(s 3 2s
(s
(s 222)) s)s+ 4+ + sff3f33s3(c (c (c + + 2)) f3 networks
22(66) (64)
wide minima that ticity
ticity
ticity
is In
asas
as
favorable these
nodes
nodes
nodes inin
in
wide
[458, aaa multi-agent
graph
graph
graph
470,minima 480]. that
that
that that systems,
move
move
The move is
fact it
collectively
collectively
favorable
that often [458, possible
according
according
according
470, 480].to
as histogram.
The fact that
3
results indicate 3
that 4
all scale-free (7
local There
summed
summed rulesacross
across
tion
have
(diffusion,
inthe
the this
There
been
entire
entireline etc.) of
have
recent
network
network as well
research
been
advances
(i.e.,
(i.e.,
ence, isas
recent
their
convection
convection
and
that
inthat
large,
advances
understanding
external due
leaders
sparsedue in
inputs
to in-
tonetworks
can understanding
in-leader
strongly with influence
4 The and 4 The manipulate
cdegree c2 distribution
of awith c3 heterogeneous
network c4 of is the a network distribution is the
+
(71) of (67)
⇥
(66)
distribution
(67)how (69) of (65)how (69) (69) (71)
birds and
andvelocity
when fish
such
theeach
manipulate
oftento to subsequent
large-scale
ato
manipulate
manipulate
global
global
when global
actthe
network as
the
suchrulescoherent
rules
rules
local athe
isnetwork
merging
large-scale
the
(i.e.,
(i.e.,
(i.e.,
birds
flows
network
controllable
motion
large-scale
large-scale
governing
governing
governing
and
with is of
considering
behavior
fish branches
[458,
behavior
large-scale
controllable
and
with
477],
behavior
physical
physical
physical
often
with act
is results
network-based
leader
with
coher-
how
promising
with leader
equations)
equations)
asequations)
and local
many with inflows
nodes ahow
when more
nodesfluid
that
based
based nodes
based with 43on
many on
ondegree
flow
thatare
large-scale
distribution
control.
thatinherently coher-
1sparse, degree distribution. (65)
summed
duced
duced velocity across from
from the entire
the Biot–Savart
Biot–Savart the(i.e.,
law).
law). convection
large-scale coherent due to in-
tomotion [458, 477],nodes ismanypromising other when
nodes
finely enact
considering
tuned
local
local aenact
enact
local larger aor
mapping.
rules
rules
rules supervisory
network-based
astrongly
larger larger
(diffusion,
(diffusion,
(diffusion, supervisory
supervisory
The control
fluid
“root”,
etc.)
etc.)
etc.) There
control
as
as asprotocol
flow
control
where
well
wellwell have
control.
protocol
asas as to
protocol
all been
create
branches
their
their
their to arecent
Acreate
create
external
external
external
many
system- ainputs
a inputs
even- system-
inputs
other
advances
system- in each node
sunderstanding
is connected feach nodeto; is this connected
fcc2c3f33f43is fcc3coften +fto; 4 (72)
this(67)
visualized
(68)
is often visualized
(70) (66)(70) (70) (72)
s11⇥results all (68) (66)
ence, and that leaders leader can “driver” influence
nodes in and manipulate + ss22 sRecent sindicate 1 ss cc4c4f11f1results21fc c1c2f2f2342The 444f4 ⇥
leader
ducedIn
Inthese or “driver”
these
velocity
wide large
large
There from
minima
wide wide
nodes
multi-agent
multi-agent
the
have
minimaminimathat
in the
Biot–Savart
been is
ence,
systems,
that that
system
systems, and law).
considering
favorable
recent
is is when
that
itit
favorable
favorable isthe
[480].
is
[458, often
advances often
such
system
leaders A possible
key
possible
network-based
470,
[458, [458,athe
can
in
[480].
network
480].
470,
strongly
observa- to
to
The
understanding
470, 480]. 480].
keyinfluence
fluid
is
fact
The
asobserva-
aflow
controllable
The that
fact
histogram.
fact
andasmanipulate
control.
that that and
a histogram.
Recent
with how
33
many that all indicate scale-free degree that distribution
networks scale-free of a networksnetwork is the distribution
themanipulate
large-scale
tionIn
manipulate tually
in these
thisthe merge,
line
the
summed
summed
summed
coherent tion of
large-scale in is
motion
research
large-scale this the
across
across
acrosslinesignal
is
behavior
the
the
[458,the
of
the
behaviorthat that
entire
entire
entire
477],
research
large-scale
large,
with
with is
network
is fed
network
network
promising
is
sparse that
leader
leader into
coherent (i.e.,
(i.e.,
(i.e.,
large,
networks
nodes
nodes when actuation.
convection
convection
convection
sparse
motion with
that
that networks
[458, due
duedue
are 477], to
toto in-
in-
with
43
inherently in-
is 4
promising
The are
sparse,degree
inherently sdistribution
when
with s2sparse,
1 heterogeneous s3 ofwith sa4 network many
heterogeneous
degree other is distribution. the nodes distribution
degree each (69) (68)
(69) node
distribution. of ishowconnected
(71) (67)(71)
(67) (71) to; this is often vis
birds largeand multi-agent
fish often act systems,
isas local There
leader itlocal
islocal
flows often
orhave with
“driver” possible
been
large-scale
nodes to
recent in advances
coher-
thecoher- system in understanding Sensors and constants
when
The duced
ducedsuch
birds
duced birds
same andaevolutionary
and
velocity
velocity
velocity network
fish fish often
from
from
from often the
the controllable
act
the act as as
Biot–Savart
Biot–Savart
Biot–Savart
operations flowsand
flows
law).
law).
law). with
with with how large-scale
large-scale many many[480].
coher- A keyeach observa- is 4connected The as adegree histogram. distribution Recent results
of visualized
a network indicate is the that all scale-free
distribution of hone
a of aelitism, replication,
considering
enact
enact aalarger
largernetwork-based
supervisory
supervisory fluid
control
control flow
considering control.
protocol
protocol tonetwork-based
to create
create system-
ainfluence
system- fluid flow control. other nodes node ⇥ to; this is often (69) (72)(68)
manipulate ence,
leader the and
ence,orlarge-scale
ence, that
“driver”
and and leaders
that that behavior
nodes can
leaders
leaders when
in can with
strongly
thetion
can such
systemin
stronglyleader
influence
this
strongly [480].
line nodes
network
influence of Aand that
keyis
research andcontrollable
manipulate
observa-
and is that
manipulate
manipulate large,and with fhow
sparse 11 ffmany
fnetworks 22 ff3+ with ff ss4s411+
many 1 + sssother
are ⇥ ⇥sss333nodes
22 inherently sss4 each sparse, node(70)
(70)
with heterogeneous
isnetworks
connected (72)
(68)
to;
(72)
thisdegree distribution
is often visualize
There cross-over,
have that been InIn
In
and these
these
these
recent mutation, large
large
large
advances multi-agent
multi-agent
multi-agent
described
There in systems,
systems,
systems,
in
understanding
have the
been it
itit is
is
previousis
recent often
often
often possible
possible
possible
section,
advances to
toto
as
in a histogram.
understanding Recent 3
results indicate 2
that 44all scale-free
wide
wide minima
minima
enact a larger that
supervisory is
is favorable
favorable control [458,
[458, 470,
leader470,
protocol 480].
480].
or to“driver” The
The
create fact
fact
anodes that
system-that inpromising
the4 The system [480]. A keyof observa- 4
themanipulate
tion large-scale
inthe this
the
manipulate
manipulate line
large-scale
large-scale coherent
of the
the research
the motion
coherent
coherent
large-scale
large-scale
large-scale is that
motion [458,
large,
motion
behavior
behavior
behavior 477],
[458,sparse
[458, is
477],
with
withwith promising
networks
477], is
leader
leader
leaderis
promising43 when
with
nodes
nodes
nodes degree
when
that
that when
thatare distribution
43
Figureinherently 27: 1a
fsparse, network
f2 with The
f3 heterogeneous is
asf4the adegree histogram. distribution distribution degree Recent ofdistribution.how
of a(71)
results (70)
network
(71) indicate is the that
(69)distribution
(69) all scale-free of how
networ
when
birds such
wideand
birds are
and
minima
afishnetwork
used
fish often
considering toconsidering
often
that
considering is
is as
advance
act
act controllable
as local
favorable local
network-based
when
individuals
flows
flows
[458,
network-based
network-based tionand
such
with
with
in
470,
fluid
with
alarge-scale
across
this network
large-scale
480].
flow
fluid
how
line
fluid
generations
Theof
control.
flow flow
many is
coher-
research
fact controllable
coher- that
control.
control.
in
many
is thatgenetic other and
large, nodes with
sparse each how node
networks many isIllustration
connected
with manyare to;other
ofthis
inherently
function
nodes is often each sparse,
tree
visualized node
representation
with is connected
heterogeneous to; this
used
degreeis often visualized
distribution.
leader orand“driver” enact
enact
enact
nodescan a a a larger
larger
larger
in the supervisory
supervisory
supervisory
system
leader [480]. control
control
control
or “driver” Aand key protocol
protocol
protocol
observa-
nodes to
toto
inFig. create
create
create
theas system aa a system-
system-
system-
[480].inA genetic
key observa- + + programming.
⇥ ⇥ f f f ff f f f f f f f (72) (71)
(72) (70)
(70)
ence,
birdsand
ence, andthat fishleaders
programming.
that leaders
often
There act These
can
have asstrongly
strongly
local
been operations influence
flows influence with are andshown
large-scale manipulate
manipulate inin480].
coher- 28 a forhistogram.ge- Recent results indicate as athat histogram. 111 all 22scale-free
43 2 33Recent 3 444 networks results indicate that all scale-free networks
tion this line ofwide
inlarge-scale wide
wide
research There
minima There
minima
minima is have
that have
that
that
thatlarge,
tion is
is isrecent
been been
infavorable
favorable
favorable
sparse
this
recent advances
recent
line [458,
networks
of[458, advances
advances
[458, research 470,
470, 470, with
understanding
ininare
480].
480].
is that The understanding
understanding
The
The fact
fact
factsparse
inherently
large, that
that
that sparse, 4 The 4degree
networks with The 4 The
withdistribution
heterogeneous
degree degree distribution
are distribution of
inherently degree a network of of distribution.
asparse, a network
network is the with isdistribution
is
thethe
heterogeneous distribution
distribution of how degree of(71)ofhow
(71) how
distribution.
the
the large-scale
netic
ence, andwhen that coherent
coherent
programming
leaders
when such
when motion
acan
such motion
network
such strongly
aoften
[458,
function [458,
anetwork
networkis 477],
477],trees.
influence
controllableis
islocal
is
is promising
promising
The
and
controllable
controllable probability
manipulate
and when
when
with
and and how
with withhowof manyeach
how 43
many many + ⇥ (72)
birds
birds
birds and
and and fish
fish
fish often
often act
actact as
asas local
local flows
flows
flows with
with
with large-scale
large-scale
large-scale coher-
coher-
coher- manymany other
many nodes
other other each
nodes nodes node
each each is
node connected
node is connected is connected to; thisto;isto; often
this this is visualized
is often
often visualized
visualized
considering
considering operation network-based
network-based
leader isand
or
leader chosen
“driver” or fluid
fluid
to nodes
“driver” flow
balanceflow in
nodes control.
control.
the exploration
system
in [480]. with A exploitation.
key Aobserva- generalized mean-field 43 + + +results model ⇥ ⇥ ⇥ indicate [328], the mixing layer (72)
(72)
with
the large-scale leader
ence, coherent
ence,
ence, and
and orthat “driver”
motion
that
that leaders
leaders
leaders nodes
[458, can
cancan 477], in
strongly
strongly
stronglyis the
the system
promising system
influence
influence
influence [480].
[480].
when and
andand A
key key
manipulate
manipulate
manipulate observa-
observa- as a histogram.
as as a histogram.
a histogram. Recent Recent results
Recent indicate
results that all
indicate that scale-free
that all all networks
scale-free
scale-free networks
networks
There
Theretion have
have been
been recent
recent advances inisunderstanding
considering Recently,in
tion this
tion
network-based
the
thethe inline
large-scale
large-scale genetic
in
large-scale this this of
line line ofadvances
programming
research
fluid
coherent
coherent
coherent of flow ismotion
research
research that isinlarge,
control.
motion
motion that understanding
control
that
[458,
[458,
[458, sparse
large,
large, 477],
477],
477], has isbeen
networks
sparse
sparse
is
is 43
promising
promising
promising applied
networks
networks 4with
4The
The when
when
when with
degree
degree pulsed
with
are inherently
areare
distribution
distribution actuation
inherently
inherently sparse,
ofof aa43 jets
with
network
sparse,
network sparse, on heterogeneous
with with
is isthe the the splitter
heterogeneous
heterogeneous distribution
distribution degree plate of
degree of [326,
distribution.
degree
howhow 262,
distribution.
distribution. 328, 264],
when
when such such aa network network is is controllable
controllable and and with with how how many many many other
othernodes each
eachnode nodeis isconnected
leader
on“driver”
There a set have of been
considering
considering
considering benchmark recent
network-based
network-based
network-based turbulence
advancesfluid incontrol
fluid
fluid understanding
flow
flow
flowcontrol. experiments
control.
control. many that
4 The degree
nodes
the backward facing connected stepto; to;
controlled this
thisisisoften often byvisualized visualized
a slotted jet [327, 264],
leader or or “driver” nodes nodes in in the the system system [480]. [480]. A A key key observa- observa- as as aa histogram.
histogram. Recent distribution
Recent results results of aindicate network
indicate thatthat is all the
all scale-free distribution
scale-free networks
networks of how
when exhibit
such a various
network There
There
There levels
ishave
have
have of
controllable been
been
been
complexity recent
recent
recent
and advances
advances
withadvances
[325, how 326, in
inin
327,
many understanding
understanding
understanding
328, many 264, other and and4 a turbulent separated boundary layer [264].
tion
tionin inthis
thisline lineof ofresearch
when
researchis
such
isthatthatlarge, large,sparse sparsenetworks networkswith with are are inherently
inherently 43 nodes 43
sparse,
sparse, 4
43 The 4 each
The
Thewith
with node
degree
degree
degree heterogeneous
heterogeneousis connected
distribution
distribution
distribution ofof
degreeof
degree to;aaathis network
network
network
distribution. is often
distribution. is
is isvisualized
the
the distribution
the distributiondistribution of of how how
leader262]. or “driver” when
Thewhen suchaa
such
nodes
interaction in network
anetwork
network
theofsystem theisisisgenetic
controllable
controllable
controllable
[480]. key and
Aprogramming and
andwith
observa- with
with how howas a many
how
control many
many many
histogram. many
manyThe
Recent other
other
other firstresults
nodes
nodes
nodes experimental
indicate
each
each eachnode node
node thatis isisall connected
connected
connected demonstration
scale-free to;
to; to;networks this
this
thisis is of visualized
often
isoftenoften machine-
visualized
tion inparadigm
this line leader
leader
leader
ofwith or
or
research a“driver”
or“driver”“driver” is that
dynamical nodes
nodes
nodes
large, inin
systemin
sparsethe
thethesystem system
system
isnetworks
illustrated [480].
[480].
[480]. with AA Ain key
keykey are
Fig. observa-
observa-
observa-
inherently
29. learning as
asasaaahistogram.
sparse, histogram.
with
histogram.
control heterogeneous Recent
Recent
Recentresults
employing results
results degree indicate
indicate
indicate
genetic distribution. that
thatthatall
programming all scale-free
allscale-free
scale-free networks networks
was per-
tion
tion
tionin in this
inthis thisline line
lineof of research
ofresearch
researchis is that
isthat thatlarge, large,
large,sparse sparse
sparsenetworks networks
networks
4343 with
with
with are are inherently
areinherently
inherentlysparse, sparse,
sparse,with with
withheterogeneous heterogeneous
heterogeneousdegree degree
degreedistribution. distribution.
distribution.
It is important to reiterate that the methods are based on ge- formed in a mixing-layer with a velocity ratio of approxi-
netic programming so that both control structure and param- mately 1:3 — both for laminar and turbulent boundary layers
43
eters are identified, as opposed to genetic algorithms, which [262].
43
43
43 The flow was actuated with 96 equidistantly spaced
are useful for parameter optimization only. These examples streamwise facing jets and sensed with a rake of 24 equidis-
demonstrate the ability of machine-learning control using ge- tantly spaced hot-wire sensors downstream. MLC yields a
netic programming to produce desired macroscopic behavior control law which increases the mixing-layer width by 67%.
(e.g., drag reduction, mixing enhancement, etc.) for a variety Thus, MLC performs 20% better as compared to the best peri-
of flow configurations. These flow configurations include a odic forcing while simultaneously reducing the volume flux

38
 e452, well 453,may
vor- 454]
453, •view
There
for K. Extremum-seeking
454]
quite
a by
leveraged Breuerpackets
have large leveraged
leveraged a[450,
been few.
for
singular bof 451,
many
these vor-
singular Similarly,
singular 452,
systems,
bperturbation
powerful
leveraged 453,
sperturbation
in
perturbation •454] as K.
aadvances
leveraged mathematics
compared fluid
quite
)singular
theory Breuer we
large
theory to and
fstheory
[450,
bin
may
external
with
perturbation
singular network
for a)sand 451, a view
these
separation
and (60) a452,
regular stimulus.
2perturbation
packets
con-
fsystems,
aespecially
b)separation
453,
stheory
separation (55) or of 454] ofThis
where
andas oftheory ))of
vor-
quite Extremum-seeking
compared a)analogy separation
! large
fand isby leveraged
the holds
for
awith cutoff
leveraged
separation these
babiological
of quite
bsingular
regularfrequency.
bsystems,
1(58) = well
singular
b̂of 2f)1
(55) or (s leveraged for
bbperturbation )as s+quite Extremum-seeking
mathematics
s1b1scompared
fperturbation
ac= f s22large singular
leveraged
(s (c fs)22theory
leveraged +sfor
with ftheorythese
b(s perturbation
and
sa
singular
22b)) regularfba(c
systems,
singular
1and
(56) separation
2(55)(60)
2a )or
sperturbation 2astheory
perturbation
separation mathematics
))f232of
compared and
(58) of theory a)with
theory separation
and aand regular
a(58)a of
sep
(64) se
large collection of individuals (represented by nodes), that that (
viduals D (Mathelin) (represented [444] bcompressed systems by = nodes),
2f)1 b+ (s b=
[469],DMD )1b̂= s+f+ s1that scsensorM (Mathelin)
f large
)22sin(!t) (s fsnetworks collection [444] systems
compressed b=(56)
)[467, of bindividuals
[469],
463], b+DMD = sensor
biological b̂s2f + (c
(Mathelin) M (represented
networks
+ sin(!t) sfregula- s(s ))
[444] [467, nodes),
463],
(58) = that
f= (62)
(s + M+ regula- csin(!t) 1s(s b= = s= b̂1= (s+ + bM 1= +f(c sin(!t) (s131(64)
s2f (c (62)
(58) 2)+ s2to s1)control.
is(64)
2(s + c2))
presented
etworks turbulence [467, • ]networks),
•nodes),
463], biological that are especially regula- fdifficult large •))) to collection
control. (c sof free 2(64)
individuals
(58) (s (62) ))
turbulence 3(60) (represented networks), by are nodes),
(58) especially
free that
turbulence b2= 1=
difficult 21(c networks),
22) + to control. (c 223))) + are )especially difficult + (s is2f2a(ssignal 2)
= 1(c + 2f 1(s
egula-
450, alsthat 451,
(represented 452, number 453, by = large
454] of• f
nodes), K.large(s collection
Breuer b2=
that
graph cfree
= [450, 1b (s of
turbulence
21
(c =(c 22451,
dynamical individuals + + (s 2f 452, 1(s )(csDC + networks),
453,
•+ csystems, (represented
454]
•(s 2K.= 1f Breuer are
=(s =+ (cby
[450, 11 + nodes),
(c 451,
2
2+
f2according (s difficult
2452,
2 )that 22 453, to454] control. phase to
1
demodulate
2
the signal. The result 1
To remove the component –including ani-
Daniel [456] and Spohnberg 1ticity as nodes
Daniel in [456] a 2of
graph and this that
2
Spohnberg signal, move the output
collectively according 3
uding move
ding
[335, s
networks
6] are collectively
trol
327?
sparse
ani- randomtheory
?
are
time-scales
Videler Sam
,
[476]
sparse
• •Dickinson
[326] graph
time-scales
time-scales et and
ticity
according
surrounding
al [476]
argument.
1
Aditya
(science withas argument. nodes
–argument.
[333] more [457]
All
and multi-agent
– [335,
time-scales
1
All in
scale-free 1
homogeneous
nature ascale-free 327? graph time-scales
Videler
23
random
Network
bio-flight) ? networks
systems
argument.
2
]23
• that, –Dickinson
[326]
1
networks
et number
degree All
2
move
al graph in
argument.control
[455] scale-free
are
(science the collectively
1
are of
sparse
distribution.
with past
–All
[333] large
sparse
refs[458, All
and
1
more networks
two [476]
scale-free
[335, scale-free
3
naturegraph [476] 327?
homogeneous
459, are
networks
460,
1
bio-flight)
2
dynamical
? networks
random ] sparse
,
Finally,461, [326] are [476]
time-scales
462? graph
degree
[455] systems,
are
the sparse sparse
time-scales
, 463] with
distribution.
high-pass including
[476] [476]
argument.
more filtered
argument. homogeneous time-scales ani- random
signal time-scales
is argument.
graph
degree
time-scales
multiplied with distribution.
argument. more
argument.
by the homogeneous degree distributi
d nature Then, bio-flight) this signal [455] passes
each execute through each their the • system, own
execute toryset and their of networks the local own out- protocols
set
[478, of 479], local in and response
protocols the internet in response
[462, 460], to name
lExtremum-seeking
e and nature bio-flight) [455] •mathematics Extremum-seeking Extremum-seeking
mathematics mathematics
n
ponse td cal
],articular,
name set
and
•of APS of
local
•
protocols
the local
DFD internet
the
Other
protocols
mals protocols
•2008,
inflocking
number response
Other eachtory
[462, DMD[443]
Other in of460], in
networks
driver
execute
References
response[458, response
References
to In
References
name
nodes
477], their
particular, [478,
Extremum-seeking
APS • (or
multi-robotic Extremum-seeking
own 479],
DFD aleaders) the each
set
Other
and 2008,
4• number of
Other
the execute
may 4local
Other
DMD[443] internet
4(57)
•cooperative
be
References of (61)
protocolstheir
driver
References
[462,
References In(65) own
control particular,
nodes mathematics
460],in set mathematics
response to
(or name
ofleaders) local
the number protocols
may (59)
),(65)of
be driver inInresponse (59) particular,
(63) nodes is either (or the leaders) mostly number may 4positive4 of be driver (57)
when nodes (61)b is (or
left (65) leaders)
(63)
(59) of the the (59) may
optimum be point (59)
(65)
f)elrences
455]
physical 6, 447, 448,
decades. equations) 449] Willcoxto based global Videler [445,
on sent
rules et 446, althrough(i.e., 447,
(science governing to
448, global high-pass
and 449] Willcox
nature rules
mals physical filter
Videler (i.e.,
flocking
bio-flight) [445, of governing
equations) the
et 446, [458,
al (59)form (63)
[455] 447,
(science (61)
based
477], physical
h(⇣) 448, and 449]on
multi-robotic
= ⇣/(⇣ equations)
nature + !this
bio-flight) based
cooperative(59) [455] on control
(science and •In
nature particular, bio-flight) networks [455] are often characterized by aanalogy 4 4
Extremum-seeking mathematics Extremum-seeking aExtremum-seeking
mathematics mathematics
on Then, this signal passes through the Then, system, this and
signal the passes out- through the
Then, 4system, signal
and the passes out- through Then,
the system, this signal and passes
the out- through system, 4 4 and the o
k
well
This
berg
network gy
ohnberg
ve
for
powerful
cal puts
put
fluid
[444]
•
control
holds
large
vor-
advances
con-
analogy
work a as
analogy
regula-
•we
J(s)
large
for
con-
compressed their
put
heterogeneous
Daniel
con-
free
may
holds also
•these
quite
advancesin
systems
Degree
Eurika
•holds
external
collection
turbulence
free
heterogeneous
heterogeneous
has
view
networkWang
well
[456]
quite
DMD
b
free
Moss
systems,
to
local
also
=
ain
There
[469],ato
•and
quite
turbulences b for
andpackets
external
inputs
turbulence of =
which
bat
to
sinusoidal
few. [328]
There
well
(c con-
network
rules
1hasnetworks),
degree
Bernd
(Mathelin) sahave well
as
Spohnberg
sensor
individuals
2a +
[338,
external
Similarly,
for
where
(cb sinusoidal
f b
network
degree
compared
have of
quite
(diffusion,
= +
[379]
been
stimulus.
(s
networks),
= a b̂
distributions
degree
337]
for
con- • s
perturbation
vor-
networks
networks),+
f [444]
)) !been stimulus.
s alarge
M
(s
many
compressed
are
to•
distributions
) is
in
(represented
is There
sin(!t)
)
with
many
Daniel
local
• the
+
controllable
distributions
external
afor
etc.)
perturbation
especially
This
put
are
More c
fluid
[467, tothese
powerful
cutoff
are s
•ahave
rules as Extremum-seeking
(such
•Degree
heterogeneous
This
at Wang
external
regular
analogy
powerful
[456]
31especially
especially
s
a Moss
There
refs
463],
DMD westimulus.
systems well
b
few.
frequency been
systems,
There
frequency.
by
also =
difficult
cb
(such
analogy
(diffusion,[464,
and
(58)
at
[481]
mayas
=
(such
[328]
f
to
advances bat
biological
asor
have
(Mathelin)
nodes), f
sparse,
(s
frequency
has
which
Similarly,
many
stimulus.
holds have
advances
difficult
c 465,
as
[338,
view
Spohnberg
(62)
their
[469],
(s )to
difficult
as
b
a c
as
This
!.
b
holds
been
+ )
sparse,
degree
= quite
been
466,
etc.)
337]
in
compared
scale-
sparse,
network
packets
powerful
regula-
external
sensor
that
c
sinusoidal
control.
= + (66)f
(60)f
to
c c f (56)
(64) (s s
to
in
network
many
(55)
[444] in
(s
quite
This
467,(s as
control.
large
many wells
scale-
distributions
)
a 468,
network
(62) well
scale-
networks
)
control.
) +
(56)+
originalis
fluidof
analogy
inputs c
(56)
well
advances
powerful for
for
with
perturbation
f put f
heterogeneous
controllable
holdsvor-
con-
powerful 469]
as
free
(c awe
these
(s mathematics
Degree
for
a
con-
their
+
perturbation
[467,
)
quite
turbulence
free s
may
holds
regular
heterogeneous
heterogeneous
free
(such
aadvances
in
systems,
)also
fto[481]
advances
external
(60)
463],
at
well
view
network quite
(56)
(66) has
turbulence
turbulence
which
oras b
degree sparse,
for
packets
as
in
biological
frequency =
(60)a
networks),
sin(!t
well
quite
inputs
bs bin
network
degree
con-
compared
network
(60)
=
sinusoidal = anetwork
(64)
(56)(c b̂
networks),
distributions
degree
s for+
+
large
+
networks),
1 b
scale-
(c of
⇤ a)vor-
regula-
cM
b fb
are
2 and
= +=
isdistributions
distributions
for
con- with
sin(!t)
(s put
with
controllable
heterogeneous
con-
sb
perturbation
c
f b f
especially
free 1area
1)) =
=
(s
these
s(s
Degree
signal
c
are
2 ff ) )
turbulence
) +
(such
(s
(s
especially
free
1
heterogeneous
aheterogeneous
possible
regular
+ systems,
calso
c
especially
free 2 f c that
)
)
turbulence
to
+
+
(such
s
2at
(c
difficult
f
[481]
as
has
(s c
c
turbulence
which
(such
s or
is
1b f
degree
sparse,
additional
c=
b
frequency
+ s)
as
mostly
a
s (s
s
networks),
as
= f
b
network
as
degree
)
f
= c
sparse,
compared
(58) (s
sinusoidal
)
to
difficult
difficult
2 2 1 b (s
=
distributions
b̂
control.
scale-
degree
sparse,
)(62)
negative
+ c
f
networks),
2
networks), to (55)
b + ) b toM=+
(s =
are
cb
control. f
c
isdistributions
distributions
scale-
control.
1 c f
sin(!t)
s = with
controllable
scale-
(66)
s
(s
(c(s
(60)
f
perturbation
)f 2 + (s(56)
(64)
especially
are when
sc 2 )
(s)
+
are) +a(60)
f (56)
+ s
)
(such
regular
f
(s
especially + c bs
especially (c is
)c
s) to
s
[481]
+
at
as
(such
difficult
(such
sor
s
sparse,
the )
frequency
asas
to
difficul
difficult
spar
(60)
right (56)
(66)
con
sps
t
al
aph
dom
ording ] 51, and
To
dynamical452,
systems,
that
graph 453,
Spohnberg
remove move
J(s)
454]
the•
systems,
including
with K.
•collectively
more DC Breuer
ticity
•
trol
number
Dickinson Daniel
componentnumber
including
ani-
homogeneous
theory
1[450,
astheory c b
nodes
according
of [333]
12
[456] = 2 451,
surrounding
large
random
c
off 21bb
ani-
1
ofin
[335,
1(s
1large
and = =
f452,
b
2c
2
graph number
this 2
degree
)
32 s
agraph
= f 1+ 1(c
Spohnberg
trol
327? graph
f 453,
(sc c
graph 4f
signal,
12 1 (s
23 f+
1 )
theory?number
2+
(c 454]
• free
)
sJ(s)
(s
of
]•that
1
distribution.
2
multi-agent
dynamical with
+c
2 K.
2+
s
,[330],
)
1
[326]large
ticity
dynamical
2
the c
s
Dickinson
) s Breuer
turbulence
1
more• move
surrounding
s
2 )1 s
2Daniel
output
of as graph
21 b
large
systems
systems,
1
2
homogeneous [450,
=
nodes 2 b
2
collectively
[333]
f
systems,
[456] = (s
issurrounding
31
graph 451,
networks),
dynamical
random inf1
multi-agent
2
in
including
[335,
1
1)41!.
and
1 (s
2
a+
the 452, 2
12graph 2c )
including
2
dynamical
2
1 + f
Spohnberg
according
327? phase
graph
past
degree 453,
2 are
c (s 1
systems,
1 32 s
ani-that
?in-
2
1 )454]
especially
systems
two
s3J(s)
]with
2
2 ani- to
move
systems,
distribution.
,convection
[326]
2demodulate
including
more in difficult
collectively
the past
including
homogeneous the
ani-
random
1 to
two control.
signal.
2 !.
according
ani-
1
graph
2
degree The 2 11 J(s)
with
2result 2
32 1
distribution. more
2 4 1 is
1
3 1
a 3 12signal
2
homogeneous
1 1 22 1
2
that
2 !. 3
1 2 1
41
degree distribution.
1 1 12 2 2
1 2 2 1 3 1
22 3
1 12 2 2
trol surrounding trol trol theory theory surrounding multi-agent multi-agent systems
systems in in the the past past two two
gent
0],
st
08, namical
rk o49]
eani- two
multi-agent
the past
(i.e.,
in-
to
•sent
systems
past
DMD[443] each
name
Willcox
two
•through systems,
two
Intory
convection
ToMoss •remove in
systems
Network
execute •In
[445, bat
particular,
the
•networks
[330],
In APSsummed
due
[338,
446, •past
including
their
the
particular,
particular, in DFD
•APS to
the
447,
the
control
[331] DC
APS
337]
mals
two [478,
in-own DFD
number past
across
2008,
the
448,
ani-
component
DFD the
flocking
479],
refs[458,set
number
449]
two
Finally,
2008,•DMD[443]
number the
2008, of of and
Willcox •entire
driver
mals DMD[443]local
[458,
459, the
summed
the
DMD[443]
•Moss
of Toof Mesbahi
driver this
flocking
internet
460,
network
high-pass
nodes
[445,
477],
•In
protocols
remove
driver •multi-agent
bat 461,
APS
signal, across
tory [338,
nodes
particular,
to 446, (or work
nodes
•global
multi-robotic
[462,
462?
(59)
[458,
[331]
•(i.e.,
networks
DFD
the
APS 447, in
the
337]
leaders) the
filtered
APS
(or DC460],
,DFD
response
(63)
[470]
(or
477],
rules
systems
entire
463]
convection
2008,
outputDFD
leaders)
the
448, component
leaders) mayto
signal
[478,
2008,
number
449] DMD[443]
multi-robotic
(i.e.,
(67)
cooperative
(61)
name
network
2008,
(57)
(65) isbe may
in 479],due
isDMD[443]
governing may
(63)
the
multiplied
DMD[443]
4of
(57) is be
(57) to and
of To
past
(i.e.,
driver
either be In this the
remove
two
particular,
cooperative
control In
nodes
physical
mostly by
internet
signal,In the
the
particular,
particular, (61)
(or the
(57)
(67) the due
[462,
DC
equations)
positive leaders)
control
(61)output
number
theto 460],
component
(61)
the in-
(65)
(57)
when number may is
number
based
of
ofbthe
to name
driver
beis To of
on Inoptimum
left of remove
this
particular,
drivernodes
driver
of Inthe signal,
In particular,
nodes point.
the
(or
particular,
nodes
optimum the the
DC
leaders)
(or This
number
(or output
component
the (59)
the
leaders)
point
demodulated
may
leaders)
number (63)
number of isbe may driver may of
(67) be (57)
(65)
(61) of this
driver benodes signal
(61)
driver(57) signal, nodes(or is leaders)
nodes then
the
(or(or inte-
(61)
output
leader (57)
(67)
leadma
7],
obotic
verning ence ed Degree multi-robotic
on and cooperative
physical
to nature which Then, a equations)
network
Videler
bio-flight) malsto
cooperative
high-pass control
decades.
this global et
flocking
signal filter
alis
[455] based
In rules control
controllable
(science passes of
particular, [458, (i.e.,
the
on
Degree and
through form governing
decades.
477], [481]
networks
nature
to mals multi-robotic
which the In
Videlerbio-flight)
decades. system,physical
flocking particular,
decades. are
network often
et and
Inal equations)
[458,
[455]
In (science
cooperative is networks
characterized
particular,the controllable
particular, 477], out- Degree and based
multi-roboticare
control
networks nature
networks to often
by
[481] onwhich a characterized
bio-flight)
Then, are are cooperative
network
this
often often [455]
signal is by
characterized
characterized passes a
control
controllable Degree through by to by [481]
a which a Then,
the network
system, this signal andis controllable
passes
the out- through [481] the system, and the
ets
d
multi-robotic
etc.)
erized
vart
references:
etworks
DMD
67,
7?
are
ntrol
etworks
cterized38, by337] often
?463],
•where
ato
oflaw).
] as
by
•vor-
[326]
Extremum-seeking
are
by
sent acharacterized
[467,
,(Mathelin)
a aoften
external
quite
•[326]
biological
Dickinson
well is
few.
More
as
•
cooperative
through •large
463],
the
•quite
quite Network
characterized
WangSimilarly,
duced
bcompressed
their [444]
cutoff
•bfor
stimulus.
•large
refs
local
biological
regula-
[333]
large
systems
= Mossdecades.
ba•1external
large
by
b̂[328]
high-pass
[464,these
systems
compressed
=
+ control
velocity
for
ftheory
rules
compressed
[335,
frequency.
collection
aM
1[469],(s
bat
original
for
b465,
in
This
these
DMD
cregula-
sin(!t)
systems,
327?
)1inputs
In
[338,
these
f+
by
a
[469],
from
filter
•2466,
references:
(diffusion,
bsthat sensor = fbDMD
particular,
fluidheterogeneous
analogyaperturbation
systems,
?K.
of
337]
(Mathelin)
b̂2bDMD
(c systems,
•467,
s])systems c32+
of the
sensor
,s3of as
large
mathematics
we
duced
[326]
individuals + M the
s(Mathelin)
•Dickinson
snetworks
Extremum-seeking
sent
compared
468,
f2etc.)
Biot–Savart
s4smove
h(⇣)
(Mathelin)
may
holds
22systems
sin(!t) form
as
networks
• velocity
through
•469]
as
quite
[469],
networks
)collection
)+ • compared
Network
f[444] aslocal
=
aWang
compressed
view
h(⇣) degree
quite
sin(!t
compared well
[333]
⇣/(⇣
• with
few. large
(represented
[467, sensor
[444]
Moss
b law).
rules
[444]
•compressed
(60)
[469],
from
a=
=
are
[467,
ofas
+
packets
[328] well
high-pass
[335,
s463],
+ ⇣/(⇣
theory
compressed)12(64)
!
b̂(55)
Similarly,
bfor =
with
often
adistributions
bat
+ f the
regular
with
networks
their
individuals
sensor
),
b̂M
)463],
cthese
(diffusion,
bbs2and DMD
for
+
327?
= with
+
biological
[338, of
Biot–Savart
!
by
sin(!t)
a1references: M characterized
external
s2454] 2b
f
DMD
vor-
abiological
a1(58)
csregular
in
filter
)?systems,
),
networks
(c or
regular 337]
asin(!t)
(Mathelin)
= DMD c1[467,
s]222graph
nodes),
aetc.)
possible
(55)
,b̂+ bof
s(represented
[326]
+
(such
fluid
or
s)⇤f(58)
(Mathelin)inputs
cM andExtremum-seeking
mathematics
sent
s4theor
463],
2(Mathelin)
law).
quite
as
[467,
that as we
regula-
)sin(!t) form
by
aslarge
additional
athrough
compared
[444] well quite
biological
signal
463],
a b view
sparse,
mayquite h(⇣)
[444]
byas a=
for
large
that
(62)
[444] =
their
nodes),
(58)
biological
(68)
b̂scale-
high-pass
large bthese
with (55)
+
⇣/(⇣for
regula-
is
=
M
for(55)
packets
fabthese
external
amostly
+(62)
that
sin(!t)
!2bfffilter
systems,
these
regular
=
(66)
(s
(58) regula- )1=), grated
(s
+
of
sbinputs
negative (55)
c=
vor-
of
b̂1c2bsystems,
systems, or
)+
1= f+Extremum-seeking
smathematics
sent
as M the
quite into
bbs21that
c(s fcompared
= through
(cform
as
)when
3sin(!t)
(c our
large
1as
ssc3quite 3(c+ +quite
ss21compared
compared fcs4h(⇣)
s+
best bfor
(s
=
))ais
b2largewith
sflarge =b̂these
to
sb(s
2))
+
estimate
high-pass for
(55)
a=
with
b)⇣/(⇣
s= the
M regular
forwith
as(60)sright
sin(!t)
systems,
these +
b(c these
a2(c filter
bb̂! a=of
regular
(64)
+ fscbsystems,
b13),
+ or
regular
=b̂b=
s!. b+
⇤
fs2of
systems, cs,(55)
f as mathematics
driving
the
4(68)
(s
M (ssor
)(62) compared
(58)
(66) ))orform
as
s212c1sin(!t)
)(c as
(62)
+(58)
2) c fthe
compared 2(cf(s23input
compared
h(⇣) with
+at =s ⇣/(⇣
2)
(55)
signal
a regula
with with
(62) +a!r
(58)
(68)
2regula-
), ,454]
nputs
represented 327?
that K. ]!
?Breuer ,is
put by [450, nodes), = also
451, fby large
(shas
452,
that )in ascollection
+ 453, c2bsinusoidal = 454]
s21cb(s 1free (c Breuer perturbation
turbulence
+ cNetwork
s11individuals (s 2[450, )) 2large b2large
•
451, s=
bscience
networks), = ats(represented
452,
collection (cfrequency
bcollection (c +
in
453, + sare
general (68)
sf(62) of (66) (s of
especially individuals
by (64)
)) (58)
individuals
nodes), put difficult that (represented
(represented also
to has
control. (62) by sinusoidal
by nodes), nodes), that put
perturbation1=
(c 2s + (c also + at has 2frequency
1)) 1 sinusoidal
12= (c 1 = s2+ perturbation
(c 3s + (s ))2 )) frequenc
dorks gula- to[467, 463], biological regula- = )(c 2+ fcollectively (s )) 3= (c + (s 3 2
iduals
odes), nodes), (represented
that that
fticity J(s) as nodes In nodes),
these a1and = graph
large (c =
3 = multi-agent f
that + 3 (s (c In these (c (s
systems, ))
+
large s[456]
1 =multi-agent
2
it is (c
according
often + !. possible systems, 1 2 2 ))
to itJ(s) is often possible 1
to 1 2 2 2 J(s) 21 2 24 21 2 2 1 2 1 1 2 2 43 2
ystems, leSpohnberg
according
333] number
[335, it
whererandom
327? often •! of
random
? Daniel
random ]possible
large
is
graph
, • the
[326] Dickinson [456]
graph
cutoff
graph
2
with
graph to phase more
with
1
dynamical
frequency.
with [333] Spohnberg 2
more
2
to
homogeneous
more tory [335,
1
demodulate
1
homogeneous where systems,
2 4
homogeneous
networks 327? • random
3
!ticity
Daniel
?
summed ] the
degree 32
• is
, including
1 2
[326]
[478, the as
Dickinsonsignal.
graph
2
degree nodes
cutoff
distribution.
degree
479],
across with
11
ani-
The
and in
frequency.
[333]
distribution. Spohnberg
1
distribution.
the
2
4
more a
theresult
entire
3
[335, internet homogeneous is where
327?
network that
a
random signal ?
[462, move
! ]
random random
,
(i.e.,isthat
graph
[326]
460], collectively
the
degree cutoff
graph
to
convection with
graph name more
with
distribution. frequency.
with according
due towards
morehomogeneous
more to where
in- random
homogeneous the
homogeneous random
! optimal
random is
graph degree the cutoff
graphvalue.
with
graph
degree distribution.
degree more
with frequency.
with distribution.
distribution. more homogeneous
more homogeneoushomogeneous degree degreedistribu
degree disd
],
nternet
460,re andnetwork
461, the
Then, [462, internet
462? this 460],
(i.e., , • f
463] [330],
convection
signal [462,
to summed
each
tory name passes [331]
tory
460],
networksexecute duenetworks
across
to through name to their
[478, in- the the [478, entire
479],each
own
Then, system, 479], tory
and setnetwork
execute
this • theof[330],
and and
networks f
signal local the
internet their
the(i.e.,[331] passes internet
out- convection
own
[478,
protocols [462, 479],
through set [462,
460], in of and 460],
due of
local
Then,
response
to the the
name the to to
system, in- name
optimum
protocols
this
internet f
signal and [462, thepoint.
in passes response
460],
out- This through
to namedemodulated Then, the system,this signal
f
signal and is
passes
the then out- inte-
through (65) the system, (63) and the out- (63)
n43]
name ,local
ponse
dure
to
in
set
[457]
446,
vior
that the in-
•response •Finally,
protocols
internet
response
of447,
with
mals
Videler
bio-flight)
To
local
Wang to
•leader
448, removethe
[462,
Mesbahi
•protocols
global
flocking
et •
al449]
[328]
[455] in
Sam high-pass
Willcox response
460], the
rules
manipulate
nodes
(science • and
[458, workeach
•Willcox inDC
to
Willcox
thatandAditya
[445,
(i.e.,477], filtered
execute
name
[470]
response component
nature the 446,
governing
[445, •[457]
[445,
multi-robotic In
large-scale signal
447,
446,
Videler their•particular,
bio-flight) 446, manipulate
448,
Wang447, of physical is
Then,
own
447, this
etpositive multiplied
449]
448,
•al behavior
•[328] Sam
448, set
thesignal,
eachthis
Willcox
cooperative
[455] to
(science 449] each global of
the
•equations)
449]and signal
number
(61)
• execute
Willcox local by
the
execute
Willcox
with (65)
Aditya
large-scale
and [445, rulesthe output
passes
control of protocols
leader
nature their
446,
based
[445, driver
[445,
(69)
(i.e., their
[457]
(63) (59)
(67) through is
behavior
447,
446,nodes ownon 446,
governing
bio-flight) (65) own
nodes (59) in
448,
447, (59) set
447, the
that To
response set
(or
449]with
448, of remove
system,
448, of
physical
[455] local
leaders)
449] leader local
449] the
and
protocols
(63)(69) protocols
may
(59) nodes
equations)DC
the (63) be out- component
(63)
that (67)
(59) in based in response response To
onthrough of remove
this signal, the the
DC output
component
(61) is (69) of (59)
(67)
(63) this (59) signal, the (59)
outpu
(69)
mayed )
37]
logy
te tremum-seeking
fluid based
Biot–Savart
put
fororiginal
vor-
lhis
well wewell view wefor
holds
aJ(s)may
analogy
for
on
amay
sent
Finally,
packets
also
Degree
•
law).
perturbation
quite
view
a through
holds view
has Moss Then,
Degree
ato
well
packets of
the
aduced few.
•quite
to
Degree
packets bat
vor-
Wang
sinusoidal
external
ato a
for
high-pass
which
this
high-passfew. to
[338,
velocity
external
of well
is
ba[450,
mathematics
to
signal
[328]
Similarly,
which
svor-
of either
network
Extremum-seeking
which
337]
Similarly,
b̂stimulus.
vor-
perturbation
heterogeneous
for
filtered
)put
cfilter
passes
from
quiteawith
stimulus.
mostly
network ain
network
put J(s)
is
few.
to ofthe
alarge
enact
signal
controllable
through
in
external
the
aoriginal This
fluid
J(s) aalso
possible
is
a
Biot–Savart – Finally,
Similarly,
• few.
at
form
This
for
is
degree
alsoto
is
controllable
fluid controllable
Small-world
duced
Moss frequency
analogy
we has
the
to
these
multiplied
Degree Then,
stimulus. when
may
external
analogy
•h(⇣) has we
a Wang
Similarly,
the
[481]
system,
bat
external
additional in
velocity
law). may
aholds
sinusoidal
distributions
systems,
=
high-pass
this
to
[338,
view ab [481]
[328]
sinusoidal
⇣/(⇣!.
is
which
fluid
stimulus.
holds
by
[481]
and Extremum-seeking
signal left
mathematics
networks
337]
view
This
in from
quite
packets
stimulus.
+
the
sfas
!
the of
we
aperturbation
put
quite
filtered
networkpasses
packets
analogy
4
fluid
compared
),+ the the
out-
well
(such may
perturbation of
grated
J(s)This
optimum
well we [471],
Biot–Savart
fc3)bsThis
for
vor-
signal
through
sent
is
view
as of
holds
alsomay
analogy a
Finally,
for at
Degree
controllable
vor-through
analogy
into
sparse,
with has
is
at
the
Scale-free
packetsquite
view
afrequency
our
multiplied
point
Then, the
Degree
law).
aholds
acreate
to
Degree
system,
best
scale-
frequency
regular holdswell
apackets
sinusoidal
high-pass
which
this
[481]
of
high-pass
to
estimate
quite
!. Extremum-seeking
or cmathematics
to
for
squite
by
signal
net-
vor- which
and
!.
network
which
ofput
the the filtered
aperturbation
well vor-
filter
passes
well
b̂M
network
of
sJ(s)
out- network
for
The is
of
⇤
for
boriginal sent,scalso
signal
athe
Finally,
controllable
theory isDegree
a through
driving at
formhas
is is
controllable Then,
frequency is
controllable
the
Degree
s4c3)the
multiplied
the
ah(⇣)
to
Degree
fsinusoidal
ainput
high-pass
[481]
relatively which
system,this
bhigh-pass
=f
to
b⇣/(⇣
mathematics
to
signal
[481]
which
signal
!.
by
network
and
+
the
straightforward
[481]
which filtered
perturbation
!ffilter
passes
the network
),
networkout- is
)fof
signal
controllable
through
the
is at
form
is to
controllable is multiplied
analyze
controllable
the
frequency
fh(⇣)
[481]
system, in
(s)=2extend
the
[481]
!.(70)
⇣/(⇣
at(64)
by
[481
and
+(
t
•a+ larger supervisory )control protocol asto ba additional
system-
local rules sin(!t +
•1(diffusion, )etc.) as well as their external b̂inputs
enact a larger supervisory control )(66) (64) (60)
Network theory f1references: Network )protocol theory bfto create creferences: afmove system- (62)
local rules (diffusion, etc.) = f+ fb(s sfc+ fbsin(!t) (c (60) + s(c )acs+
helin) protocol systems
[444] to create [469], =a•Breuer system-
ssensor networks sc[467, 463], 2biological )12(66) )regula- (55) 2as well as (64) their bexternal (64) a(55) fbinputs (55)
68, d tem-
nal 0, 469]
[379]
451, inputs original
452, 453, put
Network
•[331] perturbation
Eurika
band
454] K.J(s) cSpohnberg also
1science
K. (cand
K. 2c
Breuer + Breuer fb=
has
bBernd ⇤bc
sas in
sin(!t and
s= +
anodes
451,
[450, general scsinusoidal
[450, M
2a
b[379] 4(s =b+ signal
452,
451,)2sin(!t) = f)in 3451,
+ )f(s f453,with
452, (s
cs4[456] that
)452, f2perturbation
•454]
453,
)nodes a+
put
c(s is
Eurika
K.
453,
c4works possible
bperturbation
scf)as mostly =
Breuer
454] fb(s (62)
454]
=
K. (s f[472,
and
)K. fb(salso
f2Breuer additional
negative
(sat Bernd
[450,
Breuer sfb+ )frequency
has= =+ f (70)
451,f(64)
sin(!t
[450, (c
a (60)
(68) 4+
[379]
[450, (s
f
fwhen
sinusoidal
32452, M
451, (ss)(66) (60)
3451,
+ + s(c !.)2(60)
sin(!t) 453,
452, )1original
is with
f(c
452, s3to
454]
453,
(s + the
put
perturbation
)2453, )networks454]perturbation
possible
)right
454]
J(s) (60)
(70) also =
at
(64) has =
frequency (68)
sin(!t(c
(60) c+
b̂possible
2+ sinusoidal
case +sb1c= s=
sin(!t) of 2f)bf 2no with
=
(s4(s 2f fput)23s11)+
dynamics 3(sa+
perturbation perturbation
f22possible
J(s) (c+ 32f c(salso
+and f(c
2sb2)32additional
(s =
at a=
)has 1f)frequency
quadratic
sin(!t (s
s1b̂22=a2)+ M
sinusoidal)b11f3(s++ = 32c sf= costf3(68)
(70)
)(64)
+
!. f(s 1(s
with fsc2(s 2)
(60)
2)3
)2perturbation
+
function. c4+ possible
1c s122f(s
)2To )additio frequ
ove ph
i-agent To
ble • Daniel
remove
to •
collectively
that [330],
move
systems, •[456]
where the •collectively
according
it
DC isnumber
ticity
is often
component
the ticity
In as cutoff
2
these
according
possible of
nodes
1 = 1 3
large
2 f
large
frequency.• of in (s
1
To Daniel
to this a ticity
fgraph
• multi-agent +
graph1[330],
number signal, a
1 32 as (c
1 graph
ticity
dynamical 32[331]
that + and
of
the 1 1•
that
systems,
move
2 J(s)
large
number in
• In
Spohnberg
nodes
1
output
2 2 a
1these
move
systems, graph
graph= 2
collectivelyit 473],
in
of
2
is large
is 13
a(s
collectively
large often that
3
dynamical
graph Universality
1 12
including
1 multi-agent
+ To
2
2
graphpossible
accordingremove
2
that +
according
34 collectively
systems,
move
ani-
where
dynamical to of systems,
the
2
collectively
DC including
is according
it
component
the
systems, is [474] often
cutoff
11
ani-
according To
2
of
frequency.
including remove
1
this
13 !. to 1
signal,
whereani-
12
the DC
the is
1
4
component
output
the
3
1
cutoff
1 2
is of
frequency.
2
this
1
24 1
signal,
2 3
the output
2
is
isremove
4
•,the DC component ofentire this signal, the output isincluding
1 2 3 2 3
guding
rding
that cal
cluding
[335,
458, phase
ani-
dynamical systems,
move327?
470,
toryani- ani- to
480]. ,including
] demodulate
collectively
systems,
?networks
summed •[326]!
The Dickinson
fwide •across
fact
[478, number
including
[330], ani-
the
according
minima
that the
479], [333] signal.
[331] entire of
andfree [335,
ani-
that large
random the
2
turbulence
The
network Tofavorable
327? wide graph
internet remove
result ?graph ]minima
(i.e., is
[326]number
dynamical
networks),
Dickinson
summed
[462, the
awith
2
•signal
convection
[458, DC
that
[330],
460], more
470, of component
are
that
[333]
is
across
to 480].
[331] large
systems, 1
homogeneous
favorable
namedue especially
[335,
2
the to
The graph in-including
327? of
3
fact towards
[458, this dynamical
difficult
?that
network degree
]470, ,signal,
2
ani-
[326] the 480]. ! to (i.e., systems,
the
foptimal
control.
distribution. The output
convection fact value. that is due tothat ani- in-systems !
that
due to phase
in- To toremove demodulate the of DC the
the component signal.
optimum phase The point. of result
To this to
remove is
demodulate
signal, ademodulated signal
the the DC that
output component signal. is phase The of Toresult this toremove is(71)
demodulate
signal, a leader
signal
the the DC the that
output signal.
component phase
is The To ofresult to f
remove
this demodulate is anonlinear
signal, the signal the
DC
(63) thethat (67)
output
componentsignal. is The(65) of result this signal, is a signal (65)
the(61)
(71) th
o
448,
ysical
cale
455] erning
refs[458,
dsent
ses: that
[455] 449] equations)
behavior
throughphysical459, ••high-pass
•with
460, Network
aSimilarly,
Finally, Sam
based
equations)
Videler
to manipulate
leader
461,
mals global
the and
Videler toon
462? et
Videler Aditya
global
nodes
control
flocking
filter
high-pass based
al ,fish
rules 463]
(science
et the
of etal brules
that
refs[458, [457]
on
[458,
the
(i.e.,al (science
sent large-scale
filtered to
(science form and(i.e., global
mals
477],
governing
through 459, nature governing
signal
and to flocking
and rules
••Then,behavior
global
460,
–•multi-robotic a=manipulate
Network Sam
Videler
mals
nature •This
bio-flight)
physical
is
nature mals
high-pass a461, (i.e.,
(63) and
multiplied
Videler physical
[458,
rules 462?
Videler
flocking
bio-flight) etwith (67)
governing
flocking Aditya
control al
bio-flight) 477],
f(i.e.,
the
cooperative
[455]
equations)
filter et leader
,(science
463] by
et[458,
al large-scale
equations)
(71)(65)
governing
refs[458,
of [457]
(61)
(69)
multi-robotic
[458,
al
[455] the
(science sent
[455]
the physical
(science nodes
and
477], signal
(67)
based (61) through
477],
form (61)
control 459, based
nature
and that
physicalon
andis
behavior
multi-robotic 460, then
equations)
multi-robotic
nature on
cooperative 461,
bio-flight)
atheory
Finally,
nature high-pass inte- (65)
with
equations)
462?(61)
the based
bio-flight)
bio-flight) (65)
[455] control
,filter
f463]
high-pass
cooperative
cooperative on(65)
based (69)
(61)),of
[455] nodes
sent
[455] the this on through
filtered
control form
controlto signal a high-pass
Finally, is with multiplied
the filter
+high-pass of
by the dynamics,
the filtered form (71)
(65) (61)
(69) signal (61)Wang is⇣/(⇣ and
multiplied !Krstic by
(science
flows
oher- ontrol
],
ative
rences:
eoryll
ry
robotic
ng
ve
tc.)
•is
where
stem-
on
as
physical
multi-robotic
either
control
Moss
control
control
with
a is
their
references:
as
and
well
Extremum-seeking
cooperative
Network
few. mostly
bat
duced nature
equations)
Then,
external
protocol
isas
original as
[338,
large-scale
either sent
the
• b
–Extremum-seeking
Extremum-seeking
their
cooperative
mostly
Wang
=
bcutoff to
•bio-flight)
this
positive
337]
Small-world
velocity
birds
theory
through
enact
b̂ inputs
systems
local
perturbation +
control
•
Network
create
mals
coher-
external M
based
signal
and
[328]
local
in
positive
rules
frequency.a
fromwhen
references: a a
larger
sin(!t)
flocking
grated
a
[469], rules
passes
control
high-pass fluid
system-
on[455]theIn
theory
inputs
(diffusion,
•
often
when networks
supervisory
sensor
particular,
is
into
Moss
(diffusion,
through
Biot–Savart
we left
local
[458, sent
actfilter
b Degree
birds
Network
may
our
systems
references:
of
is mathematics
bat as
is
networks
etc.)
left
rules
477],
through
h(⇣)
the
with
either
of
bestlocal
•and
thethe
[338,
[471],
local
view mathematics
of
the
etc.)
control
to
Extremum-seeking
optimum
law).
sent
as
Network
Wang
multi-robotic
mathematics
number
system,
theory
duced thewhich
fish
(diffusion,
enact
estimate
[469], b
asEurika
systems
⇣/(⇣
337]
flows
form
rules
well =
packets
mostly through
[467,
possible
systems as
•high-pass
this
b̂
(64)
often
Scale-free
optimum
(55)
protocol
Network a [328]
sensor
+
+
well
with
h(⇣)
larger
network
and
(diffusion,
as 463],
M
of
!
point
review
signal
references:
velocity
b̂positive
of
their
(68)
[469],
act
of
as
driver
),
a
etc.)
sin(!t)=
additional
(c[469],
the
large-scale
b vor-
high-pass
to
networks
⇤
cooperative
filter
passes
as
⇣/(⇣from
point
supervisory
theory ,
their
biological
out-
create
external
sensor
paper
net-
driving
is
local
as
sensor
whennodes
etc.)
where
+
of
controllablethe
well
external ! a
the
through
references:
[475]
flows
networks
coher-
filter
fsystem-
[467,
b
),
asthe
regula-
inputs
networks
(or
Biot–Savart
asis form
control Extremum-seeking
well
control
The
original
h(⇣)
with
left
either
of
their
input mathematics
leaders)
inputs
463],
isno
Then,
the
the
[481]
h(⇣)
of
sent
the as
[467,b
=
Extremum-seeking
Extremum-seeking
system,
mostly
the
=
⇣/(⇣
large-scale
form
external
protocol
signal
biological
= may
this
law).
through
their
cutoff
[467, b̂
is
⇣/(⇣
optimum
(55)
(66)
+
perturbation
463],
+ relatively
h(⇣) be
signal
and
to
external
463],
(62) M
! +
positive
inputs
frequency.
a
!the
),
coher-
create
=
regula-
sin(!t)
biological
fpasses
high-pass
biological
s ⇣/(⇣
(66)
point out-when
inputsa
where
s
straightforward
leveraged
system-
+
through
! s is
filter
b
regula-
), Extremum-seeking
is mathematics
either
2)is
c⇤foriginal s left
2with
Then,
h(⇣)
of
the
the
sent
singular
bof
the Extremum-seeking
Extremum-seeking
mathematics
3 cutoff
=
= mathematics
system,
mostly theto
through
this
⇣/(⇣
form
ac2perturbation b̂
4possible
analyze
+ optimum
(55)
signal and
positive
perturbation
M c h(⇣)
1frequency.+b
!f ), in
passes
the
a=chigh-pass
sin(!t) =
(64)
aadditionals
⇥
the
point
⇣/(⇣
(c c
out-
when(68) f +
through
theory
+ c s ! bsfis),
filter
f (72)
3(66)
mathematics
(62)
(70)
1 )1with )
h(⇣)
leftf of
and
the
(66)mathematic
offa
the
(62)
=
mathemat
system,
theform
separation
a possible (55)
+and
optimumh(⇣) (66)
f ), thepo
additioof
=(72) ⇣/
(62)
o
b = f (70)
(s )!.regula-
+ f (c c + cs ) c c
f=
[467,
egula-
nputs as well463], a⇤as
put biological
their Eurika
external
also fsregula- 1systems
has aand inputs
a+ Bernd
csinusoidal [469], sbwhere [379] csensor where perturbation is
networks •sthe is 2the
c cutoff at cutoff
c [467,sand
2frequencyfrequency. c b463],
cfrequency.
Bernd c s(72) c c1f(70) (62)
biological[379] c f(68) (62) (62)
c regula- (72) (62) + f b ⇥ b s f = (c s f + (c s f + s s ) s ) c c s b = b c s
= s s(c c s(c + + s s s s
) )
bsput also has athe issinusoidal bput
perturbation also 4 at has frequency sinusoidal 21 + perturbation
negative when at
s(66) 2 2 frequency
tworks
gical regula- [467, 463], biological to(66)
cal regula- ! = (s s)[464, fregula-
bfands1473], sin(!t
(c 1+ s+ 1)can s)1put b2is = 1+ s2(c )2cto case f! of sin(!t c1+ 1! c31function. sin(!t
453, 454] quite large for these systems, as compared with adynamics
a469] regular or and aright quadratic cost 2To extend
also ⇥ chas cfand
a ?sinusoidal perturbation at frequency
= s2fs3Spohnberg 4(c + s34strongly )J(s) 1 2 3 4
⇤
and signal J(s) •
In that
these
ence, is large
andmostly +
that multi-agent negative
⇥ cthat =band = 3ence,
ss2Spohnberg
f s(c c!
when b4sf
systems, (c + !and + sbMore ais that s)1to
it + leaders
the
•)influence often right possible can
= strongly
!. + sSpohnberg 4influence )and J(s) and manipulate 1 2 3 4 J(s) 1 2 !. 1 1 2 2 3 4
works ]leaders and manipulate
65, 466, 2[472, Universality of networks [474] 1 2 3 3 1 1 2 22 1 3 2 4
b467, 468, aMore 469]
Daniel refs [456] 465, 466, 467, 468, Daniel 469] In refs these
[456] [464, large 465, ,1Spohnberg 466,
multi-agent 467, 468, systems, it is often possible to
3 4 24 2 1 1
ly ,•bthat
influence ticity and nodes manipulate •in Daniel graph ?[456] move Spohnberg collectively Daniel according [456] and 1
and
ulate
in)
possible Spohnberg
[444]
Dickinson to
f
and ••
where signal •[335, Daniel
1
that
is the
2
is [456]
cutoff mostly
3 3
and
12
frequency. •negative
2 1⇤
2f
and
2• –••[333]
f
when 14 1
signal
• bDaniel is 1
to
that 2
the [456] is
1
right
]mostly
2
and 2 3 4Spohnberg
3
negative 1 4⇤ f
2
when
where signal is that
is the cutoffmostly frequency. negative and where awhen signal !bf is that to
theis the mostly
cutoff right frequency. b is to the rig
•[333] 327? ,in- [326]
Dickinson [335, 327? [326]
2
]vorable [326] !to 1
•that summed
3
where
across All
wide scale-freethe minima is1
entire the 2
networks cutoff
network
that
3
is frequency.
favorable are (i.e., sparse convection [458, [476] !b fdue to in-
to470, 480]. The tofact that
7] (i.e., •(i.e.,
convection Sam
[458, and
470, [330],
due Aditya wide
480]. summed
[331]in-The [457]
minima towardsfact across that the is479],torySam
the optimal
favorable entire networks
summed and [330], Aditya
value.
network [458, ![478,
[331]
across f 470, [457] (i.e.,
479],
the 480]. convection
entire and The the network fact internet due that to
(i.e., in-
[462, convection460], to name
due time-scales
in- argument.
ct ,re network Finally, phase (i.e., the convection to
high-passtory
summed demodulate networks due
across
filtered to
the [478, the signal.
signal entire is The and
network
multiplied the
result tory internet
(i.e., is networks
by a(69) convection
signal
the [462, that
[478, 460], due 479], to
Finally, to name in-
phase
and the the internet
high-pass demodulate [462,filtered the
460], signal.
signal to
Finally, name phase isremoveThe
multiplied
the result tothe
high-pass demodulate isthe
by a the signal
filtered the that signal.
signal isthis The
multiplied result isthe bya the signal
(67)
f
e460],
etwork
0],to ditya458, internet
name
andofin-
tothe to the
name
[457]
477],
to nameTo [462,
internet
optimum
is
global convection
remove
manipulate
promising 460],
rules [462,
point.
the to
the
•[455] Sam
(i.e., tory
name
due
the
large-scale
whenDC
460], This andnetworks
to component
to
large-scale
governing in- name
demodulated random
Aditya coherent [478,
physical Finally,
[457]
behavior the of
graph To
motion 479],
Finally,
this
large-scale
signal remove the
with
equations) tory
and
signal,
with [458,
manipulate high-pass
the
is• the
more networks
then
(65) the
Sam
leaderhigh-pass
coherentthe
477], internet DC output
homogeneous
inte-
and
based is nodes
thefiltered [478,
component
motion
Aditya
promising on [462,
filtered
(67) (63)
large-scale
(71) is that 479],signal460],
[457]
(69) [458,
(63) this signal
degree
when
(63) and
of To to is 477],
to this
behavior the
name
multiplied
is
removesystems internet
multiplied
signal,
is
distribution. promising the
(67)
with with
(63) by
the DC[462, the
by
leader
(67) output
when
nonlinearthe460],
component
(67) (71)
(63) nodes isto name
dynamics,
that To of this signal,
Wang DC
and output
component
(65)
Krstic (69) is of (63)
(71)
(67) (67)
(63) signal, outpu (63)
when nodes
d ] Then,nature that
thisof the
bio-flight)
signal optimum Network
Finally,
passes point.
through the control high-passThis the refs[458,
Then, demodulated
system, filtered
this of 459, the and
signal optimum
460,
signalNetwork
thesignal 461,
Finally,
passes out- is is 462? point.
then
control
multiplied the
through , inte-
463]
high-pass This refs[458,Then,
the by demodulated
the
system, this of
filtered 459, thesignaland460, optimum
signal thesignal
461,
Finally,
passes is
out- 462? ispoint.
through
multiplied then
the , 463] high-passThis
inte-
Then, the by demodulated
system,
thethis of filtered thesignal andoptimum signal signal
Finally,
passes
the out- is point.
is
multiplied
the
through then This
high-pass inte-
the demodulated
by system, the
filtered and signal signal
the out-is is then
multiplie in
470] 8,
coher-
fluid
ckets frt
ets
dereferences:
as
t–Savart
79] 449]
a
Bernd
J(s)
337]
law).
•vor-
may
flow
local
Biot–Savart
[471],
original
grated
Wang
of
system- we
of
• view
alsovor-
control.
local
flows
is
law).
sent
Eurika
vor-
into
grated
[379]
may[328]
either
Scale-free
perturbation
enact
has
•with
law).
packets
rulesthrough
our
•
•view
original
a
into
–Mesbahi
and
a
Network
c
Moss Network
sinusoidal
abirds
•large-scale
duced
mostly
our
•
Bernd
considering
larger
(diffusion,
best c f
of
a
packets•Moss
few.
net- a batMoss
duced
sin(!t
estimate
perturbation
Eurika best
few.
vor-
high-pass work
and
velocity
positive
supervisory
ctheory
f
[338,
[379] review
bat
Similarly,
b of
etc.)
perturbation
estimate
s and
+ b
+
=
c
bat
fish
put
f
[470]
velocity
coher-
b̂
=
337]
[338,
Similarly,
b̂
The
vor-
network-based •filter
of
) [338,
references:
s b ⇥f +
often
from
when
Bernd
f J(s)
bf =Extremum-seeking
original
as bb
paper
Wang
with M
sin(!t
(s=b
theory
Degree
⇤control
=
duced
337]
b̂
s
•
There
, well
in
=
f f+
)
337]
from
original
of
driving
act
athe
sin(!t)
of b̂
also
+
a
Eurika
b athe
considering
+
[379]
f(sM
few.
b
in
at
s
[475]
[328]
is sent
grated
In
⇤
+ fas
(s
as
fluid
M , )
velocity
isthe
duced
Biot–Savart
possible left
perturbation
to
have a •local
form
protocol
frequency
sin(!t)driving
has
these
(c + )
•
relatively
•fluid
sin(!t) f
ofbirds
Mesbahi
a
perturbation
fluid
their and
through
which
the
Moss
Biot–Savart
Similarly,
Network
+fwith
original
+ a
we
enact
into
ence, been
f
h(⇣)
s
•
the
a
few.flows
velocity
input Bernd
flow
sinusoidal
large
(c s • b(c
from
we•Moss
may
few.
additional
network-based
externalthe
) to
a
(66) our
and
Eurika
=
+ and
bat
network
as =Moss
law).
optimum a
manycreate
possible
!.
+sf
work
may
larger
with
signal
theory
input
perturbation
multi-agent
s
the
straightforward
in
(70)
best
s)(s
fish
[338,
view
Similarly,
s
that
bat
[379]
high-pass
sin(!t ⇣/(⇣
control.
law).
from
Similarly,
sin(!t
)bs inputs
and
+s
abat
)
Biot–Savart
view
=
+is
powerful
a
often
[470]
put
estimate
signal
337]
[338,
large-scale
fluid
+
perturbation
+ s
leaders
packets
point
system-
supervisory ! the
references:
additional
f (68)b
s
⇥
[338,
controllable
f s (64)
(72)
Bernd
f =
(s
+
(55) mathematics
in
),
(c
J(s)
337]
original
packets
filter
fluid
) actwe
Biot–Savart
with
b̂
systems, s
in
sin(!t
s)
337]
a
(70)+)
+
can(64)
b̂
+ [379]
asto
a
with
advances
(55) of
leveraged
also
of
M
may
of
fluid
flow
(64)
s (55)
fs
law).
coher-
a local
analyze
fluid
sent
the
control
grated)at
strongly
b
vor-
is
perturbation
+
(c
of
possible
⇤
sin(!t) a
has
,
it
view
either
we
[481] vor-
driving
frequency
is
+)
flows
possible
through
form
control. law).
we in
original
a
into
with
s
may
often
in
singular
protocol mostly
may
sinusoidal
influence
)
with
the
packets
(68)
our
the
a
view
additional
network
h(⇣) a
(64)
view
additional
=
large-scale
high-pass
sin(!t
perturbation
possible!. best
input
c
possible 1(68)
ofpackets
positive
perturbation
⇣/(⇣
to andcon-
create
b
put
vor-
packets
+
=
(68)
perturbation
estimate
c f 2to (72)
signal
(64)
! b̂
J(s)c
additionalf
manipulatebf
when
original
filter
)
a+
32=
),
system-
b of
with
sM
coher-
(55)
=
+
sin(!t bc
of
also
b̂f 4=
ofvor-
theorysent
sin(!t)
+
of b̂ b
grated
s
is
a
b
vor-
+
f
s
⇥ the
M b
is
perturbation
at has
f ⇤
=
+
either
M,
(s
left
possible
through
form
frequency
sin(!t)
driving
sf s and
sin(!t)
original
)a
into(s
of
with
+
mostly
sinusoidal
s s f )
the
h(⇣)a
our
additional
+ (ca
a
the
optimum
high-pass
sin(!t
separation =
s
perturbation
s
f
positive
possible
best
!.
b + (c
1
=b
input
⇣/(⇣
s = b
fs
+ 2
+
(66)
f
=
perturbation
estimate
) s s
(s1
+ (s
b̂
signal s
)
point
of
b!
) +when
filter
)
(70)
32+
additional sf
f(55)
=
+ ) with
b ),
M+ =
sin(!t c s
b
b̂ 4⇥of f
+
of
b
sin(!t)
sf
=
sb̂
f 3
a
(55)+the
M
is
(72)
(c
(68)
f
b
sat
(s
possible
(55)
(64)⇤
+ 3M ,
sf
left
(s+form
) (68)
(64)
frequency
sin(!t)
driving
+
) )s
sin(!t) of
with
f
2+ f ) the
h(⇣)
f
(c a
the
(c
optimum
additional
+ = ⇣/(⇣
possible
!. input
+s s
) (68)
(56)
) +p!
(64)
sigad
tems,
an
berg -agent strongly itcollectively
is
systems, often influence •]possible
it More ence,
ticity often and refs
In toand
as manipulate
these
possible In
[464,
case
nodes that
1 these large of465, leaders
2in to no large 466,
aphase
multi-agent ticity
dynamics graph can multi-agent
467, •strongly
as 468,
that nodesMore
and
In asystems, 469]
1
these
move a[476] influence
systems,
refs
in 2
bquadratic 1a
large [464,
graph
3
collectively
it is and
itrecent
465,
multi-agent
2often 4
cost is[333]that often manipulate
466,
afunction.
according move possible
467, systems, 468,
collectively
To ]to
?469]
extend ]it is often according possible 1
to negative 3 1 1 4 2 1 2 2 3 3 3 4
3 3 2 1 2 1
2 32 1 1 2 43 2 43 3 3 3 2 2
pulate ispossible to]to
⇤ ⇤ ⇤
ble
ent
ph move phase
ality
ly rding
in general
to of
systems,
according
that
according bto
networks
where
move and
demodulate
it is aoften
Network
1
signal
[474]
according
f,Dickinson
collectively is the
2 1
possible the
ticity that
cutoff
Dickinson
3 2
science signal.
as
[333]
according is 4
to
3
mostly
nodes
frequency. in
[335,
1
phase
1
[333]
4
Thegeneral
2
in
3 negative
327? 21
result
[335, atheory1 4graph
2
to
3
?There
2 •is
3
•demodulate
]327? to when
,Networkticity
4
[326]3
Dickinson thatticity
?•signal
demodulate
3 23
]result
3
as move is as 2 1
the to
nodes
that
science
2
nodes
2
[333] the
1
collectively
signal. in right
signal.
in
[335,
1 2 3
in
3 aphase
general
2 3
graph
The
34graph
327? The
2 4
according
3
result ?bthat
where
3
that
and
result ,demodulate
[326] move
is 2
move
a is signal
signal
is a[326]
thesignal collectively
collectivelythe that
that
cutoff signal.
that is mostly
frequency. according
phase according
The 1
where bto and
result 2
demodulate 3
isafwhen asignal 4
issignal thebthe is
that
cutoff to
that is
1the
signal. mostly
2
frequency. right Thenegative 1
result iswhen
1 2
a signal b is that to the r
33] [335, 327? ?optimal –minima [326]
All •Dickinson
scale-free [333]
networks •[335, where
are 327? sparse ?! ]the have ,is ,,optimal
[326]
Dickinson
the [326]
Dickinson cutoff
been [333]
frequency. [335, [335,
advances 327? 327? ?phase in ,470,
,argument.
[326]
understanding
[330], [331]
wide • •! •optimal There have been
[330], [331] recent advances • in understanding ! !
advances
remove 3,
8, ding
s[458,
or emotion
[457]
when
ale
454]
•towards
fact
459,
with
summed
behavior
that
460,
459,
• the
towards
leader
[458,
of 4in
Networkthe 461,
460,
The DC
the
understanding
phase
477],
with
across
component
462?
461,
nodes
degree
optimum
the
Sam control
isto 462?
the
manipulate
leader and
the
promising to
global
value.
,distribution
463] ,that
large-scale
thatmanipulate
entire
refs[458,
demodulate
463]
Aditya
point.nodes this
of
value.
rules
is To the
favorable
network
this
when
of to
This
•remove
[457]
that
4 The 459,
acoherent
(i.e., the the
signal,
network
large-scale
systems
trol
manipulate
demodulated to460,
degree Network(i.e.,
signal.
large-scale
governing
[458,
towards
the
global motion461,
the convection
•DC
distribution
manipulate
isthrough
with the
behavior
470,
phase Thewide
surrounding
462?
output
the
Sam rules control
the component
f
[458,
behavior
distribution
physical
nonlinear
to signal
480].
(67)
and
minima
large-scale
large-scale
global
463]
(i.e., is
to
of 477],
with the
due
(71)
Aditya
aand
is
The
refs[458, is
demodulate
network
governing with
large-scale
equations)
dynamics,
rules
then
ato
is
of
leader
Tofact
that
multi-agent
value.
of in-
signal
coherent
promising
how (69) this
behavior
[457]
inte-
459,
remove
(65)
leader
(i.e.,
that
isnodes
favorable
the
the
(71) 460,
that
signal,
physical
(65)
based
time-scales
motion
behavior
Wang
governing (65)
nodes
systems
the
towards
signal.
whenwith461,
distribution
that
of onthe
and
DC
the
[458,
462?
leader
[458,
equations)
that The
output
with
in
component
f
the
Krstic
physical
optimum
the
,477],
4 The
of 463]
optimal
result
nodes
(69)
how(65)
leader tois480].
past
is
degree demodulate
based(69)
equations)
point.
two
To
isnodes
that
promising of The
value.
adistribution
(69)(65)remove
on this
signal
This
fact
that the
signal,
when
based
that that
demodulatedof the
4towardssignal.
The
of athe on
DC
the
degree
network
the
phase
optimum
The
component
output
the distributionis
signal
result
optimal to
the isdemodulate isof
distribution
point.
is ofa this
value.
(67)
then asignal
This network
(71)
inte-
the
signal, ofthat
demodulated
signal.
how(69) is
(65) the
the (69)
(65) The result
output
distribution
signal
is of
is
is(69) a(65)
how
(57)
then
sig
ontrol
erning
ollable
many cale ns) ed)
75]
,
behavior is
hysical 463]
that either
basedonbased
coher-
refs[458,
physical
and
duced on mostly
equations)
with
on
birds with
•459,
Then,
Finally,
velocityleader
• positive
equations)
and Then,
Mesbahi
howwhen
Wang
460,
based
•thisthe
Then,
fish from to
Network
nodesmany 461,
signal
[328]
Wang
global
this
suchwhen
oftenon
high-pass work
the based
this 462?
that
signal a passes
signal
act rules
control
b
Biot–Savart
[328] [470]
network is
, on as is463]
filtered left
passes either
local (i.e.,
through
passes
decades. is of
refs[458,
wheneither is the governing
signal mostly
through
flows
law). controllableoptimum
the
• mostly
Finally,
such
In
to
459, is
Mesbahi
with
birds
Wang system,
Then,
particular, •global
positive
multiplied
the a 460, thephysical
Network
positive
the
network system,
large-scale
and [328]
Wang
point461,
system,
thisand rules when
high-pass
workfish signal
networks 462?
the
by
with
[328] when
and
is (i.e.,
equations)
control
[470]
often and
coher-out-
the
b , is
is
the
controllable 463]
passes
how governing
filtered
b
act either
left
the refs[458,
is
are out- as left
out-
many of based
through
often local mostly
the of
signal and thephysical
459, on
optimum
Then,
Finally,
flows
characterized is
the positive
460,
optimum
with this
Then, the
Then,
multiplied
system,
with equations)
461,
how point
signal
thiswhen
high-pass 462?
point
this
large-scale
bymany
and signal
by a ,
passesis
signal
b 463]
the
the based
is either left
filtered
passes
out-
coher- through
passes of on mostly signalFinally,
through Then,
through optimum
the positive
is system,
Then, this
multiplied
the
theThen,the point
signal
system, when
and
high-pass
system,
this this signal passes
the
by
signal
and
b is
andthe out-
filtered left
passes
the through
passes
the of
out- out- the
signal
through throughoptimum
the is system,
multiplied
the the pointand
system,
system, aniat
the
by
d through
net- networks Themany a theory Theis
high-pass
other • either
[471], is
theory nodes •
– relatively
considering • mostly
filter
Wang
Small-worldis
Scale-free
each of
relatively [328]
node thestraightforward
positive
sent many form
network-basedis through
net- straightforward
networks
other
connected enact when
h(⇣) nodes a a = bhigh-pass
larger
to; •to
is
The
⇣/(⇣ is analyze
left
[471],
each either
considering
this
fluid theory
–•
supervisory
+ is of
to •
node Wang
Small-world
!
flow often the
analyzemostly
filter
), in
Scale-freeisis the
optimum
[328]
relatively
connected
visualized
control. of
network-basedsent
inpositive
the
control the through
form net-
networks point
to; straightforward
when
protocol
this h(⇣) is afluid high-pass
b
often
= Theis
[471],
to is
⇣/(⇣ either
left
many
flowcreatetheory
visualized + to
of other mostly
filter
analyze
the
Scale-free
control.
! a ),is
system- of
optimum
nodes sent
relatively positive
the in eachthrough
theform
net- many
straightforward
point
node when
h(⇣) a
is other
high-pass
The
connected
= b is is
⇣/(⇣ either
nodes left
theory + toof
each
to;
! mostly
filter the
analyze
isthis
), node
of optimum
relatively
is positive
the
often is
in connected
formthe visualized point
straightforward when
h(⇣) to; = b is
this
⇣/(⇣ left
is often
+ of
to ! the
analyze
), optimu
visualized (70) in
protocol
ased ature Network bio-flight)
fluid ato
There flowcreate theory
•have
control.[455]
acreate
system- references:
been enact smany aestimate larger powerful Network⇤ supervisory
c⇤local advances rules
theory control (diffusion,
references:
in network protocol etc.) con- to as create well as
system- their external inputs (66)
⇤enact aJ(s) larger supervisory control protocol asto create aestimate
More refs [464,
local 465,
arules 466, (diffusion, af467,Extremum-seeking mathematics •1468, More caetc.) 469] refs as [464,well 465,
as their466, a467,
external 468, af469]
inputs 1(55) fsystem- (68)
ry
466, 467, ⇤
control
and 468,
467, 469]
protocol
468,
signal 469] that enact
to siscreate local
mostly larger
3a leveraged
rules system- negativesupervisory (diffusion, bcb̂3singular when control
aetc.) local
perturbation
is local
to as cprotocol
the brules
f
well rules
cfright fas cfto
(diffusion, theory theircreate
(diffusion, ⇤
and
external a(72)
and system-
etc.) etc.) separation
a)strongly as
inputs
signal as well well that ofas
(70)
is their their
mostly external externalnegative ⇤
inputs b̂4inputs
f f1rightcfrequencyafhas cfsin(!t f(72)3c cf1b̂4(s c⇥ c(66) (70) f
2and caoriginal
when signal athat
is to isthe mostly negative when
well
stem-
ontrol
ternal nputs
c.)nal as
inputs their
•inputs
protocol
well grated
put
original external
as to into
their
put also
perturbation our inputs
external aalso
has best
system-
also c1sinusoidal
sinputs has a and
sinusoidal of )perturbation
and with signal3, c driving
bsignal a fthat
possible
perturbation (68) the
that fis
at = input
b̂nodes
mostly
frequency
(72) +
additional
is fmostly atM signal sin(!t)
cnegative
frequency (66) negative (66) when grated
put
original when is into
put ato also
perturbation
is our
the
to
f
(66) has
the best
sright also a
rights+ (70)
sinusoidal
(66)has asis
⇥ a)fmostly c1s1!.
sinusoidal of grated
put
perturbation
cs2112with ⇤ c
,J(s) driving
b3put into
perturbation calso
fcpossible our cthe
fat bbest
has input
1additional
also estimate 1 signal
sinusoidal
at
has afrequency f+ ac3networkssinusoidal fc4of (70) b2f⇤!.
)perturbation
with c4,3all(66)
driving is
cb3ascale-free to
c4 the
c4possible
perturbation at right
the input
frequenc
additio sig
isat tofr
f (66)
bput has cbsinusoidal original perturbation perturbation at 2frequency with bput ano possible also additional has 3sinusoidal fcfput
b3perturbation also bsat frequency sinusoidal perturbation bat freq
d b [379] J(s) sEurika and ⇥ Bernd sin(!t c+ [379] + 3leader cb4collection putor Eurika “driver” also
and chas
Bernd (70)in
fcto 4[379] bthe
sinusoidal !. ff system [480].
perturbation b⇤as
J(s) A
binfluence key observa-
at (70) frequency sin(!t1bthe + c)f
fhistogram. bindicate fJ(s) 2
(56) f!. + (58)
leader
J(s) a+
J(s) or “driver” 1c nodes ⇥ cin the bcall ence, csystem and [480].that A sin(!t
leaders key observa-
c(represented
3+can c466,
!. 4!. J(s) J(s) band manipulate J(s) !.
2
ystem [480]. aence, A
Inhistogram.
these and
key that
•observa-
large leaders
multi-agent f can stronglyf465,
systems, isf •influence
it is often and possible manipulate fthe =and (s snegative
2
erva-
ere 464,
manipulate case 465,
f] ,is of 466,
the
case
no 467,
cutoff
of
dynamics
⇤
no and 1468,
frequency.
dynamics More
2469]
andsignal refs
ascience and that
quadratic
1 4 [464,
a ais2quadratic
2
mostly large 466,
case negative
4
cost 467,
of ⇤
no
function.
and 4•468,
1 1
when
dynamicsMore
of a[331] 469]
2
individuals
signal 2
brefs
To
3
is11[331]3
extend to
where
and
that [464, 4 2
athat is 465,
quadraticright
mostly is case the 467,
negative cutoff
costof 468,
by and
function. 469]
nodes),
histogram.
a signal
frequency.
dynamics
when bTo that
where
is Recent
that
extend
to a quadratic as
results
fright isacase the cutoff
ofcost
22
⇤
no and 33
4 4Recent
that
frequency.
function.
dynamics awhen 44
all
signal = results
scale-free
To andis
that (c to2a
extend indicate
+
is
the quadratic
21
mostly 2right
3
2that
2 )) negative
cost function. when networks
To exte
where fcost the function. cutoff To
frequency. extend
2 3 2 3 4
74]
,7? 473], !
?in- as
Universality
[326] Network of Recent
works
networks results
[472, as [474] 473], in histogram.
indicate ! general
wide
2
Universality that
minima Recent 4
Network scale-free
that ofresults
works
There networks is 1 indicate
science
networks
[472,
favorable
have [474] 473],in
3
been general
[458, ! 4
fall
Universality recent scale-free
470, 480].
advances networks
of Thenetworks factinThe that [474]
understanding ! 1 1 2 1
n re nding
vorable
rk e on8,
ble recent
of
to
that 470,
sparse
the
(i.e.,
network
due due[458,
Sam to 480].
to advances
trol
[458,
optimum [476]
convection
towards
phase
in- 470,
To
and in- theory
(i.e., The
470,
removeTo
Aditya••
480]. to
convection
To [330],
fact
480].
point.
the
removein
summed
surrounding
wide
due • The the
demodulate
remove[457] that
[330],
summed
•optimal to There
[331]
understanding
[330],
wide minima
The
Thisfact
DC in-the due [331]
across
thefact minima
demodulated
time-scales
that [331]
value.
component
DC have
theto DC multi-agent
acrossthat in- of the
signal.
component Sam componentbeen
theis that of the summed
entirefavorable
argument.
tion
and optimum
the of is
entire
phase 4The recent
signal wide
in favorable
thisnetwork
systems
optimum
Aditya •
To
•
of
this result across
[330],
summed
network
ofsignal,minima
is[458,
this
remove •advances
summed
point.
to
line this •
then
[457] [330],
in
is(i.e.,
point.
signal,
demodulateofthe
[330],
[458,
470,a the
signal, entire
that
across
(i.e.,
inte-
This
signal
research
the [331] 470,
convection
past
480].
across outputThisin
the
DC isthe
convection
demodulated the
that
the understanding
twonetwork
480].
favorable
The
the of
demodulated
output
is
component entire
is
output
that signal. thedue
fact
entireThe large, is (i.e.,
due
optimumto
network
phase fact
[458,
that
towards
To
is The in-
network
signal to
of convection
that
remove
sparse 470,
in-
signal
To result
this To (i.e.,
point.
is
to the 480].
(i.e.,
removethen the
demodulate
is
remove
is
4networks
signal,
(71) thenadue
convection
convection
optimal This
inte-
DC the
signal the to
inte-
the fact
demodulated
component
with DC in-
value.the of
DC
that
output that
due the due
signal.
component component
are to optimum
is to
in-
phase towards
To
inherently of in-signal
The remove
this To of To4to
result point.
of theis
signal,
remove
The this
sparse, then
theoptimal
demodulate
remove
this is
degree This
signal, aDC
theinte-
signal,
with signal
the thedemodulated
value.
output
component
DC
distribution
(69) thethe
heterogeneous DC
the that signal.
component
output is
component
output of a of is
(67) signal
The
network
degree this
is of result is
signal,
of thisis then
this
distribution. is
the a inte-
the signal
signal,
signal,
distributio (71)
outp
(67)
the th t
tlg with
sing large,
•
Finally,
this
refs[458, when •tosparse
manipulate
are Mesbahi
the
this the
systems
459,
4 The
inherentlyoflarge-scale
networks
high-pass
to degree
the
systems
•460, Networktion
work
the
with 461, sparse,
optimum in [470]
with
large-scale
distribution
filtered
with
nonlinear
462? this
coherent with
control ,fishline point.
nonlinear
463] are
signal •
heterogeneous of
ofbehavior motion
inherently
refs[458,Finally,
dynamics, research
a This network
• Then,
each
is
birds Mesbahi
multiplied
dynamics, this [458,
demodulatedthe
459,
and with
The
execute sparse,is is
degree
Wang to that
of
high-pass477],
degree
the
460, the
Network
fish work
leader
systems
the with
bylarge,
Wang large-scale
461,
and
often (69)
is
distribution
their
distribution. the
optimum [470]
promising
distribution nodes
heterogeneous
signal
462? sparse
own
and
with
filtered
control
Krstic
act of
is that
set
Krstic
,large-scale
463] networks
coherent
of (71)
nonlinear
point. then
signal how when
(67)
refs[458, a of Finally, network
degree local
inte-
This (67)
is motion
(67) with
Crossover,
dynamics,
this
multiplied
459, the
demodulatedis
distribution.
protocols the
460, [458,
of
high-pass
to are distribution
systemsthe
461, Wang
by The
477],
inherently
C
in (67)
optimum
the
462? degree
is
response
filtered
signal with
and promising
(71) of
, 463]sparse,
isdistribution
how
(71)(67)
point.
signal
nonlinear
Krstic Finally,
then with This
inte-whenis this of
heterogeneous
multiplied
the
dynamics, a
demodulated network
high-pass
to ofsystems the by is
degree
optimum
Wang the thesignal
filtered
with distribution
distribution.
andnonlinear point.
is
(57)
signal
Krstic then This isinte- of how
(71)demodulated
multiplied
dynamics, (71)
(67) by
Wang the signal
and (59) is
Krs th
vart
eview orkws
asmanyis [470]
with
controllable
locallaw). paper large-scale
decades.
flows [475]
The with and In when
duced
birds
theory coher-
with
•particular,
large-scale
– Mesbahi birds
aduced
Network andis such
how
velocityrelatively and
velocity work
aofoften
many
networks
coher-
review network
fish from ⇤ [470] often
from
straightforward actduced
paper the are as is act
the birds
Biot–Savartlocal
often
[475] this
•controllable
velocity as when
duced
Biot–Savart signal
and
local –•create
flows
duced
characterized Mesbahi
Network
to from
fish such
flows
law). passes
with
velocity
analyze and
often
velocity the
law). aas with work
reviewlocal
network
through
with
Biot–Savart
from
in act by from
the aas
large-scale [470]howflows
the local
paper the is
the coher- many
Biot–Savart with
controllable
system,
law).
flows
Biot–Savart
[475] coher- large-scale
The with and law).
theory and
law).the
large-scale coher-
is with
out- relatively how
coher- ⇤ many straightforward many
The theory other to nodes
analyze
is relatively each in node
the isis connected to;form this ish(⇣)
often v
ocal oher- Biot–Savart flows is sentwith
either law).through
sent large-scale
sent mostly through through high-pass
positive coher-a a
high-pass high-pass when filter of
bfilter is filter
isthe left of
either form of the the form
mostly optimum
form positive point when is sent either
left through
sent
of sent mostly
the through
through
optimum asignalhigh-pass
positive a a
high-pass when
high-pass
point filter of
is
bfilter sent
is the
either
isleft
filter of through
form
sent of mostly
sent
the the through
formthrough
form a+
optimum high-pass
positive a a
high-pass pointwhen
high-pass filter f,straightforward of
filter filter theleft of formof the the form to analyze
optimum poin
⇤visualized
grated net- into our best – estimate Small-world grated , driving
networksgrated many into into the
sent
our
other input
[471],
our through
best
nodesh(⇣) – best signal
Small-world
estimate =each a
Scale-free
estimate
⇣/(⇣
h(⇣) high-pass
node +
= of
! is
⇣/(⇣ gratedconnected
), net-
networks
of⇤
, + filter
driving ⇤
f! , into
driving), of to; our
thethe [471],
this best
many
form
theinput
is estimate
often
input other
Scale-free visualizednodes
signal ofgrated
each net- , driving
node into our
connected the best input h(⇣) estimate
to; signal
this
= ⇣/(⇣
h(⇣) is of
often=+ ! ⇣/(⇣ ), driving b + ! f ), the input h(⇣) signal
= ⇣/(⇣ +
=
ory
465,dvances
eral
an general
serva-
ystems,
-agent
cience ulate
ee n
inal
ble
rongly
possible
474] Other References Other References
Eurika
•leveraged
heterogeneous
sinfluence
in the
strongly
466,
to
possible
towardsnetworks
enact
references:
perturbation
•systems,
in
wide it b
many
leveraged
in
Network
467,
large
to
as is
case
influence
considering
⇤ and
system
where
the
general
a
network
and
influence
and
to
a 468,
often
other
larger
grated
singular Bernd
collection
of•
where
optimal
histogram.
minima
are ait
!
There no
fNetwork
manipulate
[480].More
where 469]
andis
nodes
sin(!t
1science
degree
ence,
possible
signal
supervisory
singular
leaderinto
con-
dynamics
is•often
that!
Network
and A2[379]
fperturbation
refs
manipulate
the
value.
Network
have !
Network
ence,
In key
and
of
that
is
Recent
works
+each
network-based
our
fin
cutoff
is
theory
3perturbation
distributions
or
these
[464,
manipulate
to
possible
is
the In +
isthat
individuals
the
favorable
been
bestnode
)“driver”
generalsthese
observa-
and and
b̂
f1with
original
cutoff
[472,
theory
4large
results
control
465,⇥
frequency.
mostly
•
cutoff
science
references:
theory
that
leaders a
b
sestimate
to
is
sput2 1a466,
Eurika
theory
large
connected
sence,
quadratic
473],
towards
recent
to
frequency.
indicate [458,
4
nodes
•1multi-agent
leaders
fluid
negative
protocol
references:
possible
sperturbationreferences:
3Network
sJ(s) (such
as
frequency.
towards in 2 can
theory
external
s467, leveraged
multi-agent
(represented 2a
general
Universality b
and
sb̂⇤sIn
advances the
470,
flow
4ence, of
3also
in s•
can
strongly
histogram.
and
that 468,
to;
as
cost
where
3additional
these grated
bBernd
that
awhen
the
⇤
optimal
the
to
and
480].b
this
considering
control.
Network
leader
sMore
all
stimulus.
4,separation
a
has + driving
science
sparse,
and
strongly
469] 4leaders
system
ssystems,
function.
= signal
optimal
•
is
Network
singular
ssin(!t
large
f1 aa[379]
(70)
In
by
scale-free
!
into
that
influence
refs
systems,
Network
of
b
There
in The
1
often
Network
⇥separation
works
or
In
is
(s these
Recent
is
h(⇣)
sinusoidal
s+in a
nodes),
networks
value. 2
theory
the
scale-
these
to
our
[464,
)
that
the it
+
visualized
system-
can
“driver”
+ general
[480].
2multi-agent
leaders
Tos+
influence
the
value.
fact
understanding
⇥
have
This
of
sis 1
=
network-based
large
networks
theory
perturbation
3input
best
results
[472,c is
cutoff
)and
it
⇣/(⇣
originalb̂
strongly
often
that
1 large
465,
extend
right
s
science
(72)
⇥ references:
theory
with
sA
is s(68)
that
mostly 1 s4of
2analogy
can
[474]
473],
been
b̂
f b
signal
estimate
nodes
perturbation
keymanipulate
multi-agent
often 466,
towards
2
+
heterogeneous
and
indicatespossible
multi-agent
frequency.
3
in
!
references:
asystems,
(68)
breferences:
perturbation
possible
observa-
strongly 467,
b
(68)
influence
possible
negativegeneral
Universality
recent
),
sin
manipulate leveraged
f
4theory
b
holds
case
fluid
the
b̂468,
⇤that
where
the
ofinfluence
to
and it system
at
systems,
of
all
⇤
is
where
optimal
flow
quite
badditional
grated ,as
and
systems,
469]
to
when
advances a ! no
driving
frequency
degree
often (56)
signalais
scale-free
where of
control.
(72)
singular
ainto
dynamics
the
h(⇣)
well separation
(68)
manipulate
sin(!t [480].it
histogram.
networks
b! is
value.!
f
it
is
that
isto
our
the
+ b
cutoff
=
for
distributions
possible
and 1 (72)is A
!.
manipulate
often
networks
inwhen is
the =
the
⇣/(⇣
original
perturbation
f )best
input
often
is
the
(72)
key
2
and
understandings
b̂
awith
(68)
cutoffto f +
of
possible
Recent
right
[474]
1 (c
frequency.
mostly cutoff
towards a
b
estimate
3signal
observa-
heterogeneous
possible +
!
aperturbation
2quadratic+ s f
f