Paralel CFD 2007, May 21-24, 2007, Antalya, Turkey.

Published by Springer Verlag

An Approach for Parallel CFD Solutions of Moving

Boundary Problems

E. Oktaya, O. Merttopcuoglub, and H.U. Akayc

a
EDA- Engineering Design and Analysis Ltd. Co.
Ankara, Turkey
b
ROKETSAN -Missile Industries Inc.
Ankara, Turkey
c
Department of Mechanical Engineering
Indiana University-Purdue University Indianapolis
Indianapolis, Indiana 46202 USA

ABSTRACT

A new fast and accurate parallel algorithm is developed for solution of moving body
problems, with specific reference to store separation problems. The algorithm starts
with development of separate meshes for the moving body (store) and the aircraft wing,
which are then connected by using mesh blanking and mesh filling algorithms
automatically. Following the partitioning of the connected meshes for parallel
computing and obtaining a steady state flow solution, the separation starts by using a
dynamically deforming mesh algorithm coupled with the six-degree of freedom rigid
body dynamics equations for the store. The solutions continue until severe mesh
distortions are reached after which automatic remeshing and partitioning are done on a
new mesh obtained by blanking and filling operations to continue with the solutions. As
the store reaches far enough distances from the aircraft, the algorithm switches to a
relative coordinates eliminating any need for mesh deformations and remeshing. The
developed algorithms and the results are discussed with a sample problem, including the
parallel efficiency on distributed computers.

Keywords: Store Separation, Moving Boundary, Parallel CFD, Moving Grid.

1
1. INTRODUCTION

Moving body flow problems such as the simulation of the separation of an external
store from underneath of an aircraft wing to compute its influence on the performance
of the aircraft are inherently difficult to solve due to excessive computations and
memory needed as well as excessive deformations of the meshes. The problem is further
complicated for parallel computing when the meshes have to be moved during the
computations spoiling the connectivity and load balancing of the solution blocks. Yet,
these calculations are important since aerodynamicists need to predict the movements of
stores before flight tests, to avoid any disasters emanating from faulty designs. Store
separation problem is essentially a simulation problem of multi bodies in relative
motion and the determination of forces and moments acting upon the moving body
because of the airflow and its consequent displacements. The modeling of this problem
requires the interactive solution of unsteady flow and rigid-body equations together.

In this paper, a new fast and accurate method to solve an external store separation
problem in parallel computing clusters is presented. First, the moving-boundary
problem is solved using deforming mesh algorithm and ALE formulation [1-2] on pre-
partitioned blocks starting from steady state solution, whose input comes from the
dynamics of the rigid-body, and the resultant unsteady fluid flow around the body is
determined. Then, using the resultant force and moment terms in the rigid-body
equations, the motion and the trajectory of the body is calculated. The solutions are
carried out in this manner, as long as the solution mesh is not distorted. Whenever the
mesh distortion reaches a specified value which the solver can not handle, solution is
proceeded by generating a new local mesh automatically and the solutions on the new
mesh are calculated. When the moving the body reaches a position sufficiently far from
the effect of the fixed body, our parallel solver (FAPeda) [3] automatically switches
itself to the relative coordinate solution [4-5]. In this way, there is no need for either
deforming mesh or blanking and remeshing. The solution strategy and the results are
discussed with a sample problem, illustrating the accuracy of the algorithms as well as
the parallel efficiency on distributed computers.

2. GENERAL APPROACH

For the method developed, meshes for the moving and fixed bodies are generated
separately, after which they are overlapped to form an overlapped mesh as in the overset
or Chimera mesh method [6-7]. However, the instead of interpolating between the two
meshes at the overlapped regions, our method further blanks the overlaps and connects
the two meshes. One of the major contributions of this study has been to develop very
fast automatic mesh generation in small empty regions occurring after blanking of a part
of the mesh occupied by the moving body and transferring previous solution space on to
the newly created mesh accurately using a dual time approach.

2
It has no interpolation requirement that is a source of inaccuracies in both space and
time integration of Chimera mesh method. It has no difficulties in the case of multi
body overlapping as in chimera method. Blanking process requirement is less than that
of chimera method because of the deforming mesh algorithm. Since deforming mesh
algorithm handles small displacements, it does not require blanking and consequently
mesh generation. With these features, this algorithm is very appropriate for moving
boundary solutions.

Furthermore, the code can automatically switch itself to relative coordinate solution
methodology when the moving body is sufficiently far from the effect of the fixed body.
In this case, neither mesh deformations nor blanking and meshing are needed.
Therefore, the solution becomes very fast. This part of the solver is useful for tracking
the store rather than the separation problem.

Simulation of the flow is realized in terms of unsteady solution of Euler equations,

while a domain partitioning parallel solution strategy is used. The Euler solver is based
on a cell-centered finite volume scheme [8] and uses backward-Euler implicit
integration method [1-3 and 9]. The solver uses an unstructured mesh, which is
continuously deforming according to a spring-analogy scheme as the distance from the
wing to the load increases.

3. GOVERNING EQUATIONS

Euler equations for an unsteady compressible fluid flow in an Arbitrary Lagrangian

Eulerian (ALE) system can be expressed in integral form as follows [1]:

∂ ⌠ ⌠⌠ ⌠⌠
⎮ ⎮ ⎮ Q dV + ⎮⎮ F ⋅ n dS = 0 (1)
∂t ⌡⌡⌡ ⌡⌡
Ω ∂Ω

where, the velocity of the fluid is expressed as relative to the local velocity of the mesh
and
⎡ ρ ⎤ ⎡0⎤
⎢ ρu ⎥ ⎢n ⎥
⎢ ⎥ ⎢ x⎥
Q = [ρ , ρu , ρv, ρw, e] , F ⋅ n = [(V − W ) ⋅ n] ⎢ ρv ⎥ +
T
p⎢ ny ⎥ (2)
⎢ ⎥ ⎢ ⎥
⎢ ρw ⎥ ⎢ nz ⎥
⎢⎣e + p ⎥⎦ ⎢⎣Wn ⎥⎦

Also, in the above n = n x i + n y j + n z k is the unit vector normal to the surface,

V = uj + vj + wk fluid velocity vector, W = i ∂x ∂t + j ∂y ∂t + k ∂z ∂t is the grid
velocity vector, and Wn = W ⋅ n = n x ∂x ∂t + n y ∂y ∂t + n z ∂z ∂t is the component of grid

3
velocity normal to the surface, p is pressure, and e is the total energy. From the gas
law, the pressure is related to the total energy as follows:
⎡ 1 ⎤
p = ( γ − 1) ⎢e − ρ(u 2 + v 2 + w2 )⎥ (3)
⎣ 2 ⎦
For the movement of the moving body (store), the dynamic equations of the rigid-body
are express as:
dVc
m( + ω c ×Vc ) = mg + Fc (4)
dt
dVc
I + ωc × ( IVc ) = M c (5)
dt

Here, Vc and ωc are velocity and angular velocity vectors of the moving body,
respectively. Fc and M c are force and moment vectors affecting on the body and
calculated by following surface integrals, respectively:

Fc = ∫∫ pndS (6)
S

M c = ∫∫ pr × ndS (7)
S
where p is pressure distribution on the moving body surface S.

4. DEFORMING MESH ALGORITHM

The deforming mesh algorithm models the mesh movements of interior points to
conform to the known movements of boundaries. The algorithm used in this study has
been previously developed by Batina [10]. In this algorithm, the computational mesh is
moved to conform to the final position of the boundary. The algorithm treats the
computational mesh as a system of interconnected springs at every mesh point
constructed by representing each edge of every cell by a linear spring. The spring
stiffness for a given edge is taken as inversely proportional to the length of the edge.
The mesh points on the outer boundary of the domain are held fixed while final location
of the points on the boundary layer edge (i.e., moved Euler grid points on the body) is
given by the body motion. The static equilibrium equations in the x, y, and z directions
that result from a summation of forces are solved iteratively using Jacobi iterations at
each interior node i of the mesh for the displacements ∆xi , ∆yi and ∆z i [1].

4
5. MESH BLANKING AND MESH FILLING OPERATIONS

The minor unstructured grid covering the moving body firstly overlaps with the major
unstructured mesh including the other fix body. To create new mesh after the distortion
caused by excessive mesh deformations, the overlapped portion of the major mesh with
the minor mesh is blanked. It is obvious that, there is an unavoidable requirement of an
efficient and reliable search algorithm if the significant number of elements of such a
three-dimensional mesh is considered for a time dependent solution. For this purpose,
the neighbor-to-neighbor jump search algorithm [7] is efficiently utilized in the present
method. The procedure is schematically shown in Figure 1 for a triangular grid. The
only insufficiency of this efficient search method is that it may fail for convex domains
depending on the starting point. This problem has been overcome by adding subsidiary

B
C

A Target

Figure 1: Neighbor-to-neighbor search

mesh points into the moving body and outside of the computational domain [7].
The small empty region occurring after blanking of a part of the mesh occupied by the
moving and fixed bodies is filled by tetrahedral cells based on Delaunay triangulation
[11]. In this method, the local element lengths are automatically set using boundary
triangles and dimension of the hole. The mesh blanking and filling algorithms are
depicted in the schematic shown in Figure 2. The solution domain on the blanked region
is first predicted from previous solution using the quadratic Shepard method for
trivariate interpolations [12] and then corrected with respect to external flux distribution

(a) (b)
Figure 2: Mesh coupling stages: a) overlapped mesh, b) blanked mesh to be filled later

5
using the Lagrange multiplier method with the dual time stepping approach. This way,
flux conserving in the remeshed region is computed without any interpolations.

6. PARALLELIZATION OF THE CODE

For parallelization of the code, a domain-decomposition approach is used, where the

flow domain is subdivided into a number of subdomains equal or more than the number
of available processors. The interfaces serve to exchange the data between the blocks.
The governing equations for flow or mesh movements are solved in a block solver,
which updates its interface solver with newly calculated nodal variables at each time
step. The interface solver in each block, in turn, communicates with the interface solver
of the neighboring block for the same interface. Each interface solver also updates its
block after receiving information from its neighbor. In each block, the nodes on the
interfaces are flagged as either receiving or sending nodes. The ones on the interior side
of the blocks are sending nodes; the ones on the exterior side are receiving nodes. The
sending nodes send their data to the corresponding nodes in the neighboring blocks; the
receiving nodes receive the data from the corresponding nodes in the neighboring block.
The communication between the blocks is achieved by means of the message-passing
library MPI. The computational domain that is discretized with an unstructured mesh is
partitioned into subdomains or blocks using a program named General Divider, which is
a general unstructured mesh-partitioning code [13]. It is capable of partitioning both
structured (hexahedral) and unstructured (tetrahedral) meshes in three-dimensional
geometries. The interfaces between partitioned blocks are of matching and overlapping
type. The interfaces serve to exchange data among the blocks. The partitioning is done
following the merging of moving body and fixed-body meshes by blanking and
remeshing operations yielding a nearly balanced solution blocks. The same block
structure is used during mesh deformations via ALE until severe distortions are reached
at which stage a new merging operation is performed by repeating blanking and
remeshing operations followed by partitioning.

7. TEST CASE

For the validation purpose of present method, simulation results are compared with the
wind-tunnel test data of AFRL-WPS model (Air Force Research Laboratory - Generic
Wing, Pylon, and Moving Finned Store) which are given in Reference [14]. Agreement
between solution results and test data are found to be satisfactory. Figure 2 shows the
mesh coupling stages around the WPS configuration and Figure 3 shows various store
positions during separation and pressure distribution on the surfaces. The comparisons
of the computed trajectory with the experimental data for the store’s center of gravity
are shown in Figure 4, in terms of the linear displacements and Euler angles, showing
good agreements. Parallel efficiency of the code using up to 6 Intel P4 dual core
processors, with 3.2 GHz clock speed and 2 GB RAM memory are shown in Figure 5,
showing a reasonably good speedup for even a mesh of 400,000 finite volume cells.
While the problem with a single processor took 1731 seconds of elapsed time for 100

6
number of time steps the 12 processor (each core has been used as an individual
processor) solution took only 219 seconds, which translates into 65% efficiency.

Figure 3: Computed store positions and pressure distribution

Figure 4: Computed and measured trajectory of the store

Figure 5: Parallel efficiency of the code

8. CONCLUSIONS

The method developed here for parallel computing of store separation problems yields
accurate and fast solutions. The combined blanking, filling, and partitioning algorithm
provides significant advantages over the classical overset algorithm, since no
interpolations are needed between moving and fixed meshes in this case. The parallel

7
efficiency is achieved by load balancing following each remeshing stage of the
solutions.

9. REFERENCES

1. Oktay, E., and Akay, H.U. and Uzun, A., “A Parallelized 3D Unstructured
Euler Solver for Unsteady Aerodynamics,” Journal of Aircraft, Vol. 40, No. 2,
pp. 348-354, 2003.
2. Akay, H.U., Oktay, E., Li, Z. and He, X., “Parallel Computing for
Aeroelasticity Problems,” AIAA Paper: 2003-3511, 33rd AIAA Fluid
Dynamics Conference, June 20-23, 2003, Orlando, FL.
3. Akay, H.U., Baddi A., Oktay, E., “Large-Scale Parallel Computations Of
Solid-Fluid Interaction Problems For Aeroelastic Flutter Predictions,” AIAC-
2005-002, Ankara International Aerospace Conference, August 22-25, 2005,
Ankara, Turkey.
4. Kandil, O.A. and Chuang, H.A., “Computation of Steady and Unsteady
Vortex-Dominated Flows with Shock Waves,” AIAA Journal, Vol. 26, pp. 524-
531, 1998.
5. Hi, X., “Parallel Computations of Solid-Fluid Interactions Using Arbitrary
Lagrangian-Eulerian and Relative Coordinate Formulations,” Master’s Thesis,
Purdue University, May 2004.
6. Benek, J.A, Buning, P.G., and Steger, J.L., “A 3-D Chimera Grid Embedding
Technique,” AIAA Paper 85-1523, June 1985.
7. Nakahashi, K. and Togashi, F., “Intergrid-Boundary Definition Method for
Overset Unstructured Grid Approach,” AIAA Journal, Vol. 38, pp. 2077-2084,
2000.
8. Frink, N.T., Parikh, P., and Pirzadeh, S., “A Fast Upwind Solver of the Euler
Equations on Three-Dimensional Unstructured Meshes,” AIAA Paper, 91-
0102, 1991.
9. Anderson, W.K., “Grid Generation and Flow Solution Method for Euler
Equations on Unstructured Grids,” Journal of Computational Physics, Vol.
110, pp. 23-38, 1994.
10. Batina, J.T., “Unsteady Euler Algorithm with Unstructured Dynamic Mesh for
Complex Aircraft Aerodynamic Analysis,” AIAA Journal, Vol. 29, No.3, pp.
327-333, 1991.
11. Karamete, K., “A General Unstructured Mesh Generation Algorithm with its
Use in CFD Applications”, Ph.D. Thesis, METU, Dec. 1996.
12. Renka, R.J., “Quadratic Shepard Method for Trivariate Interpolation of
Scattered Data”, Transactions on Mathematical Software, Vol. 14, No. 2, p.
151, 1988.
13. Bronnenberg, C.E., “GD: A General Divider User’s Manual - An Unstructured
Grid Partitioning Program,” CFD Laboratory Report, IUPUI, 1999.
14. Fox, J.H., “Chapter 23: Generic Wing, Pylon, and Moving Finned Store,” In
“Verification and Validation Data for Computational Unsteady
Aerodynamics”, RTO Technical Report, RTO-TR-26, 2000.