Adaptive Mesh Refinement and High Performance Computing for Fluid Dynamics

Summary: In order to integrate partial differential equations from fluid dynamics at extremely high resolution, a numerical framework has been developed to obtain high precision solutions using an adaptive grid that increases the numerical resolution locally as needed. This framework is usable on massively parallel machines and shows close-to-optimal scaling on up to $10^5$ CPUs.

In fluid dynamics, physical features are often localised. In particular in a setup such as in this project, highly detailed flow structures are clustered in a very small region, while the majority of the computational domain is filled with relatively harmless flow. Yet, due to the non-local nature of incompressible fluids, even far-away parts of the flow may have important impact locally. In such a case, adaptive mesh refinement, as introduced in the pioneering work of Berger and Colella (J. Comp. Phys., 82, 64, 1989), is a natural choice.

In the implementation presented here, based on the numerical framework racoon II, a block-structured adaptive mesh refinement is used: The computational domain is devided into equally sized blocks. Whenever a chosen refinement threshold is exceeded, a block is subdevided into $2^d$ child blocks, where $d$ is the domain dimension. Similarly, child blocks are merged when the threshold is no longer exceeded. The resulting octree-structure allows the grid to adaptively follow the computationally expensive structures, while remaining relatively simple: To the program, regardless of the position and resolution, each block locally looks identical. Its dimensions may be optimized, for example, towards hardware requirements such as the cache size. This provides performance-advantages over the usually more flexible patch-based or unstructured refinement techniques.

AMR Hilbert curve AMR Flow

Above, a snapshot of a simulation of a Rayleigh-Taylor instability is shown as example, color denoting the fluid density. The blocks of the grid, shown on the right, follow the physical structure of the flow, resolution is highest at points where the gradients get large.

Parallelization is achieved by distributing the blocks among the available processors by traversing a space-filling Hilbert curve. This not allows for a mapping between the block octree-structure to a linear processor number, but optimizes for locality of the blocks: Blocks lying close together in the computational domain have a high probability of also being close along the Hilbert curve and thus tend to reside on the same compute node. This is depicted on the left of the above figures, where color denotes the processor ID.

Scaling

The computational framework can be used to solve three-dimensional partial differential equations for hydro-dynamics or magneto-hydro-dynamics on massively parallel machines. The above figure shows the parallel scaling (mixed weak and strong) up to $\approx 2\cdot10^5$ CPUs. Up to this number, the scaling is close to optimal: Doubling the computational ressources roughly halves the amount of time the computation takes.

Relevant publications

  1. T. Grafke, H. Homann, J. Dreher and R. Grauer, "Numerical simulations of possible finite time singularities in the incompressible Euler equations: comparison of numerical methods", Physica D 237 (2008), 1932

  2. T. Grafke, "Finite-time Euler Singularities: A Lagrangian perspective", PhD thesis (Jun 2012)

European Physics News Highlight: Instanton filtering for the stochastic Burgers equation

T. Grafke, R. Grauer and T. Schaefer, European Physics News (Highlight), 43 No. 3 (2013).

Abstract

Understanding intermittency in turbulent systems is still one of the open problems in classical physics. Since intermittency is governed by the non-Gaussianity of rare fluctuations, instantons might offer a way to better understand the behavior of turbulent systems. In the present work we concentrate on rare fluctuations in Burgers turbulence and we address the question whether one can identify instantons in direct numerical simulations of the stochastically driven Burgers equation. This is of special importance since this demonstrates that instantons indeed form the skeleton of rare turbulent fluctuations. For this purpose, we first solve the instanton equations using the Chernykh-Stepanov method [Phys. Rev. E 64, 026306 (2001)]. These results are then compared to direct numerical simulations by introducing a filtering technique to extract prescribed rare events from massive data sets of realizations. Using this approach we can extract the entire time history of the instanton evolution, which allows us to identify the different phases predicted by the direct method of Chernykh and Stepanov with remarkable agreement.


Link to EPN-Homepage

Turbulence properties and global regularity of a modified Navier-Stokes equation

T. Grafke, R. Grauer and T. Sideris, Physica D 254 (2013) 18.

Abstract

We introduce a modification of the Navier-Stokes equation that has the remarkable property of possessing an infinite number of conserved quantities in the inviscid limit. This new equation is studied numerically and turbulence properties are analyzed concerning energy spectra and scaling of structure functions. The dissipative structures arising in this new equation are curled vortex sheets contrary to vortex tubes arising in Navier-Stokes turbulence. The numerically calculated scaling of structure functions is compared with a phenomenological model based on the She-Lévêque approach. Finally, for this equation we demonstrate global well-posedness for sufficiently smooth initial conditions in the periodic case and in R³. The key feature is the availability of an additional estimate which shows that the L4-norm of the velocity field remains finite.


doi:10.1016/j.physd.2013.03.007

arXiv

EPN Instanton filtering for the stochastic Burgers equation

Extreme events in stochastic nonlinear systems play an essential role in nature. Characterizing their likelihood is a fundamental albeit challenging problem since the tails of the underlying probability distributions are usually non-Gaussian and governed by saddlepoints of the corresponding path integrals, so-called “instantons”.

Link

Instanton filtering for the stochastic Burgers equation

T. Grafke, R. Grauer and T. Schaefer, J. Phys. A 46 (2013) 62002 (fast track communication).

Abstract

We address the question whether one can identify instantons in direct numerical simulations of the stochastically driven Burgers equation. For this purpose, we first solve the instanton equations using the Chernykh-Stepanov method [Phys. Rev. E 64, 026306 (2001)]. These results are then compared to direct numerical simulations by introducing a filtering technique to extract prescribed rare events from massive data sets of realizations. Using this approach we can extract the entire time history of the instanton evolution which allows us to identify the different phases predicted by the direct method of Chernykh and Stepanov with remarkable agreement.


doi:10.1088/1751-8113/46/6/062002

arXiv

Finite-Time Euler singularities: A Lagrangian perspective

T. Grafke and R. Grauer, Appl. Math. Lett. 26 (2013) 500.

Abstract

We address the question whether a singularity in a three-dimensional incompressible inviscid fluid flow can occur in finite time. Analytical considerations and numerical simulations suggest high-symmetry flows being a promising candidate for a finite-time blowup. Utilizing Lagrangian and geometric non-blowup criteria, we present numerical evidence against the formation of a finite-time singularity for the high-symmetry vortex dodecapole initial condition. We use data obtained from high resolution adaptively refined numerical simulations and inject Lagrangian tracer particles to monitor geometric properties of vortex line segments. We then verify the assumptions made by analytical non-blowup criteria introduced by Deng et. al [Commun. PDE 31 (2006)] connecting vortex line geometry (curvature, spreading) to velocity increase to rule out singular behavior.


doi:10.1016/j.aml.2012.12.004

arXiv

Existence of Finite Time Singularities in the Euler equations

Summary: There has been a history of claims for and against possible numerical evidence for a finite-time singularity of the incompressible three-dimensional Euler equations. Addressing the shortcomings of simply monitoring peak vorticty to discern a singularity, this project aims to instead numerically test the assumptions posed by analytic blowup criteria connecting geometric properties of Lagrangian vortex line segments, like curvature and spreading, to possible finite-time singular behavior.

The Euler equations for incompressible fluids,

$$\begin{cases} \partial_t u + u\cdot\nabla u + \nabla p = 0\ \nabla\cdot u=0 \end{cases}$$

are known for more than 250 years. Nevertheless, the question of the global in time existence of smooth solutions for smooth initial conditions is not answered. What is known is local in time existence,

Local Existence:

For initial conditions $u_0 \in H^m$, $\nabla\cdot u_0=0$, $m\ge 7/2$, there exists a time $T \le 1/c|u_0|_{H^m}$, such that there exists a unique solution of the Euler equations with $$ u \in C([0,T];C^2(\mathbb{R}^3)) \cap C^1([0,T]; C(\mathbb{R}^3)). $$

With the advent of scientific computing, analytical results are accompanied by numerical simulations, which are used to gain insight into the evolution of flow structures and the interplay between physical quantities such as strain and vorticity, and may serve as clue, pointing in the general direction to be taken for analytical work. Most important in this direction is the classical result by Beale-Kato-Majda [BKM],

Beale-Kato-Majda:

If $T$ is the maximal time of existence of a unique, smooth solution of the Euler equations, then the vorticity $\omega=\nabla\times u$ fulfills $$ \int_0^T |\omega(\cdot,t)|_{L^\infty}\,dt = \infty $$

Therefore, if the maximum of the vorticity, $\Omega(t)=|\omega(\cdot,t)|_{L^\infty}$ scales in time like $\Omega(t)\sim 1/(T-t)^\gamma$, then there is a finite time singularity if $\gamma \ge 1$. Historically, numerical simulations have tried to prove or disprove finite time singularities by measuring the exponent $\gamma$. Physically, it is worthwhile asking if such a growth is reasonable. Generally, a singularity in the Euler equations is believed to be point-wise and supposedly locally self-similar. Analytical results investigating the bounds of such scenarios are quite scarce for the Euler equations, yet recently several cases relevant to numerical simulations have been ruled out. By far the most important process in the formation of singularities in finite time is the coupling of vorticity and strain. Starting from the vorticity formulation of the Euler equations,

$$\partial_t \omega + u \cdot \nabla \omega = \omega \cdot \nabla u,$$

the vortex stretching term $\omega \cdot \nabla u$ (which is notably absent in 2D) might provide a mechanism for critical growth: If the strain grows alongside the vorticity, $\nabla u \sim \omega$, then along an advected fluid volume one would have $D/Dt\, \omega \sim \omega^2$, resulting in the critical scaling $\omega(t) \sim 1/(T-t)$. It is not known to date, whether internal mechanisms render such amplification impossible. Yet, it is known from turbulence research that the mentioned alignment is at least unlikely in a natural context. The open question remains whether it is possible to design initial conditions which exhibit and maintain, despite its inherent instability, a period of vorticity-strain coupling long enough to cause the blowup.

Several such candidates are proposed in the literature. A perturbed vortex tube as well as anti-parallel vortex tubes are afflicted with the inconvenience to require the curvature in the plane of symmetry to blow up alongside the vorticity. This is due to the fact that an amplification of the strain with the same growth rate as the vorticity can be connected via the Biot-Savart law to the requirement to kink infinitely at the location of maximum vorticity. On the other hand, due to axial stretching of the vortex tube, a growing strain reduces the curvature in the plane of symmetry. These counteracting processes may limit the ability of the aforementioned initial conditions to maintain vorticity-strain coupling over a period of time long enough for the formation of a singularity in finite time. The class of high-symmetry flows, and most notably among them the vortex dodecapole configuration pictured below, does not suffer from the above mentioned disadvantage: Here, the strain imposed on the vortex tubes in the plane of symmetry is induced by the rotational images. Axial strain is not dictated by the curvature and the above canceling does not take place. This renders the vortex dodecapole initial condition to be one of the most promising in terms of singularity formation known today.

Kida-Pelz vortex dodecapole

Next to classical results such as BKM, recently the geometric analysis of the flow has played a role in distinguishing finite-time singularities from flows that exhibit merely fast accumulation of vorticity. This approach, applied to numerical simulations, may provide clearer insight into the possible formation of the singularity. The following theorem is a simple example of such techniques:

Deng-Hou-Yu (2006):

Let $x(t)$ be the position of maximum vorticity and $y(t)$ a point on the same vortex line $c(s)$ as $x(t)$. If $$ \left| \int_{x(t)}^{y(t)} (\nabla \cdot \xi) (c(s),t)\,ds \right| \le C$$ for $\xi=\omega/|\omega|$ being the direction of the vorticity, and $\int_0^T |\omega(y(t),t)|\,dt<\infty$, then there is no blowup up to time $T$

Most notably, theorems like this imply numerical techniques to distinguish between a point-wise blowup and the blowup of a whole vortex line segment: Monitoring a vortex line segment which maintains a fixed convergence of neighboring lines, $\int \nabla \cdot \xi\,ds=C$, the absence of the segment's collapse must coincide with a blowup of vorticity along the whole segment, assuming the formation of a finite-time singularity. Furthermore, using additional theorems, if a blowup of curvature and $\int \nabla \cdot \xi \,ds$ is not observed, then components of the velocity have to scale like $1/(T-t)$. Since in numerical simulations, velocity growth is usually far from that, this argument can be used against critical accumulation of vorticity much more clearly than the usual approach via BKM .

Relevant publications

  1. T. Grafke and R. Grauer, "Finite-Time Euler singularities: A Lagrangian perspective", Appl. Math. Letters 26 (2013), 500

  2. T. Grafke and R. Grauer, "Lagrangian and geometric analysis of finite-time Euler singularities", Procedia IUTAM 9 (2013), 32

  3. T. Grafke, H. Homann, J. Dreher and R. Grauer, "Numerical simulations of possible finite time singularities in the incompressible Euler equations: comparison of numerical methods", Physica D 237 (2008), 1932

  4. T. Grafke, "Finite-time Euler Singularities: A Lagrangian perspective", PhD thesis (Jun 2012)

PhD Thesis

Finite-time Euler Singularities: A Lagrangian perspective

Abstract

This work presents numerical evidence against the formation of a finite-time singularity for the vortex dodecapole initial condition. It uses data obtained from high resolution adaptively refined numerical simulations to test the assumptions demanded by analytic blowup criteria connecting vortex line geometry to velocity increase. In the course of this work, a numerical framework has been extended to allow the integration of the incompressible three-dimensional Euler equations on adaptively refined grids, which supports the diagnostics of geometrical and Lagrangian criteria and scales close to optimal on massively parallel machines.


Download thesis

See also this project.

Diploma Thesis

Singularities and Turbulence in Hydrodynamical Models

Three hydrodynamical models are presented in direct comparison, the Navier-Stokes equation, the Burgers' equation and a new and intermediate model, the Euler-Burgers' equation. The ultimate ambition is to shed light on the Navier- Stokes/Euler equation regarding the problem of turbulence and the question of possible finite time singularities. By proving existence and regularity of its solutions, the Euler-Burgers' equation is shown to be a prototype of a hydrodynamical model that circumvents the problems posed by the Navier-Stokes equation. With the help of numerical simulations it demonstrated, that this equation nevertheless exhibits turbulent statistics and intermittency that is successfully described by a generalized She-Leveque model.

Summary

In this thesis, three hydrodynamical models are presented in direct comparison, the Navier-Stokes equation, the Burgers' equation and a new and intermediate model, the Euler-Burgers' equation. The ultimate ambition is to shed light on the Navier- Stokes/Euler equation regarding the problem of turbulence and the question of possible finite time singularities. Since these problems are unsolved despite enduring and intensive research, the introduction of the Euler-Burgers' equation can be considered as a new and different approach to the topic. While being entirely theoretical in nature and without any application in reality, the Euler-Burgers' equation is shown to be a prototype of a hydrodynamical model that circumvents the problems posed by the Navier-Stokes equation, without turning out to be overly simple in its structure and dynamical behavior.

All three models are compared analytically and numerically under the aspects of general characteristics, turbulence and possible singularities. The close connection of the structure of Navier-Stokes and Euler-Burgers' equation is pointed out for several aspects. Both are similar in terms of the procedure of energy dissipation, since the identical viscous term of both hydrodynamical models is solely responsible for the annihilation of kinetic energy. The nonlinearity is shown to leave the total energy untouched and rather change the scale the energy is distributed on, finally leading to the process that was introduced as the Richardson cascade. Furthermore, both equations feature a nonlinearity local in position space, accompanied by a non-local projection operation. While the projection for the Navier-Stokes equation is just a circumscription for the physical pressure and concurrently causes mathematical and numerical inconvenience due to its instantaneous nature, the same operation for Euler-Burgers' equation is mathematically treatable. It is shown that global solutions of the Euler-Burgers equation exist for all times when sufficiently smooth external forcing and initial conditions. It is, furthermore, deduced that the solutions are smooth for all times and are unique. The Euler-Burgers' equation is, therefore, proved to be a full-fledged predictive hydrodynamical model that behaves well for all times.

For all further analysis and comparison of the model equations, numerical simulations were used. Two different implementations are added to the already existent framework racoon II. The first allows for accurate modeling of the maximum vorticity, maximizing the resolution by means of adaptive mesh refinement. While automatically ensuring solenoidality of the velocity field, the vorticity formulation, furthermore, provides reliable data for the most important finite time blow-up criterion, at the expense of execution speed. The second implementation focuses on the direct comparison of the three models. No adaptive refinement is employed but all model equations are implemented at once, allowing for an authentic comparison with improved execution speed. Both numerical formulations demand the solution of a Poisson-type equation that was obtained by implementing a Multigrid algorithm. It is capable of solving the Poisson equation parallel on an adaptive grid at arbitrary precision with adjustable error smoothing and interpolation methods. The simulations were mostly run on the local Linux Cluster, the obtained data was utilized to study general properties of the flows, to generate statistics and to analyze specific scenarios for possible finite time blow-ups.

No strict theory of turbulence is known today for the Navier-Stokes equation. During the last century, starting with the famous results of Kolmogorov, more and more refined phenomenological descriptions of turbulence based thereupon were developed. The most promising of these theories, regarding the successful description of numerical and experimental data, is the model of She and Leveque. This model was modified to the generalized She-Leveque model to be applied to all three considered hydrodynamical models. For Burgers' equation, the energy spectrum and structure function were measured and the numerical data fits well on the theory. For the Euler-Burgers' equation, two-dimensional folded sheets were identified as the most dissipative structures. Inserting these into the generalized She-Leveque model, predictions can be made for energy spectra and structure functions of turbulent Euler-Burgers' flows. The numerical data agrees very well with these predictions, showing that Euler-Burgers' turbulence is closely connected to Navier-Stokes turbulence.

Regarding the question of finite time singularities of the Euler equation, the mathematical criteria presented hint to monitor the peak vorticity of the flow. Exactly this was carried out in an adaptive simulation to probe the Kida-Pelz initial conditions for a finite time singularity at a very high resolution, compared to simulations run so far. Though no definite conclusion could be drawn whether a finite time singularity is possible, the numerical data allows statements regarding the necessary grid resolution. Larger simulations will be able to finally settle the issue definitely.


Download pdf