AIAA 2010-4554

28th AIAA Applied Aerodynamics Conference 28 June - 1 July 2010, Chicago, Illinois

Cartesian Adaptive Mesh Refinement for Rotorcraft Wake Resolution Andrew M. Wissink∗ 1 , Sean Kamkar† 2 , Thomas H. Pulliam‡ 3 , Jayanarayanan Sitaraman§ 4 , and Venkateswaran Sankaran¶ 1 1

U.S. Army Aeroflightdynamics Directorate (AMRDEC), Moffett Field, CA 2 Stanford University, Palo Alto, CA 3 NASA Ames Research Center, Moffett Field, CA 4 University of Wyoming, Laramie, WY

We explore the use of high-order algorithms within a block-structured Cartesian adaptive mesh refinement (AMR) solver to capture shed wakes from rotary-wing vehicles. The adaptive Cartesian solver is used as an off-body solver, in conjunction with a viscous near-body unstructured solver to capture wall boundary layer effects. The off-body solver automatically adapts to resolve time-dependent flow features, refining and coarsening throughout the unsteady simulation, with mesh refinement targeting regions of high vorticity. Accuracy and performance of the adaptive solver are tested for resolving the tip vortices from a wing at angle of attack and the wake of a quarter-scale V-22 helicopter rotor in hover conditions.

I.

Introduction

Vortical wakes introduce important aerodynamic phenomena in certain classes of aerospace vehicles. Rotary-wing vehicles, in particular, fly in their own wake and experience numerous aerodynamic effects, affecting vehicle handling qualities, vibration, and noise.1 The wake of the vehicles also complicates nearground operations, from shipboard landings to “brownout” conditions in desert flight. Fixed-wing aircraft also experience problems such as tail buffet from tip vortices emanating from the nose and swept wing in high angle-of-attack fighter jets.2 The availability of accurate and efficient computational models to better predict vortex wake behavior could help to minimize the onset of these sometimes disastrous phenomena. High-fidelity Reynolds-Averaged Navier-Stokes (RANS) CFD methods have demonstrated the ability to give accurate predictions of aerodynamic loads, but their ability to predict the wake is often limited by numerical dissipation. This can be mitigated by using very fine grids in the wake, but this quickly exhausts available computational resources. For problems where key solution features, like tip vortices, occur only in localized regions of the computational domain, spatial adaptive mesh refinement (AMR) can be an effective tool. AMR involves automatically refining and coarsening the grid locally to resolve important flow features. By focusing memory usage and computational effort in these localized regions, a highly resolved solution may be obtained much more efficiently than a globally refined grid of equal resolution. The use of AMR has been studied extensively for wakes of hovering rotors. Strawn and Barth3 first demonstrated the concept using AMR in an unstructured Euler solver. Kang and Kwon4 later applied a similar strategy within an unstructured Navier-Stokes solver. Potsdam5 also applied unstructured AMR to wind turbine wake predictions. Deitz et al.6 introduced an overset-grid based approach that moved tubular curvilinear grids to align with the tip vortices. Meakin7 proposed a Cartesian-based AMR scheme within the Overflow8 code. This approach was recently extended by Holst and Pulliam.9, 10 The aforementioned efforts ∗ Aerospace

Engineer, Research Development and Engineering Command, AIAA Member Student, Dept. Aeronautics & Astronautics, AIAA Student Member ‡ Senior Research Scientist, MS T27B, AIAA Associate Fellow § Assistant Professor, Dept. Mechanical Engineering, AIAA Member ¶ Aerospace Engineer, Research Development and Engineering Command, AIAA Member † Graduate

1 of 18 Institute of protection Aeronautics Astronautics This material is declared a work of the U.S. Government and American is not subject to copyright in theand United States.

all adopted AMR techniques targeting steady-state solutions; a solution is computed on an initial mesh, the mesh reconstructed to adapt to features in the solution, then the simulation is run again on the new mesh. This steady-AMR approach is useful for isolated rotors in hover conditions, but many of the more complex problems in rotorcraft require time-dependent moving body capability. An AMR scheme that can resolve unsteady effects like rotor-fuselage interactions or rotor vehicles in forward flight or maneuver conditions require an unsteady-AMR approach, for which the grid is adapted continually in a time dependent manner throughout the simulation. Another important strategy for CFD-based wake resolution is the use of high-order numerics. High-order schemes are effective because they resolve features like tip vortices with much fewer points across the vortex core, so it is possible to achieve better resolution of the vortex wake with a coarser mesh. The benefits of high-order schemes for rotor wakes have been shown by Hariharan,11, 12 Sankar,13 Yeshala,14 and Duque et al.15 Cartesian grids offer the most computationally efficient approach to high-order accuracy. Unlike Discontinuous Galerkin schemes on unstructured grids, which can be as much as an order of magnitude more expensive than standard second order schemes, finite-difference-based high-order schemes on Cartesian grids are only marginally more expensive than a second-order scheme. This paper presents details of the off-body solver used in the Helios16 CFD/CSD rotorcraft simulation code. Helios employs a a dual-solver paradigm with a near-body unstructured RANS solver to compute wallbounded viscous effects and an adaptive Cartesian off-body solver to resolve wakes, which is the primary focus of this paper. Section II discusses further details of the computational approach in Helios. Section III presents algorithmic details of the adaptive Cartesian off-body solver, outlining the meshing, high-order algorithms, and parallel communication strategies. Results are presented in section IV, and Sec. V summarizes conclusions and recommendations.

II.

Computational Approach

Figure 1. Near-body/off-body overset grid system.

The spatial discretization scheme employs an overset near-body/off-body approach, using unstructured body-fitted grids near the body surface and adaptive Cartesian grids away from the surface (Fig 1). The mixed-element unstructured near-body mesh is “subsetted” a certain distance from the wall. The NSU3D code17 is the solver applied to the near-body grid. NSU3D was developed specifically for high-Reynolds number external aerodynamic applications. The discretization employs a 2nd -order accurate, vertex-based approach, where the unknown fluid and turbulence variables are stored at the vertices of the mesh, and fluxes are computed on faces delimiting dual control volumes, with each dual face being associated with a mesh edge. The single-equation Spalart-Allmaras turbulence model in NSU3D is used for the high-Re flows in this work. The block structured adaptive Cartesian code SAMARC, described in detail in the next section, is

2 of 18 American Institute of Aeronautics and Astronautics

used for the off-body Cartesian calculations. Domain connectivity software transfers flow solution information between the near- and off-body solvers. The main tasks to perform are Chimera grid hole-cutting to remove points in the mesh that will not compute a solution, to identify the points that exchange data, and performing the actual interpolation and inter-processor exchange of solution data between the solvers. The Parallel Unsteady Domain Information Technology (PUNDIT) software package by Sitaraman et al.18 is used to perform these steps. Integration of these components — near-body solver, off-body solver, and domain connectivity package — is performed using a Python-based infrastructure.19, 20 The infrastructure treats each code component as an object with methods and data, providing a convenient way to assemble complex multi-disciplinary simulations in an object-oriented fashion. Data exchanges are done without memory copies or file I/O, and the infrastructure can be run in parallel on large multi-processor computer systems.21 The assembled set of these components that includes the integration framework and all the requisite solvers forms the Helios code.16 The next section provides details of the off-body adaptive Cartesian solver, which is the focus of the present article.

III.

Adaptive Cartesian Solver

Cartesian grids have a number of advantages over general unstructured volume grids for wake applications. Cartesian grid generation is fully automatic and robust. The grids have virtually no memory requirement for grid related data. Each 3D Cartesian grid block can be completely defined using 7 integers, with the number of grid blocks in complex practical applications ranging from 100’s to 1000’s. In contrast, unstructured volume grids require storage of 10’s to 100’s of millions of floating point values. Mathematical operations on Cartesian grids are fast and efficient because there are no grid metric terms, and the structured data layout maximizes cache performance. Grid quality is also never an issue and adaptive meshing is straightforward to implement. Finally, Cartesian grids facilitate straightforward implementation of multi-level algorithms such as multigrid sequencing and high-order numerical methods. In the present work, the off-body Cartesian solver is used to resolve the unsteady wake flowfield through a combination of high-order algorithms and adaptive mesh refinement. The unsteady-AMR strategy is based on the structured AMR ideas of Berger, Colella, and Oliger.22, 23 Grid levels composed of a union of 3D Cartesian blocks of like refinement are stored as a composite grid hierarchy. Details of this grid structure and the framework used to manage operations on it are discussed next. Then, the high-order numerical difference operations applied to each Cartesian block are described. Following is a discussion of the parallel implementation strategy, including load balancing, inter-block, and inter-level communication. Lastly, we show an example implementation of the Cartesian solver used to resolve the unsteady shed wake behind a sphere. III.A.

Cartesian Mesh Generation

The block structured Cartesian grid system is composed of a hierarchy of nested refinement levels, with each level formed as a union of logically-rectangular grid regions. The coarsest level defines the physical extent of the computational domain and each finer level is formed by selecting cells on the coarser level and clustering the tagged cells together to form block regions constituting the new finer level. A global indexing system is used that allows each block to be defined by seven integers - the level number and the lower and upper level indices (Fig. 2). All grid cells on a particular level have the same spacing, and the ratio of spacing between levels is a factor of two or four. The SAMRAI package24–26 from Lawrence Livermore National Lab manages the construction of the mesh and parallel MPI-based communication between blocks. SAMRAI is an object-oriented C++-based framework that provides an extensible and scalable infrastructure for parallel AMR applications. It supports a variety of data representations and algorithms and is currently used for programmatic applications at LLNL. Considerable work has gone into ensuring efficient scalable performance in SAMRAI for adaptive applications.27, 28 Cartesian grid levels are generated coarsest to finest, by first selecting cells for refinement, clustering them into blocks, then refining those blocks to form a new finer level. The criteria used to select cells for refinement is dependent on both solution-based quantities as well as near-body proximity considerations. Cells which contain flow conditions that require resolution, such as regions of high vorticity, are tagged for solution-based

3 of 18 American Institute of Aeronautics and Astronautics

Figure 2. Block structured adaptive Cartesian off-body grid.

refinement (Fig. 3). Off-body grid cells containing near-body mesh points that require resolution are also tagged for geometry-based refinement (Fig. 4). The tagged cells are then clustered to form the patches that will constitute the finer level. A consequence of this approach is that finer level patch boundaries always align with coarse cell boundaries. The spatial relationship between points on coarse and fine levels is completely defined through the global index space, which facilitates data communication between levels.

(a) coarse

(b) intermediate

(c) fine

Figure 3. Solution-based refinement. (a) solve equations on coarse level and tag cells containing features, (b) generate finer level blocks to refine features, (c) repeat until feature has been adequately resolved.

The hierarchical Cartesian grid system is automatically constructed to both resolve important solution features as well as to ensure a smooth transition of the flow variables from the near-body to the off-body mesh system. The latter is referred to as geometry-based refinement, as illustrated in Fig. 4. Points at the outer boundary of the near-body mesh that receive interpolated data from the off-body Cartesian mesh are referred to as inter-grid boundary points (IGBPs). They do not compute a solution, they simply receive data interpolated directly from the overlapping background Cartesian grids. The grid resolution of the near and off-body grid systems should be comparable at these points, and therefore, the Cartesian grids are refined until they match or slightly exceed the unstructured mesh resolution. The locations (x, y, z coordinates) and spacing ∆s of the near-body IGBPs are provided to the off-body solver. Any Cartesian cells that contain a near-body IGBP and whose grid spacing ∆x is greater than the spacing at the IGBP, ∆s, are tagged for refinement (Fig. 4(a)). All tagged cells are then clustered to construct a new finer level (Fig. 4(b)). The 4 of 18 American Institute of Aeronautics and Astronautics

Figure 4. Geometry-based refinement. (a) tag cells containing near-body IGBPs, (b) generate finer level blocks until ∆x spacing is less than or equal to near-body IGBP spacing.

process is repeated until no Cartesian cells are tagged i.e., the resolution of all near-body IGBPs has been satisfied by the off-body Cartesian grid system.

(a) cluster threshold=0.4

(b) cluster threshold=0.7

(c) cluster threshold=0.9

Figure 5. Effect of cluster threshold on construction of finer levels. Cluster efficiency defines cutoff ratio of tagged to total cells in the clustered block; (a) 40% cells tagged, (b) 70% tagged, (c) 90% tagged.

Some adjustable parameters may be set to help guide the construction of the off-body grid system. A user-specified tag buffer , which extends the tagged region to a specified layer of surrounding cells, is helpful for unsteady problems where solution features tend to migrate. If a finer grid is constructed only around regions where the feature is located at time t, by the time the solution reaches t + ∆t the feature may migrate out of the fine region into coarser grids where it will dissipate. The tag buffer effectively extends the fine mesh region around the feature, preserving the feature in the fine mesh until the next adaptation cycle. The size of the tag buffer is dependent on the adapt frequency; a small tag buffer is required when there is frequent adaptation, whereas a large tag buffer is needed if adaptation is infrequent. The adapt frequency is also a user-defined parameter, which should be commensurate with the convective scales of the problem. A cluster threshold controls the level of sub-blocks created as part of the tagged-cell clustering process, illustrated in Fig. 5. Tagged-cell clustering recursively sub-divides blocks until the ratio of tagged to total cells inside the block is equal to or greater than the prescribed cluster threshold, so the cluster threshold is effectively used to guide how deep the recursive process should go. For example, Fig. 5(a) uses a relatively low tag threshold of 0.4, meaning clustering ceases once the tagged cell block contains 40% or more tagged cells. Figures 5(b) and (c) show the effect of increasing the parameter. More blocks are created so

5 of 18 American Institute of Aeronautics and Astronautics

the new finer level more closely follows the identified feature, and contains fewer cells. However, this comes at the cost of more smaller blocks and more time spent in the clustering process. The user also can prescribe a minimum block size, which is necessary to ensure accurate implementation of high-order methods that require a minimum stencil, and a nesting buffer which specifies the minimum number of cells between nested grid levels in the hierarchical grid system. III.B.

High-Order Algorithms

Flow solution of the Euler equations on each Cartesian block is performed by ARC3DC from NASA Ames Research Center. ARC3DC is a Cartesian version of the ARC3D code developed by Pulliam29, 30 which is derived from the AIR2D code originally developed by Steger. The 3D Euler equations are solved ∂t Q + ∂x E + ∂y F + ∂z G = 0 where Q are the solution quantities and E, F , and ρ ρu Q = ρv , ρw e ρv ρuv F = ρv 2 + p ρwv v(e + p)

G are the flux quantities ρu ρu2 + p E = ρuv , ρuw u(e + p) ρw ρuw , G = ρvw ρw2 + p

(1)

(2)

w(e + p)

The equations are discretized using a uniformly-spaced Cartesian grid. Qj,k,l represents a discrete point value of Q, with j, k, l subscripts indicating the node-centered index counters in x, y, and z directions, respectively. The difference equation is ∂t Qj,k,l + δx Ej,k,l + δy Fj,k,l + δz Gj,k,l = 0

(3)

where δx , δy , and δz represent finite difference operators. Central finite difference operators of second, fourth, and sixth-order may be applied to the flux differences E, F, G (shown for E in x only, for convenience) ∂x E ≈ δx(2) Ej

=

δx(4) Ej

=

δx(6) Ej

=

Ej+1 − Ej−1 2∆x −Ej+2 + 8Ej+1 − 8Ej−1 + Ej−2 12∆x Ej+3 − 9Ej+2 + 45Ej+1 − 45Ej−1 + 9Ej−2 − Ej−3 60∆x

(4) (5) (6)

Note that the 2nd -order scheme requires a 3 point stencil, the 4th -order a 5 point stencil, and 6th -order a 7 point stencil, requiring 1, 2, and 3 fringe points at block boundaries, respectively. Artificial dissipation is used for stability. The dissipation is added explicitly to a central difference of the inviscid fluxes. The standard method used is a mixed 2nd , 4th and 6th derivative artificial dissipation employing a pressure gradient switch and spectral radius scaling. (2) (4) (6) ∇x (σj+1 + σj ) j ∆x Qj − j ∆x ∇x ∆x Qj + j ∆x ∇x ∆x ∇x ∆x Qj /∆x where ∇x Qj = (Qj − Qj−1 ),

∆x Qj = (Qj+1 − Qj ) 6 of 18

American Institute of Aeronautics and Astronautics

(7)

(2)

j

= 2 max(Υj+1 , Υj , Υj−1 )

Υj = (4,6)

j

=

|pj+1 −2pj +pj−1 | |pj+1 +2pj +pj−1 | (2)

max(0, (4,6) − j )

(8)

where typical values of the constants are 2 = 1/4, 4 = 1/100, and 6 = 0.0. Similar terms are used in the y, z directions. Note that the stencil widths and fringe points are consistent with the convective derivative requirements. The term σj is a spectral radius scaling and is defined as σj = |u| + a

(9)

with a the speed of sound. While necessary for stability, the degree of added dissipation also affects the decay of the vortex core. Reducing the coefficient reduces the core decay, but at the expense of less robustness and stability, leading to increased oscillations (non-smoothness or the generation of high frequency error). In practice, we have found the 6th -order central difference scheme coupled with 5th -order artificial dissipation is quite effective for the application in which we are most interested, resolving vortex flows rotorcraft wakes. Explicitly-added dissipation coefficients of 2 = 0.0, 4 = 0.0 and 6 = 0.5 are used. This scheme minimizes both vortex dissipation and dispersion error, relative to the 2nd -order difference scheme with 3rd -order dissipation at a cost only about 10% more. It does require 3-fringe cells rather than 2, which means a larger volume of data is exchanged at inter-block boundaries, but in general our tests have shown the improved accuracy outweighs this relatively minor extra communication cost. Between adaptive gridding steps the governing equations are numerically integrated on the patches using an explicit 3rd-order Runge Kutta (RK) scheme. At the beginning of each RK sub-step, data on fine patch boundaries are first updated either through copying data from a neighboring patch on the same level, if one exists, or through interpolation of data from a coarser level. Second, the numerical operations to advance a single RK sub-step are performed simultaneously on each patch of each level. Third, data are injected into coarse levels wherever fine patches overlap coarse ones. The data exchanges performed to update patch boundary and interior information are done between processors using MPI communication, as discussed next. III.C.

Parallel Implementation

(a) Neighboring blocks on same level

(b) Fringe cell data exchange

Figure 6. Intra-level fringe data transfer between blocks residing on the same refinement level.

SAMRAI adaptively regenerates blocks to follow time dependent features or track moving surfaces. Each “adapt” cycle involves constructing new levels and distributing them to processors. The Cartesian grid system is constructed through a combination of solution-based (Fig. 3) and geometry-based (Fig. 4) refinement. Once the blocks forming the new finer level have been generated, they are distributed to processors in a load balanced fashion before allocating data on the new set of blocks. This final step is unique to the structured AMR paradigm and is key to its efficient performance. Because the load balancing is done before the new mesh level is actually constructed, Cartesian grid generation takes place in parallel. In contrast, more traditional AMR schemes generate the newly refined mesh local to each processor before re-balancing. This can incur heavy costs in passing volume data to re-balance the workload across processors. Between adaptive gridding steps, the governing equations are numerically integrated on the blocks. Numerical operations are executed independently on each block, with the accuracy of high-order operators 7 of 18 American Institute of Aeronautics and Astronautics

(a) Overlapping blocks on different levels

(b) Fine block fringe cell update

Figure 7. Inter-level fringe data exchange between blocks on different refinement levels. (a) coarse-fine boundary, (b) fine-block fringe data interpolated from data on coarser level.

at block boundaries maintained through the use of fringe cells. After the Q data on the block has been computed the fringe overlap is exchanged with neighboring blocks. Blocks on the same refinement incur an intra-level transfer, as shown in Fig. 6. Each block is grown to add a layer of fringe cells and the resulting overlap with neighbor blocks is computed (Fig. 6(b)). Overlap data is packed into a buffer and exchanged with neighboring blocks. Blocks residing on different processors exchange this buffer through MPI, those on the same processor simply do a memory copy. The exchange of data at coarse-fine boundaries is similar except that the fringe data on the finer block is updated from data interpolated from the coarser block using an inter-level transfer, shown in Fig. 7. III.D.

Example Application - Flow over sphere

We demonstrate the adaptive gridding concepts discussed above for calculation of unsteady flow over a sphere. The physical characteristics of unsteady flow over a sphere, such as onset of instabilities and shedding frequency at different Reynolds numbers, are well known and documented both experimentally and computationally. Validation of this case for the steady low-Reynolds number (Re) flow has been presented elsewhere20 using the dual-mesh unstructured-Cartesian approach. Here, we compare results for a higher Re unsteady case as a demonstration of the use of Cartesian AMR to resolve the unsteady wake. A near-body sphere mesh is constructed by trimming a standard unstructured mixed element mesh. The trim distance is one half of the sphere diameter and the NS solver is applied to this near-body mesh. The off-body Cartesian off-body mesh extends into the far field, using a maximum of 7 levels of refinement. It adapts to match the mesh spacing at the interface to the near-body mesh and adapts time-dependently to regions of high vorticity throughout the simulation. The flow Reynolds number used is Re=800, which is unsteady but laminar so no turbulence model is used in the near-body solver. Figure 8(a) shows an iso-surface of vorticity of the wake computed using standalone NS on a purely unstructured mesh, while Fig. 8(b) shows the solution on the unstructured-Cartesian mesh. The standalone NS solution captures the near-field effects well but dissipates the vorticity in the wake, causing the wake solution to appear steady. The adaptive Cartesian solver, on the other hand, captures the are unsteady shedding behavior properly. A calculation at three time instances is shown in Fig. 9. Experimental results have demonstrated transition from large scale to small scale structures at Re=800. The computational results resolve this phenomenon. It should be noted that the unstructured mesh used for the standalone RANS solution in Fig. 8(a) is not refined to capture the wake and the solution quality could be likely be improved if it were. However, there are two drawbacks to static grid wake refinement. First, adding refinement manually at the initial grid generation stage requires the person generating the grid to know where to add points to resolve the wake. This can be difficult to know apriori, particularly for unsteady flow. It is advantageous both for automation and accuracy to allow the solver to determine where to apply refinement. Second, there is significant cost savings by using structured Cartesian grids in place of unstructured tets, as demonstrated in the timing results presented in the next section.

8 of 18 American Institute of Aeronautics and Astronautics

(a) Standalone NS

(b) unstructured-Cartesian

Figure 8. Flow over sphere at Re=800. Wake resolution comparison. Iso-surface of vorticity overlaid on mesh.

Figure 9. Unsteady shedding over sphere at Re=800 using dual mesh near-body unstructured-NS with off-body adaptive Cartesian-Euler. Vorticity contours at different solution times. Cartesian meshes adapt to regions of high vorticity magnitude.

9 of 18 American Institute of Aeronautics and Astronautics

IV.

Results

Results with the coupled unstructured-Cartesian AMR approach are demonstrated to resolve shed wakes for two applied cases. The first resolves tip vortices shed from a NACA0015 wing at angle of attack, investigating the ability of the scheme to automatically capture and maintain tip vortices downstream. The second application is the wake of a quarter-scale isolated V-22 rotor blade in hover conditions, investigating the ability of the scheme to resolve helical rotor wakes. IV.A.

NACA 0015 Wing

This test case involves steady flow around a flat-tipped NACA0015 wing at 12o angle of attack at Mach number 0.1235 with Reynolds number 1.5 million. This case was studied experimentally by McAlister and Takahashi31 in the 7’x10’ windtunnel at NASA Ames in 1991. Computational results have been presented by Sitaraman and Baeder32 and by Hariharan and Sankar.11 As with the sphere case presented previously, we show comparisons of the wake using the unstructuredCartesian grids vs. standalone unstructured RANS. Figure 10(a) shows the fully unstructured RANS mesh while Fig. 10(b) shows the unstructured-Cartesian mesh. The near-body mesh is a subsetted version of the RANS mesh, trimmed a distance of 0.5 chords from the surface. This case exhibits turbulent flow near the surface so the Spalart-Allmaras one-equation turbulence model is applied on the unstructured near-body mesh. The unstructured-Cartesian solution applies refinement to regions of high vorticity to capture wingtip vortices.

(a) Standalone RANS

(b) unstructured-Cartesian

Figure 10. Mesh systems for NACA 0015 wing calculations (a) unstructured everywhere; (b) near-body unstructured with off-body adaptive Cartesian

Figure 11 shows a comparison of the wakes computed by the standalone RANS solver and the unstructuredCartesian solver. In the near-vicinity of the wing surface the solutions for the two approaches are essentially the same, which is to be expected since the same unstructured mesh and solver are applied in both cases. Downstream of the wing, however, there is a major difference in the ability of the two schemes to capture the tip vortices emanating from the wingtips. The wing tip vortices dissipate very quickly in the standalone RANS solution while the unstructured-Cartesian solver is able to resolve the tip vortices well back from the

10 of 18 American Institute of Aeronautics and Astronautics

trailing edge. Breakdown into sub-structures is observed approximately 15 chords behind the wing.

(a) standalone RANS

(b) unstructured-Cartesian Figure 11. NACA 0015 calculation, iso-surface of vorticity at ω = 0.25 showing downstream tip vortices. (a) unstructured grid with no refinement; (b) adaptive Cartesian grid refined to regions of high vorticity.

Figure 12 presents a side-by-side comparison of the unstructured-Cartesian vs. fully unstructured computations for this case. Shown are an iso-surface of vorticity with plots comparing computed vs. experimental swirl velocity component VZ of the wingtip vortices at 2 and 6 chords downstream of the trailing edge, measured by McAlister et al.31 In addition, computed swirl velocity at 12 chords is also shown for reference although there were no measurements taken at this station. The unstructured-Cartesian solution does a much better job at preserving the vortex strength than the standalone RANS solution. Note that the computed swirl velocity is relatively unchanged from 2 chords to 12 chords downstream in the adapted Cartesian grids. The dissipation of the vortex relative to experimentally-measured values is already present at 2-chord station, behind the trailing edge, and remains largely unchanged at the 12-chord station. This implies the near-body mesh is responsible for much of the dissipation. Indeed, studies by Kamkar33 of this problem indicates most of the vortex dissipation occurs within the near-body grid, extending one-chord aft of the trailing edge. Table 1 compares the number of gridpoints and computation time for the standalone RANS and unstructuredCartesian calculations. Although the addition of the Cartesian off-body mesh more than triples the total 11 of 18 American Institute of Aeronautics and Astronautics

(a) Vorticity iso-surface

(b) Swirl Velocity Figure 12. Computed vorticity iso-surface for flow around NACA 0015 using unstructured-Cartesian (left) and standalone unstructured RANS (right). Plots of computed and experimentally-measured31 vertical velocity at 2, 6, and 12 chords downstream.

12 of 18 American Institute of Aeronautics and Astronautics

standalone RANS unstructured Cartesian AMR

nprocs 32 32

Unstruct 4.8M 6.53s 4.5M 6.15s

Cartesian – – 9.9M 1.24s

Total 4.8M 6.53s 14.4M 7.39s

Table 1. Computational performance of NACA 0015 calculation. Number of gridpoints (left column) and computational time per step (right column) in unstructured and Cartesian grid systems. Calculation run on Air Force Research Lab “hawk” system (SGI Altix 4700).

number of gridpoints, the total time per step increases by just 13%. The reason is that on a per-point perstep basis, calculations on Cartesian grids are about an order of magnitude faster than unstructured. Also, Cartesian solutions are 5th -order accurate while the unstructured solver is standard 2nd -order. In practice unstructured-Cartesian calculations typically require more iterations for convergence, due to a combination of an explicit off-body integration algorithm and the extra steps needed to develop the wake in the finer Cartesian mesh. However, this is not unique to the Cartesian solver, a fully unstructured RANS solution with finer meshes in the wake would also likely experience slower convergence. IV.B.

Isolated V-22 (TRAM) Rotor in Hover

The Tilt Rotor Aeromechanics Model (TRAM) is a 0.25 scale model of the Bell/Boeing V-22 Osprey tiltrotor aircraft right-hand 3-bladed rotor. The isolated TRAM rotor was tested in the Duits-Nederlandse Windtunnel Large Low-speed Facility (DNW-LLF) in the spring of 1998. Aerodynamic pressures, thrust and power, were measured along with structural loads and aeroacoustics data. Wake geometry, in particular the locations of tip vortices, was not part of the data collected. Further details on the TRAM experiment and extensive CFD validations can be found in the work of Potsdam and Strawn.34

(a) standalone RANS

(b) unstructured-Cartesian

Figure 13. Baseline mesh and solution for TRAM calculations. (a) fully unstructured, (b) unstructured near-body with adaptive Cartesian off-body. Wake solutions show iso-surface of Q-criterion colored by vorticity magnitude.

The TRAM geometry contains multiple components, including the 3 blades and a center-body. Each blade is 57 inches in radius with a tip chord of 5.5 inches. The tip Mach number is Mtip = 0.625, which

13 of 18 American Institute of Aeronautics and Astronautics

corresponds to a tip Reynolds number of Retip = 2.1 × 106 . The case is run to a steady-state condition using a non-inertial rotational frame. Results are first compared using a baseline 5.1M node unstructured prismatic-tetrahedral mesh for standalone RANS. The unstructured-Cartesian near-body mesh subsets the mesh to a distance of two blade chords. The off-body Cartesian mesh contains 8 levels of refinement with finest level spacing of 5% of the tip chord (Ctip ), adapting to regions of high vorticity. The standalone RANS solution and mesh are shown in Fig. 13(a), and the unstructured-Cartesian solution and mesh are shown in Fig. 13(b). The wake of the standalone RANS solution dissipates quickly, while the wake of the unstructured-Cartesian solution is well resolved with adaptive Cartesian grids. Computed computational thrust, power, and figure-of-merit (FM) for this case baseline TRAM case are shown in Table 2. Figure of merit is a measure of rotor efficiency compared to ideal (computed as FM √ 3/2 = CT /CQ 2). It is often used for comparative purposes between experiment and computation. The standalone RANS solution is 11% low in FM while the unstructured-Cartesian simulation is 5% low. While this is encouraging in that it indicates that rotor performance prediction is improved with the added wake resolution, it is also discouraging because the FM prediction is still off to a significant degree. A 5% difference in rotor FM for a full-scale helicopter like UH-60 or V-22 translates to 1000-1500 lbs variation in hover thrust conditions. To improve the load predictions we utilize a finer near-body blade mesh. The refined fine mesh contains 9.3M points total, which is about a 3X increase in size over the baseline mesh. Refinement was added around the tip and in the path of the vortex from the advancing blade tip (Fig. 14(b)). A comparison of the solutions in the immediate vicinity of the tip for the baseline grid, shown in Fig. 14(a), and the refined grid shown in Fig. 14(b), indicates the added refinement is effective at reducing dissipation of the tip vortex.

(a) Baseline mesh - 2.8M nodes

(b) Fine mesh - 9.3M nodes

Figure 14. Near-body meshes used in dual mesh TRAM calculation; (a) baseline, (b) refinement added to tip region and in the path of the vortex from the advancing blade.

The differences in the wake computation between these three meshes are shown in Fig. 15. Vorticity contours are overlaid on the mesh at the quarter-chord plane. Figure 15(a) shows the result from the standalone RANS solution using a fully unstructured grid, while Fig. 15(b) and Fig. 15(c) show the unstructured-Cartesian result with the baseline and refined near-body mesh, respectively. Note that because both unstructured-Cartesian cases (b) and (c) use the same resolution in the off-body, the differences are solely the result of improved resolution of the near-body mesh and the near-body refinement added. It is clear that a combination of refinement in both the near and off-body meshes is key to the overall problem resolution. A 3D view of the unstructured-Cartesian solution with refined near-body grid is shown in Fig. 16. Tip vortices are maintained well downstream of the rotor plane.

14 of 18 American Institute of Aeronautics and Astronautics

Figure 15. Wake vorticity contours overlaid on mesh (a) standalone RANS, (b) baseline unstructured-Cartesian, (c) refined near-body unstructured-Cartesian.

A comparison of the computed loads for the refined near-body grid is included in Table 2. It indicates that the the FM prediction is within 1% of the experimental value, which is likely within the error margins of the measurements. Thus, accurate prediction of both computed loads and wake can be obtained through a combination of refinement in both the near and off-body meshes. Statistics on the mesh sizes and computational times for the calculations with the standalone RANS and unstructured-Cartesian with baseline and refined near-body meshes are given in Table 3. It should be noted that the number of Cartesian gridpoints changes over the coarse of the simulation due to adaptation. The numbers shown in the table are at the final stage of the solution. The dual-mesh does require more time per step than fully-unstructured RANS because of its larger number of gridpoints. However, while the gridpoint count is greater by more than 10X the average time per step is only about 2.7X larger. The reason is the faster operations on Cartesian grids. In terms of time per point per step, Cartesian off-body calculations are approximately 6X-7X faster than unstructured.

Experiment baseline standalone RANS baseline unstructured-Cartesian refined unstructured-Cartesian

CT 0.0149 0.0146 2% low 0.0152 2% high 0.0151 1% high

CQ 0.00165 0.00179 8% high 0.00180 9% high 0.00171 4% high

FM 0.779 0.694 11% low 0.739 5% low 0.771 1% low

Table 2. TRAM rotor computed loads. Collective pitch θ = 14o .

baseline standalone RANS baseline unstructured-Cartesian refined unstructured-Cartesian

nprocs 64 64 128

Unstruct 5.0M 2.67s 2.8M 1.45s 9.1M 2.25s

Cartesian – – 54.0M 4.25s 116.0M 3.84s

Total 5.0M 2.67s 56.8M 7.15s 125.1M 6.15s

Table 3. Computational performance of TRAM calculation. Number of gridpoints (left column) and computational time per step (right) in unstructured and Cartesian grid systems. Calculation run on Army Research Lab “mjm” system (Linux Networx Woodcrest cluster) with four procs (cores) per node.

A more detailed study of isolated TRAM rotor in hover calculations using the unstructured-Cartesian dual mesh scheme in Helios was presented recently.35 Convergence characteristics, collective sweep results, and

15 of 18 American Institute of Aeronautics and Astronautics

Figure 16. Refined TRAM solution at θ = 14o collective. Iso-surfaces of Q-criterion colored by vorticity. (a) solution overlaid on mesh, (b) solution only.

comparison of results with the steady inertial formulation used here to non-inertial moving-blade unsteady hover results are shown in that reference.

V.

Conclusions

We present an innovative dual-mesh approach for CFD problems which require high-fidelity predictions of aerodynamic loads as well as wakes. The approach applies a traditional unstructured RANS solver to near-body meshes in order to capture complex geometry and wall-bounded viscous effects, then applies an off-body Euler solver that employs both high-order algorithms and AMR to resolve the wake. The offbody solver adopts the block-structured Cartesian AMR strategy traditionally used in the shock-physics community to enable frequent adaptation to resolve unsteady wake flows. The high-order adaptive off-body solver effectively resolves the wake at relatively low cost. The cost savings arise because of the inherent efficiency of the Cartesian off-body solver, which means that significantly larger grids can be solved with only nominal increases in CPU time. Moreover, high-order solutions in the Cartesian off-body solver incur negligible cost. We find that the off-body solver must be coupled with an adequately resolved near-body mesh in order to compute the correct aerodynamic loads and tip vortex pattern in the wake. Adding refinement in the near-body mesh to particular regions, such as around the tip, can greatly improve both the aerodynamic load and wake resolution. Future work will investigate application of AMR to near-body meshes as well. Another important future direction is the development of automated schemes to determine where to apply refinement. The AMR scheme used here targets refinement to regions of high vorticity. While this is shown to work, it requires considerable experimentation to determine the correct threshold value of vorticity to trigger refinement. Automated schemes that can determine an appropriate threshold for different problem classes will enable this scheme to be more readily used in routine production engineering applications. Some promising early work in this direction has been presented in reference.33

16 of 18 American Institute of Aeronautics and Astronautics

Acknowledgments Material presented in this paper is a product of the CREATE-AV Element of the Computational Research and Engineering for Acquisition Tools and Environments (CREATE) Program sponsored by the U.S. Department of Defense HPC Modernization Program Office. This work was conducted at the High Performance Computing Institute for Advanced Rotorcraft Modeling and Simulation (HIARMS). Rotary-wing capability was added to NSU3D by Prof. Dimitri Mavriplis, under sponsorship of the institute. The authors gratefully acknowledge additional contributions by Dr. Buvana Jayaraman, Dr. Aaron Katz, Mr. Mark Potsdam, and Dr. Roger Strawn.

References 1 Caradonna, F. X., R.C. Strawn, and J.O. Bridgeman, “An Experimental and Computational Study of Rotor-Vortex Interactions,” Vertica, Vol. 12, No. 4, 1988, pp. 315–327. 2 Morton, S., R. Cummings, and D. Kholodar, “High Resolution Turbulence Treatment of F/A-18 Tail Buffet,” AIAA2004-1676, 45th AIAA Structures, Structural Dynamics and Materials Conference, Palm Springs CA, 2004. 3 Strawn, R.C., and T.J. Barth, “A finite-volume Euler solver for computing rotary-wing aerodynamics on unstructured meshes,” Journal of the American Helicopter Society, Vol. 38, 1993, pp.61–67. 4 Kang, H.J., and O. J. Kwon,“Unstructured Mesh Navier-Stokes Calculations of the Flow Field of a Helicopter Rotor in Hover,” Journal of the American Helicopter Society, April 2002, pp.90–99. 5 Potsdam, M., and D. J. Mavriplis,“Unstructured Mesh CFD Aerodynamic Analysis of the NREL Phase VI Rotor,” AIAA-2009-1221, 47th AIAA Aerosciences Meeting, Orlando FL, Jan 2009. 6 Dietz, M., E. Karmer, and S. Wagner,“Tip Vortex Conservation on a Main Rotor in Slow Descent Flight Using VortexAdapted Chimera Grids,” AIAA-2006-3478, 24th AIAA Applied Aerodynamics Conference, San Francisco, CA, June 2006. 7 Meakin, R.L.,“Automatic Off-body Grid Generation for Domains of Arbitrary Size,” AIAA-2001-2536, 15th AIAA Computational Fluid Dynamics Conference, Anaheim CA, June 2001. 8 Buning, P.G., D.C. Jespersen, T.H. Pulliam, W.H. Chan, J.P. Slotnick, S.E. Krist, and K. J. Renze, “OVERFLOW User’s Manual, Version 1.8,” NASA Langley Research Center, 1998. 9 Holst, T., and T. Pulliam, “Overset Solution Adaptive Grid Approach Applied to Hovering Rotorcraft Flows,” AIAA2009-3519, 27th AIAA Applied Aerodynamics Conference, San Antonio, TX, June 2009. 10 Holst, T., and T. Pulliam, “Optimization of Overset Solution Adaptive Grids for Hovering Rotorcraft Flows,” 2010 AHS Specialists Meeting on Aeromechanics, San Francisco CA, January 2010. 11 Hariharan, N., “Rotary-Wing Wake Capturing: High Order Schemes Towards Minimizing Numerical Vortex Dissipation,” AIAA Journal of Aircraft, Vol. 39, No. 5, pp. 822-830, 2002. 12 Hariharan, N., and L. Sankar, “High-Order Essentially Non-oscillatory Schemes for Rotary-Wing Wake Computations,” Journal of Aircraft, Vol. 41, No. 2, 2004, pp. 258–267. 13 Sankar, L., N. Yeshala, and N. Hariharan, “Application of Spatially High Order Adaptive Methods for Unsteady Flow over Rotary Wing Configurations,” International Forum on Rotorcraft Multidisciplinary Technology, American Helicopter Society Specialists Meeting, paper No. 21-1, Seoul, Korea, October 2007. 14 Yeshala, N., A.T. Egolf, R. Vasilescu, and L. Sankar, “Application of Higher Order Spatially Accurate Schemes to Rotors in Hover,” AIAA Paper No. 2006-2818, 24th AIAA Applied Aerodynamics Conference, San Francisco, CA, June 2006. 15 Duque, E., L. Sankar, S. Menon, L. Bauchau, S. Ruffin, M. Smith, K. Ahuja, K. Brentner, L. Long, P. Morris, and F. Gandhi, “Revolutionary Physics-based Design Tools for Quiet Helicopters,” AIAA Paper 2006-1068, 44th AIAA Aerospace Sciences Meeting, Reno NV, Jan 2006. 16 Sankaran, V., J. Sitaraman, A. Wissink, A. Datta, B. Jayaraman, M. Potsdam D. Mavriplis, Z. Yang, D. O’Brien, H. Saberi, R. Cheng, N. Hariharan, and R. Strawn, “Application of the Helios Computational Platform to Rotorcraft Flowfields,” AIAA-2010-1230, 48th AIAA Aerospace Sciences Meeting, Orlando FL, Jan 2010. 17 Mavriplis, D. J., and V. Venkatakrishnan, “A Unified Multigrid Solver for the Navier-Stokes Equations on Mixed Element Meshes,” International Journal for Computational Fluid Dynamics, Vol. 8, 1997, pp. 247-263. 18 Sitaraman, J., M. Floros, A. M. Wissink, and M. Potsdam, “Parallel Unsteady Overset Mesh Methodology for a MultiSolver Paradigm with Adaptive Cartesian Grids,” AIAA-2008-7117, 26th AIAA Applied Aerodynamics Conference, Honolulu HI, Jan 2008. 19 Wissink, A. M., J. Sitaraman, V. Sankaran, D. J. Mavriplis, and T. H. Pulliam, “A Multi-Code Python-Based Infrastucture for Overset CFD with Adaptive Cartesian Grids”, AIAA-2008-0927, 46th AIAA Aerosciences Conference, Reno NV, Jan 2008. 20 Sitaraman, J., A. Katz, B. Jayaraman, A. Wissink, V. Sankaran, “Evaluation of a Multi-Solver Paradigm for CFD using Unstructured and Structured Adaptive Cartesian Grids,” AIAA-2008-0660, 46th AIAA Aerosciences Conference, Reno NV, Jan 2008. 21 Wissink, A. M., and S. Shende, “Performance Evaluation of the Multi-Language Helios Rotorcraft Simulation Software, Proceedings of the DoD HPC Users Group Conference, Seattle WA, June 2008. 22 Berger, M.J., and J. Oliger, “Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations,” Journal of Computational Physics, Vol. 53, 1984, pp. 484–512. 23 Berger, M. J., and P. Colella, “Local Adaptive Mesh Refinement for Shock Hydrodynamics,” J. Comp. Phys., 82, 1989, pp. 65–84.

17 of 18 American Institute of Aeronautics and Astronautics

24 Hornung, R. D., A. M. Wissink, and S. R. Kohn, “Managing Complex Data and Geometry in Parallel Structured AMR Applications,” Engineering with Computers, Vol. 22, No. 3-4, Dec. 2006, pp. 181-195. Also see www.llnl.gov/casc/samrai. 25 Hornung, R. D., and S. R. Kohn, “Managing Application Complexity in the SAMRAI object-oriented framework,” Concurrency and Computation: Practise and Experience, Vol. 14, 2002, pp. 347-368. 26 Wissink, A. M., R. D. Hornung, S. Kohn, S. Smith, and N. Elliott, “Large-Scale Parallel Structured AMR Calculations using the SAMRAI Framework”, Proceedings of Supercomputing 2001 (SC01), Denver CO, Nov 2001. 27 Wissink, A. M., D. A. Hysom, and R. D. Hornung, “Enhancing Scalability of Parallel Structured AMR Calculations”, Proceedings of the 17th ACM International Conference on Supercomputing (ICS03), San Francisco CA, June 2003, pp. 336–347. 28 Gunney, B. T. N., A. M. Wissink, and D. A. Hysom, “Parallel Clustering Algorithms for Structured AMR,” J. Parallel. Dist. Computing, 66, 2006, pp. 1419–1430. 29 Pulliam, T. H., “Solution Methods in Computational Fluid Dynamics,” von Karmon Institute for Fluid Mechanics Lecture Series, Numerical Techniques for Viscous Flow Computations in Turbomachinery, Rhode-St-Genese, Belgium, Jan 1986. See http://people.nas.nasa.gov/ pulliam/mypapers/vki notes/vki notes.html. 30 Pulliam, T. H., “Euler and Thin-Layer Navier-Stokes Codes: ARC2D, and ARC3D” Computational Fluid Dynamics Users Workshop, The University of Tennesse Space Institute, Tullahoma TN, March 1984. 31 McAlister, K. W., and R. K. Takahashi, “NACA 0015 Wing Pressure and Trailing Vortex Measurements,” NASA Technical Paper 3151, AVSCOM Technical Report 91-A-003, Nov 1991. 32 Sitaraman, J., and J. D. Baeder, “Evaluation of the Wake Prediction Methodologies used in CFD Based Rotor Airload Computations,” AIAA-2006-3472, 24th AIAA Applied Aerodynamics Conference, San Francisco CA, June 2006. 33 Kamkar, S.J., A. Wissink, V. Sankaran, A. Jameson, “Feature-Driven Cartesian Adaptive Mesh Refinement in the Helios Code,” AIAA-2010-171, 48th AIAA Aerospace Sciences Meeting, Orlando FL, Jan 2010. 34 Potsdam, M. A., and R. C. Strawn, “CFD Simulations of Tiltrotor Configurations in Hover,” Journal of the American Helicopter Society, Vol. 50, No. 1, 2005, pp. 82–94. 35 Wissink, A.M., M. Potsdam, V. Sankaran, J. Sitaraman, Z. Yang, and D. Mavriplis, “A Coupled Unstructured-Adaptive Cartesian CFD Approach for Hover Prediction,”American Helicopter Society 66th Annual Forum, Phoenix AZ, May 11-16, 2010.

18 of 18 American Institute of Aeronautics and Astronautics

28th AIAA Applied Aerodynamics Conference 28 June - 1 July 2010, Chicago, Illinois

Cartesian Adaptive Mesh Refinement for Rotorcraft Wake Resolution Andrew M. Wissink∗ 1 , Sean Kamkar† 2 , Thomas H. Pulliam‡ 3 , Jayanarayanan Sitaraman§ 4 , and Venkateswaran Sankaran¶ 1 1

U.S. Army Aeroflightdynamics Directorate (AMRDEC), Moffett Field, CA 2 Stanford University, Palo Alto, CA 3 NASA Ames Research Center, Moffett Field, CA 4 University of Wyoming, Laramie, WY

We explore the use of high-order algorithms within a block-structured Cartesian adaptive mesh refinement (AMR) solver to capture shed wakes from rotary-wing vehicles. The adaptive Cartesian solver is used as an off-body solver, in conjunction with a viscous near-body unstructured solver to capture wall boundary layer effects. The off-body solver automatically adapts to resolve time-dependent flow features, refining and coarsening throughout the unsteady simulation, with mesh refinement targeting regions of high vorticity. Accuracy and performance of the adaptive solver are tested for resolving the tip vortices from a wing at angle of attack and the wake of a quarter-scale V-22 helicopter rotor in hover conditions.

I.

Introduction

Vortical wakes introduce important aerodynamic phenomena in certain classes of aerospace vehicles. Rotary-wing vehicles, in particular, fly in their own wake and experience numerous aerodynamic effects, affecting vehicle handling qualities, vibration, and noise.1 The wake of the vehicles also complicates nearground operations, from shipboard landings to “brownout” conditions in desert flight. Fixed-wing aircraft also experience problems such as tail buffet from tip vortices emanating from the nose and swept wing in high angle-of-attack fighter jets.2 The availability of accurate and efficient computational models to better predict vortex wake behavior could help to minimize the onset of these sometimes disastrous phenomena. High-fidelity Reynolds-Averaged Navier-Stokes (RANS) CFD methods have demonstrated the ability to give accurate predictions of aerodynamic loads, but their ability to predict the wake is often limited by numerical dissipation. This can be mitigated by using very fine grids in the wake, but this quickly exhausts available computational resources. For problems where key solution features, like tip vortices, occur only in localized regions of the computational domain, spatial adaptive mesh refinement (AMR) can be an effective tool. AMR involves automatically refining and coarsening the grid locally to resolve important flow features. By focusing memory usage and computational effort in these localized regions, a highly resolved solution may be obtained much more efficiently than a globally refined grid of equal resolution. The use of AMR has been studied extensively for wakes of hovering rotors. Strawn and Barth3 first demonstrated the concept using AMR in an unstructured Euler solver. Kang and Kwon4 later applied a similar strategy within an unstructured Navier-Stokes solver. Potsdam5 also applied unstructured AMR to wind turbine wake predictions. Deitz et al.6 introduced an overset-grid based approach that moved tubular curvilinear grids to align with the tip vortices. Meakin7 proposed a Cartesian-based AMR scheme within the Overflow8 code. This approach was recently extended by Holst and Pulliam.9, 10 The aforementioned efforts ∗ Aerospace

Engineer, Research Development and Engineering Command, AIAA Member Student, Dept. Aeronautics & Astronautics, AIAA Student Member ‡ Senior Research Scientist, MS T27B, AIAA Associate Fellow § Assistant Professor, Dept. Mechanical Engineering, AIAA Member ¶ Aerospace Engineer, Research Development and Engineering Command, AIAA Member † Graduate

1 of 18 Institute of protection Aeronautics Astronautics This material is declared a work of the U.S. Government and American is not subject to copyright in theand United States.

all adopted AMR techniques targeting steady-state solutions; a solution is computed on an initial mesh, the mesh reconstructed to adapt to features in the solution, then the simulation is run again on the new mesh. This steady-AMR approach is useful for isolated rotors in hover conditions, but many of the more complex problems in rotorcraft require time-dependent moving body capability. An AMR scheme that can resolve unsteady effects like rotor-fuselage interactions or rotor vehicles in forward flight or maneuver conditions require an unsteady-AMR approach, for which the grid is adapted continually in a time dependent manner throughout the simulation. Another important strategy for CFD-based wake resolution is the use of high-order numerics. High-order schemes are effective because they resolve features like tip vortices with much fewer points across the vortex core, so it is possible to achieve better resolution of the vortex wake with a coarser mesh. The benefits of high-order schemes for rotor wakes have been shown by Hariharan,11, 12 Sankar,13 Yeshala,14 and Duque et al.15 Cartesian grids offer the most computationally efficient approach to high-order accuracy. Unlike Discontinuous Galerkin schemes on unstructured grids, which can be as much as an order of magnitude more expensive than standard second order schemes, finite-difference-based high-order schemes on Cartesian grids are only marginally more expensive than a second-order scheme. This paper presents details of the off-body solver used in the Helios16 CFD/CSD rotorcraft simulation code. Helios employs a a dual-solver paradigm with a near-body unstructured RANS solver to compute wallbounded viscous effects and an adaptive Cartesian off-body solver to resolve wakes, which is the primary focus of this paper. Section II discusses further details of the computational approach in Helios. Section III presents algorithmic details of the adaptive Cartesian off-body solver, outlining the meshing, high-order algorithms, and parallel communication strategies. Results are presented in section IV, and Sec. V summarizes conclusions and recommendations.

II.

Computational Approach

Figure 1. Near-body/off-body overset grid system.

The spatial discretization scheme employs an overset near-body/off-body approach, using unstructured body-fitted grids near the body surface and adaptive Cartesian grids away from the surface (Fig 1). The mixed-element unstructured near-body mesh is “subsetted” a certain distance from the wall. The NSU3D code17 is the solver applied to the near-body grid. NSU3D was developed specifically for high-Reynolds number external aerodynamic applications. The discretization employs a 2nd -order accurate, vertex-based approach, where the unknown fluid and turbulence variables are stored at the vertices of the mesh, and fluxes are computed on faces delimiting dual control volumes, with each dual face being associated with a mesh edge. The single-equation Spalart-Allmaras turbulence model in NSU3D is used for the high-Re flows in this work. The block structured adaptive Cartesian code SAMARC, described in detail in the next section, is

2 of 18 American Institute of Aeronautics and Astronautics

used for the off-body Cartesian calculations. Domain connectivity software transfers flow solution information between the near- and off-body solvers. The main tasks to perform are Chimera grid hole-cutting to remove points in the mesh that will not compute a solution, to identify the points that exchange data, and performing the actual interpolation and inter-processor exchange of solution data between the solvers. The Parallel Unsteady Domain Information Technology (PUNDIT) software package by Sitaraman et al.18 is used to perform these steps. Integration of these components — near-body solver, off-body solver, and domain connectivity package — is performed using a Python-based infrastructure.19, 20 The infrastructure treats each code component as an object with methods and data, providing a convenient way to assemble complex multi-disciplinary simulations in an object-oriented fashion. Data exchanges are done without memory copies or file I/O, and the infrastructure can be run in parallel on large multi-processor computer systems.21 The assembled set of these components that includes the integration framework and all the requisite solvers forms the Helios code.16 The next section provides details of the off-body adaptive Cartesian solver, which is the focus of the present article.

III.

Adaptive Cartesian Solver

Cartesian grids have a number of advantages over general unstructured volume grids for wake applications. Cartesian grid generation is fully automatic and robust. The grids have virtually no memory requirement for grid related data. Each 3D Cartesian grid block can be completely defined using 7 integers, with the number of grid blocks in complex practical applications ranging from 100’s to 1000’s. In contrast, unstructured volume grids require storage of 10’s to 100’s of millions of floating point values. Mathematical operations on Cartesian grids are fast and efficient because there are no grid metric terms, and the structured data layout maximizes cache performance. Grid quality is also never an issue and adaptive meshing is straightforward to implement. Finally, Cartesian grids facilitate straightforward implementation of multi-level algorithms such as multigrid sequencing and high-order numerical methods. In the present work, the off-body Cartesian solver is used to resolve the unsteady wake flowfield through a combination of high-order algorithms and adaptive mesh refinement. The unsteady-AMR strategy is based on the structured AMR ideas of Berger, Colella, and Oliger.22, 23 Grid levels composed of a union of 3D Cartesian blocks of like refinement are stored as a composite grid hierarchy. Details of this grid structure and the framework used to manage operations on it are discussed next. Then, the high-order numerical difference operations applied to each Cartesian block are described. Following is a discussion of the parallel implementation strategy, including load balancing, inter-block, and inter-level communication. Lastly, we show an example implementation of the Cartesian solver used to resolve the unsteady shed wake behind a sphere. III.A.

Cartesian Mesh Generation

The block structured Cartesian grid system is composed of a hierarchy of nested refinement levels, with each level formed as a union of logically-rectangular grid regions. The coarsest level defines the physical extent of the computational domain and each finer level is formed by selecting cells on the coarser level and clustering the tagged cells together to form block regions constituting the new finer level. A global indexing system is used that allows each block to be defined by seven integers - the level number and the lower and upper level indices (Fig. 2). All grid cells on a particular level have the same spacing, and the ratio of spacing between levels is a factor of two or four. The SAMRAI package24–26 from Lawrence Livermore National Lab manages the construction of the mesh and parallel MPI-based communication between blocks. SAMRAI is an object-oriented C++-based framework that provides an extensible and scalable infrastructure for parallel AMR applications. It supports a variety of data representations and algorithms and is currently used for programmatic applications at LLNL. Considerable work has gone into ensuring efficient scalable performance in SAMRAI for adaptive applications.27, 28 Cartesian grid levels are generated coarsest to finest, by first selecting cells for refinement, clustering them into blocks, then refining those blocks to form a new finer level. The criteria used to select cells for refinement is dependent on both solution-based quantities as well as near-body proximity considerations. Cells which contain flow conditions that require resolution, such as regions of high vorticity, are tagged for solution-based

3 of 18 American Institute of Aeronautics and Astronautics

Figure 2. Block structured adaptive Cartesian off-body grid.

refinement (Fig. 3). Off-body grid cells containing near-body mesh points that require resolution are also tagged for geometry-based refinement (Fig. 4). The tagged cells are then clustered to form the patches that will constitute the finer level. A consequence of this approach is that finer level patch boundaries always align with coarse cell boundaries. The spatial relationship between points on coarse and fine levels is completely defined through the global index space, which facilitates data communication between levels.

(a) coarse

(b) intermediate

(c) fine

Figure 3. Solution-based refinement. (a) solve equations on coarse level and tag cells containing features, (b) generate finer level blocks to refine features, (c) repeat until feature has been adequately resolved.

The hierarchical Cartesian grid system is automatically constructed to both resolve important solution features as well as to ensure a smooth transition of the flow variables from the near-body to the off-body mesh system. The latter is referred to as geometry-based refinement, as illustrated in Fig. 4. Points at the outer boundary of the near-body mesh that receive interpolated data from the off-body Cartesian mesh are referred to as inter-grid boundary points (IGBPs). They do not compute a solution, they simply receive data interpolated directly from the overlapping background Cartesian grids. The grid resolution of the near and off-body grid systems should be comparable at these points, and therefore, the Cartesian grids are refined until they match or slightly exceed the unstructured mesh resolution. The locations (x, y, z coordinates) and spacing ∆s of the near-body IGBPs are provided to the off-body solver. Any Cartesian cells that contain a near-body IGBP and whose grid spacing ∆x is greater than the spacing at the IGBP, ∆s, are tagged for refinement (Fig. 4(a)). All tagged cells are then clustered to construct a new finer level (Fig. 4(b)). The 4 of 18 American Institute of Aeronautics and Astronautics

Figure 4. Geometry-based refinement. (a) tag cells containing near-body IGBPs, (b) generate finer level blocks until ∆x spacing is less than or equal to near-body IGBP spacing.

process is repeated until no Cartesian cells are tagged i.e., the resolution of all near-body IGBPs has been satisfied by the off-body Cartesian grid system.

(a) cluster threshold=0.4

(b) cluster threshold=0.7

(c) cluster threshold=0.9

Figure 5. Effect of cluster threshold on construction of finer levels. Cluster efficiency defines cutoff ratio of tagged to total cells in the clustered block; (a) 40% cells tagged, (b) 70% tagged, (c) 90% tagged.

Some adjustable parameters may be set to help guide the construction of the off-body grid system. A user-specified tag buffer , which extends the tagged region to a specified layer of surrounding cells, is helpful for unsteady problems where solution features tend to migrate. If a finer grid is constructed only around regions where the feature is located at time t, by the time the solution reaches t + ∆t the feature may migrate out of the fine region into coarser grids where it will dissipate. The tag buffer effectively extends the fine mesh region around the feature, preserving the feature in the fine mesh until the next adaptation cycle. The size of the tag buffer is dependent on the adapt frequency; a small tag buffer is required when there is frequent adaptation, whereas a large tag buffer is needed if adaptation is infrequent. The adapt frequency is also a user-defined parameter, which should be commensurate with the convective scales of the problem. A cluster threshold controls the level of sub-blocks created as part of the tagged-cell clustering process, illustrated in Fig. 5. Tagged-cell clustering recursively sub-divides blocks until the ratio of tagged to total cells inside the block is equal to or greater than the prescribed cluster threshold, so the cluster threshold is effectively used to guide how deep the recursive process should go. For example, Fig. 5(a) uses a relatively low tag threshold of 0.4, meaning clustering ceases once the tagged cell block contains 40% or more tagged cells. Figures 5(b) and (c) show the effect of increasing the parameter. More blocks are created so

5 of 18 American Institute of Aeronautics and Astronautics

the new finer level more closely follows the identified feature, and contains fewer cells. However, this comes at the cost of more smaller blocks and more time spent in the clustering process. The user also can prescribe a minimum block size, which is necessary to ensure accurate implementation of high-order methods that require a minimum stencil, and a nesting buffer which specifies the minimum number of cells between nested grid levels in the hierarchical grid system. III.B.

High-Order Algorithms

Flow solution of the Euler equations on each Cartesian block is performed by ARC3DC from NASA Ames Research Center. ARC3DC is a Cartesian version of the ARC3D code developed by Pulliam29, 30 which is derived from the AIR2D code originally developed by Steger. The 3D Euler equations are solved ∂t Q + ∂x E + ∂y F + ∂z G = 0 where Q are the solution quantities and E, F , and ρ ρu Q = ρv , ρw e ρv ρuv F = ρv 2 + p ρwv v(e + p)

G are the flux quantities ρu ρu2 + p E = ρuv , ρuw u(e + p) ρw ρuw , G = ρvw ρw2 + p

(1)

(2)

w(e + p)

The equations are discretized using a uniformly-spaced Cartesian grid. Qj,k,l represents a discrete point value of Q, with j, k, l subscripts indicating the node-centered index counters in x, y, and z directions, respectively. The difference equation is ∂t Qj,k,l + δx Ej,k,l + δy Fj,k,l + δz Gj,k,l = 0

(3)

where δx , δy , and δz represent finite difference operators. Central finite difference operators of second, fourth, and sixth-order may be applied to the flux differences E, F, G (shown for E in x only, for convenience) ∂x E ≈ δx(2) Ej

=

δx(4) Ej

=

δx(6) Ej

=

Ej+1 − Ej−1 2∆x −Ej+2 + 8Ej+1 − 8Ej−1 + Ej−2 12∆x Ej+3 − 9Ej+2 + 45Ej+1 − 45Ej−1 + 9Ej−2 − Ej−3 60∆x

(4) (5) (6)

Note that the 2nd -order scheme requires a 3 point stencil, the 4th -order a 5 point stencil, and 6th -order a 7 point stencil, requiring 1, 2, and 3 fringe points at block boundaries, respectively. Artificial dissipation is used for stability. The dissipation is added explicitly to a central difference of the inviscid fluxes. The standard method used is a mixed 2nd , 4th and 6th derivative artificial dissipation employing a pressure gradient switch and spectral radius scaling. (2) (4) (6) ∇x (σj+1 + σj ) j ∆x Qj − j ∆x ∇x ∆x Qj + j ∆x ∇x ∆x ∇x ∆x Qj /∆x where ∇x Qj = (Qj − Qj−1 ),

∆x Qj = (Qj+1 − Qj ) 6 of 18

American Institute of Aeronautics and Astronautics

(7)

(2)

j

= 2 max(Υj+1 , Υj , Υj−1 )

Υj = (4,6)

j

=

|pj+1 −2pj +pj−1 | |pj+1 +2pj +pj−1 | (2)

max(0, (4,6) − j )

(8)

where typical values of the constants are 2 = 1/4, 4 = 1/100, and 6 = 0.0. Similar terms are used in the y, z directions. Note that the stencil widths and fringe points are consistent with the convective derivative requirements. The term σj is a spectral radius scaling and is defined as σj = |u| + a

(9)

with a the speed of sound. While necessary for stability, the degree of added dissipation also affects the decay of the vortex core. Reducing the coefficient reduces the core decay, but at the expense of less robustness and stability, leading to increased oscillations (non-smoothness or the generation of high frequency error). In practice, we have found the 6th -order central difference scheme coupled with 5th -order artificial dissipation is quite effective for the application in which we are most interested, resolving vortex flows rotorcraft wakes. Explicitly-added dissipation coefficients of 2 = 0.0, 4 = 0.0 and 6 = 0.5 are used. This scheme minimizes both vortex dissipation and dispersion error, relative to the 2nd -order difference scheme with 3rd -order dissipation at a cost only about 10% more. It does require 3-fringe cells rather than 2, which means a larger volume of data is exchanged at inter-block boundaries, but in general our tests have shown the improved accuracy outweighs this relatively minor extra communication cost. Between adaptive gridding steps the governing equations are numerically integrated on the patches using an explicit 3rd-order Runge Kutta (RK) scheme. At the beginning of each RK sub-step, data on fine patch boundaries are first updated either through copying data from a neighboring patch on the same level, if one exists, or through interpolation of data from a coarser level. Second, the numerical operations to advance a single RK sub-step are performed simultaneously on each patch of each level. Third, data are injected into coarse levels wherever fine patches overlap coarse ones. The data exchanges performed to update patch boundary and interior information are done between processors using MPI communication, as discussed next. III.C.

Parallel Implementation

(a) Neighboring blocks on same level

(b) Fringe cell data exchange

Figure 6. Intra-level fringe data transfer between blocks residing on the same refinement level.

SAMRAI adaptively regenerates blocks to follow time dependent features or track moving surfaces. Each “adapt” cycle involves constructing new levels and distributing them to processors. The Cartesian grid system is constructed through a combination of solution-based (Fig. 3) and geometry-based (Fig. 4) refinement. Once the blocks forming the new finer level have been generated, they are distributed to processors in a load balanced fashion before allocating data on the new set of blocks. This final step is unique to the structured AMR paradigm and is key to its efficient performance. Because the load balancing is done before the new mesh level is actually constructed, Cartesian grid generation takes place in parallel. In contrast, more traditional AMR schemes generate the newly refined mesh local to each processor before re-balancing. This can incur heavy costs in passing volume data to re-balance the workload across processors. Between adaptive gridding steps, the governing equations are numerically integrated on the blocks. Numerical operations are executed independently on each block, with the accuracy of high-order operators 7 of 18 American Institute of Aeronautics and Astronautics

(a) Overlapping blocks on different levels

(b) Fine block fringe cell update

Figure 7. Inter-level fringe data exchange between blocks on different refinement levels. (a) coarse-fine boundary, (b) fine-block fringe data interpolated from data on coarser level.

at block boundaries maintained through the use of fringe cells. After the Q data on the block has been computed the fringe overlap is exchanged with neighboring blocks. Blocks on the same refinement incur an intra-level transfer, as shown in Fig. 6. Each block is grown to add a layer of fringe cells and the resulting overlap with neighbor blocks is computed (Fig. 6(b)). Overlap data is packed into a buffer and exchanged with neighboring blocks. Blocks residing on different processors exchange this buffer through MPI, those on the same processor simply do a memory copy. The exchange of data at coarse-fine boundaries is similar except that the fringe data on the finer block is updated from data interpolated from the coarser block using an inter-level transfer, shown in Fig. 7. III.D.

Example Application - Flow over sphere

We demonstrate the adaptive gridding concepts discussed above for calculation of unsteady flow over a sphere. The physical characteristics of unsteady flow over a sphere, such as onset of instabilities and shedding frequency at different Reynolds numbers, are well known and documented both experimentally and computationally. Validation of this case for the steady low-Reynolds number (Re) flow has been presented elsewhere20 using the dual-mesh unstructured-Cartesian approach. Here, we compare results for a higher Re unsteady case as a demonstration of the use of Cartesian AMR to resolve the unsteady wake. A near-body sphere mesh is constructed by trimming a standard unstructured mixed element mesh. The trim distance is one half of the sphere diameter and the NS solver is applied to this near-body mesh. The off-body Cartesian off-body mesh extends into the far field, using a maximum of 7 levels of refinement. It adapts to match the mesh spacing at the interface to the near-body mesh and adapts time-dependently to regions of high vorticity throughout the simulation. The flow Reynolds number used is Re=800, which is unsteady but laminar so no turbulence model is used in the near-body solver. Figure 8(a) shows an iso-surface of vorticity of the wake computed using standalone NS on a purely unstructured mesh, while Fig. 8(b) shows the solution on the unstructured-Cartesian mesh. The standalone NS solution captures the near-field effects well but dissipates the vorticity in the wake, causing the wake solution to appear steady. The adaptive Cartesian solver, on the other hand, captures the are unsteady shedding behavior properly. A calculation at three time instances is shown in Fig. 9. Experimental results have demonstrated transition from large scale to small scale structures at Re=800. The computational results resolve this phenomenon. It should be noted that the unstructured mesh used for the standalone RANS solution in Fig. 8(a) is not refined to capture the wake and the solution quality could be likely be improved if it were. However, there are two drawbacks to static grid wake refinement. First, adding refinement manually at the initial grid generation stage requires the person generating the grid to know where to add points to resolve the wake. This can be difficult to know apriori, particularly for unsteady flow. It is advantageous both for automation and accuracy to allow the solver to determine where to apply refinement. Second, there is significant cost savings by using structured Cartesian grids in place of unstructured tets, as demonstrated in the timing results presented in the next section.

8 of 18 American Institute of Aeronautics and Astronautics

(a) Standalone NS

(b) unstructured-Cartesian

Figure 8. Flow over sphere at Re=800. Wake resolution comparison. Iso-surface of vorticity overlaid on mesh.

Figure 9. Unsteady shedding over sphere at Re=800 using dual mesh near-body unstructured-NS with off-body adaptive Cartesian-Euler. Vorticity contours at different solution times. Cartesian meshes adapt to regions of high vorticity magnitude.

9 of 18 American Institute of Aeronautics and Astronautics

IV.

Results

Results with the coupled unstructured-Cartesian AMR approach are demonstrated to resolve shed wakes for two applied cases. The first resolves tip vortices shed from a NACA0015 wing at angle of attack, investigating the ability of the scheme to automatically capture and maintain tip vortices downstream. The second application is the wake of a quarter-scale isolated V-22 rotor blade in hover conditions, investigating the ability of the scheme to resolve helical rotor wakes. IV.A.

NACA 0015 Wing

This test case involves steady flow around a flat-tipped NACA0015 wing at 12o angle of attack at Mach number 0.1235 with Reynolds number 1.5 million. This case was studied experimentally by McAlister and Takahashi31 in the 7’x10’ windtunnel at NASA Ames in 1991. Computational results have been presented by Sitaraman and Baeder32 and by Hariharan and Sankar.11 As with the sphere case presented previously, we show comparisons of the wake using the unstructuredCartesian grids vs. standalone unstructured RANS. Figure 10(a) shows the fully unstructured RANS mesh while Fig. 10(b) shows the unstructured-Cartesian mesh. The near-body mesh is a subsetted version of the RANS mesh, trimmed a distance of 0.5 chords from the surface. This case exhibits turbulent flow near the surface so the Spalart-Allmaras one-equation turbulence model is applied on the unstructured near-body mesh. The unstructured-Cartesian solution applies refinement to regions of high vorticity to capture wingtip vortices.

(a) Standalone RANS

(b) unstructured-Cartesian

Figure 10. Mesh systems for NACA 0015 wing calculations (a) unstructured everywhere; (b) near-body unstructured with off-body adaptive Cartesian

Figure 11 shows a comparison of the wakes computed by the standalone RANS solver and the unstructuredCartesian solver. In the near-vicinity of the wing surface the solutions for the two approaches are essentially the same, which is to be expected since the same unstructured mesh and solver are applied in both cases. Downstream of the wing, however, there is a major difference in the ability of the two schemes to capture the tip vortices emanating from the wingtips. The wing tip vortices dissipate very quickly in the standalone RANS solution while the unstructured-Cartesian solver is able to resolve the tip vortices well back from the

10 of 18 American Institute of Aeronautics and Astronautics

trailing edge. Breakdown into sub-structures is observed approximately 15 chords behind the wing.

(a) standalone RANS

(b) unstructured-Cartesian Figure 11. NACA 0015 calculation, iso-surface of vorticity at ω = 0.25 showing downstream tip vortices. (a) unstructured grid with no refinement; (b) adaptive Cartesian grid refined to regions of high vorticity.

Figure 12 presents a side-by-side comparison of the unstructured-Cartesian vs. fully unstructured computations for this case. Shown are an iso-surface of vorticity with plots comparing computed vs. experimental swirl velocity component VZ of the wingtip vortices at 2 and 6 chords downstream of the trailing edge, measured by McAlister et al.31 In addition, computed swirl velocity at 12 chords is also shown for reference although there were no measurements taken at this station. The unstructured-Cartesian solution does a much better job at preserving the vortex strength than the standalone RANS solution. Note that the computed swirl velocity is relatively unchanged from 2 chords to 12 chords downstream in the adapted Cartesian grids. The dissipation of the vortex relative to experimentally-measured values is already present at 2-chord station, behind the trailing edge, and remains largely unchanged at the 12-chord station. This implies the near-body mesh is responsible for much of the dissipation. Indeed, studies by Kamkar33 of this problem indicates most of the vortex dissipation occurs within the near-body grid, extending one-chord aft of the trailing edge. Table 1 compares the number of gridpoints and computation time for the standalone RANS and unstructuredCartesian calculations. Although the addition of the Cartesian off-body mesh more than triples the total 11 of 18 American Institute of Aeronautics and Astronautics

(a) Vorticity iso-surface

(b) Swirl Velocity Figure 12. Computed vorticity iso-surface for flow around NACA 0015 using unstructured-Cartesian (left) and standalone unstructured RANS (right). Plots of computed and experimentally-measured31 vertical velocity at 2, 6, and 12 chords downstream.

12 of 18 American Institute of Aeronautics and Astronautics

standalone RANS unstructured Cartesian AMR

nprocs 32 32

Unstruct 4.8M 6.53s 4.5M 6.15s

Cartesian – – 9.9M 1.24s

Total 4.8M 6.53s 14.4M 7.39s

Table 1. Computational performance of NACA 0015 calculation. Number of gridpoints (left column) and computational time per step (right column) in unstructured and Cartesian grid systems. Calculation run on Air Force Research Lab “hawk” system (SGI Altix 4700).

number of gridpoints, the total time per step increases by just 13%. The reason is that on a per-point perstep basis, calculations on Cartesian grids are about an order of magnitude faster than unstructured. Also, Cartesian solutions are 5th -order accurate while the unstructured solver is standard 2nd -order. In practice unstructured-Cartesian calculations typically require more iterations for convergence, due to a combination of an explicit off-body integration algorithm and the extra steps needed to develop the wake in the finer Cartesian mesh. However, this is not unique to the Cartesian solver, a fully unstructured RANS solution with finer meshes in the wake would also likely experience slower convergence. IV.B.

Isolated V-22 (TRAM) Rotor in Hover

The Tilt Rotor Aeromechanics Model (TRAM) is a 0.25 scale model of the Bell/Boeing V-22 Osprey tiltrotor aircraft right-hand 3-bladed rotor. The isolated TRAM rotor was tested in the Duits-Nederlandse Windtunnel Large Low-speed Facility (DNW-LLF) in the spring of 1998. Aerodynamic pressures, thrust and power, were measured along with structural loads and aeroacoustics data. Wake geometry, in particular the locations of tip vortices, was not part of the data collected. Further details on the TRAM experiment and extensive CFD validations can be found in the work of Potsdam and Strawn.34

(a) standalone RANS

(b) unstructured-Cartesian

Figure 13. Baseline mesh and solution for TRAM calculations. (a) fully unstructured, (b) unstructured near-body with adaptive Cartesian off-body. Wake solutions show iso-surface of Q-criterion colored by vorticity magnitude.

The TRAM geometry contains multiple components, including the 3 blades and a center-body. Each blade is 57 inches in radius with a tip chord of 5.5 inches. The tip Mach number is Mtip = 0.625, which

13 of 18 American Institute of Aeronautics and Astronautics

corresponds to a tip Reynolds number of Retip = 2.1 × 106 . The case is run to a steady-state condition using a non-inertial rotational frame. Results are first compared using a baseline 5.1M node unstructured prismatic-tetrahedral mesh for standalone RANS. The unstructured-Cartesian near-body mesh subsets the mesh to a distance of two blade chords. The off-body Cartesian mesh contains 8 levels of refinement with finest level spacing of 5% of the tip chord (Ctip ), adapting to regions of high vorticity. The standalone RANS solution and mesh are shown in Fig. 13(a), and the unstructured-Cartesian solution and mesh are shown in Fig. 13(b). The wake of the standalone RANS solution dissipates quickly, while the wake of the unstructured-Cartesian solution is well resolved with adaptive Cartesian grids. Computed computational thrust, power, and figure-of-merit (FM) for this case baseline TRAM case are shown in Table 2. Figure of merit is a measure of rotor efficiency compared to ideal (computed as FM √ 3/2 = CT /CQ 2). It is often used for comparative purposes between experiment and computation. The standalone RANS solution is 11% low in FM while the unstructured-Cartesian simulation is 5% low. While this is encouraging in that it indicates that rotor performance prediction is improved with the added wake resolution, it is also discouraging because the FM prediction is still off to a significant degree. A 5% difference in rotor FM for a full-scale helicopter like UH-60 or V-22 translates to 1000-1500 lbs variation in hover thrust conditions. To improve the load predictions we utilize a finer near-body blade mesh. The refined fine mesh contains 9.3M points total, which is about a 3X increase in size over the baseline mesh. Refinement was added around the tip and in the path of the vortex from the advancing blade tip (Fig. 14(b)). A comparison of the solutions in the immediate vicinity of the tip for the baseline grid, shown in Fig. 14(a), and the refined grid shown in Fig. 14(b), indicates the added refinement is effective at reducing dissipation of the tip vortex.

(a) Baseline mesh - 2.8M nodes

(b) Fine mesh - 9.3M nodes

Figure 14. Near-body meshes used in dual mesh TRAM calculation; (a) baseline, (b) refinement added to tip region and in the path of the vortex from the advancing blade.

The differences in the wake computation between these three meshes are shown in Fig. 15. Vorticity contours are overlaid on the mesh at the quarter-chord plane. Figure 15(a) shows the result from the standalone RANS solution using a fully unstructured grid, while Fig. 15(b) and Fig. 15(c) show the unstructured-Cartesian result with the baseline and refined near-body mesh, respectively. Note that because both unstructured-Cartesian cases (b) and (c) use the same resolution in the off-body, the differences are solely the result of improved resolution of the near-body mesh and the near-body refinement added. It is clear that a combination of refinement in both the near and off-body meshes is key to the overall problem resolution. A 3D view of the unstructured-Cartesian solution with refined near-body grid is shown in Fig. 16. Tip vortices are maintained well downstream of the rotor plane.

14 of 18 American Institute of Aeronautics and Astronautics

Figure 15. Wake vorticity contours overlaid on mesh (a) standalone RANS, (b) baseline unstructured-Cartesian, (c) refined near-body unstructured-Cartesian.

A comparison of the computed loads for the refined near-body grid is included in Table 2. It indicates that the the FM prediction is within 1% of the experimental value, which is likely within the error margins of the measurements. Thus, accurate prediction of both computed loads and wake can be obtained through a combination of refinement in both the near and off-body meshes. Statistics on the mesh sizes and computational times for the calculations with the standalone RANS and unstructured-Cartesian with baseline and refined near-body meshes are given in Table 3. It should be noted that the number of Cartesian gridpoints changes over the coarse of the simulation due to adaptation. The numbers shown in the table are at the final stage of the solution. The dual-mesh does require more time per step than fully-unstructured RANS because of its larger number of gridpoints. However, while the gridpoint count is greater by more than 10X the average time per step is only about 2.7X larger. The reason is the faster operations on Cartesian grids. In terms of time per point per step, Cartesian off-body calculations are approximately 6X-7X faster than unstructured.

Experiment baseline standalone RANS baseline unstructured-Cartesian refined unstructured-Cartesian

CT 0.0149 0.0146 2% low 0.0152 2% high 0.0151 1% high

CQ 0.00165 0.00179 8% high 0.00180 9% high 0.00171 4% high

FM 0.779 0.694 11% low 0.739 5% low 0.771 1% low

Table 2. TRAM rotor computed loads. Collective pitch θ = 14o .

baseline standalone RANS baseline unstructured-Cartesian refined unstructured-Cartesian

nprocs 64 64 128

Unstruct 5.0M 2.67s 2.8M 1.45s 9.1M 2.25s

Cartesian – – 54.0M 4.25s 116.0M 3.84s

Total 5.0M 2.67s 56.8M 7.15s 125.1M 6.15s

Table 3. Computational performance of TRAM calculation. Number of gridpoints (left column) and computational time per step (right) in unstructured and Cartesian grid systems. Calculation run on Army Research Lab “mjm” system (Linux Networx Woodcrest cluster) with four procs (cores) per node.

A more detailed study of isolated TRAM rotor in hover calculations using the unstructured-Cartesian dual mesh scheme in Helios was presented recently.35 Convergence characteristics, collective sweep results, and

15 of 18 American Institute of Aeronautics and Astronautics

Figure 16. Refined TRAM solution at θ = 14o collective. Iso-surfaces of Q-criterion colored by vorticity. (a) solution overlaid on mesh, (b) solution only.

comparison of results with the steady inertial formulation used here to non-inertial moving-blade unsteady hover results are shown in that reference.

V.

Conclusions

We present an innovative dual-mesh approach for CFD problems which require high-fidelity predictions of aerodynamic loads as well as wakes. The approach applies a traditional unstructured RANS solver to near-body meshes in order to capture complex geometry and wall-bounded viscous effects, then applies an off-body Euler solver that employs both high-order algorithms and AMR to resolve the wake. The offbody solver adopts the block-structured Cartesian AMR strategy traditionally used in the shock-physics community to enable frequent adaptation to resolve unsteady wake flows. The high-order adaptive off-body solver effectively resolves the wake at relatively low cost. The cost savings arise because of the inherent efficiency of the Cartesian off-body solver, which means that significantly larger grids can be solved with only nominal increases in CPU time. Moreover, high-order solutions in the Cartesian off-body solver incur negligible cost. We find that the off-body solver must be coupled with an adequately resolved near-body mesh in order to compute the correct aerodynamic loads and tip vortex pattern in the wake. Adding refinement in the near-body mesh to particular regions, such as around the tip, can greatly improve both the aerodynamic load and wake resolution. Future work will investigate application of AMR to near-body meshes as well. Another important future direction is the development of automated schemes to determine where to apply refinement. The AMR scheme used here targets refinement to regions of high vorticity. While this is shown to work, it requires considerable experimentation to determine the correct threshold value of vorticity to trigger refinement. Automated schemes that can determine an appropriate threshold for different problem classes will enable this scheme to be more readily used in routine production engineering applications. Some promising early work in this direction has been presented in reference.33

16 of 18 American Institute of Aeronautics and Astronautics

Acknowledgments Material presented in this paper is a product of the CREATE-AV Element of the Computational Research and Engineering for Acquisition Tools and Environments (CREATE) Program sponsored by the U.S. Department of Defense HPC Modernization Program Office. This work was conducted at the High Performance Computing Institute for Advanced Rotorcraft Modeling and Simulation (HIARMS). Rotary-wing capability was added to NSU3D by Prof. Dimitri Mavriplis, under sponsorship of the institute. The authors gratefully acknowledge additional contributions by Dr. Buvana Jayaraman, Dr. Aaron Katz, Mr. Mark Potsdam, and Dr. Roger Strawn.

References 1 Caradonna, F. X., R.C. Strawn, and J.O. Bridgeman, “An Experimental and Computational Study of Rotor-Vortex Interactions,” Vertica, Vol. 12, No. 4, 1988, pp. 315–327. 2 Morton, S., R. Cummings, and D. Kholodar, “High Resolution Turbulence Treatment of F/A-18 Tail Buffet,” AIAA2004-1676, 45th AIAA Structures, Structural Dynamics and Materials Conference, Palm Springs CA, 2004. 3 Strawn, R.C., and T.J. Barth, “A finite-volume Euler solver for computing rotary-wing aerodynamics on unstructured meshes,” Journal of the American Helicopter Society, Vol. 38, 1993, pp.61–67. 4 Kang, H.J., and O. J. Kwon,“Unstructured Mesh Navier-Stokes Calculations of the Flow Field of a Helicopter Rotor in Hover,” Journal of the American Helicopter Society, April 2002, pp.90–99. 5 Potsdam, M., and D. J. Mavriplis,“Unstructured Mesh CFD Aerodynamic Analysis of the NREL Phase VI Rotor,” AIAA-2009-1221, 47th AIAA Aerosciences Meeting, Orlando FL, Jan 2009. 6 Dietz, M., E. Karmer, and S. Wagner,“Tip Vortex Conservation on a Main Rotor in Slow Descent Flight Using VortexAdapted Chimera Grids,” AIAA-2006-3478, 24th AIAA Applied Aerodynamics Conference, San Francisco, CA, June 2006. 7 Meakin, R.L.,“Automatic Off-body Grid Generation for Domains of Arbitrary Size,” AIAA-2001-2536, 15th AIAA Computational Fluid Dynamics Conference, Anaheim CA, June 2001. 8 Buning, P.G., D.C. Jespersen, T.H. Pulliam, W.H. Chan, J.P. Slotnick, S.E. Krist, and K. J. Renze, “OVERFLOW User’s Manual, Version 1.8,” NASA Langley Research Center, 1998. 9 Holst, T., and T. Pulliam, “Overset Solution Adaptive Grid Approach Applied to Hovering Rotorcraft Flows,” AIAA2009-3519, 27th AIAA Applied Aerodynamics Conference, San Antonio, TX, June 2009. 10 Holst, T., and T. Pulliam, “Optimization of Overset Solution Adaptive Grids for Hovering Rotorcraft Flows,” 2010 AHS Specialists Meeting on Aeromechanics, San Francisco CA, January 2010. 11 Hariharan, N., “Rotary-Wing Wake Capturing: High Order Schemes Towards Minimizing Numerical Vortex Dissipation,” AIAA Journal of Aircraft, Vol. 39, No. 5, pp. 822-830, 2002. 12 Hariharan, N., and L. Sankar, “High-Order Essentially Non-oscillatory Schemes for Rotary-Wing Wake Computations,” Journal of Aircraft, Vol. 41, No. 2, 2004, pp. 258–267. 13 Sankar, L., N. Yeshala, and N. Hariharan, “Application of Spatially High Order Adaptive Methods for Unsteady Flow over Rotary Wing Configurations,” International Forum on Rotorcraft Multidisciplinary Technology, American Helicopter Society Specialists Meeting, paper No. 21-1, Seoul, Korea, October 2007. 14 Yeshala, N., A.T. Egolf, R. Vasilescu, and L. Sankar, “Application of Higher Order Spatially Accurate Schemes to Rotors in Hover,” AIAA Paper No. 2006-2818, 24th AIAA Applied Aerodynamics Conference, San Francisco, CA, June 2006. 15 Duque, E., L. Sankar, S. Menon, L. Bauchau, S. Ruffin, M. Smith, K. Ahuja, K. Brentner, L. Long, P. Morris, and F. Gandhi, “Revolutionary Physics-based Design Tools for Quiet Helicopters,” AIAA Paper 2006-1068, 44th AIAA Aerospace Sciences Meeting, Reno NV, Jan 2006. 16 Sankaran, V., J. Sitaraman, A. Wissink, A. Datta, B. Jayaraman, M. Potsdam D. Mavriplis, Z. Yang, D. O’Brien, H. Saberi, R. Cheng, N. Hariharan, and R. Strawn, “Application of the Helios Computational Platform to Rotorcraft Flowfields,” AIAA-2010-1230, 48th AIAA Aerospace Sciences Meeting, Orlando FL, Jan 2010. 17 Mavriplis, D. J., and V. Venkatakrishnan, “A Unified Multigrid Solver for the Navier-Stokes Equations on Mixed Element Meshes,” International Journal for Computational Fluid Dynamics, Vol. 8, 1997, pp. 247-263. 18 Sitaraman, J., M. Floros, A. M. Wissink, and M. Potsdam, “Parallel Unsteady Overset Mesh Methodology for a MultiSolver Paradigm with Adaptive Cartesian Grids,” AIAA-2008-7117, 26th AIAA Applied Aerodynamics Conference, Honolulu HI, Jan 2008. 19 Wissink, A. M., J. Sitaraman, V. Sankaran, D. J. Mavriplis, and T. H. Pulliam, “A Multi-Code Python-Based Infrastucture for Overset CFD with Adaptive Cartesian Grids”, AIAA-2008-0927, 46th AIAA Aerosciences Conference, Reno NV, Jan 2008. 20 Sitaraman, J., A. Katz, B. Jayaraman, A. Wissink, V. Sankaran, “Evaluation of a Multi-Solver Paradigm for CFD using Unstructured and Structured Adaptive Cartesian Grids,” AIAA-2008-0660, 46th AIAA Aerosciences Conference, Reno NV, Jan 2008. 21 Wissink, A. M., and S. Shende, “Performance Evaluation of the Multi-Language Helios Rotorcraft Simulation Software, Proceedings of the DoD HPC Users Group Conference, Seattle WA, June 2008. 22 Berger, M.J., and J. Oliger, “Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations,” Journal of Computational Physics, Vol. 53, 1984, pp. 484–512. 23 Berger, M. J., and P. Colella, “Local Adaptive Mesh Refinement for Shock Hydrodynamics,” J. Comp. Phys., 82, 1989, pp. 65–84.

17 of 18 American Institute of Aeronautics and Astronautics

24 Hornung, R. D., A. M. Wissink, and S. R. Kohn, “Managing Complex Data and Geometry in Parallel Structured AMR Applications,” Engineering with Computers, Vol. 22, No. 3-4, Dec. 2006, pp. 181-195. Also see www.llnl.gov/casc/samrai. 25 Hornung, R. D., and S. R. Kohn, “Managing Application Complexity in the SAMRAI object-oriented framework,” Concurrency and Computation: Practise and Experience, Vol. 14, 2002, pp. 347-368. 26 Wissink, A. M., R. D. Hornung, S. Kohn, S. Smith, and N. Elliott, “Large-Scale Parallel Structured AMR Calculations using the SAMRAI Framework”, Proceedings of Supercomputing 2001 (SC01), Denver CO, Nov 2001. 27 Wissink, A. M., D. A. Hysom, and R. D. Hornung, “Enhancing Scalability of Parallel Structured AMR Calculations”, Proceedings of the 17th ACM International Conference on Supercomputing (ICS03), San Francisco CA, June 2003, pp. 336–347. 28 Gunney, B. T. N., A. M. Wissink, and D. A. Hysom, “Parallel Clustering Algorithms for Structured AMR,” J. Parallel. Dist. Computing, 66, 2006, pp. 1419–1430. 29 Pulliam, T. H., “Solution Methods in Computational Fluid Dynamics,” von Karmon Institute for Fluid Mechanics Lecture Series, Numerical Techniques for Viscous Flow Computations in Turbomachinery, Rhode-St-Genese, Belgium, Jan 1986. See http://people.nas.nasa.gov/ pulliam/mypapers/vki notes/vki notes.html. 30 Pulliam, T. H., “Euler and Thin-Layer Navier-Stokes Codes: ARC2D, and ARC3D” Computational Fluid Dynamics Users Workshop, The University of Tennesse Space Institute, Tullahoma TN, March 1984. 31 McAlister, K. W., and R. K. Takahashi, “NACA 0015 Wing Pressure and Trailing Vortex Measurements,” NASA Technical Paper 3151, AVSCOM Technical Report 91-A-003, Nov 1991. 32 Sitaraman, J., and J. D. Baeder, “Evaluation of the Wake Prediction Methodologies used in CFD Based Rotor Airload Computations,” AIAA-2006-3472, 24th AIAA Applied Aerodynamics Conference, San Francisco CA, June 2006. 33 Kamkar, S.J., A. Wissink, V. Sankaran, A. Jameson, “Feature-Driven Cartesian Adaptive Mesh Refinement in the Helios Code,” AIAA-2010-171, 48th AIAA Aerospace Sciences Meeting, Orlando FL, Jan 2010. 34 Potsdam, M. A., and R. C. Strawn, “CFD Simulations of Tiltrotor Configurations in Hover,” Journal of the American Helicopter Society, Vol. 50, No. 1, 2005, pp. 82–94. 35 Wissink, A.M., M. Potsdam, V. Sankaran, J. Sitaraman, Z. Yang, and D. Mavriplis, “A Coupled Unstructured-Adaptive Cartesian CFD Approach for Hover Prediction,”American Helicopter Society 66th Annual Forum, Phoenix AZ, May 11-16, 2010.

18 of 18 American Institute of Aeronautics and Astronautics