Implementation of a moving immersed boundary ...

www.iconCFD.com Implementation of a moving immersed boundary method on a dynamically refining mesh with automatic load balancing

10th OpenFOAM Workshop 2 July 2015 – Ann Arbor MI

Kyle Mooney∗1 and Jacques Papper2 1 - Icon Technology & Process Consulting LLC. 4555 Lake Forest Drive, Suite 650 Cincinnati, OH USA, [email protected] 2 - Icon Technology & Process Consulting Ltd., 7 Rue Auber 31000 Toulouse France, [email protected]

Contact Us ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 Us ICON Technology & Process Consulting Ltd. Contact ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 ICON Technology & Process Consulting Ltd.

1

AGENDA

www.iconCFD.com

• • • •

Introduction Handling moving boundaries Implementation of AMR + load balancing routines Adapting solver and IB libraries to support dynamic remeshing • Initial results • Next steps

Mooney - Papper | OFW10 | Ann Arbor | 06/2015

Contact Us ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 ICON Technology & Process Consulting Ltd.

2

ICON WORLDWIDE

www.iconCFD.com

Offices Staff Locations ICON Agents



3

www.iconCFD.com

HANDLING MOVING BOUNDARIES



4

MOVING

www.iconCFD.com

BOUNDARIES IN FV Overset Grids

Robert H. Nichols, Pieter G. Buning OVERFLOW 2.2 User’s Manual 2010



5

MOVING

www.iconCFD.com

BOUNDARIES IN FV Full Re-Meshing w/ SHM: 1. Displace patches + smooth points 2. If(bad cell quality) then {re-mesh+map fields} 3. Solve NS

Image: Wikipedia.org

Alejandro Roger. Study of mesh deformation features of an open source CFD package and application to a gear pump simulation. Masters Thesis, Univseritat Politechnica de Catalunya. June 2012



6

MOVING

www.iconCFD.com

BOUNDARIES IN FV Adaptive Boundary Fitted

Parallel Adaptive Simplical Re-Meshing for Deforming Domain CFD Computations. Sandeep Menon, Kyle G. Mooney, K.G. Stapf, David P. Schmidt. JCP 2015



7

MOVING

www.iconCFD.com

BOUNDARIES IN FV Adaptive Boundary Fitted



8

www.iconCFD.com

AUTOMATIC REFINEMENT AND PARALLEL LOAD BALANCING



9

AMR

www.iconCFD.com

IN CFD Just a few examples: Gerris [open-source]: S. Popinet. Gerris: a tree-based adaptive solver the for incompressible Euler equations in complex geometries. Journal of Computational Physics 2003 Converge [commercial]: S. Quan, et. Al. Numerical simulations of supersonic diesel spray injection and the induced shock waves. SAE 2014. SpaceX [in-house]: A. Lichtl. S. Jones. GPUs to Mars - Full scale simulation of SpaceX’s Mars rocket engine. GTC 2015.



10

DYNAMIC REFINEMENT

www.iconCFD.com

QUAD – HEX BASED

Quadtree – 2D


Octree – 3D Contact Us ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 ICON Technology & Process Consulting Ltd.

11

DYNAMIC REFINEMENT

www.iconCFD.com

QUAD – HEX BASED 0

Refinement level n: 1 2 3

Quadtree – 2D

4

𝑏𝑎𝑠𝑒𝑆𝑖𝑧𝑒 ∆𝑥 = 2𝑛


Octree – 3D Contact Us ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 ICON Technology & Process Consulting Ltd.

12

DYNAMIC REFINEMENT

www.iconCFD.com

CONFIG: GRADIENT / CURL / REGION DRIVEN



13

DYNAMIC REFINEMENT

www.iconCFD.com

OCTREE REFINEMENT HISTORY

One refinement history octree per level-0 cell ‘super parent’ Contains data required for de-refinement Cannot split octree during load balancing Mooney - Papper | OFW10 | Ann Arbor | 06/2015


14

DYNAMIC REFINEMENT

www.iconCFD.com

GRAPH RE-PARTITIONING

0

1

2

3

0 1 2

Poor processor balance


3

Better processor balance


15

LOAD BALANCING

www.iconCFD.com

PTSCOTCH - GRAPH PARTITIONER

1. Recursive calculation of ‘super parents’ and ‘children’ 2. Weight sub-graph such that children in a super parent cluster are keep together in redistribution 3. Rebalance mesh + associated refinement history



16

ICON AT A

www.iconCFD.com

GLANCE



17

DYNAMIC REFINEMENT

www.iconCFD.com

LARGE SCALE CASE

Eo Oh_d Oh_c Vis ratio Density ratio

196.875 6.32E-01 2.00E-01 10 10

Gravity driven droplet acceleration / breakup Max 5 levels of refinement Equivalent to ~1.2 billion cells of uniform mesh size Actual max mesh size = ~3.5 million



18

ICON AT A

www.iconCFD.com

GLANCE



19

DYNAMIC REFINEMENT

www.iconCFD.com

LARGE SCALE CASE – CELL LEVEL


196.875 6.32E-01 2.00E-01 10 10



20

DYNAMIC REFINEMENT

www.iconCFD.com

LARGE SCALE CASE – MAG(U)


196.875 6.32E-01 2.00E-01 10 10



21

HEX GRIDS

www.iconCFD.com

INTERFACE RECONSTRUCTION

Hexahedral grids open the way for implementation of interface reconstruction methods.

SLIC

PLIC

Sandeep Menon. A Numerical Study of droplet formation and behaviour using interface tracking methods. PhD Thesis. September 2011



22

www.iconCFD.com

ADAPTING IMMERSED BOUNDARY LIBRARY FOR ADAPTIVE GRIDS



23

SOLVER

www.iconCFD.com

ALGORITHM •

Incorporated the following libraries, solvers, and utilities:

•

hexRef8+dynamicRefinedBalancedFvMesh

• • •

decompositionMethod immersedBoundary1 PIMPLE, incompressible flow solver dynamic mesh2 + IB support surfaceTransform for stl manipulation

•

w/

[1] Hrvoje Jasak, Damir Rigler, and Zeljko Tukovic. Finite volume immersed boundary method for turbulent flow simulations. 9th OpenFOAM Workshop, Zagreb Croatia, June 2014. [2] Kyle Mooney, Jacques Papper, and Tyler Voskuilen. Performance evaluation of existing and new VOF simulations techniques: solving, interface treatment, and dynamic meshes. 9th OpenFOAM Workshop, Zagreb Croatia, June 2014.



24

MOVING IB

www.iconCFD.com

STATIC MESH



25

STATIC IB

www.iconCFD.com

ADAPTIVE MESH • •

Cube IB surface in cross flow. Orientation skewed to mesh direction. Refinement triggered by mag(grad(ibMask))) field.



26

STATIC IB

www.iconCFD.com

ADAPTIVE MESH



27

MOVING IB

www.iconCFD.com

DYNAMIC MESH



28

NEXT STEPS

www.iconCFD.com

• Dig into issues with VOF solver • Fix memory issues with PT-Scotch for more efficient decomposition • Robustness Improvements • Validation • Expand to non-orthogonal hex based meshes



29

ACKNOWLEDGEMENTS

www.iconCFD.com

IB Library authors:

Hrvoje Jasak, Damir Rigler, and Zeljko Tukovic Load balancing support:

Tyler Voskuilen (Purdue) & David Martineau (Icon)



30

www.iconCFD.com

THANK YOU

Contact Us ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 Us ICON Technology & Process Consulting Ltd. Contact ICON HQ, Berkshire House, Windsor SL41QN, Windsor, UK P. +44 (0)1753 751400 / [email protected] © 2013 ICON Technology & Process Consulting Ltd.

31