Global Hurricane Modeling with NASA

Global Hurricane Modeling with NASA Supercomputing Technology: My Journey with Computational Science Bo-Wen Shen, Shen Ph Ph.D. D [email protected] Department of Mathematics and Statistics Center for Climate and Sustainability Studies Computational Science Research Center San Diego State University Climate Informatics Lab, GMCS 416 S Diego San Di St State t University U i it 16 October 2014 Modeling with NASA Supercomputing Technology

1

San Diego State Univ. Oct 16, 2014

Outline

1. Introduction 2. Supercomputing, Visualization, and Global Modeling • Initial Results with Columbia Supercomputer • Recent Results with Pleiades ((after 2006)) 3. Summary and Future Tasks

Modeling with NASA Supercomputing Technology

3


Computational Science July, 2004

•

•

June, 2005

Dec., 2007

Computational Science (CS) is defined as an inter-disciplinary field with th goals the l off understanding d t di and d solving l i complex l problems bl using i hi high-end h d computing facilities. CS is identified as one of the most important fields of the 21st century to contribute to the scientific, scientific economic, economic social and national security goals of USA by the President’s Information Technology Advisory Committee (PITAC).


4


Early Efforts with NCAR CAM March 1999 – September 2003

NCAR Community Atmosphere Model (CAM)

• Unix System and Network Programming: Unix curses, device control (serial I/O), file system control, inter-process communication (pipes, semaphore, shared memory, TCP/IP sockets), process control, signal handling. • Supercomputing (Parallel/Distributed/Cluster Computing): MPI (Message Passing Interface), MPI-2 remote memory access, MLP (MultiLevel Parallelism), OpenMP, ESMF (Earth Science Modeling Framework), POSIX Threads, and JAVA Threads. Knowledge of Gird computing. • Software: Fortran (F77/F90/F95), OOP (Object Oriented Programming), C/C++, JAVA, Basic, Pascal, UNIX Shells, UNIX m4 script, PERL, Python, PHP, HTML, XML, XHTML, CGI, AWK. CVS (Concurrent Version System), GNU Make, gdb, LaTex, MATLAB, VMWARE, Secure Shell, MS-Office, VIS5D, AVS, GrADS, NCAR Graphics, GEMPAK. Modeling with NASA Supercomputing Technology

5

1992-1994

1999-2003


Early Efforts with GEOS-4 and GEOS-5 The first paper for the high-resolution NWP with the model (Lin, Shen, Putman, Chern, 2003) March 1999 -- Sep S 2003

Monthly reports on the first EMSF grid components (IO, Solar, Radiation) of the GEOS-5 Dec 2003 Dec,

Nov 2003 Nov,

Oct 2003 Oct,

12 Jan 2004


6


High-impact Tropical Weather: Hurricanes Each year tropical cyclones (TCs) cause tremendous economic losses and many fatalities throughout the world. Examples p include Hurricanes Katrina ((2005)) and Sandy (2012) . Hurricane Katrina (2005) (Shen et al., 2006b; 2013a) • • • •

Cat 5, 902 hPa, with two stages of rapid intensification The sixth-strongest Atlantic hurricane ever recorded. The third-strongest landfalling U.S. hurricane ever recorded. recorded The costliest Atlantic hurricane in history! ($100+ billion)

Hurricane Sandy (2012) (Shen et al., 2013c) • • •

The deadliest and the most destructive TC of 2012 Atlantic hurricane season The second-costliest hurricane in United States history ($50 or 75$ billion) The largest Atlantic hurricane on record


7


Global Visualizations of Hurricanes


8


08. 02. 2004 – Initial Columbia Promising


9


Columbia Supercomputer (2004) •

•

•

•

• • •

Comprised of an integrated cluster of 20 interconnected SGI® Altix® 512processor systems, for a total of 10,240 Intel® Itanium® 2 processors, Columbia was built and installed at the NASA Advanced Supercomputing facility at Ames in less than 120 days. Columbia was ranked as the second most powerful supercomputer on the TOP500 list in November 2004 at a LINPACK rating of 51.87 teraflops, or 51 87 trillion floating point calculations per second 51.87 second. "With Columbia, scientists are already seeing dramatic improvements in the fidelity of simulations in such areas as hurricane track prediction, global ocean circulation, p prediction of large g scale structures in the universe, and the p physics y of supernova detonations," The computational efficiency allows simulations at a resolution of a quarter of a degree, which is double the resolution currently adopted by most global models i operational in ti l weather th centers. t Columbia was named to honor the crew of the Space Shuttle Columbia lost Feb. 1, 2003. http://www nasa gov/home/hqnews/2004/oct/HQ 04353 columbia html http://www.nasa.gov/home/hqnews/2004/oct/HQ_04353_columbia.html Improved Linux kernel for scalability (from 16- to 512+ CPUs)


10


Supercomputers @Top 500.org 1: Roadrunner 2: Jugar 3: NASA Pleiades (11/2008)

Japan Earth Simulator (06/2002) (38.86 Tflops)

1: IBM/DOE BG/L Beta (70.72 Tflops) 2: NASA Columbia (51.87 Tflops) 3: Japan ES (38.86 Tflops) (11/2004)


11


Professional Achievements 1. Ten (Five) awards from UMCP, SAIC and NASA/GSFC Since 2001 (2006). 2. Research results featured by Dr. Jack Kaye (Associate Director at NASA/HQs) at the Interdepartmental Hurricane Conferences in 2012, 2013, and 2014. 3 Research 3. R h results lt ffeatured t d iin a recentt P President's id t' Corner C article ti l off UCAR Magazine by Dr. Rick Anthes in 2011. The article is entitled ``Turning the Tables on Chaos: Is the atmosphere more predictable than we assume?’’ 4. Research results featured in NASA News Stories ((07/2010 and 11/2010). ) It was also translated in Chinese by Science and Technology Division, Taipei Economic and Cultural Representative Office in the United States (駐美國台北經濟文化代表處科技組). 5 Research results appeared in news medias 5. medias, such as MSNBC MSNBC, PhysOrg PhysOrg.com, com National Geographic--Indonesia, ScienceDaily, EurekAlert, Yahoo News, TechNews Daily, Scientific Computing, HPCwire, etc. (2010) 6. Research projects selected as one of top 4 demonstrations at NASA Booth for Supercomputing Conferences (SC) in 2004, 2008, 2009, and 2010. 7. 7. Selected Selected as as Journal Journal Highlights Highlights by by American American Geophysical Geophysical Union Union (07/2006) (07/2006) 8. 8. Research Research results results featured featured in in Science Science magazine magazine (08/2006) (08/2006)

“Is new science “I i being b i produced d d or just j t really ll cooll pictures?” i t ?” which was raised by Mahlman and others who have reservations Modeling with NASA Supercomputing Technology

13


NASA Supercomputing and Visualization Systems Pleiades Supercomputer (as June 2013) • one of a few petascale supercomputers • Rmax of 1,240 teraflops (LINPACK); Rpeak off 2,880 2 880 tteraflops fl • 162,496 cores in total; Intel Xeon processors, Nehalem, Westmere,Sandy Bridge, g , Ivyy Bridge, g , • 417 TB memory • 3.1 PB disk space • Largest InfiniBand network.

• Large-scale Large scale visualization system – 8x16 LCD tiled panel display – 245 million pixels • 128 nodes – 1024 cores, 128 GPUs • InfiniBand (IB) interconnect to Pleiades – 2D torus topology – High-bandwidth Modeling with NASA Supercomputing Technology

15


Predicting Genesis of Six TCs in May 2002 ``Although some aspects of the transformation of atmospheric disturbances into tropical cyclones are relatively well understood, the general problem of tropical cyclogenesis remains in large measure, one of the greatest mysteries of the tropical atmosphere.’’ – Kerry Emanuel, The Divine Wind (2005)

Init at 05/06

Init at 05/01

TC02B (May 10-12)

TC01A (May 6-10)

simulation, i l ti 2007; 2007 visualization, i li ti 2008 2008; paper submitted b itt d 2007 2007,and d published bli h d 2012 2012, 28pp 28 Kesiny (May 3-11)

Init at 05/11 Typhoon yp Hagibis (May 15-21)

May 1

May 6

May 11

May 16

May 21

May 26

Errol (May 9-14)

Kesiny 01A

Init at 05/22

Errol 02B

Hurricane Alma (May 24- June 1)

Hagibis Alma

Best tracks (observations) indicated by blue lines:


16


High-impact Tropical Weather: Hurricanes Each year tropical cyclones (TCs) cause tremendous economic losses and many fatalities throughout the world. Examples p include Hurricanes Katrina ((2005)) and Sandy (2012) . Hurricane Katrina (2005) (Shen et al., 2006b; 2013a) • • • •

Cat 5, 902 hPa, with two stages of rapid intensification The sixth-strongest Atlantic hurricane ever recorded. The third-strongest landfalling U.S. hurricane ever recorded. recorded The costliest Atlantic hurricane in history! ($100+ billion)

Hurricane Sandy (2012) (Shen et al., 2013c) • • •

The deadliest and the most destructive TC of 2012 Atlantic hurricane season The second-costliest hurricane in United States history ($50 or 75$ billion) The largest Atlantic hurricane on record


17


7-day Simulation and Visualization of Sandy

Collaboration with Dr. David Ellsworth of NASA/ARC/NAS Shen, B Sh B.-W., W B. B Nelson, N l W W.-K. K T Tao, and dY Y.-L. L Li Lin, 2013 2013a: Ad Advanced d Vi Visualizations li ti off S Scale l IInteractions t ti off Tropical Cyclone Formation and Tropical Waves. IEEE Computing in Science and Engineering, vol. 15, no. 2, pp. 47-59, March-April 2013, doi:10.1109/MCSE.2012.64. Modeling with NASA Supercomputing Technology

18


Are the simulations of TC genesis consistent with Chaos theory? • The butterfly effect of first kind: sensitive dependence on initial conditions. • The butterfly effect of second kind: a metaphor (or symbol ) that small perturbations can alter large-scale structure. • Lorenz’s studies suggested finite predictability and nonlinearity as the source of chaos. • Increased degree of nonlinearity (e.g., multiscale interactions) can stabilize solutions and thus improve simulations (Shen et al al., 2014a 2014a,b). b) r=25

Lorenz Model

High-order Lorenz Model

Lorenz (1963)

Shen, B.-W., 2014a

Negative Nonlinear Feedback

The studies by Lorenz (1963, 1972) laid the foundation for chaos theory, which was viewed as the third scientific revolution of the 20th century after relativity and quantum mechanics (e.g. Gleick, 1987; Anthes 2011). Modeling with NASA Supercomputing Technology

19


Lorenz Models D, H, and N refer to as the dissipative terms, the heating term, and nonlinear terms associated with the primary modes (low wavenumber modes), respectively. Ds, Hs, and Ns refer to as the dissipative terms, the heating term, and nonlinear terms associated with the secondary modes (high wavenumber modes), respectively. NLM refers to the non-dissipative Lorenz mode.

D

H

Linearzied 3DLM

V

V

3DLM

V

V

V

X c = Yc = ± b(r − 1)

V

V

( X c , Yc ) = ( ± 2σr ,0)

3D-NLM

N

Ds

Hs

Ns

Critical points for (X,Y)

rc

remarks

“1”

5DLM

V

V

V

V

6DLM

V

V

V

V

V V

V

X c = Yc ~ ± 2b(r − 1)

Unstable as r>1

24.74 conservative

42.9

X c = Yc = ± b( Z c + 2Z1c )

41.1

Shen, B.-W., 2014a: Nonlinear Feedback in a Five-dimensional Lorenz Model. J. of Atmos. Sci. 71, 1701–1723. doi: http://dx.doi.org/10.1175/JAS-D-13-0223.1 Shen, B.-W., 2014b: On the Nonlinear Feedback Loop p and Energy gy Cycle y of the Non-dissipative p Lorenz Model. Nonlin. Processes Geophys. Discuss., 1, 519-541, 2014. www.nonlin-processes-geophys-discuss.net/1/519/2014/ Shen, B.-W., 2014c: Nonlinear Feedback in a Six-dimensional Lorenz Model. Impact of an Additional Heating Term. (to be submitted to JAS) Modeling with NASA Supercomputing Technology

20


Reviewer’s Comments


21


AIST11 data transfer Global Multiscale Modeling

Supercomputing & Visualization Satellite Data

Multiscale Analysis Scalable framework for global EMD, ensemble EMD

Scalable Multiscale Analysis and, multi-dimensional ensemble EMDPackage Modeling with NASA Supercomputing Technology

22

NASA AIST CAMVisCAMVis MAP(PI: Shen): $1,107K, 05/2012–08/2015, Integration of the NASA CAMVis and Multiscale Analysis Package (CAMVisMAP) for Tropical Cyclone Climate Study. San Diego State Univ. Oct 16, 2014

AIST14 (pending)

NASA AIST Project (PI: Shen): $957.8K, 05/2015-04/2017 (pending) Integration of Concurrent Ensemble Hierarchical Modeling and FusionBased Multivariate Data Visualization into the NASA CAMVis for Improving Climate Simulations Modeling with NASA Supercomputing Technology

23


Published Articles since 2010 Journal Articles: 1. 2. 3. 4.

5. 6.

7 7. 8.

Shen, B.-W., 2014a: Nonlinear Feedback in a Five-dimensional Lorenz Model. J. of Atmos. Sci. 71, 1701–1723. doi: http://dx.doi.org/10.1175/JAS-D-13-0223.1 Shen, B.-W., M. DeMaria, J.-L. F. Li and S. Cheung, 2013c: Genesis of Hurricane Sandy (2012) simulated with a global mesoscale model, Geophys. Res. Lett., 40, 4944–4950, doi:10.1002/grl.50934. Shen, B.-W., B. Nelson, S. Cheung, W.-K. Tao, 2013b: Improving NASA’s Multiscale Modeling Framework for Tropical Cyclone Climate Study. IEEE Computing in Science and Engineering, vol. 15, no 5, pp 56-67. Sep/Oct 2013. Shen, B.-W., B. Nelson, W.-K. Tao, and Y.-L. Lin, 2013a: Advanced Visualizations of Scale Interactions of Tropical Cyclone Formation and Tropical Waves. IEEE Computing in Science and Engineering, vol. 15, no. 2, pp. 47-59, March-April 2013, doi:10.1109/MCSE.2012.64. Shen, B.-W., W.-K. Tao, and Y.-L. Lin, and A. Laing, 2012: Genesis of Twin Tropical Cyclones as Revealed by a Global Mesoscale Model: The Role of Mixed Rossby Gravity Waves. J. Geophys. Res. 117, D13114, doi:10.1029/2012JD017450. 28pp Shen, B.-W., W.-K. Tao, and B. Green, 2011: Coupling Advanced Modeling and Visualization to Improve High-Impact Tropical Weather Prediction (CAMVis). IEEE Computing in Science and Engineering (CiSE), vol. 13, no. 5, pp. 56-67, Sep./Oct. 2011, doi:10.1109/MCSE.2010.141. Shen B.-W., Shen, B -W W.-K. W -K Tao, Tao and M M.-L. -L Wu Wu, 2010b: African Easterly Waves in 30-day High resolution Global Simulations: A Case Study during the 2006 NAMMA Period. Geophys. Res. Lett., 37, L18803, doi:10.1029/2010GL044355. Shen, B.-W., W.-K. Tao, W. K. Lau, R. Atlas, 2010a: Predicting Tropical Cyclogenesis with a Global Mesoscale Model: Hierarchical Multiscale Interactions During the Formation of Tropical Cyclone Nargis (2008) . J. Geophys. Res.,115, D14102, doi:10.1029/2009JD013140.

Magazine g Articles: 9.

Shen, B.-W., S. Cheung, J.-L. F. Li, and Y.-L. Wu, 2013e: Analyzing Tropical Waves using the Parallel Ensemble Empirical Model Decomposition (PEEMD) Method: Preliminary Results with Hurricane Sandy (2012), NASA ESTO Showcase . IEEE Earthzine. 10. Shen, B.-W., 2013f: Simulations and Visualizations of Hurricane Sandy (2012) as Revealed by the NASA CAMVis. NASA ESTO Showcase. IEEE Earthzine. posted December 2, 2013. Papers under review/preparation: 11. Shen, B.-W., 2014b: On the Nonlinear Feedback Loop and Energy Cycle of the Non-dissipative Lorenz Model. (accepted, NPGD) 12. Shen, B.-W., 2014c: Nonlinear Feedback in a Six-dimensional Lorenz Model. Impact of an Additional Heating Term. (to be submitted) Modeling with NASA Supercomputing Technology

24
