Fourth International Workshop on Advances in High-Performance Computational Earth Sciences: Applications and Frameworks (IHPCES) Session 1

Time and Date: 14:10 - 15:50 on 2nd June 2015

Room: M104

Chair: Henry Tufo

532 2D Adaptivity for 3D Problems: Parallel SPE10 Reservoir Simulation on Dynamically Adaptive Prism Grids [abstract]
Abstract: We present an approach for parallel simulation of 2.5D applications on fully dynamically adaptive 2D triangle grids based on space-filling curve traversal. Often, subsurface, oceanic or atmospheric flow problems in geosciences have small vertical extent or anisotropic input data. Interesting solution features, such as shockwaves, emerge mostly in horizontal directions and require little vertical capturing. \samoa is a 2D code with fully dynamically adaptive refinement, targeted especially at low-order discretizations due to its cache-oblivious and memory-efficient design. We added support for 2.5D grids by implementing vertical columns of degrees-of-freedom, allowing full horizontal refinement and load balancing but restricted control over vertical layers. Results are shown for the SPE10 benchmark, a particularly hard two-phase flow problem in reservoir simulation with a small vertical extent. SPE10 investigates oil exploration by water injection in heterogenous porous media. Performance of \samoa is memory-bound for this scenario with up to 70\% throughput of the STREAM benchmark and a good parallel efficiency of 85\% for strong scaling on 512 cores and 91\% for weak scaling on 8192 cores.
Oliver Meister and Michael Bader
446 A Pipelining Implementation for High Resolution Seismic Hazard Maps Production [abstract]
Abstract: Seismic hazard maps are a significant input into emergency hazard management that play an important role in saving human lives and reducing the economic effects after earthquakes. Despite the fact that a number of software tools have been developed (McGuire, 1976, 1978; Bender and Perkins, 1982, 1987; Ordaz et al., 2013; Robinson et al. 2005, 2006; Field et al., 2003), map resolution is generally low, potentially leading to uncertainty in calculations of ground motion level and underestimation of the seismic hazard in a region. In order to generate higher resolution maps, the biggest challenge is to handle the significantly increased data processing workload. In this study, a method for improving seismic hazard map resolution is presented that employs a pipelining implementation of the existing EqHaz program suite (Assatourians and Atkinson, 2013) based on IBM InfoSphere Streams – an advanced stream computing platform. Its architecture is specifically configured for continuous analysis of massive volumes of data at high speeds and low latency. Specifically, it treats processing workload as data streams. Processing procedures are implemented as operators that are connected to form processing pipelines. To handle large processing workload, these pipelines are flexible and scalable to be deployed and run in parallel on large-scale HPC clusters to meet application performance requirements. As a result, mean hazard calculations are possible for maps with resolution up to 2,500,000 points with near-real-time processing time of approximately 5-6 minutes.
Yelena Kropivnitskaya, Jinhui Qin, Kristy F. Tiampo, Michael A. Bauer
519 Scalable multicase urban earthquake simulation method for stochastic earthquake disaster estimation [abstract]
Abstract: High-resolution urban earthquake simulations are expected to be useful for improving the reliability of the estimates of damage due to future earthquakes. However, current high-resolution simulation models involve uncertainties in their inputs. An alternative is to apply stochastic analyses using multicase simulations with varying inputs. In this study, we develop a method for simulating the responses of ground and buildings to many earthquakes. By a suitable mapping of computations among computation cores, the developed program attains 97.4% size-up scalability using 320,000 processes (40,000 nodes) on the K computer. This enables the computation of more than 1,000 earthquake scenarios for 0.25 million structures in central Tokyo.
Kohei Fujita, Tsuyoshi Ichimura, Muneo Hori, Lalith Maddegedara, Seizo Tanaka
704 Multi-GPU implementations of parallel 3D sweeping algorithms with application to geological folding [abstract]
Abstract: This paper studies some of the CUDA programming challenges in connection with using multiple GPUs to carry out plane-by-plane updates in parallel 3D sweeping algorithms. In particular, attention must be paid to masking the overhead of various data movements between the GPUs. Multiple OpenMP threads on the CPU side should be combined multiple CUDA streams per GPU to hide the data transfer cost related to the halo computation on each 2D plane. Moreover, the technique of peer-to-peer memory access can be used to reduce the impact of 3D volumetric data shuffles that have to be done between mandatory changes of the grid partitioning. We have investigated the performance improvement of 2- and 4-GPU implementations that are applicable to 3D anisotropic front propagation computations related to geological folding. In comparison with a straightforward multi-GPU implementation, the overall performance improvement due to masking of data movements on four GPUs of the Fermi architecture was 23%. The corresponding improvement obtained on four Kepler GPUs was 50\%.
Ezhilmathi Krishnasamy, Mohammed Sourouri, Xing Cai

Fourth International Workshop on Advances in High-Performance Computational Earth Sciences: Applications and Frameworks (IHPCES) Session 2

Time and Date: 16:20 - 18:00 on 2nd June 2015

Room: M104

Chair: Xing Cai

244 Big Data on Ice: The Forward Observer System for In-Flight Synthetic Aperture Radar Processing [abstract]
Abstract: We introduce the Forward Observer system, which is designed to provide data assurance in field data acquisition while receiving significant amounts (several terabytes per flight) of Synthetic Aperture Radar data during flights over the polar regions, which provide unique requirements for developing data collection and processing systems. Under polar conditions in the field and given the difficulty and expense of collecting data, data retention is absolutely critical. Our system provides a storage and analysis cluster with software that connects to field instruments via standard protocols, replicates data to multiple stores automatically as soon as it is written, and provides pre-processing of data so that initial visualizations are available immediately after collection, where they can provide feedback to researchers in the aircraft during the flight.
Richard Knepper, Matthew Standish, Matthew Link
690 Multi-Scale Coupling Simulation of Seismic Waves and Building Vibrations using ppOpen-HPC [abstract]
Abstract: In order to simulate an earthquake shock originating from the earthquake source and the damage it causes to buildings, not only the seismic wave that propagates over a wide region of several 100 km2, but also the building vibrations that occur over a small region of several 10 m2 must be resolved concurrently. Such a multi-scale simulation is difficult because such kind of modeling and implementation by only a specific application are limited. To overcome these problems, a multi-scale weak-coupling simulation of seismic wave and building vibrations using "ppOpen-HPC" libraries is conducted. The ppOpen-HPC, wherein "pp" stands for "post-peta scale", is an open source infrastructure for development and execution of optimized and reliable simulation codes on large-scale parallel computers. On the basis of our evaluation, we confirm that an acceptable result can be achieved that ensures that the overhead cost of the coupler is negligible and it can work on large-scale computational resources.
Masaharu Matsumoto, Takashi Arakawa, Takeshi Kitayama, Futoshi Mori, Hiroshi Okuda, Takashi Furumura, Kengo Nakajima
621 A hybrid SWAN version for fast and efficient practical wave modelling [abstract]
Abstract: In the Netherlands, for coastal and inland water applications, wave modelling with SWAN has become a main ingredient. However, computational times are relatively high. Therefore we investigated the parallel efficiency of the current MPI and OpenMP versions of SWAN. The MPI version is not that efficient as the OpenMP version within a single node. Therefore, in this paper we propose a hybrid version of SWAN. It combines the efficiency of the current OpenMP version on shared memory with the capability of the current MPI version to distribute memory over nodes. We describe the numerical algorithm. With initial numerical experiments we show the potential of this hybrid version. Parallel I/O, further optimization, and behavior for larger number of nodes will be subject of future research.
Menno Genseberger, John Donners
573 Numerical verification criteria for coseismic and postseismic crustal deformation analysis with large-scale high-fidelity model [abstract]
Abstract: Numerical verification of postseismic crustal deformation analysis, computed using a large-scale finite element simulation, was carried out, by proposing new criteria that consider the characteristics of the target phenomenon. Specifically, pointwise displacement was used in the verification. In addition, the accuracy of the numerical solution was explicitly shown by considering the observation error of the data used for validation. The computational resource required for each analysis implies that high-performance computing techniques are necessary to obtain a verified numerical solution of crustal deformation analysis for the Japanese Islands. Such verification in crustal deformation simulations should take on greater importance in the future, since continuous improvement in the quality and quantity of crustal deformation data is expected.
Ryoichiro Agata, Tsuyoshi Ichimura, Kazuro Hirahara, Mamoru Hyodo, Takane Hori, Chihiro Hashimoto, Muneo Hori