Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction

scalability of elliptic solvers in NWP

Research output: Contribution to journalArticle

26 Citations (Scopus)
60 Downloads (Pure)

Abstract

The demand for substantial increases in the spatial resolution of global weather and climate prediction models makes it necessary to use numerically efficient and highly scalable algorithms to solve the equations of large-scale atmospheric fluid dynamics. For stability and efficiency reasons, several of the operational forecasting centres, in particular the Met Office and the European Centre for Medium-Range Weather Forecasts (ECMWF) in the UK, use semi-implicit semi-Lagrangian time-stepping in the dynamical core of the model. The additional burden with this approach is that a three-dimensional elliptic partial differential equation (PDE) for the pressure correction has to be solved at every model time step and this often constitutes a significant proportion of the time spent in the dynamical core. In global models, this PDE must be solved in a thin spherical shell. To run within tight operational time-scales, the solver has to be parallelized and there seems to be a (perceived) misconception that elliptic solvers do not scale to large processor counts and hence implicit time-stepping cannot be used in very high-resolution global models. After reviewing several methods for solving the elliptic PDE for the pressure correction and their application in atmospheric models, we demonstrate the performance and very good scalability of Krylov subspace solvers and multigrid algorithms for a representative model equation with more than 1010 unknowns on 65 536 cores on the High-End Computing Terascale Resource (HECToR), the UK's national supercomputer. For this, we tested and optimized solvers from two existing numerical libraries (the Distributed and Unified Numerics Environment (DUNE) and Parallel High Performance Preconditioners (hypre)) and implemented both a conjugate gradient solver and a geometric multigrid algorithm based on a tensor-product approach, which exploits the strong vertical anisotropy of the discretized equation. We study both weak and strong scalability and compare the absolute solution times for all methods; in contrast to one-level methods, the multigrid solver is robust with respect to parameter variations.
Original languageEnglish
Pages (from-to)2608-2624
Number of pages17
JournalQuarterly Journal of the Royal Meteorological Society
Volume140
Issue number685
Early online date24 Mar 2014
DOIs
Publication statusPublished - Oct 2014

Fingerprint

climate prediction
weather
fluid dynamics
anisotropy
spatial resolution
shell
timescale
resource
method

Keywords

  • numerical weather prediction
  • dynamical core
  • implicit time-stepping
  • elliptic solvers
  • parallel scalability
  • multigrid
  • Krylov subspace methods

Cite this

@article{a55820d3e7ca4e50885b8c70c365a527,
title = "Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction: scalability of elliptic solvers in NWP",
abstract = "The demand for substantial increases in the spatial resolution of global weather and climate prediction models makes it necessary to use numerically efficient and highly scalable algorithms to solve the equations of large-scale atmospheric fluid dynamics. For stability and efficiency reasons, several of the operational forecasting centres, in particular the Met Office and the European Centre for Medium-Range Weather Forecasts (ECMWF) in the UK, use semi-implicit semi-Lagrangian time-stepping in the dynamical core of the model. The additional burden with this approach is that a three-dimensional elliptic partial differential equation (PDE) for the pressure correction has to be solved at every model time step and this often constitutes a significant proportion of the time spent in the dynamical core. In global models, this PDE must be solved in a thin spherical shell. To run within tight operational time-scales, the solver has to be parallelized and there seems to be a (perceived) misconception that elliptic solvers do not scale to large processor counts and hence implicit time-stepping cannot be used in very high-resolution global models. After reviewing several methods for solving the elliptic PDE for the pressure correction and their application in atmospheric models, we demonstrate the performance and very good scalability of Krylov subspace solvers and multigrid algorithms for a representative model equation with more than 1010 unknowns on 65 536 cores on the High-End Computing Terascale Resource (HECToR), the UK's national supercomputer. For this, we tested and optimized solvers from two existing numerical libraries (the Distributed and Unified Numerics Environment (DUNE) and Parallel High Performance Preconditioners (hypre)) and implemented both a conjugate gradient solver and a geometric multigrid algorithm based on a tensor-product approach, which exploits the strong vertical anisotropy of the discretized equation. We study both weak and strong scalability and compare the absolute solution times for all methods; in contrast to one-level methods, the multigrid solver is robust with respect to parameter variations.",
keywords = "numerical weather prediction, dynamical core, implicit time-stepping, elliptic solvers, parallel scalability, multigrid, Krylov subspace methods",
author = "Mueller, {Eike H.} and Robert Scheichl",
year = "2014",
month = "10",
doi = "10.1002/qj.2327",
language = "English",
volume = "140",
pages = "2608--2624",
journal = "Quarterly Journal of the Royal Meteorological Society",
issn = "0035-9009",
publisher = "Wiley-Blackwell",
number = "685",

}

TY - JOUR

T1 - Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction

T2 - scalability of elliptic solvers in NWP

AU - Mueller, Eike H.

AU - Scheichl, Robert

PY - 2014/10

Y1 - 2014/10

N2 - The demand for substantial increases in the spatial resolution of global weather and climate prediction models makes it necessary to use numerically efficient and highly scalable algorithms to solve the equations of large-scale atmospheric fluid dynamics. For stability and efficiency reasons, several of the operational forecasting centres, in particular the Met Office and the European Centre for Medium-Range Weather Forecasts (ECMWF) in the UK, use semi-implicit semi-Lagrangian time-stepping in the dynamical core of the model. The additional burden with this approach is that a three-dimensional elliptic partial differential equation (PDE) for the pressure correction has to be solved at every model time step and this often constitutes a significant proportion of the time spent in the dynamical core. In global models, this PDE must be solved in a thin spherical shell. To run within tight operational time-scales, the solver has to be parallelized and there seems to be a (perceived) misconception that elliptic solvers do not scale to large processor counts and hence implicit time-stepping cannot be used in very high-resolution global models. After reviewing several methods for solving the elliptic PDE for the pressure correction and their application in atmospheric models, we demonstrate the performance and very good scalability of Krylov subspace solvers and multigrid algorithms for a representative model equation with more than 1010 unknowns on 65 536 cores on the High-End Computing Terascale Resource (HECToR), the UK's national supercomputer. For this, we tested and optimized solvers from two existing numerical libraries (the Distributed and Unified Numerics Environment (DUNE) and Parallel High Performance Preconditioners (hypre)) and implemented both a conjugate gradient solver and a geometric multigrid algorithm based on a tensor-product approach, which exploits the strong vertical anisotropy of the discretized equation. We study both weak and strong scalability and compare the absolute solution times for all methods; in contrast to one-level methods, the multigrid solver is robust with respect to parameter variations.

AB - The demand for substantial increases in the spatial resolution of global weather and climate prediction models makes it necessary to use numerically efficient and highly scalable algorithms to solve the equations of large-scale atmospheric fluid dynamics. For stability and efficiency reasons, several of the operational forecasting centres, in particular the Met Office and the European Centre for Medium-Range Weather Forecasts (ECMWF) in the UK, use semi-implicit semi-Lagrangian time-stepping in the dynamical core of the model. The additional burden with this approach is that a three-dimensional elliptic partial differential equation (PDE) for the pressure correction has to be solved at every model time step and this often constitutes a significant proportion of the time spent in the dynamical core. In global models, this PDE must be solved in a thin spherical shell. To run within tight operational time-scales, the solver has to be parallelized and there seems to be a (perceived) misconception that elliptic solvers do not scale to large processor counts and hence implicit time-stepping cannot be used in very high-resolution global models. After reviewing several methods for solving the elliptic PDE for the pressure correction and their application in atmospheric models, we demonstrate the performance and very good scalability of Krylov subspace solvers and multigrid algorithms for a representative model equation with more than 1010 unknowns on 65 536 cores on the High-End Computing Terascale Resource (HECToR), the UK's national supercomputer. For this, we tested and optimized solvers from two existing numerical libraries (the Distributed and Unified Numerics Environment (DUNE) and Parallel High Performance Preconditioners (hypre)) and implemented both a conjugate gradient solver and a geometric multigrid algorithm based on a tensor-product approach, which exploits the strong vertical anisotropy of the discretized equation. We study both weak and strong scalability and compare the absolute solution times for all methods; in contrast to one-level methods, the multigrid solver is robust with respect to parameter variations.

KW - numerical weather prediction

KW - dynamical core

KW - implicit time-stepping

KW - elliptic solvers

KW - parallel scalability

KW - multigrid

KW - Krylov subspace methods

UR - http://www.scopus.com/inward/record.url?scp=84896553020&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1002/qj.2327

U2 - 10.1002/qj.2327

DO - 10.1002/qj.2327

M3 - Article

VL - 140

SP - 2608

EP - 2624

JO - Quarterly Journal of the Royal Meteorological Society

JF - Quarterly Journal of the Royal Meteorological Society

SN - 0035-9009

IS - 685

ER -