Parallel Distributed Computing using Python
Advances in Water Resources, 34 (9):
1124--1139 (September 2011)DOI: 10.1016/j.advwatres.2011.04.013
Abstract
This work presents two software components aimed to relieve the costs of accessing high-performance parallel computing resources within a Python programming environment: MPI for Python and PETSc for Python. MPI for Python is a general-purpose Python package that provides bindings for the Message Passing Interface (MPI) standard using any back-end MPI implementation. Its facilities allow parallel Python programs to easily exploit multiple processors using the message passing paradigm. PETSc for Python provides access to the Portable, Extensible Toolkit for Scientific Computation (PETSc) libraries. Its facilities allow sequential and parallel Python applications to exploit state of the art algorithms and data structures readily available in PETSc for the solution of large-scale problems in science and engineering. MPI for Python and PETSc for Python are fully integrated to PETSc-FEM, an MPI and PETSc based parallel, multiphysics, finite elements code developed at CIMEC laboratory. This software infrastructure supports research activities related to simulation of fluid flows with applications ranging from the design of microfluidic devices for biochemical analysis to modeling of large-scale stream/aquifer interactions. ⺠We present two packages for parallel distributed computing with Python. ⺠MPI for Python (mpi4py) provides bindings for the MPI standard. ⺠PETSc for Python (petsc4py) provides bindings for PETSc libraries. ⺠MPI and PETSc for Python target large-scale scientific application development. ⺠Performance tests confirm that the Python layer introduces acceptable overhead.