Аннотация
Growing data volumes and velocities are driving exciting new methods across
the sciences in which data analytics and machine learning are increasingly
intertwined with research. These new methods require new approaches for
scientific computing in which computation is mobile, so that, for example, it
can occur near data, be triggered by events (e.g., arrival of new data), or be
offloaded to specialized accelerators. They also require new design approaches
in which monolithic applications can be decomposed into smaller components,
that may in turn be executed separately and on the most efficient resources. To
address these needs we propose funcX---a high-performance function-as-a-service
(FaaS) platform that enables intuitive, flexible, efficient, scalable, and
performant remote function execution on existing infrastructure including
clouds, clusters, and supercomputers. It allows users to register and then
execute Python functions without regard for the physical resource location,
scheduler architecture, or virtualization technology on which the function is
executed---an approach we refer to as "serverless supercomputing." We motivate
the need for funcX in science, describe our prototype implementation, and
demonstrate, via experiments on two supercomputers, that funcX can process
millions of functions across more than 65000 concurrent workers. We also
outline five scientific scenarios in which funcX has been deployed and
highlight the benefits of funcX in these scenarios.
Пользователи данного ресурса
Пожалуйста,
войдите в систему, чтобы принять участие в дискуссии (добавить собственные рецензию, или комментарий)