Incollection,

Configure, Generate, Run: Model-based Development for Big Data Processing

, , , , , , , , , and 3 other author(s).
European Project Space on Intelligent Technologies, Software engineering, Computer Vision, Graphics, Optics and Photonics, SCITEPRESS, (2016)

Abstract

The development of efficient and robust algorithms for Big Data processing is a demanding task, which has to cope with the characteristics of this type of data (3Vs). Putting such algorithms as processing elements into larger pipelines adds an extra level of complexity, which can be alleviated by relying on a model-based approach including code generation. This allows data analysts to compose such pipelines on a higher level of abstraction, reducing the development effort as well as the risk of errors. In this chapter, we outline a model-based and adaptive approach to the development of data processing pipelines in heterogeneous processing contexts. It relies on a flexible, tool-supported approach to configuration, which embraces three levels: (a) a heterogeneous processing infrastructure - including reconfigurable hardware, (b) the pipelines as well as (c) the stakeholder applications built upon the pipelines. Furthermore, selected aspects of implementing the approach, which is validated in the context of the financial domain, are presented.

Tags

Users

  • @e.ioannou
  • @eichelbe

Comments and Reviews