Configure, Generate, Run: Model-based Development for Big Data Processing

, , , , , , , , , , , , and . European Project Space on Intelligent Technologies, Software engineering, Computer Vision, Graphics, Optics and Photonics, SCITEPRESS, (2016)


The development of efficient and robust algorithms for Big Data processing is a demanding task, which has to cope with the characteristics of this type of data (3Vs). Putting such algorithms as processing elements into larger pipelines adds an extra level of complexity, which can be alleviated by relying on a model-based approach including code generation. This allows data analysts to compose such pipelines on a higher level of abstraction, reducing the development effort as well as the risk of errors. In this chapter, we outline a model-based and adaptive approach to the development of data processing pipelines in heterogeneous processing contexts. It relies on a flexible, tool-supported approach to configuration, which embraces three levels: (a) a heterogeneous processing infrastructure - including reconfigurable hardware, (b) the pipelines as well as (c) the stakeholder applications built upon the pipelines. Furthermore, selected aspects of implementing the approach, which is validated in the context of the financial domain, are presented.

Links and resources

BibTeX key:
search on:

Comments and Reviews  

There is no review or comment yet. You can write one!


Cite this publication