Black-box Learning of Parametric Dependencies for Performance Models
V. Ackermann, J. Grohmann, S. Eismann, and S. Kounev. Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018), (October 2018)
Abstract
Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.
Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018)
%0 Conference Paper
%1 AcGrEiKo2018-MRT-DependencyModeling
%A Ackermann, Vanessa
%A Grohmann, Johannes
%A Eismann, Simon
%A Kounev, Samuel
%B Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018)
%D 2018
%K Automated_model_learning DML Optimization PRISMA Performance Self-aware-computing Statistical_estimation_and_machine_learning descartes t_workshop
%T Black-box Learning of Parametric Dependencies for Performance Models
%U http://ceur-ws.org/Vol-2245/mrt_paper_5.pdf
%X Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.
@inproceedings{AcGrEiKo2018-MRT-DependencyModeling,
abstract = {Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.},
added-at = {2020-04-06T11:24:59.000+0200},
author = {Ackermann, Vanessa and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel},
biburl = {https://www.bibsonomy.org/bibtex/226e8df0f687bdfca057b9781eb5a97b3/se-group},
booktitle = {Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018)},
interhash = {20729f84e88c925c9671f81f51e0b7ac},
intrahash = {26e8df0f687bdfca057b9781eb5a97b3},
keywords = {Automated_model_learning DML Optimization PRISMA Performance Self-aware-computing Statistical_estimation_and_machine_learning descartes t_workshop},
month = {October},
series = {CEUR Workshop Proceedings},
timestamp = {2021-01-12T13:39:42.000+0100},
title = {Black-box Learning of Parametric Dependencies for Performance Models},
url = {http://ceur-ws.org/Vol-2245/mrt_paper_5.pdf},
year = 2018
}